Markov Chains

Enter each row of the transition matrix on separate lines, separated by spaces or tabs.
Lines containing letters are ignored, with the following exceptions:
- the letter "e" may be used in scientific notification, e.g. "1.2e-3"
- A line that begins with "states" is assumed to be labels for the states. If not present, the default states are 0, 1, ....
The number of rows entered determines the number of states in the chain.
Any line with fewer columns than rows will be padded with zeros. For example, in a six-state chain, "0.5 0 0.5" will be changed to "0.5 0 0.5 0 0 0."
All rows are normalized so that the total probability is 1.

steps =

Simulated runs of the process

Initial state:
Repeat every 1 seconds

Analysis results

For each recurrent class [r₁, r₂, ..., r_k], we compute:

Stable distributions: The stable distribution is P=[p₁, p₂, ..., p_k] which is unchanged by the transition matrix; that is, PT_r = P (where T_r is restricted to the recurrent class). For non-periodic chains, P appears in T_rⁿ for large n (i.e., as n→∞). For a chain with period h, it can be found by averaging across T_rⁿ, T_rⁿ⁺¹, ..., T_r^n+h–1.
Passage times: For states r_i (left column) and r_j (the top row) within the recurrent class, we compute τ_i,j, the expected number of steps ("passage time") to go from r_i to r_j. (When i = j, τ_i,i is the expected time to return to r_i after at least 1 step, rather than 0.)

(We don't bother doing these computations for absorbing states, because both the stable distribution and the passage times would be "1.")

This chain has no recurrent classes.

Absorption probabilities: If the chain begins in a transient state, it will eventually settle into an absorbing state or a recurrent class. This table shows the distribution of the "final destination" of each transient state, as well as the expected number of steps to move to a final destination.

This chain has no transient states.

Technical and release notes

I created this page in September 2024, while teaching a unit on Markov chains in our Operations Research course. By automating the (tedious) computations involved with analyzing Markov chains, students have more time to explore the impact of changing various parameters in the chain. For example:

What happens to the stable distribution of a queueing chain when we alter the arrival distribution, or change the length of time to serve the each customer?
How are passage times changed if we increase/decrease the transition probabilities for one state? Can we identify "bottlenecks" that can be tweaked to slow down or speed up a chain?

This first release has a limited number of "presets" included. If you create a chain that you like, you can simply copy the text from the "edit" page, and paste it in later.

October 1, 2024: added the ability to show a graph of the stable distributions, and sample depictions of how the process state changes over time.

If you have bug reports or suggestions for improvement, please feel free to send me an email. And if you think you have a chain that might be general interest, you can send it to me for possible inclusion in later releases.