Markov Chain Calculator
Enter the row-stochastic transition matrix P and an initial distribution p₀. The calculator left-multiplies p₀ by P at each step to track how the distribution evolves, then solves (Pᵀ − I)·π = 0 with Σ πᵢ = 1 to find the steady-state distribution.
Two answers, reached two different ways
A Markov chain models a system moving between a fixed set of states, where the next state depends only on the current one and not on how it was reached. That memoryless property is a strong assumption and what makes the whole thing tractable: one matrix of transition probabilities describes the system entirely.
This calculator iterates such a chain from a starting distribution, showing how the probabilities shift step by step, and then solves for the steady state the chain settles into over the long run.
The step-by-step distributions and the steady state answer different questions. The first says where the system probably is after a given number of transitions; the second says where it ends up regardless of its start, and is found by solving equations rather than iterating.
Reporting both makes the relationship visible. Watching the iterated distributions converge on the algebraically derived vector demonstrates what a steady state actually is.
How to use this calculator
- Enter the transition matrix Commas between entries and semicolons between rows. The entry in row i, column j is the probability of moving from state i to state j.
- Check each row sums to 1 This is enforced. A row is a complete probability distribution over where the system goes next, so anything else is rejected with the offending row sums reported.
- Enter the initial distribution One entry per state. It is normalised automatically if the entries do not sum to 1, and left blank it defaults to certainty in the first state.
- Choose how many steps to display Up to ten are shown. The steady state does not depend on this — it is solved for directly rather than read off the last row.
How the chain is advanced and solved
The distribution is a row vector and advances by multiplying it into the matrix from the left. Each new entry is the sum over all states of the probability of being there times the probability of moving from there to the target — so column j of the matrix, not row j, determines the new probability of state j.
That orientation is the one detail worth getting right. Rows describe where you go from a state; columns describe how you arrive at one. Multiplying in the wrong order produces plausible-looking vectors that answer a different question.
The steady state is not obtained by iterating further. It is the distribution a step leaves unchanged, which makes it the solution of a linear system: the transposed matrix minus the identity, applied to the vector, gives zero. That system is underdetermined, since any multiple of a solution is also one, so the last equation is replaced by the requirement that the entries sum to 1.
The system is then solved by Gaussian elimination with partial pivoting. When a column offers no usable pivot the solution is not unique, and the tool says so rather than returning one of many — which happens for a chain that is periodic or splits into parts that never communicate.
What each input means
- P Transition matrix — form field “Transition matrix P (rows separated by ;)”
- Square, with every row summing to 1. The entry in row i, column j is the probability of moving from state i to state j in one step.
- p₀ Initial distribution — form field “Initial distribution (comma-separated, optional)”
- One probability per state, describing where the system starts. Normalised if it does not sum to 1.
- k Steps to display — form field “Steps to display”
- How many iterations to report, capped at ten. It does not affect the steady state.
- π Steady state
- The distribution a step leaves unchanged. Solved for directly, and reported as unavailable when it is not unique.
Worked examples
Every number below is produced by the same calculation engine the tool above runs. Nothing here is typed by hand, so the walkthrough cannot drift from what you get when you enter the same values yourself.
A two-state chain from a certain start
A chain that stays put with probability 0.7 from the first state and 0.6 from the second, started with certainty in the first.
Inputs Transition matrix P (rows separated by ;) = 0.7, 0.3; 0.4, 0.6, Initial distribution (comma-separated, optional) = 1, 0, Steps to display = 5
- Transition matrix 2×2 [0.7, 0.3; 0.4, 0.6]
- Initial distribution p₀ = (1, 0)
- Step 1 p1 = (0.7, 0.3)
- Step 2 p2 = (0.61, 0.39)
- Step 3 p3 = (0.583, 0.417)
- Step 4 p4 = (0.5749, 0.4251)
- Step 5 p5 = (0.57247, 0.42753)
- Steady-state distribution π = (0.571429, 0.428571)
Result p5 = (0.57247, 0.42753); π = (0.571429, 0.428571)
The distribution moves quickly at first and then barely at all: most of the journey happens in the first two or three steps. Rapid early convergence is typical when both states are reachable from each other.
By the fifth step the iterated vector is already close to the steady state reported below it. The two were computed by completely different routes, so their agreement is a genuine check on both.
A three-state chain over more steps
Three states with a mix of staying and moving probabilities, started with certainty in the first and iterated for ten steps.
Inputs Transition matrix P (rows separated by ;) = 0.6, 0.3, 0.1; 0.2, 0.5, 0.3; 0.1, 0.2, 0.7, Initial distribution (comma-separated, optional) = 1, 0, 0, Steps to display = 10
- Transition matrix 3×3 [0.6, 0.3, 0.1; 0.2, 0.5, 0.3; 0.1, 0.2, 0.7]
- Initial distribution p₀ = (1, 0, 0)
- Step 1 p1 = (0.6, 0.3, 0.1)
- Step 2 p2 = (0.43, 0.35, 0.22)
- Step 3 p3 = (0.35, 0.348, 0.302)
- Step 4 p4 = (0.3098, 0.3394, 0.3508)
- Step 5 p5 = (0.28884, 0.3328, 0.37836)
- Step 6 p6 = (0.2777, 0.328724, 0.393576)
- Step 7 p7 = (0.271722, 0.326387, 0.40189)
- Step 8 p8 = (0.2685, 0.325088, 0.406412)
- Step 9 p9 = (0.266759, 0.324377, 0.408865)
- Step 10 p10 = (0.265817, 0.323989, 0.410194)
- Steady-state distribution π = (0.264706, 0.323529, 0.411765)
Result p10 = (0.265817, 0.323989, 0.410194); π = (0.264706, 0.323529, 0.411765)
The initial certainty spreads across all three states within a couple of steps, and the third accumulates the most probability despite starting empty — its self-transition probability of 0.7 makes it hardest to leave.
Convergence is slower than in the two-state case, which is why ten steps are worth displaying. The number of states and the size of the off-diagonal probabilities both affect how quickly the starting point is forgotten.
Reading the result
What the steady state is not
It is not a state the system settles into and stops moving. The system keeps transitioning forever; the probabilities are what stop changing. Over a long run the entries give the fraction of time spent in each state.
When no unique steady state is reported
The chain either alternates on a cycle rather than settling, or it contains groups of states that cannot reach one another so that the long-run behaviour depends on where it started. Both are real properties of the chain rather than computational failures.
When you would use this
Long-run market or occupancy share
Where the probability of switching between options each period is stable, the steady state gives the eventual split, whatever split you begin from.
Weather and queue models
A day's weather modelled as depending only on the previous day's, or a queue changing state each interval, are the standard introductory chains and fit this form directly.
Assumptions and limitations
What this calculator assumes
- The matrix is square and every row sums to 1 within a small tolerance.
- The chain is memoryless: the next state depends only on the current one.
- Transition probabilities are constant over time.
- The initial distribution has one entry per state, and is rescaled if its entries do not already sum to 1.
Where it stops being the right tool
- At most ten iterations are displayed, however many steps are requested.
- No absorption analysis: expected time to absorption and absorption probabilities are not computed.
- Continuous-time chains, with rates rather than per-step probabilities, are out of scope.
Common mistakes
Entering the matrix transposed
Why it happens. Both orientations look equally natural, and a transposed matrix often still has rows summing to 1, so it passes the check and produces a full set of plausible results for the wrong chain.
How to avoid it. Read one row aloud: it should be the complete set of destinations from a single state. If your rows describe arrivals into a state instead, transpose before entering.
Reading the last displayed step as the steady state
Why it happens. After several iterations the values barely change, so the final row looks like the answer and the line below it looks like a repetition.
How to avoid it. Use the steady-state line. It is solved exactly rather than approached, and on a slowly converging chain the two can still differ noticeably.
Expecting a steady state from every chain
Why it happens. Most textbook examples have one, so its absence reads as a limitation of the tool rather than a fact about the matrix.
How to avoid it. Check whether every state can reach every other, and whether the chain can return to a state at irregular intervals rather than only on a cycle. Failing either is what removes uniqueness.
Frequently asked questions
What does row-stochastic mean?
That each row of the matrix is itself a probability distribution: every entry lies between 0 and 1, and the row sums to 1. It expresses the fact that from any state the system must go somewhere.
Why might the steady state not be unique?
Because the chain is periodic or reducible. A periodic chain cycles rather than settling; a reducible one has groups of states that cannot reach each other, so the long-run answer depends on where it began. The solver reports that rather than picking one solution.
How is the distribution updated?
By left-multiplication: the new probability of a state is the sum over all states of the current probability of being there times the probability of moving from there to it. That means column j of the matrix determines the new entry j, not row j.
Does the starting distribution affect the steady state?
No, provided the chain has a unique one. It affects only how many steps convergence takes and from which direction. That independence from the starting point is the property that makes the steady state worth computing.