This applet models a Markov chain with state space . The states are shown as balls in the top graph, with the current state colored red. At time 0, the initial state can be chosen be clicking on the desired state. As the process runs, the chain moves from state to state. The time and the sate are recorded at each update in the record table. The proportion of time that the chain is in each state is shown visually in the graph box and displayed numerically in the distribution table. In the limit, these proportions should converge to an appropriate limiting distribution.
The number of state can be varied with a scroll bar, and the transition probabilities can be specified in a dialog box that is launched by clicking on the transition probability button. Several special transition matrices can be selected from the dialog box:
The gambler's ruin chain is a simple random walk on with absorbing barriers. Thus, the transition matrix is given by
The random walk chain is a simple random walk on with reflecting barriers. Thus, the transition matrix is given by
The Ehrenfest chain models a simple diffusion process. Suppose that we have two urns containing a total of balls. At each time unit, a ball is selected at random (so that each ball is equally likely to be chosen), and then moved to the opposite urn. The state at time is the number of balls in urn 1. The transition matrix is given by
The Bernoulli-Laplace chain models a simple diffusion process. Suppose that we have two urns, each with balls. Of the balls, are black and are white. At each time unit, a ball is selected from each urn and the balls are interchanged. The state at time is the number of black balls in urn 1. The transition matrix is given by
This genetics chain models a gene that consists of sub-units, each of which can be normal or mutant. The state of the gene is the number of mutant sub-units. The Markov chain governs the splitting of the gene, and has transition matrix given by