1. Random
  2. 0
  3. 1
  4. 2
  5. 3
  6. 4
  7. 5
  8. 6
  9. 7
  10. 8
  11. 9
  12. 10
  13. 11
  14. 12
  15. 13
  16. 14
  17. 15
  18. 16
  19. 17

16. Markov Processes

Summary

A Markov process is a random process in which the future is independent of the past, given the present. Thus, Markov processes are the natural stochastic analogs of the deterministic processes described by differential and difference equations. They form one of the most important classes of random processes.

General Theory

  1. Introduction
  2. Potentials and Generators

Discrete-Time Markov Chains

  1. Discrete-Time Chains
  2. Recurrence and Transience
  3. Periodicity
  4. Limiting Behavior
  5. Time Reversal

Special Discrete-Time Chains

  1. The Ehrenfest Chains
  2. The Bernoulli-Laplace Chain
  3. Reliability Chains
  4. The Branching Chain
  5. Queuing Chains
  6. Birth Death Chains
  7. Random Walks on Graphs

Continuous-Time Markov Chains

  1. Basic Structure
  2. Transition Matrices and Generators
  3. Potential Matrices
  4. Limiting Behavior

Apps

Sources and Resources

Quote