Suppose that \( S \) is an interval of integers, either finite or infinite. A birth-death chain on \( S \) is a Markov chain \( \bs{X} = (X_0, X_1, X_2, \ldots) \) on \( S \) with transition probability matrix \( P \) of the form.
\[ P(x, x - 1) = q(x), \; P(x, x) = r(x), \; P(x, x + 1) = p(x); \quad x \in S \]where \( p \), \( q \), and \( r \) are nonnegative functions on \( S \) with \( p(x) + q(x) + r(x) = 1 \) for \( x \in S \). If the interval \( S \) has a minimum value \( a \in \Z \) then of course we must have \( q(a) = 0 \). If \( r(a) = 1 \), the boundary point \( a \) is said to be absorbing and if \( p(a) = 1 \), then \( a \) is said to be reflecting. Similarly, if the interval \( S \) has a maximum value \( b \in \Z \) then of course we must have \( p(b) = 0 \). If \( r(b) = 1 \), the boundary point \( b \) is said to be absorbing and if \( p(b) = 1 \), then \( b \) is said to be reflecting. Several other special models that we have studied are birth-death chains:
Describe each of the following as a birth-death chain.
If \( S \) is finite, classification of the states as recurrent or transient is simple, and depends only on the state graph. In particular, if the chain is irreducible, then the chain is recurrent. In this paragraph, we will study the recurrence and transience of birth-death chains when \( S = \N \). We assume that \( p(x) \gt 0 \) for all \( x \in \N \) and that \( q(x) \gt 0 \) for all \( x \in \N_+ \) (but of course we must have \( q(0) = 0 \)). Thus, the chain is irreducible. We will use the test for recurrence derived earlier with \( A = \N_+ \), the set of positive states. Essentially, we will compute the probability that the chain never hits 0, starting in a positive state.
The chain \( \bs{X} \) is recurrent if and only if
\[ \sum_{i=0}^\infty \frac{q(1) \cdots q(i)}{p(1) \cdots p(i)} = \infty \]The functional equation \( P_A u = u \) for an unknown function \( u: A \to [0, 1] \) is equivalent to the following system of equations:
\[ \begin{align} u(2) - u(1) & = \frac{q(1)}{p(1)} u(1) \\ u(x + 1) - u(x) & = \frac{q(x)}{p(x)}[u(x) - u(x - 1)], \quad x \in \{2, 3, \ldots\} \end{align} \]Solving this system of equations for the differences gives
\[ u(x + 1) - u(x) = \frac{q(1) \cdots q(x)}{p(1) \cdots p(x)} u(1), \quad x \in \N_+ \]Solving this new systems gives
\[ u(x) = \sum_{i=0}^{x-1} \frac{q(1) \cdots q(i)}{p(1) \cdots p(i)} u(1), \quad x \in \N_+ \]Finally, the test for recurrence gives the conclusion.
Note that \( r \), the function that assigns to each state \( x \in \N \) the probability of an immediate return to \( x \), plays no direct role in whether the chain is transient or recurrent. Indeed all that matters are the ratios \( q(x) / p(x) \) for \( x \in \N_+ \).
Suppose that \( p \) and \( q \) (and hence \( r \)) are constant on \( \N_+ \). The chain \( \bs{X} \) is recurrent if \( q \ge p \) and transient if \( q \lt p \).
The birth-death chain \( \bs{X} \) is positive recurrent if and only if
\[ C = \sum_{x=0}^\infty \frac{p(0) \cdots p(x - 1)}{q(1) \cdots q(x)} \lt \infty \]in which case the invariant probability density function \( f \) is given by
\[ f(x) = \frac{1}{C} \frac{p(0) \cdots p(x - 1)}{q(1) \cdots q(x)}, \quad x \in \N \]Note again that \( r \), the function that assigns to each state \( x \in \N \) the probability of an immediate return to \( x \), plays no direct role in whether the chain is transient or null recurrent, or positive recurrent.
In the positive recurrent case, the birth-death chain is reversible.
Suppose that \( p \) is constant on \( \N \) and \( q \) is constant on \( \N_+ \). Then