\( \newcommand{\P}{\mathbb{P}} \) \( \newcommand{\bs}{\boldsymbol} \)
  1. Virtual Laboratories
  2. 1. Probability Spaces
  3. 1
  4. 2
  5. 3
  6. 4
  7. 5
  8. 6
  9. 7
  10. 8
  11. Answers

4. Conditional Probability

Definitions and Interpretations

The Basic Definition

As usual, we start with a random experiment that has a sample space \(S\) and probability measure \(\P\). Suppose that we know that an event \(B\) has occurred. In general, this information should clearly alter the probabilities that we assign to other events. In particular, if \(A\) is another event then \(A\) occurs if and only if \(A\) and \(B\) occur; effectively, the sample space has been reduced to \(B\). Thus, the probability of \(A\), given that we know \(B\) has occurred, should be proportional to \(\P(A \cap B)\).

Events A and B

However, conditional probability, given that \(B\) has occurred, should still be a probability measure, that is, it must satisfy the axioms of probability. This forces the proportionality constant to be \(1 / \P(B)\). Thus, we are led inexorably to the following definition:

Let \(A\) and \(B\) be events in a random experiment with \(\P(B) \gt 0\). The conditional probability of \(A\) given \(B\) is defined to be

\[\P(A \mid B) = \frac{\P(A \cap B)}{\P(B)}\]

The Law of Large Numbers

The previous argument was based on the axiomatic definition of probability. Let's explore the idea of conditional probability from the less formal and more intuitive notion of relative frequency (or the law of large numbers). Thus, suppose that we run the experiment repeatedly. For an arbitrary event \(C\), let \(N_n(C)\) denote the number of times \(C\) occurs in the first \(n\) runs. Note that \(N_n(C)\) is a random variable in the compound experiment that consists of replicating the original experiment.

If \(N_n(B)\) is large, the conditional probability that \(A\) has occurred, given that \(B\) has occurred, should be close to the conditional relative frequency of \(A\) given \(B\), namely the relative frequency of \(A\) for the runs on which \(B\) occurred:

\[\frac{N_n(A \cap B)}{N_n(B)}\]

But

\[\frac{N_n(A \cap B)}{N_n(B)} = \frac{N_n(A \cap B) / n}{N_n(B) / n}\]

so by another application of the law of large numbers,

\[\frac{N_n(A \cap B)}{N_n(B)} \to \frac{\P(A \cap B)}{\P(B)} \text{ as } n \to \infty\]

and again we are led to the same definition.

In some cases, conditional probabilities can be computed directly, by effectively reducing the sample space to the given event. In other cases, the formula in the mathematical definition is better. In some cases, conditional probabilities are known from modeling assumptions, and then are used to compute other probabilities. We will see examples of all of these situations in the computational exercises below.

It's very important that you not confuse \(\P(A \mid B)\), the probability of \(A\) given \(B\), with \(\P(B \mid A)\), the probability of \(B\) given \(A\). Making that mistake is known as the fallacy of the transposed conditional.

Conditional Distributions

Suppose that \(X\) is a random variable for our experiment that takes values in a set \(T\). Recall that the probability distribution of \(X\) is the probability measure on \(T\) given by the function

\[A \mapsto \P(X \in A)\]

If \(B\) is an event (that is, a subset of \(S\)) with positive probability, then naturally, the conditional distribution of \(X\) given \(B\) is the probability measure on \(T\) given by the function

\[A \mapsto \P(X \in A \mid B)\]

Basic Results

For fixed \(B\), \(A \mapsto \P(A \mid B)\) is a probability measure.

Proof:

Clearly \( \P(A \mid B) \ge 0 \) for every event \( A \), and \( \P(S \mid B) = 1 \). Thus, suppose that \( \{A_i: i \in I\} \) is a countable collection of pairwise disjoint events. Then

\[ \P\left(\bigcup_{i \in I} A_i \biggm| B\right) = \frac{1}{\P(B)} \P\left[\left(\bigcup_{i \in I} A_i\right) \cap B\right] = \frac{1}{\P(B)} \P\left(\bigcup_{i \in I} (A_i \cap B)\right) \]

But the collection of events \( \{A_i \cap B: i \in I\} \) is also pairwise disjoint, so

\[ \P\left(\bigcup_{i \in I} A_i \biggm| B\right) = \frac{1}{\P(B)} \sum_{i \in I} \P(A_i \cap B) = \sum_{i \in I} \frac{\P(A_i \cap B)}{\P(B)} = \sum_{i \in I} \P(A_i \mid B) \]

Exercise 1 is the most important property of conditional probability because it means that any result that holds for probability measures in general holds for conditional probability, as long as the conditioning event remains fixed. In particular the results in Exercises 6-20 in the section on Probability Measure have analogs for conditional probability.

Suppose that \(A\) and \(B\) are events in a random experiment with \(\P(B) \gt 0\).

  1. If \(B \subseteq A\) then \(\P(A \mid B) = 1\).
  2. If \(A \subseteq B\) then \(\P(A \mid B) = \P(A) / \P(B)\).
  3. If \(A\) and \(B\) are disjoint then \(\P(A \mid B) = 0\).
Proof:

These results follow directly from the definition of conditional probability. In part (a), note that \( A \cap B = B \). In part (b) note that \( A \cap B = A \). In part (c) note that \( A \cap B = \emptyset \).

Correlation

Suppose that \(A\) and \(B\) are events in a random experiment, each having positive probability.

  1. \(\P(A \mid B) \gt \P(A)\) if and only if \(\P(B \mid A) \gt \P(B)\) if and only if \(\P(A \cap B) \gt \P(A) \P(B)\)
  2. \(\P(A \mid B) \lt \P(A)\) if and only if \(\P(B \mid A) \lt \P(B)\) if and only if \(\P(A \cap B) \lt \P(A) \P(B)\)
  3. \(\P(A \mid B) = \P(A)\) if and only if \(\P(B \mid A) = \P(B)\) if and only if \(\P(A \cap B) = \P(A) \P(B)\)
Proof:

These properties following directly from the definition of conditional probability and simple algebra. Recall that multiplying or dividing an inequality by a positive number preserves the inequality.

In case (a), \(A\) and \(B\) are said to be positively correlated. Intuitively, the occurrence of either event means that the other event is more likely. In case (b), \(A\) and \(B\) are said to be negatively correlated. Intuitively, the occurrence of either event means that the other event is less likely. In case (c), \(A\) and \(B\) are said to be uncorrelated or independent. Intuitively, the occurrence of either event does not change the probability of the other event. Independence is a fundamental concept that can be extended to more than two events and to random variables; these generalizations are studied in the next section on Independence. A much more general version of correlation, for random variables, is explored in the section on Covariance and Correlation in the chapter on Expected Value.

Suppose that \( A \) and \( B \) are events with positive probabilities. Note from Theorem 2 that if \( A \subseteq B \) or \( B \subseteq A \) then \( A \) and \( B \) are positively correlated. If \( A \) and \( B \) are disjoint then \( A \) and \( B \) are negatively correlated.

Suppose that \(A\) and \(B\) are events in a random experiment.

  1. \(A\) and \(B\) have the same correlation (positive, negative, or zero) as \(A^c\) and \(B^c\).
  2. \(A\) and \(B\) have the opposite correlation as \(A\) and \(B^c\) (that is, positive-negative, negative-positive, or 0-0).
Proof:

For part (a), we use DeMorgan's law and the complement law.

\[ \P(A^c \cap B^c) - \P(A^c) \P(B^c) = \P[(A \cup B)^c] - \P(A^c) \P(B^c) = [1 - \P(A \cup B)] - [1 - \P(A)][1 - \P(B)] \]

Using the inclusion-exclusion law and algebra,

\[ \P(A^c \cap B^c) - \P(A^c) \P(B^c) = \P(A \cap B) - \P(A) \P(B) \]

For part (b) we use the difference rule and the complement rule:

\[ \P(A \cap B^c) - \P(A) \P(B^c) = \P(A) - \P(A \cap B) - \P(A) [1 - \P(B)] = -[\P(A \cap B) - \P(A) \P(B)]\]

The Multiplication Rule

Sometimes conditional probabilities are known and can be used to find the probabilities of other events. Note first that if \( A \) and \( B \) are events with positive probability, then by the very definition of conditional probability,

\[ \P(A \cap B) = \P(A) \P(B \mid A) = \P(B) \P(A \mid B) \]

The following generalization is known as the multiplication rule of probability:

Suppose that \((A_1, A_2, \ldots, A_n)\) is a sequence of events in a random experiment whose intersection has positive probability. Then

\[\P(A_1 \cap A_2 \cap \cdots \cap A_n) = \P(A_1) \P(A_2 \mid A_1) P(A_3 \mid A_1 \cap A_2) \cdots \P(A_n \mid A_1 \cap A_2 \cap \cdots \cap A_{n-1})\]
Proof:

The product on the right a collapsing product in which only the probability of the intersection of all \(n\) events survives. The product of the first two factors is \( \P(A_1 \cap A_2) \), and hence the product of the first three factors is \( \P(A_1 \cap A_2 \cap A_3) \), and so forth. The proof can be made more rigorous by induction on \( n \).

The multiplication rule is particularly useful for experiments that consist of dependent stages, where \(A_i\) is an event in stage \(i\). Compare the multiplication rule of probability with the multiplication rule of combinatorics.

Conditioning and Bayes' Theorem

Suppose that \(\mathscr{A} = \{A_i: i \in I\}\) is a countable collection of events that partition the sample space \(S\), and \( \P(A_i) \gt 0 \) for each \( i \in I \).

Image: Total.png

The law of total probability: If \( B \) is an event then

\[\P(B) = \sum_{i \in I} \P(A_i) \P(B \mid A_i)\]
Proof:

Recall that \(\{A_i \cap B: i \in I\}\) is a partition of \(B\). Hence

\[ \P(B) = \sum_{i \in I} \P(A_i \cap B) = \sum_{i \in I} \P(A_i) \P(B \mid A_i) \]

Bayes' Theorem, named after Thomas Bayes: If \( B \) is an event then

\[\P(A_j \mid B) = \frac{\P(A_j) \P(B \mid A_j)}{\sum_{i \in I}\P(A_i) \P(B \mid A_i)}, \quad j \in I\]
Proof:

Again the numerator is \(\P(A_j \cap B)\) while the denominator is \(\P(B)\) by Exercise 6.

These two theorems are most useful, of course, when we know \(\P(A_i)\) and \(\P(B \mid A_i)\) for each \(i \in I\). When we compute the probability of \(\P(B)\) by the law of total probability in Exercise 6, we say that we are conditioning on the partition \(\mathscr{A}\). Note that we can think of the sum as a weighted average of the conditional probabilities \(\P(B \mid A_i)\) over \(i \in I\), where \(\P(A_i)\), \(i \in I\) are the weight factors. In the context of Bayes theorem in Exercise 7, \(\P(A_j)\) is the prior probability of \(A_j\) and \(\P(A_j \mid B)\) is the posterior distribution of \(A_j\). We will study more general versions of conditioning and Bayes theorem in the section on Discrete Distributions in the chapter on Distributions, and again in the section on Conditional Expected Value in the chapter on Expected Value.

Examples and Applications

Basic Rules

Suppose that \(A\) and \(B\) are events in an experiment with \(\P(A) = \frac{1}{3}\), \(\P(B) = \frac{1}{4}\), \(\P(A \cap B) = \frac{1}{10}\). Find each of the following:

  1. \(\P(A \mid B)\)
  2. \(\P(B \mid A)\)
  3. \(\P(A^c \mid B)\)
  4. \(\P(B^c \mid A)\)
  5. \(\P(A^c \mid B^c)\)

Suppose that \(A\), \(B\), and \(C\) are events in a random experiment with \(\P(A \mid C) = \frac{1}{2}\), \(\P(B \mid C) = \frac{1}{3}\), and \(\P(A \cap B \mid C) = \frac{1}{4}\). Find each of the following:

  1. \(\P(A \setminus B \mid C)\)
  2. \(\P(A \cup B \mid C)\)
  3. \(\P(A^c \cap B^c \mid C)\)

Suppose that \(A\) and \(B\) are events in a random experiment with \(\P(A) = \frac{1}{2}\), \(\P(B) = \frac{1}{3}\), and \(\P(A \mid B) =\frac{3}{4}\).

  1. Find \(\P(A \cap B)\)
  2. Find \(\P(A \cup B)\)
  3. Find \(\P(B \cup A^c)\)
  4. Find \(\P(B \mid A)\)
  5. Are \(A\) and \(B\) positively correlated, negatively correlated, or independent?

Simple Populations

In a certain population, 30% of the persons smoke and 8% have a certain type of heart disease. Moreover, 12% of the persons who smoke have the disease.

  1. What percentage of the population smoke and have the disease?
  2. What percentage of the population with the disease also smoke?
  3. Are smoking and the disease positively correlated, negatively correlated, or independent?

A company has 200 employees: 120 are women and 80 are men. Of the 120 female employees, 30 are classified as managers, while 20 of the 80 male employees are managers. Suppose that an employee is chosen at random.

  1. Find the probability that the employee is female.
  2. Find the probability that the employee is a manager.
  3. Find the conditional probability that the employee is a manager given that the employee is female.
  4. Find the conditional probability that the employee is female given that the employee is a manager.
  5. Are the events female and manager positively correlated, negatively correlated, or indpendent?

Dice and Coins

Consider the experiment that consists of rolling 2 standard, fair dice and recording the sequence of scores \(\bs{X} = (X_1, X_2)\). Let \(Y\) denote the sum of the scores. For each of the following pairs of events, find the probability of each event and the conditional probability of each event given the other. Determine whether the events are positively correlated, negatively correlated, or independent.

  1. \(\{X_1 = 3\}\), \(\{Y = 5\}\)
  2. \(\{X_1 = 3\}\), \(\{Y = 7\}\)
  3. \(\{X_1 = 2\}\), \(\{Y = 5\}\)
  4. \(\{X_1 = 3\}\), \(\{X_1 = 2\}\)

Note that positive correlation is not a transitive relation. From the previous exercise, for example, note that \(\{X_1 = 3\}\) and \(\{Y = 5\}\) are positively correlated, \(\{Y = 5\}\) and \(\{X_1 = 2\}\) are positively correlated, but \(\{X_1 = 3\}\) and \(\{X_1 = 2\}\) are negatively correlated (in fact, disjoint!).

In dice experiment, set \(n = 2\). Run the experiment 1000 times. Compute the empirical conditional probabilities corresponding to the conditional probabilities in the last exercise.

Consider again the experiment that consists of rolling 2 standard, fair dice and recording the sequence of scores \(\bs{X} = (X_1, X_2)\). Let \(Y\) denote the sum of the scores, \(U\) the minimum score, and \(V\) the maximum score.

  1. Find \(\P(U = u \mid V = 4)\) for the appropriate values of \(u\).
  2. Find \(\P(Y = y \mid V = 4)\) for the appropriate values of \(y\).
  3. Find \(\P(V = v \mid Y = 8)\) for appropriate values of \(v\).
  4. Find \(\P(U = u \mid Y = 8)\) for the appropriate values of \(u\).
  5. Find \(\P[(X_ 1, X_2) = (x_1, x_2) \mid Y = 8]\) for the appropriate values of \((x_1, x_2)\).

In the die-coin experiment, a standard, fair die is rolled and then a fair coin is tossed the number of times showing on the die. Let \(N\) denote the die score and \(H\) the event that all coin tosses result in heads.

  1. Find \(\P(H)\).
  2. Find \(\P(N = n \mid H)\) for \(n \in \{1, 2, 3, 4, 5, 6\}\).
  3. Compare the results in (b) with \(\P(N = n)\) for \(n \in \{1, 2, 3, 4, 5, 6\}\). In each case, note whether the events \(H\) and \(\{N = n\}\) are positively correlated, negatively correlated, or independent.

Run the die-coin experiment 1000 times. Let \(H\) and \(N\) be as defined in the previous exercise.

  1. Compute the empirical probability of \(H\). Compare with the true probability in the previous exercise.
  2. Compute the empirical probability of \(\{N = n\}\) given \(H\), for \(n \in \{1, 2, 3, 4, 5, 6\}\). Compare with the true probabilities in the previous exercise.

Suppose that a bag contains 12 coins: 5 are fair, 4 are biased with probability of heads \(\frac{1}{3}\); and 3 are two-headed. A coin is chosen at random from the bag and tossed.

  1. Find the probability that the coin is heads.
  2. Given that the coin is heads, find the conditional probability of each coin type.

Compare Exercises 16 and Exercise 18. In Exercise 16, we toss a coin with a fixed probability of heads a random number of times. In Exercise 18, we effectively toss a coin with a random probability of heads a fixed number of times. The random experiment of tossing a coin with a fixed probability of heads \(p\) a fixed number of times \(n\) is known as the binomial experiment with parameters \(n\) and \(p\). This is a very basic and important experiment that is studied in more detail in the section on the binomial distribution in the chapter on Bernoulli Trials. Thus, the experiments in Exercises 16 and 18 can be thought of as modifications of the binomial experiment in which a parameter has been randomized. In general, interesting new random experiments can often be constructed by randomizing one or more parameters in another random experiment.

In the coin-die experiment, a fair coin is tossed. If the coin lands tails, a fair die is rolled. If the coin lands heads, an ace-six flat die is tossed (faces 1 and 6 have probability \(\frac{1}{4}\) each, while faces 2, 3, 4, and 5 have probability \(\frac{1}{8}\) each). Let \(H\) denote the event that the coin lands heads, and let \(Y\) denote the score when the chosen die is tossed.

  1. Find \(\P(Y = y)\) for \(y \in \{1, 2, 3, 4, 5, 6\}\).
  2. Find \(\P(H \mid Y = y)\) for \(y \in \{1, 2, 3, 4, 5, 6,\}\).
  3. Compare each probability in part (b) with \(\P(H)\). In each case, note whether the events \(H\) and \(\{Y = y\}\) are positively correlated, negatively correlated, or independent.

Run the coin-die experiment 1000 times. Let \(H\) and \(Y\) be as defined in the previous exercise.

  1. Compute the empirical probability of \(\{Y = y\}\), for each \(y\), and compare with the true probability in the previous exercise
  2. Compute the empirical probability of \(H\) given \(\{Y = y\}\) for each \(y\), and compare with the true probability in the previous exercise.

Cards

Consider the card experiment that consists of dealing 2 cards from a standard deck and recording the sequence of cards dealt. For \(i \in \{1, 2\}\), let \(Q_i\) be the event that card \(i\) is a queen and \(H_i\) the event that card \(i\) is a heart. For each of the following pairs of events, compute the probability of each event, and the conditional probability of each event given the other. Determine whether the events are positively correlated, negatively correlated, or independent.

  1. \(Q_1\), \(H_1\)
  2. \(Q_1\), \(Q_2\)
  3. \(Q_2\), \(H_2\)
  4. \(Q_1\), \(H_2\)

In the card experiment, set \(n = 2\). Run the experiment 500 times. Compute the conditional relative frequencies corresponding to the conditional probabilities in the last exercise.

Consider the card experiment with \(n = 3\) cards. Find the probability of the following events:

  1. All three cards are all hearts.
  2. The first two cards are hearts and the third is a spade.
  3. The first and third cards are hearts and the second is a spade.

In the card experiment, set \(n = 3\) and run the simulation 1000 times. Compute the empirical probability of each event in the previous exercise and compare with the true probability.

Buffon's Coin

Recall that Buffon's coin experiment consists of tossing a coin with radius \(r \le \frac{1}{2}\) randomly on a floor covered with square tiles of side length 1. The coordinates \((X, Y)\) of the center of the coin are recorded relative to axes through the center of the square, parallel to the sides.

  1. Find \(\P(Y \gt 0 \mid X \lt Y)\)
  2. Find the conditional distribution of \((X, Y)\) given that the coin does not touch the sides of the square.

Run Buffon's coin experiment 500 times. Compute the empirical probability that \(Y \gt 0\) given that \(X \lt Y\) and compare with the probability in the last exercise.

Reliability

A plant has 3 assembly lines that produces memory chips. Line 1 produces 50% of the chips and has a defective rate of 4%; line 2 has produces 30% of the chips and has a defective rate of 5%; line 3 produces 20% of the chips and has a defective rate of 1%. A chip is chosen at random from the plant.

  1. Find the probability that the chip is defective.
  2. Given that the chip is defective, find the conditional probability for each line.

Suppose that \(T\) denotes the lifetime of a light bulb (in 1000 hour units), and that \(T\) has the following exponential distribution:

\[\P(T \in A) = \int_A e^{-t} dt, \quad A \subseteq [0, \infty)\]
  1. Find \(\P(T \gt 3)\)
  2. Find \(\P(T \gt 5 \mid T \gt 2)\)

Suppose again that \(T\) denotes the lifetime of a light bulb (in 1000 hour units), but that \(T\) is uniformly distributed on the interal \([0, 10]\).

  1. Find \(\P(T \gt 3)\)
  2. Find \(\P(T \gt 5 \mid T \gt 2)\)

Genetics

Recall our earlier discussion genetics in the section on Probability Measures.

In the following exercise, suppose that a certain type of pea plant has either green pods or yellow pods, and that the green-pod gene is dominant. Thus, a plant with genotype \(gg\) or \(gy\) has green pods, while a plant with genotype \(yy\) has yellow pods.

Suppose that a green-pod plant and a yellow-pod plant are bred together. Suppose further that the green-pod plant has a \(\frac{1}{4}\) chance of carrying the recessive yellow-pod gene.

  1. Find the probability that a child plant will have green pods.
  2. Given that a child plant has green pods, find the updated probability that the green-pod parent has the recessive gene.

Suppose that two green-pod plants are bred together. Suppose further that with probability \(\frac{1}{3}\) neither plant has the recessive gene, with probability \(\frac{1}{2}\) one plant has the recessive gene, and with probability \(\frac{1}{6}\) both plants have the recessive gene.

  1. Find the probability that a child plant has green pods.
  2. Given that a child plant has green pods, find the updated probability that both parents have the recessive gene.

Recall that a sex-linked hereditary disorder is associated with a gene on the X chromosome. As before, let \(h\) denote the dominant healthy gene and \(d\) the recessive defective gene. Thus, a woman of gene type \(hh\) is normal; a woman of genotype \(hd\) is free of the disease, but is a carrier; and a woman of genotype \(dd\) has the disease. A man of genotype \(h\) is normal and a man of genotype \(d\) has the disease. Dichromatism, a form of color-blindness, is a sex-linked hereditary disorder, and is thus more common in males than females.

Suppose that in a certain population, 50% are male and 50% are female. Moreover, suppose that 10% of males are color-blind but only 1% of females are color-blind.

  1. Find the percentage of color-blind persons in the population.
  2. Find the percentage of color-blind persons that are male.

A man and a woman do not have a certain sex-linked hereditary disorder, but the woman has a \(\frac{1}{3}\) chance of being a carrier.

  1. Find the probability that a son born to the couple will be normal.
  2. Find the probability that a daughter born to the couple will be a carrier.
  3. Given that a son born to the couple is normal, find the updated probability that the mother is a carrier.

Urn Models

Urn 1 contains 4 red and 6 green balls while urn 2 contains 7 red and 3 green balls. An urn is chosen at random and then a ball is chosen from the selected urn.

  1. Find the probability that the ball is green.
  2. Given that the ball is green, find the conditional probability that urn 1 was selected.

Urn 1 contains 4 red and 6 green balls while urn 2 contains 6 red and 3 green balls. A ball is selected at random from urn 1 and transferred to urn 2. Then a ball is selected at random from urn 2.

  1. Find the probability that the ball from urn 2 is green.
  2. Given that the ball from urn 2 is green, find the conditional probability that the ball from urn 1 was green.

An urn initially contains 6 red and 4 green balls. A ball is chosen at random from the urn and its color is recorded. It is then replaced in the urn and 2 new balls of the same color are added to the urn. The process is repeated.

  1. Find the probability of the event that the balls 1 and 2 are red and ball 3 is green.
  2. Find the probability of the event that balls 1 and 3 are red and ball 2 is green.
  3. Find the probability of the event that ball 1 is green and balls 2 and 3 are red.
  4. Find the probability that the second ball is red.
  5. Find the probability that the first ball is red given that the second ball is red.

Think about the results in the previous exercise. Note in particular that the answers to parts (a), (b), and (c) are the same, and that the probability that the second ball is red in part (d) is the same as the probability that the first ball is red. More generally, the probabilities of events do not depend on the order of the draws. For example, the probability of an event involving the first, second, and third draws is the same as the probability of the corresponding event involving the seventh, tenth and fifth draws. Technically, the sequence of events \((R_1, R_2, \ldots)\) is exchangeable. The random process described in this exercise is a special case of Pólya's urn scheme, named after George Pólya. We sill study Pólya's urn in more detail in the chapter on Finite Sampling Models

An urn initially contains 6 red and 4 green balls. A ball is chosen at random from the urn and its color is recorded. It is then replaced in the urn and two new balls of the other color are added to the urn. The process is repeated.

  1. Find the probability of the event that the balls 1 and 2 are red and ball 3 is green.
  2. Find the probability of the event that balls 1 and 3 are red and ball 2 is green.
  3. Find the probability of the event that ball 1 is green and balls 2 and 3 are red.
  4. Find the probability that the second ball is red.
  5. Find the probability that the first ball is red given that the second ball is red.

Think about the results in the previous exercise, and compare with Exercise 36. Note that the answers to parts (a), (b), and (c) are not all the same, and that the probability that the second ball is red in part (d) is not the same as the probability that the first ball is red. In short, the sequence of events \((R_1, R_2, \ldots)\) is not exchangeable.

Diagnostic Testing

Suppose that we have a random experiment with an event \(A\) of interest. When we run the experiment, of course, event \(A\) will either occur or not occur. However, suppose that we are not able to observe the occurrence or non-occurrence of \(A\) directly. Instead we have a diagnostic test designed to indicate the occurrence of event \(A\); thus the test that can be either positive for \(A\) or negative for \(A\). The test also has an element of randomness, and in particular can be in error. Here are some typical examples of the type of situation we have in mind:

Let \(T\) be the event that the test is positive for the occurrence of \(A\). The conditional probability \(\P(T \mid A)\) is called the sensitivity of the test. The complementary probability

\[\P(T^c \mid A) = 1 - \P(T \mid A)\]

is the false negative probability. The conditional probability \(\P(T^c \mid A^c)\) is called the specificity of the test. The complementary probability

\[\P(T \mid A^c) = 1 - \P(T^c \mid A^c)\]

is the false positive probability. In many cases, the sensitivity and specificity of the test are known, as a result of the development of the test. However, the user of the test is interested in the opposite conditional probabilities, namely \(\P(A \mid T)\), the probability of the event of interest, given a positive test, and \(\P(A^c \mid T^c)\), the probability of the complementary event, given a negative test. Of course, if we know \( \P(A \mid T) \) then we also have \( \P(A^c \mid T) = 1 - \P(A \mid T) \), the probability of the complementary event given a positive test. Similarly, if we know \( \P(A^c \mid T^c) \) then we also have \( \P(A \mid T^c) \), the probability of the event given a negative test. Computing the probabilities of interest is simply a special case of Bayes' theorem.

The probability that the event occurs, given a positive test is

\[\P(A \mid T) = \frac{\P(A) \P(T \mid A)}{\P(A) \P(T \mid A) + \P(A^c) \P(T \mid A^c)}\]

The probability that the event does not occur, given a negative test is

\[\P(A^c \mid T^c) = \frac{\P(A^c) \P(T^c \mid A^c)}{\P(A) \P(T^c \mid A) + \P(A^c) \P(T^c \mid A^c)}\]

There is often a trade-off between sensitivity and specificity. An attempt to make a test more sensitive may result in the test being less specific, and an attempt to make a test more specific may result in the test being less sensitive. As an extreme example, consider the worthless test that always returns positive, no matter what the evidence. Then \( T = S \) so the test has sensitivity 1, but specificity 0. At the opposite extreme is the worthless test that always returns negative, no matter what the evidence. Then \( T = \emptyset \) so the test has specificity 1 but sensitivity 0. In between these extremes are helpful tests that are actually based on evidence of some sort.

For a concrete example, suppose that the sensitivity of the test is 0.99 and the specificity of the test is 0.95. Superficially, the test looks good. Find \(\P(A \mid T)\) as a function of \(\P(A)\) and verify the table and graph given below:

\(\P(A)\) \(\P(A \mid T)\) P(A \mid T) as a function of P(A)
0.001 0.019
0.01 0.167
0.1 0.688
0.2 0.832
0.3 0.895
0.4 0.930
0.5 0.952

The small value of \(\P(A \mid T)\) for small values of \(\P(A)\) is striking. The moral, of course, is that \(\P(A \mid T)\) depends critically on \(\P(A)\) not just on the sensitivity and specificity of the test. Moreover, the correct comparison is \(\P(A \mid T)\) with \(\P(A)\), as in the table, not \(\P(A \mid T)\) with \(\P(T \mid A)\). In terms of this correct comparison, the test does indeed work well; \(\P(A \mid T)\) is significantly larger than \(\P(A)\) in all cases.

A woman initially believes that there is an even chance that she is or is not pregnant. She takes a home pregnancy test with sensitivity 0.95 and specificity 0.90. Find the updated probability that the woman is pregnant in each of the following cases.

  1. The test is positive.
  2. The test is negative.

Suppose that 70% of defendants brought to trial for a certain type of crime are guilty. Moreover, historical data show that juries convict guilty persons 80% of the time and convict innocent persons 10% of the time. Suppose that a person is tried for a crime of this type. Find the updated probability that the person is guilty in each of the following cases:

  1. The person is convicted.
  2. The person is acquitted.

The Check Engine light on your car has turned on. Without the information from the light, you believe that there is a 10% chance that your car has a serious engine problem. You learn that if the car has such a problem, the light will come on with probability 0.99, but if the car does not have a serious problem, the light will still come on, under circumstances similar to yours, with probability 0.3. Find the updated probability that you have an engine problem.

The ELISA test for HIV has a sensitivity and specificity of 0.999. Suppose that a person is selected at random from a population in which 1% are infected with HIV, and given the ELISA test. Find the probability that the person has HIV in each of the following cases:

  1. The test is positive.
  2. The test is negative.

The ELISA test for HIV is a very good one. Let's look another test, this one for prostate cancer, that's rather bad.

The PSA test for prostate cancer is based on a blood marker known as the Prostate Specific Antigen. An elevated level of PSA is evidence for prostate cancer. To have a diagnostic test, in the sense that we are discussing here, we must decide on a definite level of PSA, above which we declare the test to be positive. A positive test would typically lead to other more invasive tests (such as biopsy) which, of course, carry risks and cost. The PSA test with cutoff 2.6 ng/ml has sensitivity 0.40 and specificity 0.81. The overall incidence of prostate cancer among males is 156 per 100000. Suppose that a man, with no particular risk factors, has the PSA test. Find the probability that the man has prostate cancer in each of the following cases:

  1. The test is positive.
  2. The test is negative.

Diagnostic testing is closely related to a general statistical procedure known as hypothesis testing. A separate chapter on hypothesis testing explores this procedure in detail.

Data Analysis Exercises

For the M&M data set, find the empirical probability that a bag has at least 10 reds, given that the weight of the bag is at least 48 grams.

Consider the Cicada data.

  1. Find the empirical probability that a cicada weighs at least 0.25 grams given that the cicada is male.
  2. Find the empirical probability that a cicada weighs at least 0.25 grams given that the cicada is the tredecula species.

Exchangeability

In this subsection, we will discuss briefly a somewhat specialized, theoretical topic. You may want to omit this section if you are a new student of probability.

Exchangeable Events

Suppose that \(\mathscr{A} = \{A_i: i \in I\}\) is a collection of events in random experiment, where \(I\) is a countable index set. The collection is said to be exchangeable if the probability of the intersection of a finite number of the events depends only on the number of events. That is, if \(J\) and \(K\) are finite subsets of \(I\) and \(\#(J) = \#(K)\) then

\[\P\left( \bigcap_{j \in J} A_j\right) = \P \left( \bigcap_{k \in K} A_k\right)\]

Clearly, exchangeability has the basic inheritance property. Suppose that \(\mathscr{A}\) is a collection of events.

  1. If \(\mathscr{A}\) is exchangeable then \(\mathscr{B}\) is exchangeable for every \(\mathscr{B} \subseteq \mathscr{A}\).
  2. Conversely, if \(\mathscr{B}\) is exchangeable for every finite \(\mathscr{B} \subseteq \mathscr{A}\) then \(\mathscr{A}\) is exchangeable.

For a collection of exchangeable events, the inclusion exclusion law for the probability of a union is much simpler than the general version.

Suppose that \(\{A_1, A_2, \ldots, A_n\}\) is an exchangeable collection of events. For \(J \subseteq \{1, 2, \ldots, n\}\) with \(\#(J) = k\), let \(p_k = \P\left( \bigcap_{j \in J} A_j\right)\). Then

\[\P\left(\bigcup_{i = 1}^n A_i\right) = \sum_{k=1}^n (-1)^{k-1} \binom{n}{k} p_k\]

In Pólya's urn problem of Exercise 36, recall that \(R_i\) denote the event that the \(i\)th ball chosen is red. Then \(\{R_1, R_2, \ldots\}\) is an exchangeable collection of events.

Exchangeable Random Variables

The concept of exchangeablility can be extended to random variables in the natural way. Suppose that \(\mathscr{C}\) is a collection of random variables for the experiment, each taking values in a set \(T\). The collection \(\mathscr{C}\) is said to be exchangeable if for any \(\{X_1, X_2, \ldots, X_n\} \subseteq \mathscr{C}\), the distribution of the random vector \((X_1, X_2, \ldots, X_n)\) depends only on \(n\). Thus, the distribution of the random vector is unchanged if the coordinates are permuted.

Suppose that \(\mathscr{A}\) is a collection of events for a random experiment, and let \(\mathscr{C} = \{\bs{1}_A: A \in \mathscr{A}\}\) denote the corresponding collection of indicator random variables. Then \(\mathscr{A}\) is an exchangeable collection of events if and only if \(\mathscr{C}\) is exchangeable collection of random variables.