As usual, we start with a random experiment that has a sample space and probability measure . Suppose that we know that an event has occurred. In general, this information should clearly alter the probabilities that we assign to other events. In particular, if is another event then occurs if and only if and occur; effectively, the sample space has been reduced to . Thus, the probability of , given that we know has occurred, should be proportional to .
However, conditional probability, given that has occurred, should still be a probability measure, that is, it must satisfy the axioms of probability. This forces the proportionality constant to be . Thus, we are led inexorably to the following definition:
Let and be events in a random experiment with . The conditional probability of given is defined to be
The previous argument was based on the axiomatic definition of probability. Let's explore the idea of conditional probability from the less formal and more intuitive notion of relative frequency (or the law of large numbers). Thus, suppose that we run the experiment repeatedly. For an arbitrary event , let denote the number of times occurs in the first runs. Note that is a random variable in the compound experiment that consists of replicating the original experiment.
If is large, the conditional probability that has occurred, given that has occurred, should be close to the conditional relative frequency of given , namely the relative frequency of for the runs on which occurred:
But so by another application of the law of large numbers,
and again we are led to the same definition.
In some cases, conditional probabilities can be computed directly, by effectively reducing the sample space to the given event. In other cases, the formula above is better.
Suppose that is a random variable for our experiment that takes values in a set . Recall that the probability distribution of is the probability measure on given by
If is an event (that is, a subset of ) with positive probability, the conditional distribution of given is the probability measure on given by
Show that for fixed , is a probability measure.
Exercise 1 is the most important property of conditional probability because it means that any result that holds for probability measures in general holds for conditional probability, as long as the conditioning event remains fixed. In particular the results in Exercises 6-20 in the section on Probability Measure have analogs for conditional probability.
Suppose that and are events in a random experiment with . Prove each of the following:
Suppose that and are events in a random experiment, each having positive probability. Show that
In case (a), and are said to be positively correlated. Intuitively, the occurrence of either event means that the other event is more likely. In case (b), and are said to be negatively correlated. Intuitively, the occurrence of either event means that the other event is less likely. In case (c), and are said to be uncorrelated or independent. Intuitively, the occurrence of either event does not change the probability of the other event. Independence is a fundamental concept that can be extended to more than two events and to random variables; these generalizations are studied in the next section on Independence. A much more general version of correlation, for random variables, is explored in the section on Covariance and Correlation in the chapter on Expected Value.
Sometimes conditional probabilities are known and can be used to find the probabilities of other events.
Suppose that is a sequence of events in a random experiment whose intersection has positive probability. Prove the multiplication rule of probability.
Hint: Use the definition for each conditional probability on the right. You will have a collapsing product in which only the probability of the intersection of all events survives.
The multiplication rule is particularly useful for experiments that consist of dependent stages, where is an event in stage . Compare the multiplication rule of probability with the multiplication rule of combinatorics.
Suppose that is a countable collection of events that partition the sample space , and let be another event.
Prove the law of total probability:
Hint: Note that the term in the sum is and recall that is a partition of .
Prove Bayes' Theorem, named after Thomas Bayes:
Hint: Again the numerator is while the denomintor is by Exercise 5.
These two theorems are most useful, of course, when we know and for each . When we compute the probability of by the law of total probability in Exercise 5, we say that we are conditioning on the partition . Note that we can think of the sum as a weighted average of the conditional probabilities over , where , are the weight factors. In the context of Bayes theorem in Exercise 6, is the prior probability of and is the posterior distribution of . We will study more general versions of conditioning and Bayes theorem in the section on Discrete Distributions in the chapter on Distributions, and again in the section on Conditional Expected Value in the chapter on Expected Value.
Suppose that and are events in a random experiment with , , and .
In a certain population, 30% of the persons smoke and 8% have a certain type of heart disease. Moreover, 12% of the persons who smoke have the disease.
Suppose that the time required to perform a certain job (in minutes) is uniformly distributed on the interval .
Consider the experiment that consists of rolling 2 standard, fair dice and recording the sequence of scores . Let denote the sum of the scores. For each of the following pairs of events, find the probability of each event and the conditional probability of each event given the other. Determine whether the events are positively correlated, negatively correlated, or independent.
Note that positive correlation is not a transitive relation. From the previous exercise, for example, note that and are positively correlated, and are positively correlated, but and are negatively correlated (in fact, disjoint!).
In dice experiment, set . Run the experiment 500 times. Compute the empirical conditional probabilities corresponding to the conditional probabilities in the last exercise.
Consider again the experiment that consists of rolling 2 standard, fair dice and recording the sequence of scores . Let denote the sum of the scores. Find the conditional distribution of given that .
In the die-coin experiment, a standard, fair die is rolled and then a fair coin is tossed the number of times showing on the die.
Run the die-coin experiment 200 times.
Suppose that a bag contains 12 coins: 5 are fair, 4 are biased with probability of heads ; and 3 are two-headed. A coin is chosen at random from the bag and tossed.
Compare Exercises 15 and Exercise 17. In Exercise 15, we toss a coin with a fixed probability of heads a random number of times. In Exercise 17, we effectively toss a coin with a random probability of heads a fixed number of times. The random experiment of tossing a coin with a fixed probability of heads a fixed number of times is known as the binomial experiment with parameters and . This is a very basic and important experiment that is studied in more detail in the section on the binomial distribution in the chapter on Bernoulli Trials. Thus, the experiments in Exercises 15 and 17 can be thought of as modifications of the binomial experiment in which a parameter has been randomized. In general, interesting new random experiments can often be constructed by randomizing one or more parameters in another random experiment.
In the coin-die experiment, a fair coin is tossed. If the coin lands tails, a fair die is rolled. If the coin lands heads, an ace-six flat die is tossed (faces 1 and 6 have probability each, while faces 2, 3, 4, and 5 have probability each).
Run the coin-die experiment 500 times.
Consider the card experiment that consists of dealing 2 cards from a standard deck and recording the sequence of cards dealt. For , let be the event that card is a queen and the event that card is a heart. For each of the following pairs of events, compute the probability of each event, and the conditional probability of each event given the other. Determine whether the events are positively correlated, negatively correlated, or independent.
In the card experiment, set . Run the experiment 500 times. Compute the conditional relative frequencies corresponding to the conditional probabilities in the last exercise.
Consider the card experiment with cards. Find the probability of the following events:
In the card experiment, set run the simulation 1000 times. Compute the empirical probability of each event in the previous exercise and compare with the true probability.
Recall that Buffon's coin experimentconsists of tossing a coin with radius randomly on a floor covered with square tiles of side length 1. The coordinates of the center of the coin are recorded relative to axes through the center of the square, parallel to the sides.
Run Buffon's coin experiment 500 times. Compute the empirical probability that given that and compare with the probability in the last exercise.
For the M&M data set, find the empirical probability that a bag has at least 10 reds, given that the weight of the bag is at least 48 grams.
Consider the Cicada data.
A plant has 3 assembly lines that produces memory chips. Line 1 produces 50% of the chips and has a defective rate of 4%; line 2 has produces 30% of the chips and has a defective rate of 5%; line 3 produces 20% of the chips and has a defective rate of 1%. A chip is chosen at random from the plant.
Recall our earlier discussion genetics in the section on Probability Measures.
In the following exercise, suppose that a certain type of pea plant has either green pods or yellow pods, and that the green-pod gene is dominant. Thus, a plant with genotype or has green pods, while a plant with genotype has yellow pods.
Suppose that a green-pod plant and a yellow-pod plant are bred together. Suppose further that the green-pod plant has a chance of carrying the recessive yellow-pod gene.
Suppose that two green-pod plants are bred together. Suppose further that with probability neither plant has the recessive gene, with probability one plant has the recessive gene, and with probability both plants have the recessive gene.
Recall that a sex-linked hereditary disorder is associated with a gene on the X chromosome. As before, let denote the dominant normal gene and the recessive defective gene. Thus, a woman of gene type is normal; a woman of genotype is free of the disease, but is a carrier; and a woman of genotype has the disease. A man of genotype is normal and a man of genotype has the disease. Dichromatism, a form of color-blindness, is a sex-linked hereditary disorder, and is thus more common in males than females.
Suppose that in a certain population, 50% are male and 50% are female. Moreover, suppose that 10% of males are color-blind but only 1% of females are color-blind.
A man and a woman do not have a certain sex-linked herediatary disorder, but the woman has a chance of being a carrier.
Urn 1 contains 4 red and 6 green balls while urn 2 contains 7 red and 3 green balls. An urn is chosen at random and then a ball is chosen from the selected urn.
Urn 1 contains 4 red and 6 green balls while urn 2 contains 6 red and 3 green balls. A ball is selected at random from urn 1 and transferred to urn 2. Then a ball is selected at random from urn 2.
An urn initially contains red and green balls, where and are positive integers. A ball is chosen at random from the urn and its color is noted. It is then replaced in the urn and new balls of the same color are added to the urn. The process is repeated. The parameter is an integer, and if it is negative, the balls are removed.
The random process in described in the previous exercise is known as Pólya's urn scheme, named after George Pólya. Note that corresponds to sampling without replacement and corresponds to sampling with replacement (at least with respect to the colors of the balls). We sill study Pólya's urn in more detail in the chapter on Finite Sampling Models
An urn initially contains 6 red and 4 green balls. A ball is chosen at random from the urn and then replaced along with a ball of the other color. The process is repeated.
Suppose that we have a random experiment with an event of interest. When we run the experiment, of course, event will either occur or not occur. However, suppose that we are not able to observe the occurrence or non-occurrence of directly. Instead we have a diagnostic test designed to indicate the occurrence of event ; thus the test that can be either positive for or negative for . The test also has an element of randomness, and in particular can be in error. Here are some typical examples of the type of situation we have in mind:
Let be the event that the test is positive for the occurrence of . The conditional probability is called the sensitivity of the test. The complementary probability
is the false negative probability. The conditional probability is called the specificity of the test. The complementary probability
is the false positive probability. In many cases, the sensitivity and specificity of the test are known, as a result of the development of the test. However, the user of the test is interested in the opposite conditional probabilities, namely , the probability of the event of interest, given a positive test, and , the probability of the complementary event, given a negative test.
Use Bayes' Theorem to show that
For a concrete example, suppose that the sensitivity of the test is 0.99 and the specificity of the test is 0.95. Superficially, the test looks good. Find as a function of and verify the table and graph given below:
 |
||
| 0.001 | 0.019 | |
| 0.01 | 0.167 | |
| 0.1 | 0.688 | |
| 0.2 | 0.832 | |
| 0.3 | 0.895 | |
| 0.4 | 0.930 | |
| 0.5 | 0.952 |
The small value of for small values of is striking. The moral, of course, is that depends critically on not just on the sensitivity and specificity of the test. Moreover, the correct comparison is with , as in the table, not with . In terms of this correct comparison, the test does indeed work well; is significantly larger than in all cases.
A woman initially believes that there is an even chance that she is or is not pregnant. She takes a home pregnance test with sensitivity 0.95 and specificity 0.90. The test is positive. Find the updated probability that she is pregnant.
Suppose that 70% of defendants brought to trial for a certain type of crime are guilty. Moreover, historical data show that juries convict guilty persons 80% of the time and convict innocent persons 10% of the time. Find the probability that a person convicted of a crime of this type is guilty.
The Check Engine
light on your car has turned on. Without the information from the light, you believe that there is a 10% chance that your car has a serious engine problem. You learn that if the car has such a problem, the light will come on with probability 0.99, but if the car does not have a serious problem, the light will still come on, under circumstances similar to yours, with probability 0.3. Find the updated probability that you have an engine problem.
The ELISA test for HIV has a sensitivity and specificity of 0.999. Suppose that a person is selected at random from a population in which 1% are infected with HIV. If the person has a positive test, find the probability the person has HIV.
Diagnostic testing is closely related to a general statistical procedure known as hypothesis testing. A separate chapter on hypothesis testing explores this procedure in detail.
In this subsection, we will discuss briefly a somewhat specialized topic, but one that is still very important.
Suppose that is a collection of events in random experiment, where is a countable index set. The collection is said to be exchangeable if the probability of the intersection of a finite number of the events depends only on the number of events. That is, if and are finite subsets of and then
Clearly, exchangeability has the basic inheritance property. Suppose that is a collection of events.
For a collection of exchangeable events, the inclusion exclusion law for the probability of a union is much simpler than the general version.
Suppose that is an exchangeable collection of events. For with , let . Show that
In Pólya's urn scheme, let denote the event that the ball chosen is red. Show that is an exchangeable collection of events.
The concept of exchangeablility can be extended to random variables in the natural way. Suppose that is a collection of random variables for the experiment, each taking values in a set . The collection is said to be exchangeable if for any , the distribution of the random vector depends only on . Thus, the disstribution of the random vector is unchanged if the coordinates are permuted.
Suppose that is a collection of events for a random experiment, and let denote the corresponding collection of indicator random variables. Show that is an exchangeable collection of events if and only if is exchangeable collection of random variables.