In this section we discuss probability spaces from a more advanced point of view. The section on Measure Theory in the chapter on Foundations is an essential prerequisite.
As usual, suppose that we have a random experiment with sample space . It is sometimes impossible to include all subsets of as events. Our ultimate goal is to assign probabilities to events in a random experiment. This cannot be done arbitrarily; the probabilities must be mathematically consistent in the sense of the Kolmogorov axioms. Roughly speaking, the more events that we include in the mathematical model of our random experiment, the harder it is to assign probabilities in a consistent way. However, we naturally want our collection of events to be closed under the set operations in a certain sense. Technically, the collection of events is required to be a σ-algebra.
Formally, a positive measure on is a nonnegative function defined on the σ-algebra that satisfies the countable additivity axiom: If is a countable, pairwise disjoint collection of sets in then
Thus, a probability measure on is a positive measure on with the additional requirement that . For a general measure , it's possible, of course, that for some . On the other hand, if , then can be re-scaled into a probability measure..
Formally then, a probability space , the basic mathematical model of a random experiment, consists of three essential parts:
Moreoever, σ-algebras are not just important for theoretical and foundational purposes, but are important for practical purposes as well. A σ-algebra can be used to specify partial information about an experiment--a concept of fundamental importance in probability, statistics, and especially random processes. Specifically, suppose that is a collection of events in the experiment, and that we know whether or not occurred for each . Then in fact, we can determine whether or not occurred for each , the σ-algebra generated by .
Suppose that is a random variable for the experiment, taking values in a set . Almost always, will have a natural σ-algebra of admissable subsets . Technically, is required to be measurable as a function from into . This ensures that is a valid event (that is, a member of the σ-algebra ) for each . Therefore, the probability distribution of , that is the mapping , really is a probability measure on the on the σ-algebra .
Also, is a sub σ-algebra of , and in fact is the σ-algebra generated by , denoted . If we observe the value of , then we know whether or not each event in has occurred. More generally, suppose is a random variable for each in an index set (the random variables might take values in different spaces). If we observe the value of for each then we know whether or not each event in has occurred. This idea is very important in the study of random processes; see the chapter on Markov Chains for an example.
Show that the following collection of null and almost sure events (essentially deterministic events) forms a sub σ-algebra.
Hint: Boole's inequality will be helpful.
Let be a sequence of random variables for a random experiment. The tail sigma algebra of the sequence is
and an event is a tail event for the sequence. Thus, a tail event is an event that can be defined in terms of for each . The tail sigma algebra for a sequence of events is defined analogously (let , the indicator variable of . for each ). The limit of a sequence of events that is either increasing or decreasing is a tail event of the sequence. More generally, the limit inferior and superior of a sequence of events are tail events of the sequence, and the event that a sequence of real-valued random variables converges is a tail event of the sequence. The concepts are studied in the section on Convergence.
Suppose that is a sequence of events.
Show that and are tail events for a sequence of events
Show that the event is a tail event for a sequence of real-valued random variables
The following exercise gives the Kolmogorov zero-one law, named for Andrey Kolmogorov. It states that the tail σ-algebra of an indpendent sequence is a sub σ-algebra of the σ-algebra of essentially deterministic events.
Suppose that is a tail event for a sequence of independent random variables Show that or .
From Exercise 3 and Exercise 5, note that if is a sequence of independent events, then must have probability 0 or 1. The second Borel-Cantelli lemma gives a condition under which the probability is in fact 1.
In most cases, it is impossible to define a probability measure
on a σ-algebra
explicitly, by giving a formula
for computing
for each
.
Rather, we usually know how the probability measure
should work on some class of events
.
We would then like to know that
can be extended to a probability measure on the σ-algebra generated by
,
and that this extension is unique.
We will now give a basic existence and uniqueness theorem. For a proof, see for example the book, Probability and Measure. Recall first that an algebra of subsets of is a collection of subsets that contains and is closed under complements and finite unions (and hence also finite intersections). A probability measure on is a nonnegative function with that satisfies countable additivity axiom whenever the countable union happens to be in . Thus, is finitely additive and partially countably additive. The basic extension and uniqueness theorem states that a probability measure on an algebra can be uniquely extended to a probability measure on .
Next, a collection of subsets of is a π-system if is closed under finite intersections: if and then The basic uniqueness theorem states that if and are probability measures on and for all where is a π-system with then for any .
For example, the standard (Borel) σ-algebra on is generated by the collection of all open intervals of finite length, which is clearly closed under intersection. Thus, a probability measure on is completely determined by its values on the finite open intervals. In addition, the σ-algebra on is generated by the collection of closed, infinite intervals of the form . Thus, a probability measure on is completely determined by its values on these intervals. This is important in the study of distribution functions.
Next, suppose that we have a sequence of sets , with σ-algebras , respectively. Recall that the product set
is a natural sample space for an experiment that consists of multiple measurements, or for a compound experiment that consists of performing basic experiments in sequence. Usually, we give the σ-algebra generated by the collection of product sets of the form
This collection of product sets is closed under intersection, and hence a probability measure on is completely determined by its values on these product sets. An important special case occurs when and for each . In this case, is the natural sample space for the experiment that consists of repetitions of a basic experiment.
Generalizing, suppose that we have an infinite sequence of sets with σ-algebras respectively. The product set
is a natural sample space for an experiment that consists of infinitely many measurements, or for a compound experiment that consists of combining an infinite sequence of basic experiments. Usually, we give the σ-algebra generated by the collection of cylinder sets of the form
This collection of product sets is closed under intersection, and hence a probability measure on is completely determined by its values on these product sets. Again, an important special case occurs when and for each . In this case, is the natural sample space for the experiment that consists of infinite repetitions of a basic experiment.