]> Bayesian Interval Estimation
  1. Virtual Laboratories
  2. 8. Set Estimation
  3. 1
  4. 2
  5. 3
  6. 4
  7. 5

5. Bayesian Set Estimation

Definitions

As usual, our starting point is a random experiment with an underlying sample space and a probability measure . In the basic statistical model, we have an observable random variable X taking values in a set S . In general, X can have quite a complicated structure. For example, if the experiment is to sample n objects from a population and record various measurements of interest, then

X X 1 X 2 X n

where X i is the vector of measurements for the i object. The most important special case occurs when X X 1 X 2 X n are independent and identically distributed. In this case, we have a random sample of size n from the common distribution.

Suppose also that the distribution of X depends on a parameter θ taking values in a parameter space Θ . The parameter may also be vector valued, in which case Θ k for some k and the parameter has the form θ θ 1 θ 2 θ k

Recall that in Bayesian analysis, the unknown parameter θ is treated as a random variable. Specifically, suppose that the conditional probability density function of the data vector X given θ is denoted g x θ . Moreover, the parameter θ is given a prior distribution with probability density function h . (The prior distribution is chosen to reflect our knowledge, if any of the parameter). The joint probability density function of the data vector and the parameter is

f x θ h θ g x θ ,  x S ,  θ Θ

Next, the (unconditional) probability density function of X is the function g given by

g x θ θ Θ f x θ θ θ Θ h θ g x θ ,  x S

if the parameter has a discrete distribution, or by

g x θ Θ f x θ θ Θ h θ g x θ ,  x S

if the parameter has a continuous distribution. Finally, by Bayes' theorem, the posterior probability density function of θ given x is

h θ x h θ g x θ g x ,  θ Θ ,  x S

Now let C X be a confidence set (that is, a subset of the parameter space that depends on the data variable X . but no unknown parameters). One possible definition of a 1 α level Bayesian confidence set requires that

θ C x X x 1 α

In this definition, only θ is random and thus the probability above is computed using the posterior probability density function h θ x . Another possible definition requires that

θ C X 1 α

In this definition, X and θ are both random, and so the probability above would be computed using the joint probability density function f x θ . Whatever the philosophical arguments may be, the first definition is certainly the easier one from a computational viewpoint, and hence is the one most commonly used.

Let us compare the classical and Bayesian approaches. In the classical approach, the parameter is deterministic, but unknown. Before the data are collected, the confidence set (which is random) will contain the parameter with probability 1 α . After the data are collected, the computed confidence set either contains the parameter or does not, and we will usually never know which. By contrast in a Bayesian confidence set, the random parameter θ falls in the computed, deterministic confidence set with probability 1 α .

Applications

The Bernoulli Distribution

Suppose that X X 1 X 2 X n is a random sample from the Bernoulli distribution with success parameter p . Thus, X i 1 if trial i resulted in success, and X i 0 if trial i resulted in failure. Moreover, suppose that p has a prior beta distribution with left parameter a 0 and right parameter b 0 . Denote the number of successes by

Y i 1 n X i

Recall that for a given value of p , Y has the binomial distribution with parameters n and p .

Show that given Y y ,

  1. The posterior distribution of p is beta with left parameter a y and right parameter b n y
  2. A 1 α level Bayesian confidence interval for p is L y U y where L y is the quantile of order α 2 and U y is the quantile of order 1 α 2 for the beta distribution in (a).

Specifically, suppose that we have a coin with an unknown probability p of heads and that we give p the uniform prior. We then toss the coin 10 times, observing 7 heads. Compute the 90% Bayesian confidence interval for p .

The Poisson Distribution

Suppose that X X 1 X 2 X n is a random sample of size n from the Poisson distribution with parameter μ . Moreover, suppose that μ has a prior gamma distribution with shape parameter k 0 and scale parameter b 0 . Denote the sum of the sample values by

Y i 1 n X i .

Show that given Y y ,

  1. The posterior distribution of μ is gamma with shape parameter k y and scale parameter b n b 1
  2. A 1 α level Bayesian confidence interval for μ is L x U x where L x is the quantile of order α 2 and U x is the quantile of order 1 α 2 for the gamma distribution in (a).

Suppose that the number of defects in a manufactured item has the Poisson distribution with parameter μ and we give μ the exponential distribution with parameter 1. We sample 5 items and observe a total of 8 defects. Compute the 90% Bayesian confidence interval.

  1. Virtual Laboratories
  2. 8. Set Estimation
  3. 1
  4. 2
  5. 3
  6. 4
  7. 5