Estimation in the Normal Model

As usual, we will construct the confidence sets by finding appropriate pivot variables. As a first step, we will standardize our outcome variables to form a basic pivot vector from which all of our other pivot variables will be constructed. Thus, for each

i

, let

Show or recall that $Z Z 1 Z 2 Z n$ is a random sample of size $n$ from the standard normal distribution, and hence is a pivot vector for $μ σ$ .

Confidence Sets Based on a Normal Pivot Variable

The Pivot Variable

Show or recall that $M$ has the normal distribution with mean $μ$ and variance $σ 2 n$ . Hence, the corresponding standard score has the standard normal distribution, and is a pivot variable for $μ σ$ :

Z M μ σ n

Show that the pivot variable in Exercise 2 can be written in terms of our basic pivot variables in Exercise 1 as follows:

Z 1 n i 1 n Z i

As usual, we will let

φ

denote the standard normal probability density function and

Φ

the standard normal distribution function. For

p 01

, let

z p

denote the quantile of order

p

for the standard normal distribution. That is,

z p Φ p

. For selected values of

p

z p

can be obtained from the last row of the table of the

t

distribution, from the table of the standard normal distribution, from the quantile applet, or from most statistical software packages.

Show or recall the following properties. Use the inverse function theorem of calculus for part (d):

$z 1 p z p$
$z p$ as $p 0$
$z p$ as $p 1$
$z p 1 φ z p$

Confidence Sets

Use the pivot variable in Exercise 2 to show that for any $p 01$ and any $α 01$ , a $1 α$ level confidence set for $μ σ$ is

Z α p μ σ M z 1 p α σ n μ M z α p α σ n >

Show that the confidence set in Exercise 5 is a cone in the $μ σ$ parameter space, with vertex at $M 0$ and boundary lines of slopes $n z 1 p α$ and $n z α p α$ , as shown in the graph below. (Note, however, that both slopes might be negative or both positive.)

Note that the confidence set is unbounded, not surprising since we are using a single real-valued pivot variable to estimate two real parameters. However, if

σ

is known, the confidence set gives a bounded, confidence interval for

μ

. Geometrically, this interval corresponds to the horizontal cross section at

σ

. The assumption that

σ

is known is usually, but not always, artificial. Exercise 35 gives an example of a setting in which the assumption may be reasonable. In principle, we could also get confidence sets for

σ

assuming that

μ

is known; these sets would correspond to vertical cross sections. However, the assumption that

μ

is known is almost always unrealistic, and moreover, the confidence sets for

σ

would usually be unbounded.

Let us study the size of our confidence cone. One way to do this is to study

L

, the length of the cross section at

σ

. Of course, if

σ

is known,

L

is simply the length of the confidence interval for

μ

Show that $L z 1 p α z α p α σ n$ . Note in particular that the $L$ deterministic. Now show that

$L$ is a decreasing function of $α$ for fixed $n$ , $p$ , and $σ$ .
$L 0$ as $α 1$ and $L$ as $α 0$ for fixed $n$ , $p$ , and $σ$ .
$L$ is a decreasing function of $n$ for fixed $α$ , $p$ , and $σ$ .
$L 0$ as $n$ for fixed $α$ , $p$ , and $σ$ .
$L$ is an increasing function of $σ$ for fixed $α$ , $p$ , and $n$ .
$L 0$ as $σ 0$ and $L$ as $σ$ for fixed $α$ , $p$ , and $n$ .
$L$ is symmetric with respect to $p 1 2$ for fixed $n$ , $α$ , and $σ$ .
$L$ decreases as $p$ increases form 0 to $1 2$ and $L$ increases as $p$ increases from $1 2$ to 1 for fixed $n$ , $α$ , and $σ$ .
$L$ is minimized when $p 1 2$ for fixed $n$ , $α$ , and $σ$ .
$L$ as $p 0$ and as $p 1$ for fixed $n$ , $α$ , and $σ$ .

Exercise 7 shows again that there is a tradeoff between the confidence level and the size of the confidence set. If

n

and

p

are fixed, we can decrease

L

and hence tighten our estimate, only at the expense of decreasing our confidence in the estimate. Conversely, we can increase our confidence in the estimate only at the expense of increasing the size of the set. In terms of

p

, the best of the two-sided

1 α

confidence sets (and the one that is always used) is the equal-tailed set:

σ

is known, this set gives the equal-tailed confidence interval for

μ

. Note that this interval is symmetric about the sample mean

M

Let $p 1$ and let $p 0$ in Exercise 5 to show that the following are $1 α$ confidence sets for $μ σ$

$Z α 1 μ σ M z 1 α σ n μ$
$Z α 0 μ σ μ M z 1 α σ n$

σ

is known, part (a) gives a

1 α

confidence lower bound for

μ

and part (b) gives a

1 α

confidence upper bound for

μ

Use the mean estimation experiment to explore the procedure. Select the normal distribution and select normal pivot. Use various parameter values, confidence levels, sample sizes, and interval types. For each configuration, run the experiment 1000 times with an update frequency of 10. As the simulation runs, note that the confidence interval successfully captures the mean if and only if the value of the pivot variable is between the quantiles. Note the size and location of the confidence intervals and note how well the proportion of successful intervals approximates the theoretical confidence level.

Design of the Experiment

Suppose now that

σ

is known. Let

d

denote the distance between the sample mean

M

and one of the confidence bounds. That is,

where

z α z 1 α 2

for the standard two-sided interval and

z α z 1 α

for the upper or lower confidence interval. Note that

d

is deterministic, and the length of the standard two-sided interval is

L 2 d

. The number

d

is sometimes called the margin of error. In many cases, the first step in the design of the experiment is to determine the sample size needed to estimate

μ

with a given margin of error and a given confidence level.

Show that the sample size needed to estimate $μ$ with confidence $1 α$ and margin of error $d$ is

n z α 2 σ 2 d 2

Note that

n

varies directly with

z α 2

and with

σ 2

and inversely with

d 2

. This last fact implies a law of diminishing return in reducing the margin of error. For example, if we want to reduce a given margin of error by a factor of

1 2

, we must increase the sample size by a factor of 4.

Confidence Sets Based on Chi-Square Pivot Variables

The Pivot Variables

is sometimes used as a special version of the sample variance, when the distribution mean

μ

is known. On the other hand, the usual version of the sample variance is

Recall that one of the most important special properties of normal samples is that the sample mean

M

and the sample variance

S 2

are independent. Now let

Show that $U$ can be written in terms of the basic pivot variables in Exercise 1 as given below. Thus, show that this variable is a pivot variable for $μ σ$ and has the chi-square distribution with $n$ degrees of freedom.

U i 1 n Z i 2

Show that $V U Z 2$ . Thus, note that $V$ can also be written in terms of the basic pivot variables of Exercise 1 and hence is a pivot variable for $μ σ$ and for $σ$ . Finally, show that this variable has the chi-square distribution with $n 1$ degrees of freedom.

Thus, let

g k

denote the probability density function and

G k

the distribution function for the chi-square distribution with

k

degrees of freedom. In addition, for

p 01

, let

k p

denote the quantile of order

p

for the distribution, so that by definition,

k p G k p

. For selected values of

k

and

p

k p

can be obtained from the table of the chi-square distribution, from the quantile applet, or from most statistical software packages.

Show or recall the following properties. For part (c), use the inverse function theorem of calculus.

$k p 0$ as $p 0$
$k p$ as $p 1$
$p k p 1 g k k p$

Confidence Sets Based on

U

Use the pivot variable $U$ to show that for any $p 01$ and any $α 01$ , a $1 α$ level confidence set for $μ σ$ is

U α p μ σ n W 2 μ n 1 p α σ 2 n W 2 μ n α p α

Show that the boundary curves of the confidence set have the form given in the equation below, where $c$ is the appropriate chi-square quantile. Thus, the confidence set is the region between the positive branches of two hyperbolas, each centered at $M 0$ as shown in the picture.

c σ 2 n μ M 2 n 1 S 2

As before, the confidence set is unbounded, again not surprising since we are estimating two real parameters with a single real pivot variable. However, if

μ

is known, the confidence set gives a bounded, confidence interval for

σ 2

. Geometrically, this interval corresponds to the vertical cross section at

μ

. If we take square roots of the confidence bounds, then we have a confidence interval for

σ

. The assumption that

μ

is known is almost always artificial; the confidence sets that we will construct using the

V

pivot variable will yield confidence intervals for

σ

without requiring the assumption that

μ

is known. In principle, we can also obtain a confidence set for

μ

, assuming that

σ

is known; this set corresponds to the horizontal cross section at

σ

. Note however that in general, this set is not be an interval but rather a union of two disjoint intervals. Thus our construction in the previous subsection is better in this case.

Let $p 1$ and let $p 0$ in the confidence set in Exercise 14 to show that the following are $1 α$ confidence set for $μ σ$

$U α 1 μ σ n W 2 μ n 1 α σ 2$
$U α 0 μ σ 0 σ 2 n W 2 μ n α$

μ

is known, then part (a) gives a

1 α

confidence lower bound for

σ 2

and part (b) gives a

1 α

confidence upper bound for

σ 2

Of the confidence sets in Exercise 14, with fixed confidence level, we prefer the one with the smallest size, because this set gives the most information about the parameters. In this case, it's not clear how to measure the size of the set. When

μ

is known, of course, our measure would be the length of the confidence interval. Note however, that this length is random. More importantly, minimizing the length as a function of

p

is computationally difficult. For all of the reasons, we usually use the equal tailed confidence set corresponding to

p 1 2

Try to minimize the length of the interval in Exercise 14 as a function of $p$ for fixed $μ$

Confidence Sets Based on

V

Use the pivot variable $V$ to show that for any $p 01$ and any $α 01$ , a $1 α$ level confidence set for $μ σ$ is

V α p μ σ n 1 S 2 n 1 1 p α σ 2 n 1 S 2 n 1 α p α

By design, this confidence set gives no information about

μ

, but gives a bounded, confidence interval for

σ 2

. Again, if we take square roots of the confidence bounds, then we have a confidence interval for

σ

Let $p 1$ and let $p 0$ in the confidence set in Exercise 18 to show that the following are $1 α$ confidence set for $μ σ$

$V α 1 μ σ n 1 S 2 n 1 1 α σ 2$
$V α 0 μ σ 0 σ 2 n 1 S 2 n 1 α$

Part (a) gives a

1 α

confidence lower bound for

σ 2

and part (b) gives a

1 α

confidence upper bound for

σ 2

Of the confidence sets in Exercise 18, with fixed confidence level, we prefer the one with the smallest size, because this set gives the most information about the parameters. In this case, of course, we should measure the size of the confidence set by the length of the confidence interval (the vertical cross section). Note that this length is random. More importantly, minimizing the length as a function of

p

is computationally difficult. Thus, we usually use the equal tailed confidence set corresponding to

p 1 2

Try to minimize the length of the interval in Exercise 18 as a function of $p$ for fixed $μ$

Use variance estimation experiment to explore the procedure. Select the normal distribution. Use various parameter values, confidence levels, sample sizes, and interval types. For each configuration, run the experiment 1000 times with an update frequency of 10. As the simulation runs, note that the confidence interval successfully captures the standard deviation if and only if the value of the pivot variable is between the quantiles. Note the size and location of the confidence intervals and note how well the proportion of successful intervals approximates the theoretical confidence level.

Confidence Sets Based on a Student

t

Pivot Variable

The Pivot Variable

For our last basic confidence set, we will modify the standard normal pivot that we first studied, by replacing the distribution standard deviation

σ

with the sample standard deviation

S

. This will lead to confidence intervals for

μ

without the assumption that

σ

is known. Fortunately, this works because of some special properties of the statistics when the sampling distribution is normal. Let

Show that

T Z V n 1

Use the previous exercise to show that $T$ can be written in terms of the basic pivot variables of Exercise 1 and that this variable has the student $t$ distribution with $n 1$ degrees of freedom. Thus, this variable is a pivot variable for $μ σ$ and for $μ$ alone.

For

k 0

, we will let

φ k

denote the probability density function and

Φ k

the distribution function for the

t

distribution with

k

degrees of freedom. Additionally, for

p 01

, let

t k p

denote the quantile of order

p

for this distribution, so that

t k p Φ k p

. For selected values of

k

and

p

, values of

t k p

can be obtained from the table of the

t

distribution or from the quantile applet.

Show or recall the following properties. Part (d) follows from the inverse function theorem of calculus.

$t k p t k 1 p$
$t k p$ as $p 0$
$t k p$ as $p 1$
$t k p 1 φ k t k p$

Confidence Sets

Show that for any $p 01$ and any $α 01$ , a $1 α$ confidence interval for $μ σ$ is

T α p μ σ M t n 1 1 p α S n μ M t n 1 α p α S n

By design, this confidence set gives no information about

σ

, but gives a bounded confidence interval for

μ

. The size of the confidence set is clearly measured by

L

, the length of the horizontal cross section, or equivalently, the length of the confidence interval for

μ

Show that $L t n 1 1 p α t n 1 α p α S n$ . Note in particular that the $L$ random. Now show that

$L$ is a decreasing function of $α$ for fixed $n$ , and $p$ .
$L 0$ as $α 1$ and $L$ as $α 0$ for fixed $n$ , and $p$ .
$L$ is a decreasing function of $n$ for fixed $α$ , and $p$ .
$L 0$ as $n$ for fixed $α$ and $p$ .
$L$ is symmetric with respect to $p 1 2$ for fixed $n$ and $α$ .
$L$ decreases as $p$ increases form 0 to $1 2$ and $L$ increases as $p$ increases from $1 2$ to 1 for fixed $n$ , $α$ , and $σ$ .
$L$ is minimized when $p 1 2$ for fixed $n$ and $α$ .
$L$ as $p 0$ and as $p 1$ for fixed $n$ and $α$ .

Exercise 26 shows again that there is a tradeoff between the confidence level and the size of the confidence set. If

n

and

p

are fixed, we can decrease

L

p

, the best of the two-sided

1 α

confidence sets (and the one that is always used) is the equal-tailed set:

This set gives the equal-tailed confidence interval for

μ

. Note that this interval is symmetric about the sample mean

M

, but again the length of the interval is random.

Let $p 1$ and let $p 0$ in the confidence set in Exercise 25 to show that the following are $1 α$ confidence sets for $μ σ$

$T α 1 μ σ M t n 1 1 α S n μ$
$T α 0 μ σ μ M t n 1 1 α S n$

Part (a) gives a

1 α

confidence lower bound for

μ

and part (b) gives a

1 α

confidence upper bound for

μ

Use the mean estimation experiment to explore the procedure. Select the normal distribution and select student pivot. Use various parameter values, confidence levels, sample sizes, and interval types. For each configuration, run the experiment 1000 times with an update frequency of 10. As the simulation runs, note that the confidence interval successfully captures the mean if and only if the value of the pivot variable is between the quantiles. Note the size and location of the confidence intervals and note how well the proportion of successful intervals approximates the theoretical confidence level.

Intersections

We can now form intersections of some of the confidence sets constructed above to obtain bounded confidence sets for

μ σ

. We will use the fact that the sample mean

M

and the sample variance

S 2

are independent, one of the most important special properties of a normal sample. We will also need Exercise 1 from the Introduction which is based on Bonferroni's inequality. In the following exercises, suppose that

α β p q 014

with

α β 1

Show that the sets below are conservative $1 α β$ confidence set for $μ σ$ :

$Z α p U β q$
$T α p V β q$

Use independence to show that $Z α p V β q$ is a $1 α 1 β$ confidence set for $μ σ$ . This confidence set has a wedge shape as shown in the picture below.

It is interesting to note that the confidence set in Exercise 29(b) is a product set as a subset of the parameter space, but is not a product set as a subset of the sample space. By contrast, the confidence set in Exercise 30 is not a product set as a subset of the parameter space, but is a product set as a subset of the sample space.

Exercises

Robustness

The main assumption that we made was that the underlying sampling distribution is normal. Of course, in real statistical problems, we are unlikely to know much about the sampling distribution, let alone whether or not it is normal. When a statistical procedure works reasonably well, even when the underlying assumptions are violated, the procedure is said to be robust. In this subsection, we will explore the robustness of the estimation procedures for

μ

and

σ

Suppose in fact that the underlying distribution is not normal. When the sample size

n

is relatively large, the distribution of the sample mean will still be approximately normal by the central limit theorem. Thus, our interval estimates of

μ

may still be approximately valid.

Use the simulation of the mean estimation experiment to explore the procedure. Select the gamma distribution and select student pivot. Use various parameter values, confidence levels, sample sizes, and interval types. For each configuration, run the experiment 1000 times with an update frequency of 10. Note the size and location of the confidence intervals and note how well the proportion of successful intervals approximates the theoretical confidence level.

In the mean estimation experiment, repeat the previous exercise with the uniform distribution.

How large

n

needs to be for the interval estimation procedures of

μ

to work well depends, of course, on the underlying distribution; the more this distribution deviates from normality, the larger

n

must be. Fortunately, convergence to normality in the central limit theorem is rapid and hence, as you observed in the exercises, we can get away with relatively small sample sizes (30 or more) in most cases.

In general, the interval estimation procedures for

σ

is not as robust as the ones for

μ

In variance estimation experiment, select the gamma distribution. Use various parameter values, confidence levels, sample sizes, and interval types. For each configuration, run the experiment 1000 times with an update frequency of 10. Note the size and location of the confidence intervals and note how well the proportion of successful intervals approximates the theoretical confidence level.

In variance estimation experiment, select the uniform distribution. Use various parameter values, confidence levels, sample sizes, and interval types. For each configuration, run the experiment 1000 times with an update frequency of 10. Note the size and location of the confidence intervals and note how well the proportion of successful intervals approximates the theoretical confidence level.

Computational Exercises

In the following exercises, use the equal-tailed construction for two-sided confidence intervals, unless otherwise instructed.

The length of a certain machined part is supposed to be 10 centimeters but due to imperfections in the manufacturing process, the actual length is a normally distributed with mean $μ$ and variance $σ 2$ . The variance is due to inherent factors in the process, which remain fairly stable over time. From historical data, it is known that $σ 0.3$ . On the other hand, $μ$ may be set by adjusting various parameters in the process and hence may change to an unknown value fairly frequently. A sample of 100 parts has mean 10.2.

Construct the 95% confidence interval for $μ$ .
Construct the 95% confidence upper bound for $μ$ .
Construct the 95% confidence lower bound for $μ$ .

Suppose that the weight of a bag of potato chips (in grams) is a normally distributed random variable with mean $μ$ and standard deviation $σ$ , both unknown. A sample of 75 bags has mean 250 and standard deviation 10.

Construct the 90% confidence interval for $μ$ .
Construct the 90% confidence interval for $σ$ .
Construct a conservative 90% confidence rectangle for $μ σ$ .

At a telemarketing firm, the length of a telephone solicitation (in seconds) is a normally distributed random variable with mean $μ$ and standard deviation $σ$ , both unknown. A sample of 50 calls has mean length 300 and standard deviation 60.

Construct the 95% confidence upper bound for $μ$ .
Construct the 95% confidence lower bound for $σ$ .

At a certain farm the weight of a peach (in ounces) at harvest time is a normally distributed random variable with standard deviation 0.5. How many peaches must be sampled to estimate the mean weight with a margin of error $0.2$ and with 95% confidence.

The hourly salary for a certain type of construction work is a normally distributed random variable with standard deviation $1.25 and unknown mean $μ$ . How many workers must be sampled to construct a 95% confidence lower bound for $μ$ with margin of error $0.25?

In Michelson's data, assume that the measured speed of light has a normal distribution with mean $μ$ and standard deviation $σ$ , both unknown.

Construct the 95% confidence interval for $μ$ . Is the true value of the speed of light in this interval?
Construct the 95% confidence interval for $σ$ .
Explore, in an informal graphical way, the assumption that the underlying distribution is normal.

In Cavendish's data, assume that the measured density of the earth has a normal distribution with mean $μ$ and standard deviation $σ$ , both unknown.

Construct the 95% confidence interval for $μ$ . Is the true value of the density of the earth in this interval?
Construct the 95% confidence interval for $σ$ .
Explore, in an informal graphical way, the assumption that the underlying distribution is normal.

In Short's data, assume that the measured parallax of the sun has a normal distribution with mean $μ$ and standard deviation $σ$ , both unknown.

Construct the 95% confidence interval for $μ$ . Is the true value of the parallax of the sun in this interval?
Construct the 95% confidence interval for $σ$ .
Explore, in an informal graphical way, the assumption that the underlying distribution is normal.

Suppose that the length of an iris petal of a given type (Setosa, Verginica, or Versicolor) is normally distributed. Use Fisher's iris data to construct 90% two-sided confidence intervals for each of the following parameters.

The mean length of a Sertosa iris petal.
The mean length of a Vergnica iris petal.
The mean length of a Versicolor iris petal.

2. Estimation in the Normal Model

Preliminaries

The Normal Model

The Basic Pivot Variables

Confidence Sets Based on a Normal Pivot Variable

The Pivot Variable

Confidence Sets

Design of the Experiment

Confidence Sets Based on Chi-Square Pivot Variables

The Pivot Variables

Confidence Sets Based on $U$

Confidence Sets Based on $V$

Confidence Sets Based on a Student $t$ Pivot Variable

The Pivot Variable

Confidence Sets

Intersections

Exercises

Robustness

Computational Exercises