\(\newcommand{\R}{\mathbb{R}}\) \(\newcommand{\N}{\mathbb{N}}\) \(\newcommand{\E}{\mathbb{E}}\) \(\newcommand{\var}{\text{var}}\) \(\newcommand{\sd}{\text{sd}}\) \(\newcommand{\skew}{\text{skew}}\) \(\newcommand{\kurt}{\text{kurt}}\)
  1. Virtual Laboratories
  2. 4. Special Distributions
  3. 1
  4. 2
  5. 3
  6. 4
  7. 5
  8. 6
  9. 7
  10. 8
  11. 9
  12. 10
  13. 11
  14. 12
  15. 13
  16. 14
  17. 15
  18. Answers

2. The Normal Distribution

The normal distribution holds an honored role in probability and statistics, mostly because of the central limit theorem, one of the fundamental theorems that forms a bridge between the two subjects. In addition, as we will see, the normal distribution has many nice mathematical properties. The normal distribution is also called the Gaussian distribution, in honor of Carl Friedrich Gauss, who was among the first to use the distribution.

The Standard Normal Distribution

A random variable \(Z\) has the standard normal distribution if it has the probability density function \(\phi\) given by

\[ \phi(z) = \frac{1}{\sqrt{2 \, \pi}} e^{-\frac{1}{2} z^2}, \quad z \in \R \]

\(\phi\) is a probability density function. That is, \(\sqrt{2 \, \pi}\) is the normalzing constant for the function \(z \mapsto e^{-\frac{1}{2} z^2}\):

\[ \int_{-\infty}^{\infty} e^{-\frac{1}{2} z^2} dz = \sqrt{2 \, \pi} \]
Proof:

Let \(c\) denote the integral. Express \(c^2\) as a double integral over \(\R^2\) and then convert to polar coordinates.

The standard normal density function \(\phi\) satisfies the following properties:

  1. \(\phi\) is symmetric about \(z = 0\).
  2. \(\phi\) is increasing on \((-\infty, 0)\) and decreasing on \((0, \infty)\).
  3. The mode occurs at \(z = 0\).
  4. \(\phi\) is concave upward on \((-\infty, -1)\) and on \((1, \infty)\) and is concave downward on \((-1, 1)\).
  5. The inflection points of \(\phi\) occur at \(z = \pm 1\).
  6. \(\phi(z) \to 0\) as \(z \to \infty\) and as \(z \to -\infty\).

In the Special Distribution Simulator, select the normal distribution and keep the default settings. Note the shape and location of the standard normal density function. Run the simulation 1000 times, and note the apparent convergence of the empirical density function to the true density function.

The standard normal distribution function \(\Phi\), given by

\[ \Phi(z) = \int_{-\infty}^z \phi(t) \, dt = \int_{-\infty}^z \frac{1}{\sqrt{2 \, \pi}} e^{-\frac{1}{2} z^2} \, dz \]

and its inverse, the quantile function \(\Phi^{-1}\), cannot be expressed in closed form in terms of elementary functions. However approximate values of these functions can be obtained from the special distribution calculator, and from most mathematics and statistics software.

The standard normal distribution function \(\Phi\) satisfies the following properties:

  1. \(\Phi(-z) = 1 - \Phi(z)\) for \(z \in \R\)
  2. \(\Phi^{-1}(p) = -\Phi^{-1}(1 - p)\) for \(p \in (0, 1)\)
  3. \(\Phi(0) = \frac{1}{2}\), so the median is 0.

In the special distribution calculator, select the standard normal distribution.

  1. Note the shape of the density function and the distribution function.
  2. Find the first and third quartiles.
  3. Compute the interquartile range.

Use the special distribution calculator to find the quantiles of the following orders for the standard normal distribution:

  1. \(p = 0.001\), \(p = 0.999\)
  2. \(p = 0.05\), \(p = 0.95\)
  3. \(p = 0.1\), \(p = 0.9\)

The General Normal Distribution

The general normal distribution is the location-scale family associated with the standard normal distribution. Specifically, suppose that \(\mu \in \R\) and \( \sigma \in (0, \infty) \) and that \(Z\) has the standard normal distribution. Then \(X = \mu + \sigma \, Z\) has the normal distribution with location parameter \(\mu\) and scale parameter \(\sigma\). The basic properties of the density function and distribution function follow easily from general results for location scale families.

The normal distribution with location parameter \(\mu\) and scale parameter \(\sigma\) has probability density function \(f\) given by

\[ f(x) = \frac{1}{\sqrt{2 \, \pi} \, \sigma} \exp \left[ -\frac{1}{2} \left( \frac{x - \mu}{\sigma} \right)^2 \right], \quad x \in \R \]

The normal density function \(f\) satisfies the following properties:

  1. \(f\) is symmetric about \(x = \mu\).
  2. \(f\) is increasing on \((-\infty, \mu)\) and is decreasing on \((\mu, \infty)\)
  3. The mode occurs at \(x = \mu\).
  4. \(f\) is concave upward on \((-\infty, \mu - \sigma)\) and on \((\mu + \sigma, \infty)\) and is concave downward on \((\mu - \sigma, \mu + \sigma)\).
  5. The inflection points of \(f\) occur at \(x = \mu \pm \sigma\).
  6. \(f(x) \to 0\) as \(x \to \infty\) and as \(x \to -\infty\).

In the special distribution simulator, select the normal distribution. Vary the parameters and note the shape and location of the density function. With your choice of parameter settings, run the simulation 1000 times and note the apparent convergence of the empirical density function to the true probability density function.

Let \(F\) denote the distribution function for the normal distribution with location parameter \(\mu\) and scale parameter \(\sigma\), and as above, let \(\Phi\) denote the standard normal distribution function.

The normal distribution function \(F\) satsifies the following properties:

  1. \(F(x) = \Phi \left( \frac{x - \mu}{\sigma} \right)\) for \(x \in \R\).
  2. \(F^{-1}(p) = \mu + \sigma \, \Phi^{-1}(p)\) for \(p \in (0, 1)\).
  3. \(F(\mu) = \frac{1}{2}\) so the median occurs at \(x = \mu\).

In the special distribution calculator, select the normal distribution. Vary the parameters and note the shape of the density function and the distribution function.

Moments

The important properties of the normal distribution are most easily obtained using the moment generating function.

If \(Z\) has the standard normal distribution then \(Z\) has moment generating function

\[ \E(e^{t \, Z}) = e^{\frac{1}{2} t^2}, \quad t \in \R\]
Proof:

In the integral for \(\E(e^{t \, Z})\), complete the square in \(z\) and look for a normal probability density function.

If \(X\) has the normal distribution with location parameter \(\mu\) and scale parameter \(\sigma\) then \(X\) has moment generating function

\[ \E(e^{t \, X}) = \exp \left( \mu + \frac{1}{2} \sigma^2 \, t^2 \right) \quad t \in \R \]
Proof:

Use the result of the previous exercise and basic properties of expected value.

As the notation suggests, the location and scale parameters are also the mean and standard deviation, respectively.

If \(X\) has the normal distribution with location parameter \(\mu\) and scale parameter \(\sigma\) then

  1. \(\E(X) = \mu\)
  2. \(\var(X) = \sigma^2\)

In particular, the standard normal distribution has mean 0 and variance 1. More generally, we can compute all of the central moments of \(X\):

If \(X\) has the normal distribution with mean \(\mu\) and standard deviation \(\sigma\), then for \(n \in \N\),

\[ \begin{align} \E \left[ (X - \mu)^{2 \, n} \right] & = \frac{(2 \, n)!}{n! \, 2^n} \, \sigma^{2 \, n} \\ \E \left[ (X - \mu)^{2 \, n + 1} \right] & = 0 \end{align} \]

All of the odd central moments of \(X\) are 0, a fact that also follows from the symmetry of the probability density function.

In the special distribution simulator select the normal distribution. Vary the mean and standard deviation and note the size and location of the mean/standard deviation bar. With your choice of parameter settings, run the simulation 1000 times and note the apparent convergence of the empirical moments to the true moments.

The following exercise gives the skewness and kurtosis of the normal distribution.

If \(X\) has the normal distribution with mean \(\mu\) and standard deviation \(\sigma\) then

  1. \(\skew(X) = 0\)
  2. \(\kurt(X) = 3\)

Because of the last result, (and the use of the normal distribution as a standard), the excess kurtosis of a random variable is defined to be the ordinary kurtosis minus 3. Thus, the excess kurtosis of the normal distribution is 0.

Transformations

The normal family of distributions satisfies two very important properties: invariance under linear transformations and invariance with respect to sums of independent variables. The first property is essentially a restatement of the fact that the normal distribution is a location-scale family. The proofs are easy using the moment generating function.

Suppose that \(X\) is normally distributed with mean \(\mu\) and variance \(\sigma^2\). If \(a \in \R\) and \(b \in \R \setminus \{0\}\), then \(a + b \, X\) is normally distributed with mean \(a + b \, \mu\) and variance \(b^2 \, \sigma^2\).

In particular

  1. If \(X\) has the normal distribution with mean \(\mu\) and standard deviation \(\sigma\) then \(Z = \frac{X - \mu}{\sigma}\) has the standard normal distribution.
  2. If \(Z\) has the standard normal distribution and if \(\mu \in \R\) and \(\sigma \in (0, \infty)\) are constants, then \(X = \mu + \sigma \, Z\) has the normal distribution with mean \(\mu\) and standard deviation \(\sigma\).

Suppose that \(X_1\) and \(X_2\) are independent random variables, and that \(X_i\) is normally distributed with mean \(\mu_i\) and variance \(\sigma_i^2\) for \(i \in \{1, 2\}\). Then \(X_1 + X_2\) is normally distributed with

  1. \(\E(X_1 + X_2) = \mu_1 + \mu_2\)
  2. \(\var(X_1 + X_2) = \sigma_1^2 + \sigma_2^2\)

The result of the previous exercise generalizes to a sum of \(n\) independent, normal variables. The important part is that the sum is still normal; the expressions for the mean and variance are standard results that hold for the sum of independent variables generally.

Suppose that \(X\) has the normal distribution with mean \(\mu\) and variance \(\sigma^2\). The distribution is a two-parameter exponential family with natural parameters \(\left( \frac{\mu}{\sigma^2}, -\frac{1}{2 \, \sigma^2} \right)\), and natural statistics \((X, X^2)\).

Computational Exercises

Suppose that the volume of beer in a bottle of a certain brand is normally distributed with mean 0.5 liter and standard deviation 0.01 liter.

  1. Find the probability that a bottle will contain at least 0.48 liter.
  2. Find the volume that corresponds to the 95th percentile

A metal rod is designed to fit into a circular hole on a certain assembly. The radius of the rod is normally distributed with mean 1 cm and standard deviation 0.002 cm. The radius of the hole is normally distributed with mean 1.01 cm and standard deviation 0.003 cm. The machining processes that produce the rod and the hole are independent. Find the probability that the rod is to big for the hole.

The weight of a peach from a certain orchard is normally distributed with mean 8 ounces and standard deviation 1 ounce. Find the probability that the combined weight of 5 peaches exceeds 45 ounces.

  1. Virtual Laboratories
  2. 4. Special Distributions
  3. 1
  4. 2
  5. 3
  6. 4
  7. 5
  8. 6
  9. 7
  10. 8
  11. 9
  12. 10
  13. 11
  14. 12
  15. 13
  16. 14
  17. 15
  18. Answers