Transformations of Variables

As usual, we start with a random experiment with probability measure

on an underlying sample space. Suppose that we have a random variable

X

for the experiment, taking values in

S

, and a function

r S T

. Then

Y r X

is a new random variable taking values in

T

. If the distribution of

X

is known, how do we find the distribution of

Y

? This is a very basic and important question, and in a superficial sense, the solution is easy.

Show that $Y B X r B$ for $B T$ .

However, frequently the distribution of

X

is known either through its distribution function

F

or its density function

f

, and we would similarly like to find the distribution function or density function of

Y

. This is a difficult problem in general, because as we will see, even simple transformations of variables with simple distributions can lead to variables with complex distributions. We will solve the problem in various special cases.

Transformed Variables with Discrete Distributions

Suppose that $X$ has a discrete distribution with probability density function $f$ (and hence $S$ is countable). Show that $Y$ has a discrete distribution with probability density function $g$ given by

g y x r y f x, y T

Suppose that $X$ has a continuous distribution on a subset $S n$ with probability density function $f$ , and that $T$ is countable. Show that $Y$ has a discrete distribution with probability density function $g$ given by

g y x r y f x, y T

Transformed Variables with Continuous Distributions

Suppose that

X

has a continuous distribution on a subset

S n

and that

X r X

has a continuous distributions on a subset

T m

. Suppose also that

X

has a known probability density function

f

. In many cases, the density function of

Y

can be found by first finding the distribution function of

Y

(using basic rules of probability) and then computing the appropriate derivatives of the distribution function. This general method is referred to, appropriately enough, as the distribution function method.

The Change of Variables Formula

When the transformation

r

is one-to-one and smooth, there is a formula for the probability density function of

Y

directly in terms of the probability density function of

X

. This is known as the change of variables formula.

We will explore the one-dimensional case first, where the concepts and formulas are simplest. Thus, suppose that random variable

X

has a continuous distribution on an interval

S

. with distribution function

F

and density function

f

. Suppose that

Y r X

where

r

is a differentiable function from

S

onto an interval

T

. As usual, we will let

G

denote the distribution function of

Y

and

g

the density function of

Y

Suppose that $r$ is strictly increasing on $S$ . Show that for $y T$ ,

$G y F r y$
$g y f r y y r y$

Suppose that $r$ is strictly decreasing on $S$ . Show that for $y T$ ,

$G y 1 F r y$
$g y f r y y r y$

The density function formulas in Exercises 4 and 5 can be combined: if

r

is a strictly monotone on

S

then the density function

g

Y

is given by

Letting

x r y

, the change of variables formula can be written more compactly as

The generalization this result is basically a theorem in multivariate calculus. Suppose that

X

is a random variable taking values in

S n

, and that

X

has a continuous distribution with probability density function

f

. Suppose that

Y r X

where

r

is a one-to-one, differentiable function form

S

onto

T n

. The first derivative of the inverse function

x r y

is the

n n

matrix of first partial derivatives:

The Jacobian (named in honor of Karl Gustav Jacobi) of the inverse function is the determinant of the first derivative matrix

With this compact notation, the multivariate change of variables formula states that the density

g

Y

is given by

Special Transformations

Linear Transformations

Linear transformations (or more technically affine transformations) are among the most common and important transformations. Moreover, this type of transformation leads to a simple application of the change of variable theorem. Suppose that

X

is a random variable taking values in

S

and that

X

has a continuous distribution on

S

with probability density function

f

. Let

Y a b X

where

a

and

b 0

Note that and that

Y

takes values in

T a b x x S

Apply the change of variables theorem to show that $Y$ has probability density function

g y 1 b f y a b, y T

When

b 0

(which is often the case in applications), this transformation is known as a location-scale transformation;

a

is the location parameter and

b

is the scale parameter. Location-scale transformations are studied in more detail in the chapter on Special Distributions.

The multivariate version of this result has a simple and elegant form when the linear transformation is expressed in matrix-vector form. Thus suppose that

X

is a random variable taking values in

S n

and that

X

has a continuous distribution on

S

with probability density function

f

. Let

Y a B X

where

a n

and

B

is an invertible

n n

matrix. Note that and that

Y

takes values in

T a B x x S

Show that

The transformation $y a B x$ maps $n$ one-to-one and onto $n$ .
The inverse transformation is $x B y a$ .
The Jacobian of the inverse transformation is the constant function $B 1 B$ .

Apply the change of variables theorem to show that $Y$ has probability density function

g y f B y a B, y T

Sums and Convolution

Simple addition of random variables is perhaps the most important of all transformations. Suppose that

X

and

Y

are real-valued random variables for a random experiment. Our goal is to find the distribution of

Z X Y

Suppose that $X Y$ has a discrete distribution with probability density function $f$ . Show that $Z$ has a discrete distribution with probability density function $u$ given by

u z x f x z x

Suppose that $X Y$ has a continuous distribution with probability density function $f$ . Show that the $Z$ has a continuous distribution with probability density function $u$ given by

u z x f x z x

Suppose that $X$ and $Y$ are independent and have discrete distributions with densities $g$ and $h$ respectively. Show that $Z$ has a probability density function

g h z x g x h z x

The sum is actually over the countable set

x g x 0 h z x 0

. The probability density function

g h

defined in the previous exercise is called the discrete convolution of

g

and

h

Suppose that $X$ and $Y$ are independent and have continuous distributions with densities $g$ and $h$ respectively. Show that $Z$ has a probability density function given by

g h z x g x h z x

The integral is actually over the set

x g x 0 h z x 0

. The probability density function

g h

is called the continuous convolution of

f

and

g

Show that convolution (either discrete or continuous) satisfies the properties below (where $f$ , $g$ . and $h$ are probability density functions). Give an analytic proof, based on the definition of convolution, and a probabilistic proof, based on sums of independent random variables

$f g g f$ (the commutative property)
$f g h f g h$ (the associative property

Thus, in part (b) we can write

f g h

without ambiguity. Note that if

X 1 X 2 X n

is a sequence of independent and identically distributed random variables, with common probability density function

f

. then

has probability density function

f n

, the

n

-fold convolution power of

f

. When appropriately scaled and centered, the distribution of

Y n

converges to the standard normal distribution as

n

. The precise statement of this result is the central limit theorem, one of the fundamental theorems of probability. The central limit theorem is studied in detail in the chapter on Random Samples.

Minimum and Maximum

Suppose that

X 1 X 2 X n

is a sequence of independent real-valued random variables. The minimum and maximum transformations are very important in a number of applications.

For example, recall that in the standard model of structural reliability, a system consists of

n

components that operate independently. Suppose that

X i

represents the lifetime of component

i

. Then

U

is the lifetime of the series system which operates if and only if each component is operating. Similarly,

V

is the lifetime of the parallel system which operates if and only if at least one component is operating.

A particularly important special case occurs when the random variables are identically distributed, in addition to being independent. In this case, the sequence of variables is a random sample of size

n

from the common distribution. We usually think of the random variables as independent copies of an underlying random variable. The minimum and maximum variables are the extreme examples of order statistics. Order statistics are studied in detail in the chapter on Random Samples.

Let

F i

denote the distribution function of

X i

for each

i 12 n

, and let

G

and

H

denote the distribution functions of

U

and

V

respectively.

Show that for $x$ ,

$V x X 1 x X 2 x X n x$
$H x F 1 x F 2 x F n x$

Show that for $x$ ,

$U x X 1 x X 2 x X n x$
$G x 1 1 F 1 x 1 F 2 x 1 F n x$

From Exercise 14, note that the product of

n

distribution functions is another distribution function. From Exercise 15, the product of

n

right-tail distribution functions is a right-tail distribution function. In the reliability setting, where the random variables are nonnegative, the product of

n

reliability functions is a reliability function. If

X i

has a continuous distribution with probability density function

f i

for each

i 12 n

then

U

and

V

also have continuous distributions, and their probability density functions can be obtained by differentiating the distribution functions in Exercises 14 and 15.

The formulas are particularly nice when the random variables

X i

are identically distributed, in addition to being independent. Suppose that this is the case, and let

F

be the common distribution function.

Show that for $x$ ,

$H x F n x$
$G x 1 1 F x n$

In particular, it follows that a positive integer power of a distribution function is a distribution. More generally, it's easy to see that every positive power of a distribution function is a distribution function. How could we construct a non-integer power of a distribution function in a probabilistic way? Now, in addition to the independent and identically distributed assumptions, suppose that the common distribution of the variables

X i

is continuous, with density function

f

. Let

g

and

h

denote the density functions of

U

and

V

. respectively.

Show that for $x$ ,

$h x n F n 1 x f x$
$g x n 1 F x n 1 f x$

Sign and Absolute Value

Suppose that $X$ has a continuous distribution on with distribution function $F$ and density function $f$ . Show that

$X$ has distribution function $G y F y F y, y 0$
$X$ has density function $g y f y f y, y 0$

Suppose that the probability density $f$ of $X$ is symmetric with respect to 0, so that $f x f x$ for all $x$ . Show that

$X$ has distribution function $G y 2 F y 1, y 0$
$X$ has density function $g y 2 f y, y 0$ .
$J$ is uniformly distributed on $11$
$X$ and $J$ are independent.

Examples and Applications

This subsection contains computational exercises, many of which involve special parametric families of distributions. It is always interesting when a random variable from one parametric family can be transformed into a variable from another family. It is also interesting when a parametric family is closed or invariant under some transformation on the variables in the family. Often, such properties are what make the parametric families special in the first place. Please note these properties when they occur.

Dice

Recall that a standard die is an ordinary 6-sided die. A fair die is one in which the faces are equally likely. An ace six flat die is a standard die in which faces 1 and 6 occur with probability

1 4

each and the other faces with probability

1 8

Suppose that two standard dice are rolled and the sequence of scores $X 1 X 2$ is recorded. Find the probability density function of $Y X 1 X 1$ , the sum of the scores, in each of the following cases:

fair dice
ace-six flat dice

In the dice experiment, select two dice and select the sum random variable. Run the simulation 1000 times, updating every 10 runs and note the apparent convergence of the empirical density function to the probability density function for each of the following cases:

fair dice
ace-six flat dice

A fair die and an ace-six flat die are rolled. Find the probability density function of the sum of the scores.

Suppose that $n$ standard, fair dice are rolled. Find the probability density function of the following variables:

the minimum score
the maximum score.

In the dice experiment, select fair dice and select each of the following random variables. Vary $n$ with the scroll bar and note the shape of the density function. With $n 4$ , run the simulation 1000 times, updating every 10 runs. Note the apparent convergence of the empirical density function to the probability density function.

minimum score
maximum score.

The Uniform Distribution

Let $Y X 2$ . Find the density function of $Y$ and sketch the graph in each of the following cases:

$X$ is uniformly distributed on the interval $22$
$X$ is uniformly distributed on the interval $13$
$X$ is uniformly distributed on the interval $24$ .

Compare the distributions in the last exercise. Note that even a simple transformation of a simple distribution can produce a complicated distribution. On the other hand, the uniform distribution is preserved under a linear transformation of the random variable.

Suppose that $X$ is uniformly distributed on $S n$ . Let $Y a B X$ , where $a n$ and $B$ is an invertible $n n$ matrix. Show that $Y$ is uniformly distributed on $T a B x x S$ .

Suppose that $X Y$ is uniformly distributed on the square $012$ . Let $U V X Y X Y$ .

Sketch the range of $X Y$ and the range of $U V$
Find the density function of $U V$
Find the density function of $U$ .
Find the density function of $V$ .

Suppose that $X Y Z$ is uniformly distributed on the cube $013$ . Find the probability density function of $U V W X Y Y Z X Z$ .

Suppose that $X 1 X 2 X n$ is a sequence of independent random variables, each uniformly distributed on $01$ . Find the distribution and density functions of the following variables. Both distributions are beta distributions.

$U X 1 X 2 X n$
$V X 1 X 2 X n$

In the order statistic experiment, select the uniform distribution.

Set $k 1$ (this gives the minimum $U$ ). Vary $n$ with the scroll bar and note the shape of the density function. With $n 5$ , run the simulation 1000 times, updating every 10 runs. Note the apparent convergence of the empirical density function to the true density function.
Vary $n$ with the scroll bar, set $k n$ each time (this gives the maximum $V$ ), and note the shape of the density function. With $n 5$ run the simulation 1000 times, updating every 10 runs. Note the apparent convergence of the empirical density function to the true density function.

Let $f$ denote the density function of the uniform distribution on $01$ . Compute $f 2$ and $f 3$ . Graph the three density functions on the same set of axes. Note the behavior predicted by the central limit theorem beginning to emerge.

Simulations

A remarkable fact is that the uniform distribution on

01

can be transformed into almost any other distribution on

. This is particularly important for simulations, since many computer languages have an algorithm for generating random numbers, which are simulations of independent variables, each uniformly distributed on

01

. Conversely, any continuous distribution supported on an interval of

can be transformed into the uniform distribution on

01

Suppose first that

F

is a distribution function for a distribution on

(which may be discrete, continuous, or mixed), and let

F

denote the quantile function.

Suppose that $U$ is uniformly distributed on $01$ . Show that $X F U$ has distribution function $F$ .

Assuming that we can compute

F

, the previous exercise shows how we can simulate a distribution with distribution function

F

. To rephrase the result, we can simulate a variable with distribution function

F

by simply computing a random quantile. Most of the applets in this project use this method of simulation.

Suppose that $X$ has a continuous distribution on an interval $S$ and that the distribution function $F$ is strictly increasing on $S$ . Show that $U F X$ has the uniform distribution on $01$ .

Show how to simulate the uniform distribution on the interval $a b$ with a random number. Using your calculator, simulate 5 values from the uniform distribution on the interval $210$ .

Bernoulli Trials

Recall that a Bernoulli trials sequence is a sequence

X 1 X 2

of independent, identically distributed indicator random variables. In the usual terminology of reliability theory,

X i 0

means failure on trial

i

. while

X i 1

means success on trial

i

. The basic parameter of the process is the probability of success

p X i 1

. The random process is named for Jacob Bernoulli and is studied in detail in the chapter on Bernoulli trials.

Show that the common probability density function of the trial variables $X i$ is $f k p k 1 p 1 k, k 01$ .

Let $Y n$ denote the number of successes in the first $n$ trials. Use an argument based on combinatorics and independence to show that $Y n$ has the probability density function $f n$ given below. This defines the binomial distribution with parameters $n$ and $p$ .

f n k n k p k 1 p n k, k 01 n

As above, let $Y n$ denote the number of successes in the first $n$ trials.

Show that $Y n X 1 X 2 X n$
Conclude from (a) that $f n f n$
Conclude from (b) that $f m f n f m n$
Prove the result in (c) directly, using the definition of convolution.

In particular, it follows that if

Y

and

Z

are independent variables, and that

Y

has the binomial distribution with parameters

n

and

p

while

Z

has the binomial distribution with parameter

m

and

p

. Show that

Y Z

has the binomial distribution with parameter

n m

and

p

Find the probability density function of the difference between the number of successes and the number of failures in $n$ Bernoulli trials.

The Poisson Distribution

Recall that the Poisson distribution with parameter

t 0

has probability density function

This distribution is named for Simeon Poisson and is widely used to model the number of random points in a region of time or space. The Poisson distribution is studied in detail in the chapter on The Poisson Process.

Suppose that $X$ and $Y$ are independent variables, and that $X$ has the Poisson distribution with parameter $a 0$ while $Y$ has the Poisson distribution with parameter $b 0$ . Show that $X Y$ has the Poisson distribution with parameter $a b$ . Equivalently, show that $f a f b f a b$ . Hint: You will need to use the binomial theorem.

The Exponential Distribution

Recall that the exponential distribution with rate parameter

r 0

has probability density function

f t r r t, t 0

. These distributions are often used to model random times such as failure times and arrival times. Exponential distributions are studied in more detail in the chapter on Poisson Processes.

Show how to simulate, with a random number, the exponential distribution with rate parameter $r$ . Using your calculator, simulate 5 values from the exponential distribution with parameter $r 3$ .

Suppose that $T$ has the exponential distribution with rate parameter $r$ . Find the density function of the following random variables:

$T$ , the largest integer less than or equal to $T$ .
$T$ , the smallest integer greater than or equal to $T$ .

Note that the distributions in the previous exercise are geometric distributions on

and on

, respectively. In many respects, the geometric distribution is a discrete version of the exponential distribution.

Suppose that $X$ and $Y$ are independent random variables, each having the exponential distribution with parameter 1. Let $Z Y X$ .

Find the distribution function of $Z$ .
Find the density function of $Z$ .

Suppose that $X$ has the exponential distribution with rate parameter $a 0$ , $Y$ has the exponential distribution with rate parameter $b 0$ , and that $X$ and $Y$ are independent. Find the probability density function of $Z X Y$ and sketch the graph.

Suppose that $T 1 T 2 T n$ is a sequence of independent random variables, and that $T i$ has the exponential distribution with rate parameter $r i 0$ for each $i 12 n$ .

Find the distribution function of $U T 1 T 2 T n$ .
Find the distribution function of $V T 1 T 2 T n$ .
Find the density function and reliability function of $U$ and $V$ in the special case that $r i r$ for each $i 12 n$

Note that the minimum

U

in part (a) has the exponential distribution with parameter

r 1 r 2 r n

. In particular, suppose that a series system has independent components, each with an exponentially distributed lifetime. Then the lifetime of the system is also exponentially distributed, and the failure rate of the system is the sum of the component failure rates.

In the order statistic experiment, select the exponential distribution.

Set $k 1$ (this gives the minimum $U$ ). Vary $n$ with the scroll bar and note the shape of the density function. With $n 5$ , run the simulation 1000 times, updating every 10 runs. Note the apparent convergence of the empirical density function to the true density function.
Vary $n$ with the scroll bar, set $k n$ each time (this gives the maximum $V$ ), and note the shape of the density function. With $n 5$ , run the simulation 1000 times, updating every 10 runs. Note the apparent convergence of the empirical density function to the true density function.

In the setting of Exercise 44, show that for $i 12 n$ ,

j i T i T j r i j 1 n r j

Show first that if $X$ and $Y$ have exponential distributions with parameters $a$ and $b$ . respectively, and are independent, then $X Y a a b$
Next, note that $T i T j$ for all $j i$ if and only if $T i T j j i$ .
Note that he minimum on the right is independent of $T i$ and, by Exercise 44, has an exponential distribution with parameter $j i r j$

The result in the previous exercise is very important in the theory of continuous-time Markov chains.

The Gamma Distribution

Recall that for

n 12

, the gamma distribution with shape parameter

n

has probability density function

This distribution is widely used to model arrival times under certain basic assumptions.. The gamma distribution is studied in detail in the chapter on The Poisson Process.

Show that if $X$ has the gamma distribution with shape parameter $m$ and $Y$ has the gamma distribution with shape parameter $n$ . and if $X$ and $Y$ are independent, then $X Y$ has the gamma distribution with shape parameter $m n$ . Equivalently, show that $g m g n g m n$ .

Suppose that $T$ has the gamma distribution with shape parameter $n$ . Find the density function of $X T$ .

The Pareto Distribution

Recall that the Pareto distribution with shape parameter

a 0

has probability density function

The distribution is named for Vilfredo Pareto. It is a heavy-tailed distribution often used for modeling income and other financial variables. The Pareto distribution is studied in more detail in the chapter on Special Distributions.

Suppose that $X$ has the Pareto distribution with shape parameter $a$ . Find the density function of $Y X$ . Note that $Y$ has the exponential distribution with rate parameter $a$ .

Suppose that $X$ has the Pareto distribution with shape parameter $a$ . Find the probability density function of $Y 1 X$ . The distribution of $Y$ is the beta distribution with parameters $a$ and $b 1$ .

Show how to simulate, with a random number, the Pareto distribution with shape parameter $a$ . Using your calculator, simulate 5 values from the Pareto distribution with shape parameter $a 2$ .

The Normal Distribution

Suppose that $Z$ has the standard normal distribution, and that $μ$ and $σ 0$ . Find the density function of $X μ σ Z$ and sketch the graph.

Random variable

X

has the normal distribution with location parameter

μ

and scale parameter

σ

. The normal distribution is perhaps the most important distribution in probability and mathematical statistics. It is widely used to model physical measurements of all types that are subject to small, random errors. The normal distribution is studied in detail in the chapter on Special Distributions.

Suppose that $Z$ has the standard normal distribution. Find the density function of $V Z 2$ and sketch the graph.

Suppose that $X$ has the normal distribution with location parameter $μ$ and scale parameter $σ$ , $Y$ has the normal distribution with location parameter $ν$ and scale parameter $τ$ . and that $X$ and $Y$ are independent. Show that $X Y$ has the normal distribution with location parameter $μ ν$ and scale parameter $σ 2 τ 2$

The Cauchy Distribution

Suppose that $X$ and $Y$ are independent random variables, each with the standard normal distribution. Show that $T X Y$ has probability density function $f$ given below, and sketch the graph.

f t 1 1 t 2, t

For $c 0$ , show that the probability density function of $c T$ is

f c t c c 2 t 2, t

Show that if $U$ has the Cauchy distribution with scale parameter $c$ , $V$ has the Cauchy distribution with scale parameter $d$ . and $U$ and $V$ are independent, then $U V$ has the Cauchy distribution with scale parameter $c d$ . Equivalently, show that $f c f d f c d$

The Beta Distribution

Suppose that $X$ has the probability density function $f x 3 x 2, 0 x 1$ . Find the probability density function of $Y 3 X$ and sketch its graph.

Random variables

X

and

Y

both have beta distributions, which are widely used to model random proportions and probabilities. The family of beta distributions is studied in detail in the chapter on Special Distributions.

7. Transformations of Variables

Basic Theory

The Problem

Transformed Variables with Discrete Distributions

Transformed Variables with Continuous Distributions

The Change of Variables Formula

Special Transformations

Linear Transformations

Sums and Convolution

Minimum and Maximum

Sign and Absolute Value

Examples and Applications

Dice

The Uniform Distribution

Simulations

Bernoulli Trials

The Poisson Distribution

The Exponential Distribution

The Gamma Distribution

The Pareto Distribution

The Normal Distribution

The Cauchy Distribution

The Beta Distribution