\(\newcommand{\R}{\mathbb{R}}\)
\(\newcommand{\N}{\mathbb{N}}\)
\(\newcommand{\E}{\mathbb{E}}\)
\(\newcommand{\P}{\mathbb{P}}\)
\(\newcommand{\var}{\text{var}}\)
\(\newcommand{\sd}{\text{sd}}\)
\(\newcommand{\cov}{\text{cov}}\)
\(\newcommand{\cor}{\text{cor}}\)
\(\newcommand{\skew}{\text{skew}}\)
\(\newcommand{\kurt}{\text{kurt}}\)

- Random
- 4. Special Distributions
- The Logistic Distribution

The logistic distribution is used for various growth models, and is used in a certain type of regression, known appropriately as logistic regression.

The function \(F\) given below is a distribution function. \[ F(x) = \frac{e^x}{1 + e^x}, \quad x \in \R \] The probability distribution defined by \( F \) is the standard logistic distribution, and is a continuous distribution on \( \R \)

Note that \( F \) is continuous, and \( F(x) \to 0 \) as \( x \to -\infty \) and \( F(x) \to 1 \) as \( x \to \infty \). Moreover, \( F^\prime(x) \gt 0 \) for \( x \in \R \) (see the next theorem), so \( F \) is increasing.

The probability density function \(f\) of the standard logistic distribution is given by \[ f(x) = \frac{e^x}{\left(1 + e^x\right)^2}, \quad x \in \R \]

- \(f\) is symmetric about \(x = 0\).
- \(f\) increases and then decreases with the mode \(x = 0\).
- \( f \) is concave upward, then downward, then upward again with inflection points at \( x = \pm \ln\left(2 + \sqrt{3}\right) \approx = \pm 1.317 \).

These result follow from standard calculus. First recall that \( f = F^\prime \).

- The symmetry of \( f \) is not obvious at first, but note that \[ f(-x) = \frac{e^{-x}}{\left(1 + e^{-x}\right)^2} \frac{e^{2x}}{e^{2x}} = \frac{e^x}{\left(1 + e^x\right)^2} = f(x) \]
- The first derivative of \( f \) is \[ f^\prime(x) = \frac{e^x (1 - e^x)}{(1 + e^x)^3} \]
- The second derivative of \( f \) is \[ f^{\prime \prime}(x) = \frac{e^x \left(1 - 4 e^x + e^{2x}\right)}{(1 + e^x)^4} \]

In the special distribution simulator, select the logistic distribution. Keep the default parameter values and note the shape and location of the probability density function. Run the simulation 1000 times and compare the empirical density function to the probability density function.

The quantile function \( F^{-1} \) of the standard logistic distribution is given by \[ F^{-1}(p) = \ln \left( \frac{p}{1 - p} \right), \quad p \in (0, 1) \]

- The first quartile is \( -\ln(3) \approx -1.0986\).
- The median is 0.
- The third quartile is \( \ln(3) \approx 1.0986 \)

The formula for \( F^{-1} \) follows by solving \( p = F(x) \) for \( x \) in terms of \( p \).

Recall that \(p : 1 - p\) are the odds in favor of an event with probability \(p\). Thus, the logistic distribution has the interesting property that the quantiles are the logarithms of the corresponding odds ratios. Indeed, this function of \(p\) is sometimes called the logit function. The fact that the median is 0 also follows from symmetry, of course.

In the special distribution calculator, select the logistic distribution. Keep the default parameter values and note the shape and location of the probability density function and the distribution function. Find the quantiles of order 0.1 and 0.9.

Suppose that \( X \) has the standard logistic distribution. The moment generating function of \( X \) has a simple representation in terms of the beta function, and hence also in terms of the gamma function.

The moment generating function of \( X \) can be written in terms of the beta function and in terms of the gamma function:

\[ M(t) = B(1 + t, 1 - t) = \Gamma(1 + t) \, \Gamma(1 - t), \quad t \in (-1, 1) \]Note that \[ M(t) = \int_{-\infty}^\infty e^{t x} \frac{e^x}{\left(1 + e^x\right)^2} dx \] Let \(u = \frac{e^x}{1 + e^x}\) so that \( du = \frac{e^x}{\left(1 + e^x\right)^2} dx \) and \( e^x = \frac{u}{1 - u} \). Hence \[ M(t) = \int_0^1 \left(\frac{u}{1 - u}\right)^t du = \int_0^1 u^t (1 - u)^{-t} \, du \] The last integral, by definition, is \( B(1 + t, 1 - t) \) for \( t \in (-1, 1) \)

Since the moment generating function is finite on an open interval containing 0, random variable \( X \) has moments of all orders. By symmetry, the odd order moments are 0. The even order moments can be represented in terms of Bernoulli numbers, named of course for Jacob Bernoulli.

For \( n \in \N \), let \( \beta_n \) denote the \( n \)th Bernoulli number.

- If \( n \in \N \) is odd then \( \E(X^n) = 0 \).
- If \( n \in \N \) is even then \( \E\left(X^n\right) = (2^n - 2) \pi^n \left|\beta_n\right| \)

- Again, this follows from symmetry
- Recall that the moments of \( X \) can be computed by integrating powers of the quantile function of \( X \). Hence \[ \E\left(X^n\right) = \int_0^1 \left[\ln\left(\frac{p}{1 - p}\right)\right]^n dp \] This integral evaluates to the expression above involving the Bernoulli numbers.

In particular, we have the mean and variance.

The mean and variance of \( X \) are

- \(\E(X) = 0\)
- \(\var(X) = \frac{\pi^2}{3}\)

- Again, \( \E(X) = 0 \) by symmetry.
- The second Bernoulli number is \( \beta_2 = \frac{1}{6} \). Hence \( \var(X) = \E\left(X^2\right) = (2^2 - 2) \pi^2 \frac{1}{6} = \frac{\pi^2}{ 3 } \).

In the special distribution simulator, select the logistic distribution. Keep the default parameter values and note the shape and location of the mean \( \pm \) standard deviation bar. Run the simulation 1000 times and compare the empirical mean and standard deviation to the distribution mean and standard deviation.

The skewness and kurtosis of \( X \) are

- \( \skew(X) = 0 \)
- \( \kurt(X) = \frac{21}{5} \)

- Again, \( \skew(X) = 0 \) by the symmetry of the distribution.
- Recall that by symmetry, \( \E(X) = \E\left(X^3\right) = 0 \). Also, \( \left|\beta_4\right| = \frac{1}{30} \), so \( \E\left(X^4\right) = (2^4 - 2) \pi^4 \frac{1}{30} = \frac{7 \pi^4}{15} \). Hence from the usual computational formula for kurtosis, \[ \kurt(X) = \frac{\E\left(X^4\right)}{[\var(X)]^2} = \frac{7 \pi^4 / 15}{\pi^4 / 9} = \frac{21}{5} \]

It follows that the excess kurtosis of \( X \) is \( \kurt(X) - 3 = \frac{6}{5} \).

The standard logistic distribution has the usual connections with the standard uniform distribution by means of the distribution and quantile functions. Recall that the standard uniform distribution is the continuous uniform distribution on the interval \( (0, 1) \).

Connections with the standard uniform distribution.

- If \( X \) has the standard logistic distribution then \[ U = F(X) = \frac{e^X}{1 + e^X} \] has the standard uniform distribution.
- If \( U \) has the standard uniform distribution then \[ X = F^{-1}(U) = \ln\left(\frac{U}{1 - U}\right) = \ln(U) - \ln(1 - U) \] has the standard logistic distribution.

Since the quantile function has a simple closed form, we can use the usual random quantile method to simulate the standard logistic distribution.

Open the random quantile experiment and select the logistic distribution. Keep the default parameter values and note the shape of the probability density and distribution functions. Run the simulation 1000 times and compare the empirical density function, mean, and standard deviation to their distributional counterparts.

The standard logistic distribution also has several simple connections with the standard exponential distribution (the exponential distribution with rate parameter 1).

Connections with the standard exponential distribution:

- If \( X \) has the standard logistic distribution, then \( Y = \ln\left(e^X + 1\right) \) has the standard exponential distribution.
- If \( Y \) has the standard exponential distribution then \( X = \ln\left(e^Y - 1\right) \) has the standard logistic distribution.

These results follow from the standard change of variables formula. The transformations, inverses of each other of course, are \( y = \ln\left(e^x + 1\right) \) and \( x = \ln\left(e^y - 1\right) \) for \( x \in \R \) and \( y \in (0, \infty) \). Let \( f \) and \( g \) denote the PDFs of \( X \) and \( Y \) respectively.

- By definition, \( f(x) = e^x \big/ (1 + e^x)^2 \) for \( x \in \R \) so \[ g(y) = f(x) \frac{dx}{dy} = \frac{\exp\left[\ln\left(e^y - 1\right)\right]}{\left(1 + \exp\left[\ln\left(e^y - 1\right)\right]\right)^2} \frac{e^y}{e^y - 1} = e^{-y}, \quad y \in (0, \infty) \] which is the PDF of the standard exponential distribution.
- By definition, \( g(y) = e^{-y} \) for \( y \in (0, \infty) \) so \[ f(x) = g(y) \frac{dy}{dx} = \exp\left[-\ln\left(e^x + 1\right)\right] \frac{e^x}{e^x + 1} = \frac{e^x}{\left(e^x + 1\right)^2}, \quad x \in \R \] which is the PDF of the standard exponential distribution.

Suppose that \( X \) and \( Y \) are independent random variables, each with the standard exponential distribution. Then \( Z = \ln(X / Y) \) has the standard logistic distribution.

For \( z \in \R \), \[ \P(Z \le z) = \P[\ln(X / Y) \le z] = \P\left(X / Y \le e^z\right) = \P\left(Y \ge e^{-z} X\right) \] Recall that \( \P(Y \ge y) = e^{-y} \) for \( y \in (0, \infty) \) and \( X \) has PDF \( e^{-x} \) for \( x \in (0, \infty) \). We condition on \( X \): \[ \P(Z \le z) = \E\left[\P\left(Y \ge e^{-z} X \mid X\right)\right] = \int_0^\infty e^{-e^{-z} x} e^{-x} dx = \int_0^\infty e^{(e^{-z} + 1)x} dx = \frac{1}{e^{-z} + 1} = \frac{e^z}{1 + e^z} \] As a function of \( z \), this is the distribution function of the standard logistic distribution.

There are also simple connections between the standard logistic distribution and the Pareto distribution.

Connections with the Pareto distribution:

- If \( X \) has the standard logistic distribution, then \( Y = e^X + 1 \) has the Pareto distribution with shape parameter 1.
- If \(Y\) has the Pareto distribution with shape parameter 1, then \(X = \ln(Y - 1)\) has the standard logistic distribution.

These results follow from the basic change of variables theorem. The transformation, inverses of one another of course, are \( y = e^x + 1 \), \( x = \ln(y - 1) \) for \( x \in \R \) and \( y \in (1, \infty) \). Let \( f \) and \( g \) denote PDFs of \( X \) and \( Y \) respectively.

- By definition, \( f(x) = e^x \big/ \left(1 + e^x\right)^2 \) for \( x \in \R \). Hence \[ g(y) = f(x) \frac{dx}{dy} = \frac{\exp[\ln(y - 1)]}{\left(1 + \exp[\ln(y - 1)]\right)^2} \frac{1}{y - 1} = \frac{1}{y^2}, \quad y \in (1, \infty) \] which is the PDF of the Pareto distribution with shape parameter 1.
- By definition, \( g(y) = 1 / y^2 \) for \( y \in (1, \infty) \). Hence \[ f(x) = g(y) \frac{dy}{dx} = \frac{1}{(e^x + 1)^2} e^x, \quad x \in \R\] which is the PDF of the standard logistic distribution.

Finally, there are simple connections to the extreme value distribution.

If \( X \) and \( Y \) are independent and each has the standard Gumbel distribution, them \( Z = Y - X \) has the standard logistic distribution.

The distribution function of \( Y \) is \( G(y) = \exp\left(-e^{-y}\right) \) for \( y \in \R \) and the density function of \( X \) is \( g(x) = e^{-x} \exp\left(-e^{-x}\right) \) for \( x \in \R \). For \( z \in \R \), conditioning on \( X \) gives \[ \P(Z \le z) = \P(Y \le X + z) = \E[\P(Y \le X + z \mid X)] = \int_{-\infty}^\infty \exp\left(-e^{-(x + z)}\right) e^{-x} \exp\left(-e^{-x}\right) dx\] Substituting \( u = -e^{-(x + z)} \) gives \[ \P(Z \le z) = \int_{-\infty}^0 e^u \exp(e^z u) e^z du = e^z \int_{-\infty}^0 \exp\left[u(1 + e^z)\right] du = \frac{e^z}{1 + e^z}, \quad z \in \R \] As a function of \( z \), this is the standard logistic distribution function.

The general logistic distribution is the location-scale family associated with the standard logistic distribution.

If \(Z\) has the standard logistic distribution, then for \(a \in \R\) and \( b \in (0, \infty) \), \( X = a + b Z \) has the logistic distribution with location parameter \(a\) and scale parameter \(b\).

Analogies of the results above for the general logistic distribution follow easily from basic properties of the location-scale transformation. Suppose that \( X \) has the logistic distribution with location parameter \( a \in \R \) and scale parameter \( b \in (0, \infty) \).

The probability density function \( f \) of \( X \) is given by \[ f(x) = \frac{\exp \left(\frac{x - a}{b} \right)}{b \left[1 + \exp \left(\frac{x - a}{b} \right) \right]^2}, \quad x \in \R \]

- \( f \) is symmetric about \( x = a \).
- \( f \) increases and then decreases, with mode \( x = a \).
- \( f \) is concave upward, then downward, then upward again, with inflection points at \( x = a \pm \ln\left(2 + \sqrt{3}\right) b \).

Recall that \[ f(x) = \frac{1}{b} g\left(\frac{x - a}{b}\right), \quad x \in \R \] where \( g \) is the standard logsitic PDF, given above.

In the special distribution simulator, select the logistic distribution. Vary the parameters and note the shape and location of the probability density function. For selected values of the parameters, run the simulation 1000 times and compare the empirical density function to the probability density function.

The distribution function \( F \) of \( X \) is given by \[ F(x) = \frac{\exp \left( \frac{x - a}{b} \right)}{1 + \exp \left( \frac{x - a}{b} \right)}, \quad x \in \R \]

Recall that \[ F(x) = G\left(\frac{x - a}{b}\right), \quad x \in \R \] where \( G \) is the CDF of the standard logistic distribution, given above.

The quantile function \( F^{-1} \) of \( X \) is given by \[ F^{-1}(p) = a + b \, \ln \left( \frac{p}{1 - p} \right), \quad p \in (0, 1) \]

- The first quartile is \( a - \ln(3) b\).
- The median is a.
- The third quartile is \( a + \ln(3) b \)

Recall that \( F^{-1}(p) = a + b G^{-1}(p) \) for \( p \in (0, 1) \), where \( G^{-1} \) is the standard logistic quantile function, given above.

In the special distribution calculator, select the logistic distribution. Vary the parameters and note the shape and location of the probability density function and the distribution function. For selected values of the parameters, find the quantiles of order 0.1 and 0.9.

Suppose again that \( X \) has the logistic distribution with location parameter \( a \in \R \) and scale parameter \( b \in (0, \infty) \).

The moment generating function \( M \) of \( X \) is given by \[ M(t) = e^{a t} B(1 + b t, 1 - b t) = e^{a t} \Gamma(1 + b t) \, \Gamma(1 - b t), \quad t \in (-1, 1) \] where as before, \( B \) is the beta function and \( \Gamma \) is the gamma function.

Recall that \( M(t) = e^{a t} m(b t) \) where \( m \) is the standard logistic MGF, given above.

The mean and variance of \( X \) are

- \(\E(X) = a\)
- \(\var(X) = b^2 \frac{\pi^2}{3}\)

Let \( X = a + b Z \) where \( Z \) has the standard logistic distribution. Then

- \( \E(X) = a + b \E(Z) = a \)
- \( \var(X) = b^2 \var(Z) = b^2 \frac{\pi^2}{3} \)

In the special distribution simulator, select the logistic distribution. Vary the parameters and note the shape and location of the mean \( \pm \) standard deviation bar. For selected values of the parameters, run the simulation 1000 times and compare the empirical mean and standard deviation to the distribution mean and standard deviation.

The skewness and kurtosis of \( X \) are

- \( \skew(X) = 0 \)
- \( \kurt(X) = \frac{21}{5} \)

Recall that skewness and kurtosis are defined in terms of the standard score, and hence are invariant under location-scale transformations.

Once again, it follows that the excess kurtosis of \( X \) is \( \kurt(X) - 3 = \frac{6}{5} \). The central moments of \( X \) can be given in terms of the Bernoulli numbers.

Again, for \( n \in \N \), let \( \beta_n \) denote the \( n \)th Bernoulli number.

- If \( n \in \N \) is odd then \( \E\left[(X - a)^n\right] = 0 \).
- If \( n \in \N \) is even then \( \E\left[(X - a)^n\right] = (2^n - 2) \pi^n b^n \left|\beta_n\right| \)

Let \( X = a + b Z \) where \( Z \) has the standard logistic distribution. Then \( \E\left[(X - a)^n\right] = b^n \E(Z^n) \) so the results follow from the moments of \( Z \) given above.

The general logistic distribution is a location-scale family, so it is trivially closed under location-scale transformations.

Suppose that \( X \) has the logistic distribution with location parameter \( a \) and scale parameter \( b \), and that \( c \in \R \) and \( d \in (0, \infty) \). Then \( Y = c + d X \) has the logistic distribution with location parameter \( c + a d \) and scale parameter \( b d \).