Moment Generating Functions

The moment generating function (MGF) is a powerful tool that encodes all the moments of a distribution in a single function. It can be used to find means and variances, prove distributional results, and identify distributions. This topic is part of the AQA Further Mathematics specification.

---

Definition

The moment generating function of a random variable $X$X is defined as:

$M_X(t) = E(e^{tX})$MX​(t)=E(etX)

For a discrete random variable:

$M_X(t) = \sum_x e^{tx} P(X = x)$MX​(t)=∑x​etxP(X=x)

For a continuous random variable:

$M_X(t) = \int_{-\infty}^{\infty} e^{tx} f(x) \, dx$MX​(t)=∫−∞∞​etxf(x)dx

The MGF exists (is finite) for all $t$t in some interval containing 0.

---

Why "Moment Generating"?

Expanding $e^{tX}$etX as a Taylor series:

$e^{tX} = 1 + tX + \frac{(tX)^2}{2!} + \frac{(tX)^3}{3!} + \cdots$etX=1+tX+2!(tX)2​+3!(tX)3​+⋯

Taking expectations:

$M_X(t) = 1 + tE(X) + \frac{t^2}{2!}E(X^2) + \frac{t^3}{3!}E(X^3) + \cdots$MX​(t)=1+tE(X)+2!t2​E(X2)+3!t3​E(X3)+⋯

So the $n$n-th moment is obtained by differentiating $n$n times and evaluating at $t = 0$t=0:

$E(X^n) = M_X^{(n)}(0)$E(Xn)=MX(n)​(0)

Derivative	At $t = 0$t=0	Gives
$M_X'(0)$MX′​(0)	$E(X)$E(X)	Mean
$M_X''(0)$MX′′​(0)	$E(X^2)$E(X2)	Second moment
$M_X'''(0)$MX′′′​(0)	$E(X^3)$E(X3)	Third moment

And $\text{Var}(X) = M_X''(0) - (M_X'(0))^2$Var(X)=MX′′​(0)−(MX′​(0))2.

Exam Tip: The MGF provides a systematic way to find all moments. Even if you could compute $E(X)$E(X) directly, the MGF method is valuable because it gives $E(X^2)$E(X2) (and hence variance) with minimal extra work.

---

Worked Example: Poisson MGF

Let $X \sim \text{Po}(\lambda)$X∼Po(λ).

$M_X(t) = \sum_{r=0}^{\infty} e^{tr} \frac{e^{-\lambda}\lambda^r}{r!} = e^{-\lambda} \sum_{r=0}^{\infty} \frac{(\lambda e^t)^r}{r!} = e^{-\lambda} \cdot e^{\lambda e^t} = e^{\lambda(e^t - 1)}$MX​(t)=∑r=0∞​etrr!e−λλr​=e−λ∑r=0∞​r!(λet)r​=e−λ⋅eλet=eλ(et−1)

Finding the mean:

$M_X'(t) = \lambda e^t \cdot e^{\lambda(e^t - 1)}$MX′​(t)=λet⋅eλ(et−1)

$M_X'(0) = \lambda \cdot 1 \cdot e^0 = \lambda$MX′​(0)=λ⋅1⋅e0=λ

So $E(X) = \lambda$E(X)=λ.

Finding the second moment:

$M_X''(t) = (\lambda e^t + \lambda^2 e^{2t}) e^{\lambda(e^t - 1)}$MX′′​(t)=(λet+λ2e2t)eλ(et−1)

$M_X''(0) = (\lambda + \lambda^2) \cdot 1 = \lambda + \lambda^2$MX′′​(0)=(λ+λ2)⋅1=λ+λ2

So $E(X^2) = \lambda + \lambda^2$E(X2)=λ+λ2 and $\text{Var}(X) = \lambda + \lambda^2 - \lambda^2 = \lambda$Var(X)=λ+λ2−λ2=λ.

---

Worked Example: Exponential MGF

Let $X \sim \text{Exp}(\lambda)$X∼Exp(λ), so $f(x) = \lambda e^{-\lambda x}$f(x)=λe−λx for $x \geq 0$x≥0.

$M_X(t) = \int_0^{\infty} e^{tx} \lambda e^{-\lambda x} \, dx = \lambda \int_0^{\infty} e^{-(\lambda - t)x} \, dx = \frac{\lambda}{\lambda - t} \quad \text{for } t < \lambda$MX​(t)=∫0∞​etxλe−λxdx=λ∫0∞​e−(λ−t)xdx=λ−tλ​for t<λ

Mean:

$M_X'(t) = \frac{\lambda}{(\lambda - t)^2} \implies M_X'(0) = \frac{\lambda}{\lambda^2} = \frac{1}{\lambda}$MX′​(t)=(λ−t)2λ​⟹MX′​(0)=λ2λ​=λ1​

Second moment: