Probability Generating Functions

The probability generating function (PGF) is a tool for working with discrete random variables that take non-negative integer values. It encodes the entire probability distribution and provides elegant methods for finding means, variances, and distributions of sums.

---

Definition

For a discrete random variable $X$X taking values $0, 1, 2, \ldots$0,1,2,…, the PGF is:

$G_X(t) = E(t^X) = \sum_{r=0}^{\infty} P(X = r) \cdot t^r$GX​(t)=E(tX)=∑r=0∞​P(X=r)⋅tr

This is a power series in $t$t, where the coefficient of $t^r$tr is $P(X = r)$P(X=r).

---

Recovering Probabilities

The probabilities can be recovered from the PGF:

$P(X = r) = \frac{G_X^{(r)}(0)}{r!}$P(X=r)=r!GX(r)​(0)​

In particular:

• $P(X = 0) = G_X(0)$P(X=0)=GX​(0)
• $P(X = 1) = G_X'(0)$P(X=1)=GX′​(0)
• $P(X = 2) = G_X''(0)/2$P(X=2)=GX′′​(0)/2

Also, $G_X(1) = \sum P(X = r) = 1$GX​(1)=∑P(X=r)=1 (always).

---

Standard PGFs

Distribution	PGF $G_X(t)$GX​(t)
$\text{Bernoulli}(p)$Bernoulli(p)	$1 - p + pt = q + pt$1−p+pt=q+pt
$B(n, p)$B(n,p)	$(q + pt)^n$(q+pt)n
$\text{Geometric}(p)$Geometric(p) (starting from 1)	$\frac{pt}{1 - qt}$1−qtpt​, $
$\text{Po}(\lambda)$Po(λ)	$e^{\lambda(t - 1)}$eλ(t−1)

Derivation for Poisson:

$G_X(t) = \sum_{r=0}^{\infty} \frac{e^{-\lambda}\lambda^r}{r!} t^r = e^{-\lambda} \sum_{r=0}^{\infty} \frac{(\lambda t)^r}{r!} = e^{-\lambda} \cdot e^{\lambda t} = e^{\lambda(t-1)}$GX​(t)=∑r=0∞​r!e−λλr​tr=e−λ∑r=0∞​r!(λt)r​=e−λ⋅eλt=eλ(t−1)

---

Finding the Mean Using the PGF

$E(X) = G_X'(1)$E(X)=GX′​(1)

Proof:

$G_X'(t) = \sum_{r=1}^{\infty} r \cdot P(X = r) \cdot t^{r-1}$GX′​(t)=∑r=1∞​r⋅P(X=r)⋅tr−1

$G_X'(1) = \sum_{r=1}^{\infty} r \cdot P(X = r) = E(X)$GX′​(1)=∑r=1∞​r⋅P(X=r)=E(X)

---

Finding the Variance Using the PGF

$E(X(X-1)) = G_X''(1)$E(X(X−1))=GX′′​(1)

Since $E(X(X-1)) = E(X^2) - E(X)$E(X(X−1))=E(X2)−E(X):

$E(X^2) = G_X''(1) + G_X'(1)$E(X2)=GX′′​(1)+GX′​(1)

$\text{Var}(X) = G_X''(1) + G_X'(1) - (G_X'(1))^2$Var(X)=GX′′​(1)+GX′​(1)−(GX′​(1))2

Worked Example: Poisson PGF

$G_X(t) = e^{\lambda(t-1)}$GX​(t)=eλ(t−1)

$G_X'(t) = \lambda e^{\lambda(t-1)} \implies G_X'(1) = \lambda \implies E(X) = \lambda$GX′​(t)=λeλ(t−1)⟹GX′​(1)=λ⟹E(X)=λ

$G_X''(t) = \lambda^2 e^{\lambda(t-1)} \implies G_X''(1) = \lambda^2$GX′′​(t)=λ2eλ(t−1)⟹GX′′​(1)=λ2

$\text{Var}(X) = \lambda^2 + \lambda - \lambda^2 = \lambda$Var(X)=λ2+λ−λ2=λ

Confirming the Poisson property: mean = variance = $\lambda$λ.

---

PGF of a Sum of Independent Variables

If $X$X and $Y$Y are independent, then:

$G_{X+Y}(t) = G_X(t) \cdot G_Y(t)$GX+Y​(t)=GX​(t)⋅GY​(t)

Application: Prove that $B(n_1, p) + B(n_2, p) = B(n_1 + n_2, p)$B(n1​,p)+B(n2​,p)=B(n1​+n2​,p) for independent binomials with the same $p$p.

$G_{X+Y}(t) = (q + pt)^{n_1} \cdot (q + pt)^{n_2} = (q + pt)^{n_1 + n_2}$GX+Y​(t)=(q+pt)n1​⋅(q+pt)n2​=(q+pt)n1​+n2​