Combinations of Random Variables

This lesson consolidates results about linear combinations of random variables, covering both the general case and the special case of normal distributions. It also addresses the distribution of sample statistics and their role in inference.

---

General Results (Any Distribution)

For any random variables $X$X and $Y$Y (not necessarily independent):

Result	Formula
$E(aX + bY)$E(aX+bY)	$aE(X) + bE(Y)$aE(X)+bE(Y)
$\text{Var}(aX + bY)$Var(aX+bY)	$a^2\text{Var}(X) + b^2\text{Var}(Y) + 2ab\text{Cov}(X,Y)$a2Var(X)+b2Var(Y)+2abCov(X,Y)

If $X$X and $Y$Y are independent, then $\text{Cov}(X,Y) = 0$Cov(X,Y)=0:

$\text{Var}(aX + bY) = a^2\text{Var}(X) + b^2\text{Var}(Y)$Var(aX+bY)=a2Var(X)+b2Var(Y)

Special Cases

Combination	$E$E	$\text{Var}$Var (independent)
$X + Y$X+Y	$E(X) + E(Y)$E(X)+E(Y)	$\text{Var}(X) + \text{Var}(Y)$Var(X)+Var(Y)
$X - Y$X−Y	$E(X) - E(Y)$E(X)−E(Y)	$\text{Var}(X) + \text{Var}(Y)$Var(X)+Var(Y)
$3X$3X	$3E(X)$3E(X)	$9\text{Var}(X)$9Var(X)
$X + 5$X+5	$E(X) + 5$E(X)+5	$\text{Var}(X)$Var(X)

Exam Tip: The variance of $X - Y$X−Y is $\text{Var}(X) + \text{Var}(Y)$Var(X)+Var(Y) (plus, not minus) when $X$X and $Y$Y are independent. This is tested frequently and is a common source of errors.

---

Sum of Independent Identically Distributed (i.i.d.) Variables

If $X_1, X_2, \ldots, X_n$X1​,X2​,…,Xn​ are i.i.d. with mean $\mu$μ and variance $\sigma^2$σ2:

$E\left(\sum_{i=1}^n X_i\right) = n\mu$E(∑i=1n​Xi​)=nμ

$\text{Var}\left(\sum_{i=1}^n X_i\right) = n\sigma^2$Var(∑i=1n​Xi​)=nσ2

---

The Sample Mean

$\bar{X} = \frac{1}{n}\sum_{i=1}^n X_i$Xˉ=n1​∑i=1n​Xi​

Property	Value
$E(\bar{X})$E(Xˉ)	$\mu$μ
$\text{Var}(\bar{X})$Var(Xˉ)	$\sigma^2/n$σ2/n
$\text{SD}(\bar{X})$SD(Xˉ)	$\sigma/\sqrt{n}$σ/n​ (standard error)

The sample mean is an unbiased estimator of the population mean: $E(\bar{X}) = \mu$E(Xˉ)=μ.

As $n$n increases, $\text{Var}(\bar{X})$Var(Xˉ) decreases, meaning $\bar{X}$Xˉ becomes a more precise estimator of $\mu$μ.

---

Worked Example: Two Stages

A toy is assembled in two stages. Stage 1 takes time $X \sim N(10, 4)$X∼N(10,4) minutes and Stage 2 takes time $Y \sim N(15, 9)$Y∼N(15,9) minutes, independently.

Total time: $T = X + Y \sim N(10 + 15, 4 + 9) = N(25, 13)$T=X+Y∼N(10+15,4+9)=N(25,13).

$P(T > 30) = P\left(Z > \frac{30 - 25}{\sqrt{13}}\right) = P(Z > 1.387) = 1 - 0.9173 = 0.0827$P(T>30)=P(Z>13​30−25​)=P(Z>1.387)=1−0.9173=0.0827

Difference: $D = Y - X \sim N(15 - 10, 9 + 4) = N(5, 13)$D=Y−X∼N(15−10,9+4)=N(5,13).

$P(D < 0) = P\left(Z < \frac{0 - 5}{\sqrt{13}}\right) = P(Z < -1.387) = 0.0827$P(D<0)=P(Z<13​0−5​)=P(Z<−1.387)=0.0827

---

Sums and Scalar Multiples Revisited

Let $X \sim N(10, 4)$X∼N(10,4).