Type I and Type II Errors, Power of a Test

Hypothesis tests can make two kinds of errors. Understanding these errors and the power of a test is essential for designing studies and interpreting results.

---

Types of Errors

	$H_0$H0​ is true	$H_0$H0​ is false
Do not reject $H_0$H0​	Correct decision	Type II error ($\beta$β)
Reject $H_0$H0​	Type I error ($\alpha$α)	Correct decision (power)

Type I Error

A Type I error occurs when $H_0$H0​ is rejected even though it is true (a "false positive").

$P(\text{Type I error}) = \alpha = \text{significance level}$P(Type I error)=α=significance level

Choosing $\alpha = 0.05$α=0.05 means we accept a 5% chance of rejecting a true $H_0$H0​.

Type II Error

A Type II error occurs when $H_0$H0​ is not rejected even though it is false (a "false negative").

$P(\text{Type II error}) = \beta$P(Type II error)=β

$\beta$β depends on the true value of the parameter, the sample size, and the significance level.

Exam Tip: Type I error = rejecting a true $H_0$H0​. Type II error = not rejecting a false $H_0$H0​. A useful mnemonic: Type I = "crying wolf" (false alarm); Type II = "missing the wolf" (failure to detect).

---

The Trade-Off

Reducing $\alpha$α (making the test more stringent) increases $\beta$β (making it harder to detect a real effect). The only way to reduce both $\alpha$α and $\beta$β simultaneously is to increase the sample size $n$n.

---

Power of a Test

The power of a test is the probability of correctly rejecting $H_0$H0​ when it is false:

$\text{Power} = 1 - \beta = P(\text{reject } H_0 \mid H_0 \text{ is false})$Power=1−β=P(reject H0​∣H0​ is false)

Higher power means the test is better at detecting a real effect.

Factors Affecting Power

Factor	Effect on power
Larger sample size $n$n	Increases power
Larger significance level $\alpha$α	Increases power (but increases Type I error)
Larger true effect size	Increases power
Smaller population variance $\sigma^2$σ2	Increases power

---

Calculating Type II Error and Power

Worked Example (z-test)

$H_0: \mu = 50$H0​:μ=50, $H_1: \mu > 50$H1​:μ>50, $\sigma = 4$σ=4, $n = 16$n=16, $\alpha = 0.05$α=0.05.

Step 1: Find the critical value.

Under $H_0$H0​: $\bar{X} \sim N(50, 1)$Xˉ∼N(50,1). Reject if $\bar{X} > 50 + 1.645 \times 1 = 51.645$Xˉ>50+1.645×1=51.645.