Probability Fundamentals

Probability quantifies uncertainty. It is the mathematical language that underlies all of inferential statistics, from hypothesis testing to Bayesian reasoning. This lesson covers the core rules and concepts.

---

What is Probability?

Probability is a number between 0 and 1 (or equivalently 0% and 100%) that describes the likelihood of an event occurring:

Probability	Interpretation
0	Impossible — the event cannot occur
0.5	Equally likely — a coin toss
1	Certain — the event must occur

---

Key Terminology

Term	Definition
Experiment	A process that produces well-defined outcomes (e.g., rolling a die)
Sample space (S)	The set of all possible outcomes
Event (A)	A subset of the sample space — a specific outcome or set of outcomes
Complement (A')	All outcomes in S that are not in A

Example: Rolling a standard six-sided die

Sample space  S = {1, 2, 3, 4, 5, 6}
Event A       = "rolling an even number" = {2, 4, 6}
Complement A' = {1, 3, 5}

---

Approaches to Probability

1. Classical (Theoretical)

When all outcomes are equally likely:

P(A) = Number of favourable outcomes / Total number of outcomes

Example: P(even on a fair die) = 3/6 = 0.5

2. Frequentist (Empirical)

Based on the relative frequency of an event in a large number of trials:

P(A) ≈ Number of times A occurred / Total number of trials

3. Subjective

A personal degree of belief. Used in Bayesian statistics and decision-making under uncertainty.

---

Fundamental Rules

Rule 1: Complement Rule

P(A') = 1 − P(A)

If the probability of rain is 0.3, the probability of no rain is 0.7.

Rule 2: Addition Rule

For any two events A and B:

P(A ∪ B) = P(A) + P(B) − P(A ∩ B)

If A and B are mutually exclusive (cannot both happen), then P(A ∩ B) = 0:

P(A ∪ B) = P(A) + P(B)

Rule 3: Multiplication Rule

For two events A and B:

P(A ∩ B) = P(A) × P(B | A)

If A and B are independent (one does not affect the other):