Inferential Statistics and Hypothesis Testing

Descriptive statistics (means, standard deviations, graphs) summarise data but do not tell us whether differences or relationships in our data are statistically significant — that is, whether they are unlikely to have occurred by chance alone. Inferential statistics allow psychologists to make inferences about the population from sample data by testing hypotheses and calculating the probability that results are due to chance.

Key Definition: Inferential statistics are statistical tests used to determine whether the results of a study are statistically significant — that is, unlikely to have occurred by chance — and can therefore be generalised from the sample to the wider population.

---

Probability and Significance

The Significance Level

In psychology, the conventional significance level is p ≤ 0.05 (5%). This means that the probability of the observed results being due to chance is less than or equal to 5% — or, equivalently, we can be at least 95% confident that the results reflect a real effect.

Significance Level	Interpretation
p ≤ 0.05	Standard level — results are considered statistically significant
p ≤ 0.01	More stringent — only a 1% probability of the results being due to chance; used when the consequences of a Type I error are serious
p ≤ 0.10	More lenient — sometimes used in pilot studies or exploratory research

Key Definition: The significance level is the probability threshold below which the null hypothesis is rejected. In psychology, p ≤ 0.05 is the standard — meaning there is a 5% or less probability that the results occurred by chance.

---

Type I and Type II Errors

Statistical testing involves the risk of making errors:

Error Type	Description	When It Occurs	Risk Factor
Type I error (false positive)	Rejecting the null hypothesis when it is actually true — concluding there is a significant effect when there is not	More likely when the significance level is too lenient (e.g., p ≤ 0.10)	Significance level too high
Type II error (false negative)	Accepting (failing to reject) the null hypothesis when it is actually false — concluding there is no significant effect when there really is one	More likely when the significance level is too stringent (e.g., p ≤ 0.01) or the sample size is too small	Significance level too low or low statistical power

Exam Tip: Think of Type I errors as "seeing something that isn't there" (a false alarm) and Type II errors as "missing something that is there" (a miss). The p ≤ 0.05 level is a compromise — strict enough to minimise false positives but lenient enough to detect real effects.

Example:

A researcher tests whether a new therapy reduces anxiety. They set p ≤ 0.05.

• Type I error — the therapy does not actually work, but the test produces a significant result by chance. The researcher wrongly concludes the therapy is effective.
• Type II error — the therapy does work, but the test fails to detect the effect (perhaps due to a small sample). The researcher wrongly concludes the therapy is ineffective.

---

The Sign Test

The sign test is the simplest inferential statistical test at A-Level. It is used when:

• The hypothesis predicts a difference between two conditions (not a correlation).
• The design is repeated measures or matched pairs.
• The data are at least nominal level.

How to Calculate the Sign Test

1. For each participant, record the score in Condition A and Condition B.
2. Calculate the difference (A − B) for each participant.
3. Record the sign of the difference: + (positive), − (negative), or 0 (no difference).
4. Exclude any participants with a score of 0 (no difference).
5. Count the number of remaining participants — this is N.
6. Count the number of times the less frequent sign appears — this is S (the calculated/observed value).
7. Compare S to the critical value of S for the relevant N and significance level (p ≤ 0.05, one-tailed or two-tailed).
8. If S is less than or equal to the critical value, the result is significant — reject the null hypothesis.

Worked Example:

A researcher tests whether a relaxation technique reduces stress ratings. 10 participants rate their stress before and after the technique.

Participant	Before	After	Difference	Sign
1	8	5	−3	−
2	7	6	−1	−
3	6	6	0	(excluded)
4	9	4	−5	−
5	5	3	−2	−
6	8	7	−1	−
7	6	5	−1	−
8	7	8	+1	+
9	9	6	−3	−
10	8	5	−3	−

• Participant 3 excluded (difference = 0). N = 9.
• Signs: 8 minus signs, 1 plus sign.
• S = 1 (the less frequent sign).
• From the critical values table for a one-tailed test at p ≤ 0.05 with N = 9, the critical value is 1.
• S (1) ≤ critical value (1), so the result is significant.
• Conclusion: The relaxation technique significantly reduced stress ratings (p ≤ 0.05, one-tailed).

Exam Tip: In the sign test, the calculated value (S) must be equal to or less than the critical value for significance. This is the opposite of many other tests, where the calculated value must be equal to or greater than the critical value. Make sure you state this clearly.