Correlation and Regression (Further)

This lesson extends correlation and regression to include Spearman's rank correlation coefficient, hypothesis testing for correlation, and deeper analysis of the product-moment correlation coefficient (PMCC).

---

Recap: Pearson's Product-Moment Correlation Coefficient

The PMCC $r$r measures the strength and direction of a linear relationship between two variables:

$r = \frac{S_{xy}}{\sqrt{S_{xx} S_{yy}}}$r=Sxx​Syy​​Sxy​​

where $S_{xy} = \sum(x_i - \bar{x})(y_i - \bar{y})$Sxy​=∑(xi​−xˉ)(yi​−yˉ​), $S_{xx} = \sum(x_i - \bar{x})^2$Sxx​=∑(xi​−xˉ)2, $S_{yy} = \sum(y_i - \bar{y})^2$Syy​=∑(yi​−yˉ​)2.

Value of $r$r	Interpretation
$r = 1$r=1	Perfect positive linear correlation
$r = -1$r=−1	Perfect negative linear correlation
$r = 0$r=0	No linear correlation

---

Hypothesis Test for the PMCC

To test whether there is a significant linear correlation in the population:

$H_0: \rho = 0$H0​:ρ=0 (no linear correlation in the population) $H_1: \rho \neq 0$H1​:ρ=0 (two-tailed) or $\rho > 0$ρ>0 / $\rho < 0$ρ<0 (one-tailed)

Compare the sample $r$r with the critical value from the PMCC table for $n$n data points at the chosen significance level.

If $|r| > \text{critical value}$∣r∣>critical value, reject $H_0$H0​.

Worked Example

For $n = 10$n=10 pairs, $r = 0.65$r=0.65. Test at 5% (two-tailed).

Critical value ($n = 10$n=10, two-tailed 5%): 0.6319.

Since $0.65 > 0.6319$0.65>0.6319, reject $H_0$H0​. There is significant evidence of linear correlation.

---

Spearman's Rank Correlation Coefficient

Spearman's rank correlation coefficient $r_s$rs​ measures the strength and direction of a monotonic relationship (not necessarily linear). It is calculated by ranking the data and applying the PMCC formula to the ranks.

Formula

If there are no tied ranks:

$r_s = 1 - \frac{6\sum d_i^2}{n(n^2 - 1)}$rs​=1−n(n2−1)6∑di2​​

where $d_i = \text{rank}(x_i) - \text{rank}(y_i)$di​=rank(xi​)−rank(yi​) is the difference between the ranks.

Procedure

1. Rank each variable separately (smallest = rank 1).
2. For tied values, assign the average of the ranks they would have occupied.
3. Calculate $d_i$di​ for each pair.
4. Apply the formula.

Worked Example

Student	Maths score	Science score	Rank (Maths)	Rank (Science)	$d$d	$d^2$d2
A	85	78	1	2	-1	1
B	72	65	3	4	-1	1
C	90	85	0	0	—	—

Let me redo with a proper dataset: