Data Cleaning & Preparation

This lesson covers the essential processes of data cleaning and preparation — the steps required to transform raw data into a form suitable for statistical analysis. In real-world statistics (and in A-Level exam questions), data is rarely perfect. Learning to identify and handle problems systematically is a key skill.

---

Why Data Cleaning Matters

Raw data collected from real-world sources almost always contains imperfections. If these are not addressed before analysis, the results may be misleading, inaccurate, or invalid. Data cleaning is the process of detecting and correcting (or removing) errors, inconsistencies, and gaps in a data set.

In the context of the AQA large data set:

• Weather stations may have missing readings due to equipment failure.
• Values may be recorded inconsistently (e.g., different date formats).
• Outliers may represent genuine extreme weather or errors.
• Special codes (such as $\text{tr}$tr for trace rainfall) need careful handling.

---

Handling Missing Values

Missing values are one of the most common issues in real data. There are several strategies for dealing with them, each with advantages and disadvantages.

Strategy 1: Omission (Listwise Deletion)