Model Evaluation and Validation

Building a machine learning model is only half the job — you also need to evaluate how well it performs and validate that it will generalise to new, unseen data. Choosing the right evaluation metrics and validation strategies is critical to building reliable ML systems.

---

Why Evaluation Matters

A model that appears to perform well on training data may fail completely on new data. Proper evaluation answers these questions:

• How accurate are the model's predictions?
• Will the model generalise to unseen data?
• Is the model biased towards certain classes?
• Which model or hyperparameter setting is the best?

---

Classification Metrics

Confusion Matrix

The confusion matrix is the foundation of classification evaluation. For a binary classifier:

	Predicted Positive	Predicted Negative
Actual Positive	True Positive (TP)	False Negative (FN)
Actual Negative	False Positive (FP)	True Negative (TN)

from sklearn.metrics import confusion_matrix, ConfusionMatrixDisplay
import matplotlib.pyplot as plt

cm = confusion_matrix(y_test, y_pred)
disp = ConfusionMatrixDisplay(cm, display_labels=['Negative', 'Positive'])
disp.plot(cmap='Blues')
plt.title('Confusion Matrix')
plt.show()

Key Classification Metrics