Decision Trees and Random Forests

Decision trees are one of the most intuitive and widely used machine learning algorithms. They model decisions as a tree-like structure of rules, making them easy to interpret and visualise. Random forests extend decision trees into a powerful ensemble method that reduces overfitting and improves accuracy.

---

Decision Trees

A decision tree is a flowchart-like structure where:

• Each internal node represents a test on a feature (e.g., "Is age > 30?")
• Each branch represents the outcome of that test
• Each leaf node represents a prediction (class label or value)

How a Decision Tree Learns

The algorithm recursively splits the data by choosing the feature and threshold that best separates the classes (or reduces prediction error). At each node, it asks: "Which feature split produces the purest subgroups?"

Splitting Criteria