Neural Network Fundamentals

This lesson covers the foundational building blocks of neural networks — from the single neuron (perceptron) through multi-layer architectures, activation functions, and the backpropagation algorithm that makes learning possible.

---

The Perceptron

The perceptron is the simplest form of a neural network — a single artificial neuron that takes multiple inputs, multiplies each by a weight, sums them, adds a bias, and passes the result through an activation function.

Mathematical Formulation

output = activation(w1*x1 + w2*x2 + ... + wn*xn + b)

Where:

• x1, x2, ..., xn are the input features
• w1, w2, ..., wn are the learnable weights
• b is the bias term
• activation is the activation function

Perceptron in Python

import numpy as np

class Perceptron:
    def __init__(self, n_features, lr=0.01):
        self.weights = np.zeros(n_features)
        self.bias = 0.0
        self.lr = lr

    def predict(self, x):
        z = np.dot(x, self.weights) + self.bias
        return 1 if z >= 0 else 0

    def train(self, X, y, epochs=100):
        for _ in range(epochs):
            for xi, yi in zip(X, y):
                pred = self.predict(xi)
                error = yi - pred
                self.weights += self.lr * error * xi
                self.bias += self.lr * error