Recurrent Neural Networks (RNNs, LSTMs, GRUs)

Recurrent Neural Networks are designed to process sequential data — data where the order matters. While CNNs are ideal for spatial data like images, RNNs excel at temporal and sequential data such as text, time series, audio, and video frames.

---

Why Recurrent Networks?

Standard feedforward networks treat each input independently. They have no memory of previous inputs. But many real-world problems involve sequences where context from earlier elements influences later ones:

Domain	Sequential Data
NLP	Words in a sentence, characters in a word
Time Series	Stock prices, sensor readings, weather data
Speech	Audio waveforms, phoneme sequences
Music	Note sequences, chord progressions
Video	Frames in a video sequence

---

The Vanilla RNN

A Recurrent Neural Network maintains a hidden state that acts as memory. At each time step, the network takes the current input and the previous hidden state, producing a new hidden state and (optionally) an output.

Mathematical Formulation

h_t = tanh(W_hh * h_{t-1} + W_xh * x_t + b_h)
y_t = W_hy * h_t + b_y