Language Models

A language model is a probabilistic model that assigns a probability to a sequence of words. Language models answer the question: "How likely is this sequence of words?" They are the foundation of many NLP applications — from autocomplete and spell checking to machine translation and text generation.

---

What is a Language Model?

Given a sequence of words w1, w2, ..., wn, a language model estimates:

P(w1, w2, ..., wn) — the probability of the entire sequence

Or equivalently, the probability of the next word given all previous words:

P(wn | w1, w2, ..., wn-1) — the next-word prediction

Why Are Language Models Useful?

Application	How Language Models Help
Autocomplete	Predicts the most likely next word
Spell checking	"teh cat" is less probable than "the cat"
Speech recognition	Disambiguates similar-sounding words using context
Machine translation	Scores fluency of candidate translations
Text generation	Generates coherent text word by word

---

N-gram Language Models

The simplest language models. An n-gram model approximates the probability of a word using only the previous n-1 words (the Markov assumption).