Big Data Concepts

Big Data refers to datasets that are so large, complex, or fast-changing that traditional data processing methods cannot handle them effectively. The functional programming paradigm plays a key role in big data processing because its emphasis on immutability, statelessness, and parallelism makes it well-suited for distributed computing.

---

The Three Vs (and Beyond)

Big data is commonly characterised by the "Three Vs":

Volume

The amount of data being generated and stored. Modern organisations may deal with terabytes, petabytes, or even exabytes of data.

Examples:

• Social media platforms generating billions of posts per day.
• Scientific instruments (e.g. the Large Hadron Collider) producing petabytes of data.
• Retail companies storing years of transaction data for millions of customers.

Velocity

The speed at which data is generated, collected, and processed. Some applications require real-time or near-real-time processing.

Examples:

• Stock market trading systems processing millions of transactions per second.
• IoT sensors sending continuous streams of data.
• Social media feeds updating in real time.

Variety

The different types and formats of data. Big data includes structured, semi-structured, and unstructured data.