An independent study reference written by Dr Phuc V. Nguyen. It is not official subject material — for assessment requirements always follow your subject outline and vUWS.
What big data means
Big data is not a size threshold. Data becomes big when its volume, velocity or variety breaks the way an organisation currently stores, moves or analyses it. Laney named those three properties in 2001, and later writers added veracity (how far it can be trusted) and value (whether it repays the cost of keeping it). The practical test is relative. A dataset a laptop handles comfortably is not big, and the same dataset arriving every second may be. Calling data big describes a constraint on your tooling and your process, not a badge of importance.
Why it matters
Think about the loading dock behind a supermarket. Volume is how many pallets arrive. Velocity is how fast the trucks turn up and whether the dock clears before the next one pulls in. Variety is whether everything comes on a standard pallet or whether some of it is loose, frozen, or an odd shape nobody has a place for. A dock that copes with one truck an hour fails at one a minute, even though each truck is the same size.
A retailer stores 900 million loyalty transactions and queries them comfortably. It also receives 40 in-store sensor readings a second as nested documents nobody has landed anywhere. Which is the better description of its big data problem?
Formulas
Worked examples
A logistics firm says it has a big data problem because it holds 12 years of delivery records, roughly 400 million rows. The analytics team can query the whole table in under a minute.
Volume alone does not settle it. Four hundred million rows sit comfortably inside an ordinary columnar warehouse, so nothing in the current process is breaking. The firm's real constraint turns out to be variety and velocity. Telematics from 3,000 vehicles arrives every ten seconds as nested documents whose fields change without notice, and nobody has a place to land it. That stream, not the historical table, is what forces new tooling. Naming the specific property that binds keeps the conversation on a fixable constraint instead of a slogan.
A marketing manager asks whether the firm should describe its customer database as big data in a board paper.
The honest answer depends on what follows the label. If it introduces a request for a distributed platform, the paper needs to show which property is breaking and what it costs today. If it is decoration, it invites a decision nobody can evaluate. A better formulation states the constraint directly: the customer table is queried comfortably, while the clickstream arrives faster than it can be processed and is the reason for the investment.
Common mistakes
- ✗Big data just means a lot of rows. Volume is only one property; a modest table arriving as a continuous stream of nested documents can break a pipeline that handles billions of tidy rows without complaint.
- ✗There is a fixed threshold, such as a terabyte, above which data counts as big. The threshold moves with hardware and with the tools a team already runs, so the same dataset can be big for one organisation and routine for another.
- ✗Bigger datasets automatically give better answers. Size reduces random sampling error and does nothing about systematic bias, so a large biased sample can be more confidently wrong than a small representative one.
- ✗Any big data problem requires a specialised distributed platform. Most business questions are answered from aggregates that fit in ordinary tools, and reaching for a cluster before the constraint is identified adds cost without adding insight.
Revision bullets
- •Volume, velocity and variety were the original three properties (Laney, 2001)
- •Veracity and value came later and concern trust and payoff, not size
- •Big is relative to current tooling, not an absolute byte count
- •Velocity is a race between arrival rate and processing rate
- •More rows shrink random error and never shrink systematic bias
Quick check
A retailer stores 900 million loyalty transactions and queries them comfortably. It also receives 40 in-store sensor readings a second as nested documents nobody has landed anywhere. Which is the better description of its big data problem?
A team argues that moving from a 5,000-person survey to 5 million scraped social posts will make its estimate of national brand sentiment far more accurate. What is the strongest objection?
Connected topics
More in Data Foundations
Sources
- Laney (2001)Laney, D. "3D Data Management: Controlling Data Volume, Velocity, and Variety." META Group Research Note 949, 6 February 2001.The research note that introduced the three-property framing later shortened to the three Vs.
- NIST Big Data Public Working Group. NIST Big Data Interoperability Framework: Volume 1, Definitions. NIST Special Publication 1500-1r2, National Institute of Standards and Technology, 2019.A standards-body definition that treats big data as data whose scale or rate exceeds what conventional approaches handle, rather than as a fixed size.