An independent study reference written by Dr Phuc V. Nguyen. It is not official subject material — for assessment requirements always follow your subject outline and vUWS.
Metadata
Metadata is the recorded description of data: what a field means, where it came from, how it is structured, when it was last refreshed and who may see it. It usually splits three ways. Descriptive metadata names and defines things, including the business definition of a measure. Structural metadata records types, keys and how tables join. Administrative metadata covers source, owner, refresh time, access rights and retention. Lineage, the traced path from a source field to a reported number, runs across all three. Without it a dashboard is a figure with no provenance, and disagreements about it cannot be settled.
Why it matters
A jar in a pantry with no label is not useless, but you have to open it, taste it, and guess when it was made. Multiply that by four thousand jars and nobody cooks anything. Metadata is the label: what this is, who made it, when, and what it is safe to use it for. The most valuable label is not the technical type of the field. It is the sentence saying what the number actually counts.
A catalogue lists every column, its data type, and its primary and foreign keys, and is regenerated nightly from the database. With the affected query or report already in front of you, which recurring problem still cannot be diagnosed from the catalogue?
Worked examples
Two dashboards, both labelled monthly revenue, differ by about 12 per cent every month. Each team insists its figure is the right one.
Neither figure is wrong. One measure is gross of tax and counts an order at the point of sale, the other is net of tax and returns, and counts at dispatch. Nothing on either dashboard says so. The fix is a written definition attached to the measure, a named owner accountable for it, and lineage showing which source fields feed it. Once both definitions are visible the argument becomes a five-minute decision about which one belongs on the executive report, instead of a dispute that recurs every month.
A source system is about to rename a field and change its unit from cents to dollars. The engineering team asks what will break.
With lineage recorded, the answer is a query. Every table, measure and report downstream of that field is listed, so the change becomes a scoped piece of work with a testable list. Without lineage the team either guesses or ships the change and waits for someone to notice a number a hundred times too small. Lineage earns its cost mainly at moments like this, which is also why it is hard to fund before the first incident.
Common mistakes
- ✗Metadata is documentation, so it is optional. It is the only thing that lets two people agree on what a number counts, which makes it part of the number rather than an accessory to it.
- ✗A data dictionary generated from the database is enough. Generated dictionaries capture types and keys and not meaning, and meaning is where the disagreements live.
- ✗Metadata is only useful to engineers. Owners, definitions, refresh times and retention rules are business and governance facts, and they are exactly what a decision maker needs before trusting a figure.
- ✗Metadata is written once and stays true. Definitions drift when processes change, so an unmaintained catalogue eventually misleads more confidently than no catalogue at all.
Revision bullets
- •Descriptive: names, definitions, the business meaning of a measure
- •Structural: types, keys, joins, how tables fit together
- •Administrative: source, owner, refresh time, access rights, retention
- •Lineage traces a reported number back to its source fields
- •Two identically named measures with different definitions is the failure metadata prevents
Quick check
A catalogue lists every column, its data type, and its primary and foreign keys, and is regenerated nightly from the database. With the affected query or report already in front of you, which recurring problem still cannot be diagnosed from the catalogue?
Which piece of metadata most directly determines whether an executive should act on a figure this morning?
Connected topics
More in Data Foundations
Sources
- Riley, J. Understanding Metadata: What is Metadata, and What is it For? National Information Standards Organization, 2017.A standards-body primer distinguishing descriptive, structural and administrative metadata.
- Dublin Core Metadata Initiative. DCMI Metadata Terms.A long-standing worked example of an agreed descriptive metadata vocabulary.
- DAMA-DMBOK (2017)DAMA International. DAMA-DMBOK: Data Management Body of Knowledge, 2nd ed. Technics Publications, 2017.Places metadata management, lineage and business glossaries inside the wider governance function.