An independent study reference written by Dr Phuc V. Nguyen. It is not official subject material — for assessment requirements always follow your subject outline and vUWS.
Structured and unstructured data
Structured data arrives with its shape declared in advance: named columns, fixed types, one value per cell, so a query engine knows where everything is before it reads a row. Unstructured data carries no such declaration. Free text, images, audio and video have plenty of internal organisation, but nothing a database can filter on without further work. Semi-structured data sits between them, describing itself through tags or keys as in documents and log files, so fields can vary between records. The distinction matters because conventional tabular analysis needs columns, so unstructured sources are usually turned into features first, or handed to models that learn their own numeric representation of the raw text or image. Either way that conversion is a modelling decision with its own errors.
Why it matters
A spreadsheet tells you what every column means before you look at a single value. A pile of customer emails does not. The emails are full of meaning, and a person reading them understands it instantly, but a computer cannot filter on a feeling. Someone has to decide that "took nine days" becomes a delivery-time column and "box was crushed" becomes a damage flag. Unstructured does not mean messy. It means the structure has not been declared yet.
A team stores product reviews in a database column called review_text. Does putting the text in a relational table make it structured data?
Worked examples
An insurer wants to know which product lines generate the most complaint effort. Its claims system holds 2.1 million structured rows, and its contact centre holds 480,000 free-text case notes.
The structured table answers who, when and how much, so counts by product line are immediate. The case notes hold the reason, which is the part management actually wants. Reading them all is not feasible, so the team derives columns from the text: a complaint category, a flag for a named process step, a word count as a rough proxy for effort. Every one of those columns is an inference. The team keeps the original note beside each derived field and hand-checks a few hundred to confirm the conversion holds.
A team stores scanned delivery dockets as image files and reports that it now has the delivery data in the warehouse.
The files are in the warehouse and the data is not. Nothing in a scanned image can be filtered, joined or aggregated until the docket number, date and quantity are extracted into fields. Recognition on handwritten dockets is imperfect, so the extraction step needs an accuracy estimate from a hand-checked sample and a rule for what happens to low-confidence reads. Storing a source and having usable data are different milestones, and confusing them is a common way for a project to look further along than it is.
Common mistakes
- ✗Unstructured data has no structure. Sentences have grammar and images have geometry; what is missing is a declared schema the query engine can rely on, not organisation itself.
- ✗Semi-structured data is just untidy structured data. Self-describing records carry their own field names, so fields can differ from record to record without breaking the file, which is exactly what a fixed table cannot tolerate.
- ✗Converting text to columns is a mechanical step with no consequences. Each derived field encodes a judgement about what counts as a complaint or a mention, and errors in that judgement propagate into every later analysis.
- ✗Structured data is automatically more reliable. A tidy column full of default values keyed to get past a mandatory field is weaker evidence than a note written by someone who cared.
Revision bullets
- •Structured: schema declared before the data arrives, fixed columns and types
- •Semi-structured: self-describing keys or tags, fields may vary per record
- •Unstructured: no declared schema a query engine can use (text, image, audio, video)
- •Conventional tabular analysis needs columns, so unstructured sources are converted into features first
- •The conversion is a modelling decision and carries its own error rate
Quick check
A team stores product reviews in a database column called review_text. Does putting the text in a relational table make it structured data?
A logistics firm receives partner shipment files where some records carry a customs field and others do not, and new fields appear without notice. Which storage choice fits best?
Connected topics
More in Data Foundations
Sources
- DAMA-DMBOK (2017)DAMA International. DAMA-DMBOK: Data Management Body of Knowledge, 2nd ed. Technics Publications, 2017.Sets out the structured, semi-structured and unstructured distinction alongside the management practices each form requires.