An independent study reference written by Dr Phuc V. Nguyen. It is not official subject material — for assessment requirements always follow your subject outline and vUWS.

Text and Social Dataintermediate

Where social data comes from

Q: A team increases its collection from 50,000 to 5 million posts and reports that the margin of error is now negligible, so its sentiment estimate is reliable. What is the strongest objection?

Random error shrank, but any difference between posters and customers is untouched. Sample size controls only the random part of the gap between an estimate and the truth. If people who post in public differ from the customer base, that coverage bias is a fixed offset which more data cannot reduce. The interval gets tighter around a quantity that was never the one being asked about.

Q: Which source is most likely to let you connect what someone said to what they subsequently bought?

The organisation's own review and support records, joined to its transaction history. Public social data rarely carries a reliable identity link to a customer record, so it cannot be joined to outcomes. First-party channels are already tied to an account or an order, which is what makes them the workhorse for questions that link what someone said to what they later did. The join establishes an association, not a cause. Showing that what someone said changed what they bought still needs a randomised or quasi-experimental design.

Social data reaches an analyst by a small number of routes, and each route decides what you can conclude. Platform interfaces return a filtered, rate-limited slice under terms that change without notice. Licensed data providers resell fuller archives at a price. Public web pages can be collected directly, subject to site terms and copyright. First-party channels such as your own pages, reviews, support inbox and app telemetry are the most complete and the least glamorous. Panels and surveys still supply the demographic detail platform data almost never carries. The route sets the coverage, and coverage sets which questions are answerable at all.

Why it matters

Think of the difference between reading every letter your customers send you and standing outside a train station overhearing conversations. The letters are few, but they are complete and clearly yours. The overheard talk is plentiful, but you do not know who those people are or whether any of them buy from you. Most social data projects go wrong when someone treats the second kind as if it were the first.

Before you read on — recall

A team increases its collection from 50,000 to 5 million posts and reports that the margin of error is now negligible, so its sentiment estimate is reliable. What is the strongest objection?

Formulas

Standard error of a proportion in a sample of posts

\mathrm{SE}(\hat{p}) = \sqrt{\frac{\hat{p}(1-\hat{p})}{n}}

The expression assumes the collected posts behave like an independent random sample. On that assumption, with

n = 5{,}000

posts of which 4 per cent mention the brand favourably, the standard error is about 0.28 percentage points, so a 95 per cent interval runs roughly from 3.5 to 4.5 per cent. Posts rarely meet the assumption, because one account writes many of them and a reshare copies another, so the effective sample is smaller than the raw count. Clustered or weighted standard errors, taking the account or the original post as the cluster, give a wider and more defensible interval.

Why a larger sample does not repair a biased source

\hat{p} - p \;=\; \underbrace{(p_{S} - p)}_{\text{coverage bias}} \;+\; \underbrace{(\hat{p} - p_{S})}_{\text{sampling error}}

Let

p

be the value in the population you care about and

p_{S}

the value among people active on the platform. Only the second term shrinks as

n

grows. If posters differ from customers, the first term stays exactly where it was, and a tighter interval around the wrong quantity is still wrong.

Worked examples

Scenario

A bank wants to measure customer satisfaction and proposes collecting one month of public posts mentioning its name, expecting several hundred thousand items.

Solution

Ask who is missing before asking what the posts say. Public posters skew young, urban and connected, and dissatisfied customers post far more readily than satisfied ones, so the collection over-represents complaints and under-represents the silent majority. It is incomplete a second way as well, because the interface returns a slice rather than everything matching. A defensible design pairs the public collection with first-party sources the bank already owns, complaint records and post-call surveys, and reports the social figure as a directional signal about what to investigate rather than as a satisfaction rate.

Scenario

A team builds a monthly report on a platform interface that changes its access tier, and three months of trend data suddenly cannot be extended.

Solution

This is the ordinary risk of building a permanent report on infrastructure you do not own. Treat access as a dependency with a named owner, a documented rate limit, and a stored copy of everything already collected. Record what each query returned as well as what it asked for, with the collection date, because a later re-run against a changed interface will not reproduce it. Where a trend has to survive, anchor it in a first-party source you control and use platform data as the supplement rather than the spine.

Common mistakes

✗Public means free to use. Public visibility is not a licence. Site terms, copyright and privacy law can all restrict the collection, storage and republication of material anyone can read, and research ethics adds its own requirement even where the law is silent.
✗A very large collection of posts removes sampling worries. Size fixes random error only. If the people posting differ systematically from the people you care about, the bias is identical whether you gather ten thousand posts or ten million.
✗What an interface returns is the full set of matching posts. Most return a capped, rate-limited or sampled slice, apply their own relevance filtering, and omit content the author later deleted. Record the query, the limits and the collection date, or the figures are not reproducible.
✗First-party data matters less than social data because there is less of it. Your own reviews, support tickets and transaction records are linked to identified customers and to outcomes, which is exactly what anonymous public posts lack. Volume is not the same thing as evidential value.

Revision bullets

•Routes: platform interfaces, licensed providers, direct web collection, first-party channels, panels
•Each route sets coverage, and coverage sets which questions are answerable
•Interfaces return filtered, rate-limited, sometimes sampled slices
•Sampling error falls as n grows; coverage bias does not move at all
•Public visibility is not a licence to collect, store or republish
•Log the query, the limits and the collection date or the result is not reproducible

Quick check

A team increases its collection from 50,000 to 5 million posts and reports that the margin of error is now negligible, so its sentiment estimate is reliable. What is the strongest objection?

Which source is most likely to let you connect what someone said to what they subsequently bought?

Connected topics

Forms of Data Sampling Bias Weighted Ties Social Analytics Web Mining GDPR

More in Text and Social Data

NLP Sentiment Analysis Text Mining Pipeline

Sources

Ruths & Pfeffer (2014)
Ruths, D., & Pfeffer, J. "Social media for large studies of behavior." Science, 346(6213), 1063-1064, 2014.
On platform sampling, non-representative user populations, and the limits they place on inference.
Zimmer (2010)
Zimmer, M. "But the data is already public: on the ethics of research in Facebook." Ethics and Information Technology, 12(4), 313-325, 2010.
Case-based argument that public availability does not settle the ethics or the legality of collection.
OAIC, Australian Privacy Principles
Office of the Australian Information Commissioner. Australian Privacy Principles (Privacy Act 1988, Schedule 1).
The Australian obligations on collecting, holding and using personal information, including from public sources.

How to cite this page

Dr. Phil's Quant Lab. (2026). Where social data comes from. Derivatives Atlas. https://phucnguyenvan.com/concept/ba-social-data-sources