An independent study reference written by Dr Phuc V. Nguyen. It is not official subject material — for assessment requirements always follow your subject outline and vUWS.

Text and Social Dataintermediate

Sentiment analysis

Q: A hotel group is choosing between a lexicon method and a fine-tuned language model for scoring guest reviews. Which consideration most favours the lexicon?

It needs no labelled training data, and every score traces back to specific words. The lexicon trade is transparency and zero labelling cost against accuracy that usually lags on hard cases. You can point at the exact words that produced a score, which matters when a manager disputes a result. It is weaker on sarcasm and negation, needs manual updating for new slang, and aspect-based scoring is open to either approach.

Q: Of 10,000 reviews, 3 per cent are negative. Model X has 97 per cent accuracy and recall of 0.05 on the negative class. Model Y has 91 per cent accuracy and recall of 0.72. The triage team can work through about 1,000 flagged items a month. Which model should the complaint-alerting system use?

Model Y, because it catches 216 of the 300 complaints and its roughly 1,030 alerts sit close to the stated capacity of about 1,000. Work the confusion matrices. Model X flags 30 items, 15 of them genuine, so precision is 0.50 but 285 of the 300 complaints are missed. Model Y flags about 1,032 items, 216 of them genuine, so precision falls to 0.21 while recall reaches 0.72. Higher precision does not settle the choice on its own. What settles it here is that Y surfaces 216 complaints instead of 15 while producing about 1,032 alerts, roughly 3 per cent above the stated capacity of about 1,000, a gap a slightly stricter threshold closes. Halve that capacity and the choice can change, which is why an alerting decision needs the cost and capacity figures alongside the classification scores.

Sentiment analysis assigns an attitude to a piece of text: positive, negative or neutral, sometimes with an intensity and sometimes broken out by aspect, so one review can be positive about battery and negative about screen. Three approaches are common. Lexicon methods look words up in a polarity dictionary and apply negation and intensifier rules. Classical supervised classifiers learn weights from human-labelled examples. Fine-tuned pretrained models are supervised too, but start from a model that has already read a large corpus, so they carry context. None dominates on every criterion: fine-tuning usually buys accuracy and costs labelled data, computation and traceability, while a lexicon is cheap and fully inspectable.

Why it matters

Reading a thousand reviews is easy for one person and impossible for a hundred thousand. Sentiment analysis is a machine doing the skim. It handles the obvious cases well and fails exactly where people are interesting: irony, understatement, faint praise, and complaints written politely. Treat the output as a tide gauge rather than a thermometer. It shows the direction of the water, not the temperature to two decimals.

Before you read on — recall

A hotel group is choosing between a lexicon method and a fine-tuned language model for scoring guest reviews. Which consideration most favours the lexicon?

Formulas

Net sentiment score

\text{NSS} = \frac{n_{+} - n_{-}}{n_{+} + n_{-} + n_{0}}

Out of 2,000 classified posts, 620 positive, 380 negative and 1,000 neutral give

\text{NSS} = (620-380)/2{,}000 = 0.12

. Dropping the neutrals from the denominator would report 0.24 for the same data, which is why the definition has to travel with the number.

Precision, recall and F1 for the class you act on

P = \frac{TP}{TP + FP}, \qquad R = \frac{TP}{TP + FN}, \qquad F_{1} = \frac{2PR}{P + R}

A model flags 400 posts as negative and 300 of them really are, so

P = 0.75

. The data holds 500 truly negative posts, so

R = 300/500 = 0.60

and

F_{1} = 0.67

. Report these per class. A single overall accuracy hides the class that matters, because complaints are usually the rare one.

Worked examples

Scenario

A team reports 97 per cent accuracy for a model that flags negative reviews, and the operations manager still receives no useful alerts.

Solution

Check the class balance first. If 3 per cent of the 10,000 reviews are negative, a model that labels everything positive scores 97 per cent accuracy and catches zero complaints. Accuracy on an imbalanced problem mostly measures the majority class. The right report is precision and recall for the negative class alone, plus the confusion matrix, so the manager can see the trade-off between missed complaints and false alarms and can set the decision threshold where the triage team can live with it.

Scenario

A research team needs an investor sentiment measure for a stock market where most commentary is written in Vietnamese, and off-the-shelf English tools score close to random.

Solution

The work starts with representation and labels rather than with a model. A pretrained language model that has read a large volume of Vietnamese text is fine-tuned on financial commentary labelled by people who know the market vocabulary, because general annotators mislabel phrases that are only positive inside a trading context. The index built from those scores is then validated against market variables rather than accepted on its classification accuracy alone. Both choices matter more than the architecture, and both are the parts a reader of the finished index should be able to inspect.

Common mistakes

✗Sentiment analysis tells you what people feel. It classifies the wording of a text. A polite complaint reads as neutral, sarcasm often reads as positive, and a one-star rating captioned "just great" defeats the model entirely. Sentiment is a measure of expression, not of emotion.
✗A high overall accuracy means the model is fit for use. When one class is rare, accuracy is dominated by the majority class. Report precision and recall for the class you act on, and choose the threshold from the cost of a miss weighed against the cost of a false alarm.
✗One sentiment score per document is enough. Reviews frequently praise one attribute and attack another, so a single score averages away the actionable part. Aspect-based sentiment attaches a polarity to each attribute mentioned, and that is the form a product team can use.
✗A model that works on product reviews will work on financial posts. Polarity is domain-specific. "Volatile" is negative in a customer review and close to neutral in a trading discussion, and "cheap" flips meaning between a product and a share price. Lexicons and training data have to match the domain.

Revision bullets

•Three approaches: lexicon rules, classical supervised classifiers, fine-tuned pretrained models (supervised too)
•Aspect-based sentiment scores each attribute rather than the whole document
•Net sentiment score depends on whether neutrals sit in the denominator
•Accuracy misleads on imbalanced data; use per-class precision, recall and F1
•Polarity is domain-specific and language-specific
•Sarcasm, understatement and polite complaint are the standing failure modes

Quick check

A hotel group is choosing between a lexicon method and a fine-tuned language model for scoring guest reviews. Which consideration most favours the lexicon?

Of 10,000 reviews, 3 per cent are negative. Model X has 97 per cent accuracy and recall of 0.05 on the negative class. Model Y has 91 per cent accuracy and recall of 0.72. The triage team can work through about 1,000 flagged items a month. Which model should the complaint-alerting system use?

Connected topics

Supervised vs Unsup.Social Analytics NLP Text Mining Pipeline Algorithmic Bias

More in Text and Social Data

Social Data Sources Web Mining

Sources

Pang & Lee (2008)
Pang, B., & Lee, L. "Opinion Mining and Sentiment Analysis." Foundations and Trends in Information Retrieval, 2(1-2), 1-135, 2008.
The standard survey of the task, its evaluation, and why polarity is harder than it looks.
Liu (2012)
Liu, B. Sentiment Analysis and Opinion Mining. Synthesis Lectures on Human Language Technologies. Morgan & Claypool, 2012.
Sets out aspect-based sentiment and the structure of an opinion as target, aspect, holder and polarity.
Hutto & Gilbert (2014)
Hutto, C. J., & Gilbert, E. "VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text." Proceedings of ICWSM-14, 2014.
A worked example of the lexicon-plus-rules family, including its handling of negation, intensifiers and emoticons.
Devlin et al. (2019)
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." NAACL-HLT, 2019. arXiv:1810.04805.
The pretrained model family behind the fine-tuned sentiment classifiers described here.

How to cite this page

Dr. Phil's Quant Lab. (2026). Sentiment analysis. Derivatives Atlas. https://phucnguyenvan.com/concept/ba-sentiment-analysis