An independent study reference written by Dr Phuc V. Nguyen. It is not official subject material — for assessment requirements always follow your subject outline and vUWS.

Text and Social Dataintermediate

Sentiment analysis

Q: A hotel group is choosing between a lexicon method and a fine-tuned language model for scoring guest reviews. Which consideration most favours the lexicon?

It needs no labelled training data, and every score traces back to specific words. The lexicon trade is transparency and zero labelling cost against accuracy that usually lags on hard cases. You can point at the exact words that produced a score, which matters when a manager disputes a result. It is weaker on sarcasm and negation, needs manual updating for new slang, and aspect-based scoring is open to either approach.

Q: Of 10,000 reviews, 3 per cent are negative. Model X has 97 per cent accuracy and recall of 0.05 on the negative class. Model Y has 91 per cent accuracy and recall of 0.72. The triage team can work through about 1,000 flagged items a month. Which model should the complaint-alerting system use?

Model Y, because it catches 216 of the 300 complaints and its roughly 1,030 alerts sit close to the stated capacity of about 1,000. Work the confusion matrices. Model X flags 30 items, 15 of them genuine, so precision is 0.50 but 285 of the 300 complaints are missed. Model Y flags about 1,032 items, 216 of them genuine, so precision falls to 0.21 while recall reaches 0.72. Higher precision does not settle the choice on its own. What settles it here is that Y surfaces 216 complaints instead of 15 while producing about 1,032 alerts, roughly 3 per cent above the stated capacity of about 1,000, a gap a slightly stricter threshold closes. Halve that capacity and the choice can change, which is why an alerting decision needs the cost and capacity figures alongside the classification scores.

Sentiment analysis assigns an attitude to a piece of text: positive, negative or neutral, sometimes with an intensity and sometimes broken out by aspect, so one review can be positive about battery and negative about screen. Three approaches are common. Lexicon methods look words up in a polarity dictionary and apply negation and intensifier rules. Classical supervised classifiers learn weights from human-labelled examples. Fine-tuned pretrained models are supervised too, but start from a model that has already read a large corpus, so they carry context. None dominates on every criterion: fine-tuning usually buys accuracy and costs labelled data, computation and traceability, while a lexicon is cheap and fully inspectable.

Try it yourself

Accuracy and the class you act on

A month brings 10,000 customer reviews. A model gives each review a score for how negative it looks, and a decision threshold turns that score into an alert for the operations team. Move the threshold and watch two figures that answer different questions: overall accuracy, and how many real complaints reach a human.

This threshold (0.750)

97.5%

96 of 300 complaints caught

Label everything positive

97.0%

0 of 300 complaints caught

This threshold calls 9,752 of the 10,000 reviews correctly against 9,700 for the rule that flags nothing, 52 more, and it reaches 96 of the 300 complaints against none.

negative reviews (complaints), 300, drawn above the line. positive reviews, 9,700, drawn below the line. A faded band sits below the threshold and is not flagged. Both halves share one scale of reviews per pixel. The complaint side peaks at 19 reviews against 597 reviews on the positive side, so it is a thin strip. That is what a rare class looks like, and it is why a measure built on all 10,000 reviews barely notices it.

Precision (negative class) 68.6%Recall (negative class) 32.0%F1 (negative class) 0.436

Precision = 96 caught of 140 flagged. Recall = 96 caught of 300 negative reviews. F1 = 2TP / (2TP + FP + FN) = 192 / 440 = 0.436. That count form gives the same number as 2PR / (P + R) wherever precision is defined.

Decision threshold0.750

Reviews that are genuinely negative3% (300 of 10,000)

Flagged reviews the team can read1,000 a month

The 140 alerts fit inside the stated capacity of 1,000 a month, with room for 860 more.

The lowest threshold that fits 1,000 alerts a month is 0.550. It raises 935 alerts and catches 234 of the 300 complaints, a recall of 78.0%.

Confusion matrix at a threshold of 0.750. Rows are what the model did, columns are what the review actually was, and every cell is a whole number of reviews. The four cells sum to 10,000. Being flagged is not the same as being read. All 140 alerts fit inside what the team can read, so the flagged row is the row that reaches a human and the 9,860 not flagged never do.
Model	Actually negative	Actually positive	Row total
Flagged as negative	96 complaints caught	44 false alarms	140 alerts
Not flagged	204 complaints missed	9,656 correctly left alone	9,860 not flagged
Column total	300	9,700	10,000

Accuracy is 97.5%, which is 9,752 of 10,000 reviews called correctly. Precision on the negative class is 68.6%, which is 96 of the 140 flagged, and recall is 32.0%, which is 96 of the 300 negative reviews. The alert list holds 96 real complaints mixed with 44 false alarms, and 204 complaints are never flagged at all. On accuracy this threshold calls 52 more reviews correctly than the rule that flags nothing, 9,752 against 9,700 of 10,000, and it surfaces 96 of the 300 complaints against none. Both figures are reported because either one on its own hides the other. The 140 alerts fit inside the stated capacity of 1,000 a month, with room for 860 more. The lowest threshold that fits 1,000 alerts a month is 0.550. It raises 935 alerts and catches 234 of the 300 complaints, a recall of 78.0%. Negative reviews are 3 per cent of the batch, so the 300 negative reviews carry 3 per cent of the accuracy figure and the 9,700 positive ones carry the rest. A rule that never flags anything loses every complaint and still scores 97.0%. Sweeping the threshold, the highest accuracy available here is 97.5% at a threshold of 0.775, and that setting catches 79 of the 300 complaints on 104 alerts. It prints the same figure as the setting on screen at one decimal place, and the exact counts are 9,754 reviews called correctly there against 9,752 here. Compare it with the setting on screen before deciding which measure the choice should turn on.

Stylised scores. The two distributions come from a fixed seeded draw, one shape for negative reviews and one for positive reviews, and no real model, corpus or product is involved. Nothing here claims that any particular method reaches any of these figures. What is real is the arithmetic. The confusion matrix, the rates and the alert volume all follow exactly from the whole-review counts shown. The class shapes never change, so the balance slider moves only how many reviews are allotted to each shape.

Why it matters

Reading a thousand reviews is easy for one person and impossible for a hundred thousand. Sentiment analysis is a machine doing the skim. It handles the obvious cases well and fails exactly where people are interesting: irony, understatement, faint praise, and complaints written politely. Treat the output as a tide gauge rather than a thermometer. It shows the direction of the water, not the temperature to two decimals.

Before you read on — recall

A hotel group is choosing between a lexicon method and a fine-tuned language model for scoring guest reviews. Which consideration most favours the lexicon?

Formulas

Net sentiment score

\text{NSS} = \frac{n_{+} - n_{-}}{n_{+} + n_{-} + n_{0}}

Out of 2,000 classified posts, 620 positive, 380 negative and 1,000 neutral give

\text{NSS} = (620-380)/2{,}000 = 0.12

. Dropping the neutrals from the denominator would report 0.24 for the same data, which is why the definition has to travel with the number.

Precision, recall and F1 for the class you act on

P = \frac{TP}{TP + FP}, \qquad R = \frac{TP}{TP + FN}, \qquad F_{1} = \frac{2PR}{P + R}

A model flags 400 posts as negative and 300 of them really are, so

P = 0.75

. The data holds 500 truly negative posts, so

R = 300/500 = 0.60

and

F_{1} = 0.67

. Report these per class. A single overall accuracy hides the class that matters, because complaints are usually the rare one.

Worked examples

Scenario

A team reports 97 per cent accuracy for a model that flags negative reviews, and the operations manager still receives no useful alerts.

Solution

Check the class balance first. If 3 per cent of the 10,000 reviews are negative, a model that labels everything positive scores 97 per cent accuracy and catches zero complaints. Accuracy on an imbalanced problem mostly measures the majority class. The right report is precision and recall for the negative class alone, plus the confusion matrix, so the manager can see the trade-off between missed complaints and false alarms and can set the decision threshold where the triage team can live with it.

Scenario

A research team needs an investor sentiment measure for a stock market where most commentary is written in Vietnamese, and off-the-shelf English tools score close to random.

Solution

The work starts with representation and labels rather than with a model. A pretrained language model that has read a large volume of Vietnamese text is fine-tuned on financial commentary labelled by people who know the market vocabulary, because general annotators mislabel phrases that are only positive inside a trading context. The index built from those scores is then validated against market variables rather than accepted on its classification accuracy alone. Both choices matter more than the architecture, and both are the parts a reader of the finished index should be able to inspect.

Common mistakes

✗Sentiment analysis tells you what people feel. It classifies the wording of a text. A polite complaint reads as neutral, sarcasm often reads as positive, and a one-star rating captioned "just great" defeats the model entirely. Sentiment is a measure of expression, not of emotion.
✗A high overall accuracy means the model is fit for use. When one class is rare, accuracy is dominated by the majority class. Report precision and recall for the class you act on, and choose the threshold from the cost of a miss weighed against the cost of a false alarm.
✗One sentiment score per document is enough. Reviews frequently praise one attribute and attack another, so a single score averages away the actionable part. Aspect-based sentiment attaches a polarity to each attribute mentioned, and that is the form a product team can use.
✗A model that works on product reviews will work on financial posts. Polarity is domain-specific. "Volatile" is negative in a customer review and close to neutral in a trading discussion, and "cheap" flips meaning between a product and a share price. Lexicons and training data have to match the domain.

Revision bullets

•Three approaches: lexicon rules, classical supervised classifiers, fine-tuned pretrained models (supervised too)
•Aspect-based sentiment scores each attribute rather than the whole document
•Net sentiment score depends on whether neutrals sit in the denominator
•Accuracy misleads on imbalanced data; use per-class precision, recall and F1
•Polarity is domain-specific and language-specific
•Sarcasm, understatement and polite complaint are the standing failure modes

Quick check

A hotel group is choosing between a lexicon method and a fine-tuned language model for scoring guest reviews. Which consideration most favours the lexicon?

Of 10,000 reviews, 3 per cent are negative. Model X has 97 per cent accuracy and recall of 0.05 on the negative class. Model Y has 91 per cent accuracy and recall of 0.72. The triage team can work through about 1,000 flagged items a month. Which model should the complaint-alerting system use?

Connected topics

Supervised vs Unsup.Social Analytics NLP Text Mining Pipeline Algorithmic Bias

More in Text and Social Data

Social Data Sources Web Mining

Sources

Pang & Lee (2008)
Pang, B., & Lee, L. "Opinion Mining and Sentiment Analysis." Foundations and Trends in Information Retrieval, 2(1-2), 1-135, 2008.
The standard survey of the task, its evaluation, and why polarity is harder than it looks.
Liu (2012)
Liu, B. Sentiment Analysis and Opinion Mining. Synthesis Lectures on Human Language Technologies. Morgan & Claypool, 2012.
Sets out aspect-based sentiment and the structure of an opinion as target, aspect, holder and polarity.
Hutto & Gilbert (2014)
Hutto, C. J., & Gilbert, E. "VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text." Proceedings of ICWSM-14, 2014.
A worked example of the lexicon-plus-rules family, including its handling of negation, intensifiers and emoticons.
Devlin et al. (2019)
Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding." NAACL-HLT, 2019. arXiv:1810.04805.
The pretrained model family behind the fine-tuned sentiment classifiers described here.

How to cite this page

Dr. Phil's Quant Lab. (2026). Sentiment analysis. Business Analytics Atlas. https://phucnguyenvan.com/analytics_atlas/concept/ba-sentiment-analysis