MGMLaboratory
Back

Menu

01Home
02About Us
03Projects
04Publications
05Member
06Articles
07Contact

Let's Talk

Open in Mail app

Socials

Malang (ID) --:--Privacy PolicyTerms of Service

February 15, 20182 min read

Likes and Retweets as Extra Signal for Sentiment AnalysisLikes and Retweets as Extra Signal for Sentiment Analysis

Adding engagement data to a text classifier pushed the F1 score to 0.838.

  • Journal
  • Publications

Written by

  • SRShirasaka RenIT & Infrastructure · Student Member

In this article

  1. 01Two kinds of signal
  2. 02How the two were merged

Sentiment analysis reads opinion out of text. On Twitter that job is harder than it looks, because posts are short, informal, and full of sarcasm. A study from Universitas Brawijaya asked a simple question: if the text is not enough on its own, what else on the platform can help?

Opinions now arrive in short bursts on a phone. Image: Rawpixel, CC0, via Wikimedia Commons.
Fig. 01Opinions now arrive in short bursts on a phone. Image: Rawpixel, CC0, via Wikimedia Commons.

01/02

Two kinds of signal

The team split the problem in two. Text was classified with Naive Bayes, a long standing workhorse for this kind of task. People also react to a post in ways the words do not capture, so the researchers treated likes and retweets as a second set of features, measured rather than read.

Not every count is worth keeping. Fisher Score was used to test which of those engagement features actually carried information about sentiment, so that weak signals could be filtered out before they added noise.

02/02

How the two were merged

Text and engagement were combined with weights named alpha and beta. Choosing those weights is the whole design decision, because they set how much the model trusts the words and how much it trusts the crowd's reaction.

A town hall audience posts questions in real time. Image: Pete Souza, public domain, via Wikimedia Commons.
Fig. 02A town hall audience posts questions in real time. Image: Pete Souza, public domain, via Wikimedia Commons.

The best configuration put slightly more weight on the text than on the engagement features, and the combined model reached an F1 score of 0.838. The result is a useful reminder that engagement data is a supplement, not a replacement. Words still carry most of the meaning, and reactions sharpen the picture around the edges.

  • Text classified with Naive Bayes, engagement handled as separate numeric features
  • Fisher Score used to select which like and retweet features to keep
  • Best result at alpha 0.6 and beta 0.4, with F1 at 0.838

For teams building anything on social data, the takeaway is that a platform gives you more than the sentences it stores. The surrounding behaviour is cheap to collect and, with the right weighting, it earns its place in the model.

Source

  1. Rizal Setya Perdana and Aryo Pinandito (2018). Combining Likes-Retweet Analysis and Naive Bayes Classifier within Twitter for Sentiment Analysis. Journal of Telecommunication, Electronic and Computer Engineering.

Next articleWhy a Like Carries Less Weight Than a Sentence(Keep scrolling)(Read it next)(Loading)