MGMLaboratory
Back

Menu

01Home
02About Us
03Projects
04Publications
05Member
06Articles
07Contact

Let's Talk

Open in Mail app

Socials

Malang (ID) --:--Privacy PolicyTerms of Service

January 31, 20182 min read

Adding One Criterion Can Flip Your Whole RankingAdding One Criterion Can Flip Your Whole Ranking

The Malang culinary study is most useful as a catalogue of the ways a recommendation algorithm can quietly misbehave.

  • Journal
  • Publications

Written by

  • SRShirasaka RenIT & Infrastructure · Student Member

In this article

  1. 01Watch out for rank reversal
  2. 02The nine item ceiling
  3. 03The pattern worth copying

Recommendation apps look like a solved problem. Collect a few criteria, weight them, sort the list, ship it. The study from Brawijaya University shows where that plan breaks down.

The team ran three multi criteria methods over 30 real cases from Malang, all scored against the same three criteria: distance, price, and rating. The interesting material is not which method won. It is the failure mode each one carries.

01/03

Watch out for rank reversal

AHP builds its ranking from pairwise comparisons. When you add a restaurant to the database, or drop a criterion you no longer care about, the relative order of the alternatives can change even though none of them changed. The literature calls this rank reversal, and the authors list it among the known weaknesses of the method.

For an app that updates its database constantly, that is a real problem. A user who sees a restaurant at the top of the list today may find it third tomorrow, for no reason they can observe.

A food court with many competing options. Image: Stephan van Helden, CC BY-SA 4.0, via Wikimedia Commons.
Fig. 01A food court with many competing options. Image: Stephan van Helden, CC BY-SA 4.0, via Wikimedia Commons.

02/03

The nine item ceiling

AHP also caps how much you can compare at once. The paper notes that the number of criteria or alternatives is limited to nine. Past that point, the consistency of human judgments falls away, and the derived weights get worse.

TOPSIS carries the opposite weakness. It ranks alternatives well, but its attribute weighting is highly subjective, because whoever configures the system hands out the weights directly, with no check that they are consistent with one another.

03/03

The pattern worth copying

The hybrid method in this study covers both gaps. AHP sets the criteria weights, which enforces consistency through pairwise comparison, and TOPSIS does the ranking, where its normalization and weighting step costs only quadratic order. That split is why AHP TOPSIS came out fastest in the simulation while still beating plain AHP on accuracy.

The lesson for anyone building a ranked list: pick your weighting method for consistency and your ranking method for cost. Do not expect a single algorithm to do both jobs well.

Fritters for sale in Jakarta. Image: Sakurai Midori, CC BY 3.0, via Wikimedia Commons.
Fig. 02Fritters for sale in Jakarta. Image: Sakurai Midori, CC BY 3.0, via Wikimedia Commons.

Source

  1. Dewi et al. (2018). A Comparison Between AHP and Hybrid AHP for Mobile Based Culinary Recommendation System. International Journal of Interactive Mobile Technologies.

    DOI10.3991/ijim.v12i1.7561 (opens in a new tab)

Next articleThree Ways to Rank a Restaurant List, Compared(Keep scrolling)(Read it next)(Loading)