25 April 2016 – Around 20% of online hotel customer reviews may be unreliable or fake, according to Dr Markus Schuckert and Professor Rob Law of the School of Hotel and Tourism Management (SHTM) at The Hong Kong Polytechnic University and a co-researcher. After examining more than forty thousand TripAdvisor reviews of all star-rated hotels in Hong Kong in a recently published study, they warn that the high proportion of suspicious reviews should ring an “alarm bell” for customers. Although these reviews are not necessarily fake, they can be misleading and waste customers’ decision-making time. Given that many customers depend on online hotel reviews to make their purchasing decisions, the problem has serious implications for the hotel sector.
E-commerce platforms have become so popular that online sales of travel products, particularly by hotels and airlines, have become the “biggest part of their business”, write the researchers. One reason for this rising popularity is the availability of online reviews and feedback posted by customers, which help potential customers to make informed decisions. From the service provider’s perspective, online reviews provide “fast, instant and easily accessible customer feedback”, and good reviews can increase their revenue.
Understandably, customers want to book with the hotel or restaurant with the highest ratings. The researchers note that these people are prepared to spend a considerable amount of time reading online reviews before making decisions because they “want to find the right place and be sure about it”. Customers prefer “large feedback platforms and consumer-centric sites”, which are perceived as offering the most “objective, true or authentic opinions”.
However, the researchers warn that although “every e-commerce platform has a system and procedure to ensure the authenticity” of reviews, there is growing concern that companies, or customers, manipulate reviews or give false ratings for various reasons. Owners, for instance, may post positive reviews themselves, or get friends or others to do so, to attract customers and boost sales. Conversely, they may post bad reviews to “defame competitors”.
Although false and misleading reviews can lead to consumers making the wrong purchasing decisions, there is still insufficient evidence on the extent of such practices. One reason, the researchers note, is the lack of a reliable method for detecting fake reviews. Detecting them by differences in writing style, for instance, “assumes that the writing styles of manipulators will be different from those of genuine customers”. The researchers thus designed a method of identifying suspicious reviews and conducted a study to determine the presence and proportion of suspicious reviews and to “explore who tends to post” them.
The focus of that study was TripAdvisor, which allows reviewers to leave two types of numerical ratings for hotels – one overall rating, and six specific ratings for service, value, sleep quality, cleanliness, location and rooms. The ratings can range from 1 for the worst quality to 5 for the best. The anomaly that some reviews show a considerable discrepancy between the overall rating and the specific ratings prompted the researchers to consider whether this might be an indicator of a suspicious or unreliable review.
The researchers provide two examples in which the overall hotel rating is 5 but the six specific qualities are only rated as 1 or 2. Comparing the ratings with the written comments, one of the reviews seems to fit the overall rating of 5, suggesting that the specific ratings are misleading, whereas the other review seems more consistent with an overall rating of only 1 or 2, suggesting that the high overall rating is misleading. That self-contradiction prompted the researchers to explore whether this is a widespread phenomenon and, if so, what causes it.
Using specially developed software, the researchers retrieved both types of ratings for all Hong Kong star-rated hotels listed on TripAdvisor. There were 185 hotels – 18 five-star, 80 four-star, 75 three-star, 11 two-star and 1 one-star – with a total of 41,572 ratings. Next, they measured the “gap” between the overall rating and the average of the six specific ratings as the “index to measure whether a review or a rating is suspicious”.
The average overall rating of the reviews was around 4.2, which the researchers note indicates that “travellers are very satisfied with their experience in Hong Kong hotels in general”. More than three quarters of reviewers gave ratings of 4 or 5, whereas only 8% gave ratings of 1 or 2.
Comparing the overall ratings with the average for the six specific ratings, the researchers found that almost 20% of reviews had a discrepancy, or “gap”, of more than 0.5. This gap, they write, is an indication of a “suspicious” or at least “low-quality” online review, describing the number of reviews with this gap as a “large and considerable proportion”.
Turning to “who tends to post suspicious ratings”, the researchers examined the relationship between the size of the gap and the overall rating. They found that reviewers who gave excellent ratings were less likely to have a gap between their overall and specific ratings, whereas those who “dislike giving excellent ratings” tended to create big gaps. They offer two possible explanations for this finding.
First, it is possible that reviewers who tend not to give excellent ratings are less careful about the scores they give and merely “post them randomly”. Alternatively, those who tend to give excellent ratings might “include more manipulators whose job is to constantly post positive reviews” for hotels. In other words, they could be professional reviewers paid by hotels to generate positive reviews who are required to give both overall and specific ratings of 5.
Finally, the researchers examined whether different types of hotels are more likely to be associated with suspicious reviews. In general, higher class hotels not only received better reviews, but also had less of a gap in their ratings than lower class hotels. Hence, “the problem of suspicious online ratings” may be more serious among the lower class hotels.
The researchers conclude that suspicious reviews can be generated in two ways – through either deliberate manipulation or “perfunctory rating behaviour”, with the two being indistinguishable. Consequently, online customers should “pay more attention to the rating gap” on TripAdvisor.
They also suggest that TripAdvisor provide a warning to reviewers who “may have made a mistake or may not be taking the rating seriously” if they try to post a review with ratings that differ by more than 0.5. Although other sites aggregate category scores to produce a final score, circumventing this problem, the researchers suggest that their findings should ultimately remind all “e-commerce platforms about the problem of fake or low-quality reviews”.
Schuckert, Markus, Liu, Xianwei and Law, Rob. (2016). Insights into Suspicious Online Ratings: Direct Evidence from TripAdvisor. Asia Pacific Journal of Tourism Research, 16(3), 259-272.
