Beware the Star Rating
Most comments on consumer-driven travel web sites tend to be favorable, but an appreciable minority is quite negative. A recently-published study by the Cornell Center for Hospitality Research took a detailed look at an unnamed site that posts reviews from people who have rented vacation homes to determine how accurately those reviews reflected guests’ true sentiments, as well as the implications that they can have on the delivery of hotel services.
The study, “Unscrambling the Puzzling Matter of Online Consumer Ratings: An Exploratory Analysis,” by Pradeep Racherla, Daniel Connolly, and Natasa Christodoulidou (available at no charge from Cornell) focused specifically on vacation-home rentals. In their analysis of nearly 3,200 reviews posted on the home-rental review site, nearly 2,600 were positive and some 600 were fair or negative. The division was based on the site’s star rating system: four or five stars were considered positive and three or fewer stars were termed negative.
Since consumers undoubtedly rely on the star ratings when renting a home, the authors investigated the extent to which these star ratings actually reflect consumers’ comments and their ratings of certain attributes, such as check-in and check-out, cleanliness, comfort, location, service, and value. The result of the comparison was puzzling, to say the least.
The correlation between the overall star ratings and the specific attribute scores was a modest 0.40 in most cases. Posters might rate a place with four or five stars, but their average scores for these six attributes would be much lower. Even though we know that all this involves a self-selected sample, it appears that the star rating may not be the most accurate indicator of customers’ perceptions of their accommodations.
The researchers next considered the text of the reviews to see what differentiates the positive from the negative. Here, they looked at what the consumers had written about such features as the bathrooms, the beach, the bedroom, the kitchen, and the pool. The numerous positive reviews tended to be relatively brief and generally reinforced the numerical star ratings. Negative reviews, however, tended to be more descriptive and were directed to other consumers, using such phrases as “not as expected.”
What does this mean for hoteliers?
This study implies that both consumers and hoteliers should look past the general star ratings to see what guests have actually written about their accommodations. I also see a cautionary tale for lodging operators. In short, don’t rest on your laurels. If your operation is pulling mostly high ratings, there is a good chance that they can be misleading, since a closer look reveals that satisfaction with certain attributes is lower than what would be expected for a four or five star location. What’s more, we’re also seeing that happy guests are more likely to post than unhappy ones, refuting the notion that dissatisfied guests are more likely to post a comment. I have seen this trend in other online site studies, including, “Electronic Meal Experience: A Content Analysis of Online Restaurant Comments,” published in the Cornell Hospitality Quarterly. There, 2,500 posts to a London-based restaurant review site were analyzed and the vast majority were favorable.
Travel review sites involve more than just a casual conversation among friends, and writing the reviews does involve some work; as such, people are taking this “job” seriously. Even though negative posters are relatively less common, lodging operators might want to seek out those negative reviews, because they tend to specify the reason for the guest’s discontent. Perhaps the problem was a one-time failure of facilities or service, but there may also be systemic issues that require attention.
As with all online phenomena, the creation and use of online reviews remain a work in progress. You can, however, use the information in those reviews to help you avoid the cardinal sin of disappointing your guest.
About: Glenn Withiam,
Director of Publications, Cornell Center for Hospitality Research
What was your first job?
Picking green beans at the Cornell Agricultural Experiment Station
Who inspires you?
Anyone who has persevered in the face of adversity
What are your hobbies?
Bicycle riding and running
What technology excites you?
We live in an age of miracles that we can electronically communicate, look up infor-mation, and so forth in a heartbeat. Medical advances are also quite impressive of late.
Words of Wisdom:
Whether you’re speaking of publishing or hospitality the answer
is the same, don’t let technology get in the way of being a mensch.
What is one other field that you would like to try?
Professional chef
Who would you invite to lunch?
Mark Twain & Theodore Roosevelt
Top movie/book:
Some Like It Hot
Favorite vacation spot:
New York Adirondacks, but I love the Grand Canyon, Crater lake, Acadia, and the Oregon Coast.
Glenn Withiam is the director of publications for the Cornell Center for Hospitality Research. To download complimentary copies of any of the research reports from the Center for Hospitality Research, visit www.hotelschool.cornell.edu/research/chr.
The study, “Unscrambling the Puzzling Matter of Online Consumer Ratings: An Exploratory Analysis,” by Pradeep Racherla, Daniel Connolly, and Natasa Christodoulidou (available at no charge from Cornell) focused specifically on vacation-home rentals. In their analysis of nearly 3,200 reviews posted on the home-rental review site, nearly 2,600 were positive and some 600 were fair or negative. The division was based on the site’s star rating system: four or five stars were considered positive and three or fewer stars were termed negative.
Since consumers undoubtedly rely on the star ratings when renting a home, the authors investigated the extent to which these star ratings actually reflect consumers’ comments and their ratings of certain attributes, such as check-in and check-out, cleanliness, comfort, location, service, and value. The result of the comparison was puzzling, to say the least.
The correlation between the overall star ratings and the specific attribute scores was a modest 0.40 in most cases. Posters might rate a place with four or five stars, but their average scores for these six attributes would be much lower. Even though we know that all this involves a self-selected sample, it appears that the star rating may not be the most accurate indicator of customers’ perceptions of their accommodations.
The researchers next considered the text of the reviews to see what differentiates the positive from the negative. Here, they looked at what the consumers had written about such features as the bathrooms, the beach, the bedroom, the kitchen, and the pool. The numerous positive reviews tended to be relatively brief and generally reinforced the numerical star ratings. Negative reviews, however, tended to be more descriptive and were directed to other consumers, using such phrases as “not as expected.”
What does this mean for hoteliers?
This study implies that both consumers and hoteliers should look past the general star ratings to see what guests have actually written about their accommodations. I also see a cautionary tale for lodging operators. In short, don’t rest on your laurels. If your operation is pulling mostly high ratings, there is a good chance that they can be misleading, since a closer look reveals that satisfaction with certain attributes is lower than what would be expected for a four or five star location. What’s more, we’re also seeing that happy guests are more likely to post than unhappy ones, refuting the notion that dissatisfied guests are more likely to post a comment. I have seen this trend in other online site studies, including, “Electronic Meal Experience: A Content Analysis of Online Restaurant Comments,” published in the Cornell Hospitality Quarterly. There, 2,500 posts to a London-based restaurant review site were analyzed and the vast majority were favorable.
Travel review sites involve more than just a casual conversation among friends, and writing the reviews does involve some work; as such, people are taking this “job” seriously. Even though negative posters are relatively less common, lodging operators might want to seek out those negative reviews, because they tend to specify the reason for the guest’s discontent. Perhaps the problem was a one-time failure of facilities or service, but there may also be systemic issues that require attention.
As with all online phenomena, the creation and use of online reviews remain a work in progress. You can, however, use the information in those reviews to help you avoid the cardinal sin of disappointing your guest.
About: Glenn Withiam,
Director of Publications, Cornell Center for Hospitality Research
What was your first job?
Picking green beans at the Cornell Agricultural Experiment Station
Who inspires you?
Anyone who has persevered in the face of adversity
What are your hobbies?
Bicycle riding and running
What technology excites you?
We live in an age of miracles that we can electronically communicate, look up infor-mation, and so forth in a heartbeat. Medical advances are also quite impressive of late.
Words of Wisdom:
Whether you’re speaking of publishing or hospitality the answer
is the same, don’t let technology get in the way of being a mensch.
What is one other field that you would like to try?
Professional chef
Who would you invite to lunch?
Mark Twain & Theodore Roosevelt
Top movie/book:
Some Like It Hot
Favorite vacation spot:
New York Adirondacks, but I love the Grand Canyon, Crater lake, Acadia, and the Oregon Coast.
Glenn Withiam is the director of publications for the Cornell Center for Hospitality Research. To download complimentary copies of any of the research reports from the Center for Hospitality Research, visit www.hotelschool.cornell.edu/research/chr.
