Chicago, Illinois (PRWEB) November 01, 2011
A couple of months ago Hotel Compete published an article: Hotel Reviews – The Hotel Industry Must Learn from the Wisdom of Crowds, which discussed the relevancy of user review scores. That article highlighted a fallacy in current industry commentary: that review scores are too biased or unreliable to be used for analytical purposes. This article digs deeper into the important insights that can be gleaned from hotel review scores.
User review sites use different numerical scales – some 1-5, some 1-10, and so on. That complicates analysis somewhat, but the real analytical challenge is that users post scores differently on different scales. To understand hotel scores we must understand how actual review scores are distributed across each scale.
For example, when analyzing reviews on a 1-10 scale, the average score is not necessarily 5. In fact, if most reviews score in the 7-8 range the average score would be between 7 and 8. This is important, because a score of six in this scenario is relatively weak, but would look positive if we were to ignore the distribution of actual scores across the scale.
Hotel Compete used this rationale in a detailed analysis whose results were reported in the previous article. It demonstrated that review scores are normally distributed – ie. they follow a pattern with lots of reviews around the average, with fewer and fewer further away from the average.
This is important because normal distributions are statistically interesting. They enable us to establish patterns and predict things, like how we should expect a hotel to perform financially. To get to that level of analysis for a specific hotel, we must first develop a robust view of the typical range of scores for hotels of that type.
Imagine, for example, analyzing a small limited service property surrounded by upscale competitors. If the reference hotel scored 6 on a scale of 1-10 and its competitors averaged 7, that may not matter too much from a performance perspective – the other hotels are superior, after all. But if the same hotel were surrounded by other limited service hotels that averaged 7, its score of 6 could adversely affect its business. Hence to understand what these numbers mean we must get a handle on the relationship between scores and hotel classes.
Hotel class is another analytical challenge. Star ratings don’t work – there are too many sources, and they all use different rating rationales. Chain scales are too broad for meaningful analysis, and the consistency within chains means that we have to find a way of allocating individual hotels to individual classes. Hotel Compete dealt with this problem by developing its own scale of hotel classes, and an automated approach that allocates individual hotels to classes based on individual hotel characteristics. A more detailed description of this hotel classification approach can be found here.
To ensure a representative sample of reviews, 25,000 hotels from Hotel Compete's database were selected for this analysis. The hotels were grouped into five broad classifications: Moderate, Limited Service, Boutique, Full-Service and Luxury. The attached graph presents the results of that analysis when actual review scores across all sites are normalized to a 1-10 scale.
The fascinating thing about this analysis is that it uncovers so clearly the impact of hotel class upon average scores. This latest analysis suggests that that the overall distribution featured in the previous post is composed of a series of similar, lower-level distributions whose averages move up the points scale in a manner roughly consistent with the change in hotel class.
To put it another way, not only are hotel review scores distributed evenly overall, they also appear to be distributed normally within hotel classes. This has some interesting implications for performance analysis – not least that we can now understand the range of scores that hotels of a particular type are supposed to achieve. This insight would bring some clarity to the example we described above – the range of scores for moderate and full-service hotels are demonstrably different, hence performance expectations should be adjusted accordingly.
Perhaps the most interesting finding of this analysis is the alignment between the quality of the product and the average reported quality of experience. Each hotel class’s scores fit into just a few points on the scale, suggesting that guests grade hotels in line with their expectations, which are largely rational. This finding could scarcely be further from the current body of expert opinion on hotel reviews, as we discussed in our previous post.
It also seems to issue a stark warning to hotel operators: if a property’s scores fall substantially below the average scores of hotels in its class, it is likely falling far short of customer expectations, and may suffer financially as a result. Conversely, hotels that out-perform their normal class will likely perform well. In fact a small number of properties achieved perfect scores. All were in the luxury category – a good reminder that it’s possible to exceed even the loftiest expectations!
This is just one further slice of this powerful data source. Further analysis can, of course, dig deeper still into individual comp sets, as well as geographical differences. But so far with every turn more evidence of the analytical power of user review scores seems to emerge . Hotel Compete will publish more insights in the coming months, but in the meantime you can learn more about user reviews and the Wisdom of Crowds by downloading our latest Industry Viewpoint on the subject.