Stanford and Harvard Professors Demystify Online Buying Decisions with Statistically Adjusted Star/Bar Graphs

Share Article

Suppose you are shopping at Amazon.com for a digital camera. One camera has a whopping 146 ratings with an average of four stars; another has only 10 ratings with an average of five stars. How do you decide which camera to buy? An article in the November issue of The American Statistician (TAS), a journal of the American Statistical Association, addresses the statistical challenges of how to show and interpret such online ratings and presents a system - utilizing the now-traditional star ratings -- to make those ratings more meaningful.

Suppose you are shopping at Amazon.com for a digital camera. One camera has a whopping 146 ratings with an average of four stars; another has only 10 ratings with an average of five stars. How do you decide which camera to buy? An article in the November issue of The American Statistician (TAS), a journal of the American Statistical Association, addresses the statistical challenges of how to show and interpret such online ratings and presents a system - utilizing the now-traditional star ratings -- to make those ratings more meaningful.

"The decision of which product to buy is a dilemma faced by Internet users every day," said article co-author Daniel Ho, assistant professor of law and Robert E. Paradise Faculty Fellow for Excellence in Teaching and Research at Stanford Law School. "How do we draw comparisons and make a decision from online ratings of products and services when users' ratings each may mean something drastically different?"

Which camera is truly better given that different raters may be rating the product? "Without statistical adjustments, it's hard to tell," said Kevin Quinn, associate professor at the Department of Government and the Institute for quantitative Social Science at Harvard University, "and we hope that our graphical approach facilitates the adoption of more sophisticated data analysis and presentation online. Our approach is can be applied to any web site presenting data from user ratings."

"The Internet has provided a deluge of information," Ho said, "and one of the key challenges is to craft statistical solutions to convey such information succinctly and powerfully to users."

The prevailing practice on web sites like Amazon.com, Epinions.com, and YouTube.com is to present the average number of stars and the number of ratings submitted. Yet such practice, Ho and Quinn argue, can be misleading for reasons readily apparent to statisticians.

In their November TAS paper, Ho and Quinn address the statistical challenges of how to present and interpret online ratings. First, raters use the 5-star scale differently. Some might submit ratings only for products they like, leading to exclusive use of four or five stars. Other raters might do the reverse, submitting ratings only for products they don't like. Averaging these raw ratings across users that differ in their discrimination, therefore, can mislead the buyer. Second, the average number of stars doesn't convey statistical uncertainty, and requires the users themselves to assess how much more variable a mean rating of five stars from 10 ratings is compared to a mean rating of four stars from 146 ratings. More confusion is caused by the fact that some web sites discard key information by rounding the average up or down.

In a survey experiment with Harvard undergraduates, Ho and Quinn showed that a graphical solution led to better information acquisition. The graphical display thereby potentially improves inferences that consumers draw about the quality of products and services from ratings data.

Free, easy-to-use software implementing Ho and Quinn's model and figures are available at http://cran.r-project.org/ as the "Ratings" package

[Note to editors: The complete TAS paper is available by contacting rosanne @ amstat.org.]

About the American Statistical Association
The American Statistical Association (ASA), a scientific and educational society founded in Boston in 1839, is the second oldest professional society in the United States. For more than 160 years, ASA has been providing its 18,000 members serving in academia, government, and industry and the public with up-to-date, useful information about statistics. The ASA has a proud tradition of service to statisticians, quantitative scientists, and users of statistics across a wealth of academic areas and applications. For additional information about the American Statistical Association, please visit the association's web site at http://www.amstat.org or call 703.684.1221.

####

Share article on social media or email:

View article via:

Pdf Print

Contact Author

Visit website