ChaCha Proves Best in Butler Study of Mobile Search Engines, Question and Answer Platforms

Share Article

Bests 10 competitors, including Google, Siri, and Yahoo Answers

News Image

A new study by the Butler Business Accelerator found that ChaCha answered questions better than 10 other mobile search engines and question-and-answer platforms, including Ask.com, Answers.com, Google, Quora, Siri, and Yahoo Answers.

The Butler University Q&A Intelligence Index measured the likelihood that a user could expect to receive a correct answer in a timely manner to any random query using natural language. ChaCha’s mobile application for the Apple iPhone outperformed all the services tested, garnering the highest score on the Butler University Q&A Intelligence Index by responding to 99 percent of the study’s 180 randomly selected questions with the highest degree of accuracy.

The rankings are in the accompanying chart. (*Mean accuracy of responses originally graded on a five-point scale.)

The questions were asked via mobile services and randomized to cover both popular and less common questions. The coverage score measured whether an answer was returned within a three-minute window and assessed the first non-sponsored search result in Q&A platforms. The accuracy score measured how correct answers were in relation to an objective third party’s answer key that accounted for logic and bias.

Questions asked were in the areas of advice (“What if a girl doesn’t want to talk to you?”), objective (“What are the 10 most common names?”), and subjective (“Who would win in a fight, The Hulk or Superman?”).

“ChaCha delivered the highest quality responses consistently across the largest group of categories and question types,” said Trent Ritzenthaler, operating director of the Butler Business Accelerator.

Other findings of the study:

-Ask.com performed better than all others in the single category of questions labeled objective/temporal, such as “When does summer end?”

-Quora was proficient at answering difficult questions that require expert and extensive explanations, but it was generally unable to deliver answers within three minutes for most information searches on mobile devices. Quora answered only 24 percent of the questions at all, and often the match found did not include a viable answer.

-Siri accurately answered only 37.5 percent of the questions posed, but Siri’s biggest strengths are considered to be in local discovery and operating system commands, which were not highly represented in Butler’s study of more mainstream questions.

-Google’s response rate was 100 percent, but the first non-sponsored result on the search results page—which often times was not fully visible as an organic search result on the presented page on a mobile device—only presented an accurate answer about 50 percent of the time.

“Through our extensive study, ChaCha showed the highest level of performance,” Ritzenthaler said. “As the market matures, we anticipate that ChaCha and other mobile search companies will lift the standard of natural language processing and semantic understanding to enhance the user experience with mobile Q&A.”

For further information on the research methodology and analysis of test data, visit http://www.butler.edu/q-a-study/.

***

About the Butler Business Accelerator
The Butler Business Accelerator (BBA) is an extension of the Butler University College of Business. The BBA’s experienced professionals leverage the University’s resources to help solve business challenges. From strategy to marketing, sales, operations, and financial management, the BBA’s network of subject matter experts works alongside clients to build solutions that have immediate and long-term results.

About Butler University
Challenging and enabling students to meet their personal and professional goals has guided Butler University since 1855. Today, Butler is a nationally recognized comprehensive university that blends the liberal arts with first-rate pre-professional programs. It seeks to prepare each graduate not simply to make a living but to make a life of purpose, in which personal flourishing is intertwined with the welfare of others. Butler is known for its vibrant campus, superior academics and dedicated faculty. The University enrolls more than 4,700 undergraduate and graduate students in six academic colleges: Arts, Business, Communication, Education, Liberal Arts and Sciences, and Pharmacy and Health Sciences. Located just six miles from downtown Indianapolis, Butler’s urban setting affords students internship opportunities that provide excellent graduate school and career preparation.

Experts Guide: To find the ideal expert for your story, visit our Experts Guide at http://www.butler.edu/experts/

Contact: Marc Allan
Office Phone (317) 940-9822
Cell Phone (317) 501-7999
Email mallan(at)butler(dot)edu

Share article on social media or email:

View article via:

Pdf Print

Contact Author

Marc Allan
Butler University
(317) 940-9822
Email >