Acobot’s AI Built Another AI that Outperforms Human Made Chatbot
(PRWEB) May 07, 2018 -- Acobot LLC, the AI startup that recently published acobot.ai as the artificial intelligence platform for conversational AI, benchmarked two conversational AI chatbots built as virtual agents for one and the same website. One chatbot was made by Acobot's artificial intelligence, the other was made by a highly experienced human developer. The test results show a better performance of AI made virtual agent regarding the accuracy of answering customer questions.
The website used for the benchmark is the AI’s home page, https://acobot.ai. The human player, an engineer with over 10 years of experience in natural language processing and chatbot building, read the web pages and manually populated a “brain,” the knowledge base for the chatbot. With the aid of conversational AI aPaaS, BrainShop.ai, another product of Acobot, the engineer completed her chatbot in 40 work hours, the maximum time set for this test, whereas the AI, using BrainShop.ai, too, completed the job in 9 seconds.
An A/B test was then implemented. Website visitors were randomly split into two groups. The first group were served by human made chatbot and the second group, the one built by AI. When each chatbot finished delivering 500 chat sessions their performances were assessed respectively by a panel of three engineers. They reviewed all chat logs without knowing the maker and calculated accuracy rate by identifying two types of error the chatbots made: false positive, which means giving false answers to questions uncovered by the website; false negative, giving no answer to questions covered by the website.
The outcome is revealing. Though human made chatbot performed moderately better with a slightly lower rate of making the first type error, AI made chatbot outplayed it's rival by making 40% less of the second type error. As a whole AI made chatbot exceled itself.
“The advantage of AI in speed is obvious. That’s not a surprise.” Explained Vic Duan, Acobot founder and CEO. “What counts more is accuracy, which has remained low for a long time and prevented businesses from adopting chatbots for customer services automation. Now AI has brought it to a level higher than ever. The chatbots are ready to work. ”
According to Duan, humans still stay ahead of AI in reading and comprehending free-form text. They are doing especially well in distilling the points conveyed through contexts or based on common senses. It is the limited psychological endurance, however, that constrains humans from doing repetitious and long-time work as well as AI. Taking the engineer took part in the test as an example, despite her initial perfect working state, an increasing ratio of omitting the points covered by the website, induced mostly by psychological tiredness, crept into her later stage of working. Another human weakness is the subjective bias in predicting customer questions. The engineer omitted certain questions simply because she believed in the minimal likelihood of those questions being asked. In contrast, AI never gets tired. AI never leaves out any contents it extracts from the website no matter what is the frequency of their being inquired by customers. These enable AI to build a better chatbot even it is not so capable in comprehending the text.
BrainShop, the conversational AI aPaaS used for the above test, is available at http://BrainShop.ai. It’s designed for enterprises to build their own advanced conversational applications. The AI made chatbot is live at https://acobot.ai, answering questions about Acobot. With its help website owners can get their own free chatbots to engage web users and increase results from the websites.
Vic Duan, Acobot, http://acobot.com, +86 13601276247, [email protected]
Share this article