Elasticsearch 'Learning to Rank' Released, Bringing Open Source AI to Search Teams

Share Article

OpenSource Connections, Snagajob, and Wikimedia Foundation bring cutting edge open source ‘cognitive search’ techniques in Elasticsearch to push past the toughest search relevance challenges.

News Image
Learning to Rank increased our search relevance 20-40% across Wikipedia.

Search experts at OpenSource Connections, the Wikimedia Foundation, and Snagajob, deliver open source cognitive search capabilities to the Elasticsearch community. The open source Learning to Rank plugin allows organizations to control search relevance ranking with machine learning. The plugin is currently delivering search results at Wikipedia and Snagajob, providing significant search quality improvements over legacy solutions.

Learning to Rank lets organizations:

  • Directly optimize sales, conversions and user satisfaction in search
  • Personalize search for users
  • Drive deeper insights from a knowledge base
  • Customize ranking down for complex nuance
  • Avoid the sticker shock & lock-in of a proprietary "cognitive search" product

“Our mission is to empower search teams. This plugin gives teams deep control of ranking, allowing machine learning models to be directly deployed to the search engine for relevance ranking” said Doug Turnbull, author of Relevant Search and CTO, OpenSource Connections.

“We see clients struggle with 'mystical' proprietary cognitive search and machine learning infrastructure ” said Eric Pugh, CEO of OpenSource Connections. "This release is another example of how we aim to democratize these domains. There's more on the horizon!”

OpenSource Connections collaborated with Snagajob Engineering and the Wikimedia Foundation to develop the plugin for Elasticsearch.

Snagajob, the #1 hourly job marketplace, uses the plugin to deliver personalized jobs to job seekers.

"Matching our workers with the ideal job is a challenging problem. Matching mixes text search, geography, marketplace, and personalization factors" said Jason Kowalewski, Senior Director of Engineering at Snagajob. "Learning to Rank lets us carefully balance these factors and directly maximize matched jobs for the workers on our platform. Since deploying learning to rank, we’ve seen a net 32% increase in conversion metrics across our historically lowest performing use-cases."

The Wikimedia Foundation deployed the plugin to Wikipedia, one of the most visited sites on the Web, and uses the plugin to search Wikipedia’s extensive and diverse knowledge base.

"Learning to Rank increased our search relevance 20-40% across Wikipedia," said Erik Bernhardson Senior Software Engineer on the Wikimedia Foundation’s search platform team. "This sets up our team to continue to innovate in delivering insights to our users.” Learn more in Erik's interview about the plugin.

In addition to the plugin itself, OpenSource Connections created a comprehensive learning to rank guide to help teams explore the technology, demonstrating their desire to bring this information down to earth.

“Organizations need increasingly deep control of how they answer user’s questions. Topics like learning to rank can’t stay hidden away in obscure academic journals or multi-million dollar products” said Turnbull “We’re making this stuff accessible to all search teams: opening up the black box for everyone to use”.

“We feel a little bit like we’re stealing fire from Zeus... And this is just the beginning!”

Getting the Plugin
Learn more about getting started with the Elasticsearch Learning to Rank at http://elasticsearch-learning-to-rank.readthedocs.io

About OpenSource Connections
OpenSource Connections (OSC) empowers the world's best search teams. OSC helps organizations with their toughest search, relevance, and personalization problems. OSC's clients across industries and include the Wikimedia Foundation, O'Reilly Media, the US Patent and Trademark Office, Snagajob, Zappos, and many others.

About Wikimedia Foundation
The Wikimedia Foundation is the non-profit organization that supports and operates Wikipedia and its sister free knowledge projects. Wikipedia is the world’s free knowledge resource, spanning more than 45 million articles across nearly 300 languages. Every month, more than 200,000 people edit Wikipedia and the Wikimedia projects, collectively creating and improving knowledge that is accessed by more than 1 billion unique devices every month. This all makes Wikipedia one of the most popular web properties in the world. Based in San Francisco, California, the Wikimedia Foundation is a 501(c)(3) charity that is funded primarily through donations and grants.

About Snagajob
With more than 80 million registered hourly workers and 300,000 employer locations, Snagajob is America’s #1 hourly marketplace. We focus on creating instant and quality connections so workers can get jobs and employers can hire workers—in minutes. Since 2000, our mission has been to put people in right fit-positions so they can maximize their potential and live more fulfilling lives.

Contact OSC to evaluate the plugin for your problems:
Eric Pugh, epugh@opensourceconnections.com, 434-466-1467

Share article on social media or email:

View article via:

Pdf Print

Contact Author

Eric Pugh
OpenSource Connections

Visit website