Datafiniti Relaunches 80legs with Major Platform Update to Make Web Crawling Easier and Faster
Austin, TX (PRWEB) October 01, 2014 -- Datafiniti, the first search engine for data, today announced a major platform update to 80legs, a service platform for web crawling and processing Web content. The relaunch makes 80legs simpler and easier to use, complete with a new back-end design that helps the platform crawl with increased power, speed, and reliability.
The updated 80legs service offers a number of new features that improve a company’s ability to gather and process data effectively and efficiently. 80legs can crawl an unlimited number of URLs a month to meet user demands and auto-scales crawls to eliminate delays in processing jobs. With the new architecture, crawling speed for any single website has increased 10x, which means customers could use 80legs to scrape the homepage of every .com domain in 6 days (113 million pages). The update comes after Datafiniti took over the product last year. Since then, the Datafiniti team has been working to improve the power and sale of the web crawling platform.
Since the 80legs update, Datafiniti has grown its 80legs customer base by over 30%. 80legs is currently used by companies such as Acxiom, HP, Integral Ads, and others for reviews management, competitive intelligence, social media monitoring, ad and pricing optimization, fraud detection, predictive modeling, and more. MailChimp uses 80legs to capture email address on every website for spam prevention, and PayPal uses 80legs to identify fraudulent uses of their platform. In addition, Datafiniti has grown its team by 15% to support the increasing customer growth.
“It’s an exciting time for Datafiniti, and we’re proud with what we’ve accomplished with the 80legs relaunch,” said Shion Deysarkar, CEO, Datafiniti. “The new 80legs platform is more powerful and even more simple to use. We can give any company the ability quickly browse and index data from the entire Internet, as the 80legs grid model allows essentially unbounded web crawling with a few clicks.”
As the quantity of information available on the Web continues to grow, collecting and processing that information is an increasingly difficult and expensive task. 80legs powers companies from a wide variety of industries to leverage data to drive decision-making, improve customer experience, open up new revenue streams, gain a competitive edge, without the high price or infrastructure investment.
The new 80legs features include:
- Redesigned back-end architecture growls the 80legs crawl grid from 50,000 to over 70,000 unique computers and virtually an unlimited number of crawls per month.
- Improved speed and reliability. The maximum crawl speed grew from 1B URLs/month to 10B URLs/month.
- Simple web interface and heavy customization available with JavaScript applications.
- RESTful API that provides users with the information needed to interact with 80legs.
- Automatic rate-limiting that prevents any website from traffic overload and web crawls from being blocked, resulting in a 100% reduction from websites being overwhelmed.
- New user-agent for the 80legs crawl, from "008" to "voltron" -- the new crawler gives users the chance to crawl websites previously inaccessible to the old crawler.
80legs offers free and paid service plans designed to fit the needs of companies of all sizes. For more information about pricing and features, and to contact the 80legs team, please visit http://www.80legs.com.
About Datafiniti
Datafiniti is the first search engine for data. It combines the familiarity of a search engine with the breadth of the Web’s data, creating custom data sets generated in real-time to match user queries. It is the simplest way to search and access all types of data, ranging from identity data, to web page data, to news, to products to ratings and more.
Datafiniti is based in Austin, TX. For more information visit http://www.datafiniti.net or follow Datafiniti on Twitter.
About 80legs
80legs is a service platform for web crawling and processing Web content. By harnessing a unique distributed architecture of over 70,000 computers, 80legs delivers exceptional crawling and processing capabilities at incredibly low costs. The service is completely customizable and easy to use, providing any company, independent of size, access to the crawling and processing power previously reserved for only the largest infrastructures.
80legs is brought to you by the Datafiniti team, and powers Datafiniti’s search engine for data. For more information about 80legs, visit http://80legs.com.
Nate Gandomi, Jones-Dilworth, Inc., +1 510-999-6283, [email protected]
Share this article