Finding Big Fish Online: How Wild Me Is Turning Vacation Videos Into Shark Science with Artificial Intelligence
Portland, Oregon (PRWEB) May 18, 2017 -- Since 2003, a global community of shark researchers and divers has united in studying the world’s biggest fish: the endangered whale shark (Rhincodon typus). Their global research platform—Wildbook for Whale Sharks (https://www.whaleshark.org)—has combined over 120 researchers and volunteers with 5000+ citizen scientists into a global community focused on studying the population size and migrations of this rare and gentle giant. Thought to live 60 to 120 years and grow up to 60 feet (20 m) in length, whale sharks are tracked by combining human data collection with computer vision, allowing each animal to be individually identified by the unique pattern of spots on its body. Divers and snorkelers have been serving as “citizen scientists” for years, contributing their whale shark photos to whaleshark.org, where each animal receives its own page of data, much like a Facebook profile. For example, “JimmiXS” is a whale shark that scientists and divers have joined together in tracking between Mexico, Belize, and the United States. See the profile page for JimmiXS on whaleshark.org.
This successful research community now has a new member: a machine learning solution called “Wildbook A.I.” Based on a deep convolutional neural network (a form of artificial intelligence) developed at Rensselaer Polytechnic Institute, Wildbook A.I. quietly reviews videos on YouTube, looking for the telltale spot patterns of whale sharks and adding the extracted photos, dates, and locations to Wildbook.
Jason Holmberg, a director at Wild Me and co-founder of whaleshark.org, provided the inspiration for the project. “A surprising number of vacation videos that include whale sharks are posted to YouTube every day. I spent two years looking through the vacation videos of divers and snorkelers in the Philippines, Maldives, Mexico, and many other locations. The volume of whale shark sightings and identifications that I created by data-mining social media in my spare time quickly eclipsed that of my colleagues in the field. But it was a very manual process, and it was a great opportunity to push the boundaries of wildlife research and computer science. Wild Me partnered with Rensselaer Polytechnic Institute and the University of Illinois-Chicago on a National Science Foundation grant to explore how much wildlife data could be extracted from social media. For whale sharks, it turned out to be a lot.”
Jon Van Oast, lead software developer for Wild Me, took on the task of building the software to find and identify whale sharks on YouTube, turning vacation videos into a wildlife data source. “My hope is to use technology to reduce the work involved in finding and processing potentially useful data, with the goal of leaving people free to have fun taking photos or do their research.”
Jon was joined by Jason Parham, a student at Rensselaer Polytechnic Institute in the lab of Dr Chuck Stewart. Jason designed and trained the neural network to find whale sharks in these videos from different poses and angles. The A.I. can find frames of the video with whale sharks and intelligently group them. Wild Me then added natural language–processing software to try to determine where and when the whale sharks were sighted based on YouTube comments. The resulting structured data are then fed back to the research community for further analysis, augmenting the breadth and depth of data available for migration and population analysis. The team’s collaborative work is being presented this week at the 2017 Citizen Science Association Conference in St. Paul, Minnesota.
Dr Alistair Dove, Vice President of Research and Conservation at Georgia Aquarium in Atlanta, said, “As a whale shark researcher, the power of the Wildbook technology to make connections for us and grow our understanding of this species is invaluable. When you add in the citizen science component, then whale shark conservation becomes something that everyone can be a part of, which is so important for this majestic but endangered species.”
According to Jason Holmberg, “One of the limitations of citizen science has always been outreach. People have to know about your project to participate. And if you’re successful in outreach, then your project has to effortlessly scale to incorporate the flood of data that can be gathered. By sending A.I. out into social media, we have the potential to get a lot of wildlife data that we would normally miss. And in the future we want to report back on what we find and interact with users who have no idea that their vacation videos have scientific value and contain rare and unique animals that scientists are tracking around the globe. A.I. and computer vision provide the technology foundation for the citizen science projects of the future. And it’s truly amazing that our whale shark research community now has a full-time member who isn’t human. We will no doubt increasingly rely on A.I. to help us understand and protect wildlife.”
You can learn more about cutting edge science for whale sharks at http://www.whaleshark.org and even adopt and follow the individual sharks in our study.
Photos
High resolution whale shark photos for use in publication can be found at:
https://www.whaleshark.org/contactus.jsp
About Wildbook for Whale Sharks
The Wildbook for Whale Sharks photo-identification library is a visual database of whale shark (Rhincodon typus) encounters and of individually catalogued whale sharks. The library is maintained and used by marine biologists to collect and analyze whale shark sighting data to learn more about these amazing creatures.
The Wildbook uses photographs of each shark’s unique skin patterning behind its gills and its scars to distinguish between individual animals. Cutting-edge software supports rapid identification using pattern recognition and photo management tools. You, too, can assist with whale shark research by submitting photos and sighting data. The information you submit will be used in population studies to help with the global conservation of this threatened species.
http://www.whaleshark.org info(at)whaleshark(dot)org
About Wild Me
Whaleshark.org is developed by the Portland, Oregon–based nonprofit Wild Me as part of the joint Wildbook™ project, which includes biologists, data scientists, and computer vision researchers at Princeton University (Dr Dan Rubenstein), University of Illinois at Chicago (Dr Tanya Berger-Wolf), and Rensselaer Polytechnic Institute (Dr Chuck Stewart).
Wildbook is the first multi-disciplinary, multi-institution project to blend artificial intelligence, computer vision, and citizen science for wildlife research. Wildbook has been developed with the support of the National Science Foundation and Amazon’s AWS Cloud Credits for Research Program.
@IBEIS_org http://www.facebook.com/wildmeorg
Jon Van Oast, Wild Me, http://www.whaleshark.org, +1 503-662-2170, [email protected]
Share this article