Teams Complete Bureau of Reclamation's Sub-Seasonal Climate Forecast Rodeo — Outperforming the Baseline Forecasts
WASHINGTON (PRWEB) March 08, 2019 -- After a year of competitive forecasting, the Bureau of Reclamation has selected three winning teams to share $525,000 in the Sub-Seasonal Climate Forecast Rodeo, which sought to crowdsource advancements in the three-to-six-week sub-seasonal forecast timeframe. Each of the three teams demonstrated novel approaches and outperformed the baseline forecasts
"We are very excited with the outcome of this competition,” Reclamation’s Science Advisor David Raff, PhD said. “The innovative American public is demonstrating that we can improve sub-seasonal forecasts for temperature and precipitation which will allow water managers to better prepare for shifts in hydrologic regimes, such as the onset of drought or occurrence of wet weather extremes."
The Sub-Seasonal Climate Forecast Rodeo required teams to develop temperature and precipitation forecasts for three-to-four week and five-to-six-week periods every two weeks for a year. The teams also submitted an 11-year hind-cast for the same categories. The goal was to develop systems that perform better than the existing baseline forecasts. To be eligible for prizes the solvers also needed to provide their code for testing and supporting documentation.
Six teams completed the year-long portion of the competition and three ultimately met all criteria to qualify to win an award. Quantitative scoring of real-time forecasts and hind-casts were the primary mechanism for evaluating the solvers’ forecasting systems. In addition, a judging panel reviewed solver prepared documentation to determine if it met the requirements of the competition. The judging panel consisted of subject matter experts from NOAA Earth System Research Laboratory, NOAA Climate Prediction Center, California Department of Water Resources, and Reclamation.
Salient, a Massachusetts family team comprised of oceanographer Raymond Schmitt of Woods Hole and twin engineer sons Eric of Boston and Stephen of Cambridge, received a total prize of $250,000 for their forecasting of precipitation during both time frames. They focused on rainfall by incorporating new insights into the role of the ocean in the global water cycle. Their artificial intelligence model was trained on ocean data to make their forecasts, winning both first place awards and additional awards for their 11-year hind-casts.
The team StillLearning included Lester Mackey of Arlington, Massachusetts, Judah Cohen and Ernest Fraenkel of Newton, Massachusetts, and Jessica Hwang and Paulo Orenstein of Stanford, California. They were awarded $200,000 for their first-place performance in forecasting temperature during weeks three-to-four, second place forecasting precipitation during weeks five-to-six, and additional awards for their 11-year temperature hind-casts. They combined two methods to produce their forecasts. Their primary approach was to use a backwards stepwise local linear regression with feature selection shared across grid points using a diverse pool of predictor datasets, which included meteorological observations, sub-seasonal forecasts from numerical weather models, and climate indices. They also employed an analog forecast technique when periods from previous years were sufficiently similar to the conditions preceding issuing a forecast.
The Lupoa13 team, comprised of Anthony Lupo and Joseph Renken of Columbia, Missouri, and Joshua Herman of Oshkosh, Wisconsin, used global pressure anomalies to identify analog periods from the historical record. Precipitation values from four analog periods were blended together to produce their forecasts. They were awarded $75,000 based on their method’s second and third place performances in forecasting precipitation for the three-to-four week and five-to-six-week periods, respectively.
In addition to these teams, two other teams outperformed the benchmarks over the 12-month competition. Team bgzimmerman had the highest average score in each of the temperature categories and was the only team to beat the benchmarks in the five-to-six-week temperature category. Team PRXWX outperformed the benchmarks in all categories except for five-to-six-week temperature category. Rather than seek prize purses in exchange for their forecast methods, these teams elected to withdraw from the competition and formed a partnership to commercialize their techniques. Team bgzimmerman is Brian Zimmerman and Team PRXWX was led by Richard James of Prescient Weather Ltd. Mr. Zimmerman’s solution is being integrated into Prescient Weather’s commercial long-range forecasting system, called the World Climate Service.
This partnership between competitors highlights the benefits of prize competitions and spurring innovation and relationships that would not have been otherwise formed. This also allowed for direct commercialization of a research methodology which is another a positive outcome of these competitions.
Reclamation is planning to launch Sub-Seasonal Forecast Rodeo II this summer to further improve sub-seasonal forecast skill and engage an even broader community of solvers.
Reclamation collaborated with the National Oceanic and Atmospheric Administration Earth System Research Laboratory in Boulder, Colorado, NOAA's Climate Prediction Center in College Park, Maryland, and California Department of Water Resources to design and judge this challenge. In addition, the U.S. Geological Survey and U.S. Army Corps of Engineers contributed subject matter experts to review and assist with competition design. Learn more by visiting https://www.usbr.gov/research/challenges/forecastrodeo.html.
To learn more about this and other prize competitions at Reclamation, please visit Reclamation's Water Prize Competition Center at http://www.usbr.gov/research/challenges.
Peter Soeth, Bureau of Reclamation, https://www.usbr.gov, +1 (303) 445-3615, [email protected]
Share this article