Posted by Vector and Vector-borne Disease Committee
January 19, 2023
Evaluation of an open forecasting challenge to assess skill of West Nile virus neuroinvasive disease prediction.
Holcomb, Karen M., et al. Centers for Disease Control, Email: kholcolb@cdc.gov
- Parasites & Vectors https://doi.org/10.1186/s13071-022-05630-y
Condensed abstract:
This study used forecasting models submitted to the 2020 WNV Forecasting Challenge, an open challenge organized by the Centers for Disease Control and Prevention, to assess the status of WNV neuroinvasive disease (WNND) prediction and identify avenues for improvement. Researchers performed a multi-model comparative assessment of probabilistic forecasts submitted by 15 teams for annual WNND cases in US counties for 2020 and assessed forecast accuracy, calibration, and discriminatory power. Results showed that simple models based on historical WNND cases generally scored better than more complex models and combined higher discriminatory power with better calibration of uncertainty. Among models using additional data, inclusion of climate or human demographic data was associated with higher skill, while inclusion of mosquito or land use data was associated with lower skill. Although opportunities might exist to specifically improve predictions for areas with large populations and low or high winter temperatures, areas with high case-count variability are intrinsically more difficult to predict. Further improvements to prediction could be obtained with improved calibration of forecast uncertainty and access to real-time data streams (e.g., current weather and preliminary human cases).
Note: This study presents an analysis of a challenge to predict the number of cases of WNV neuroinvasive disease in 2020 across all US counties and discusses the difficulties of creating a nationwide model.