A four-week influenza activity forecast from FluNet
Suitability and evaluation
Retained for retrospective model-development research with documented timing limitations. Later evaluation rows expose earlier outcomes; predictions require restricted access.
Supported uses: Retrospective train/validation model-development experiments under the documented timing and selection limitations; Final evaluation only under a protocol that isolates holdout data from development and prevents later rows influencing earlier predictions
Not supported: Claims of verified real-time or operational forecasting validity; Treating development/validation scores as untouched final-test results; Forecasting scores from unrestricted cross-row access to this artifact
- Historical feature availability
- assumed
- Training-label timing
- unverified
- Data vintage
- current snapshot
- Row selection
- unverified
- Required evaluation access
- chronological
Cross-row audit: recovery demonstrated. Independent operator-commissioned audit; predictions frozen before labels were opened; holdouts sealed; two AUC implementations in exact agreement
The labeled eval split is development/validation data. Isolate final holdout data from development and prevent later rows influencing earlier predictions. Published data is not a secure hidden test service.
- join (country, W) -> (country, W+28 days): the next row's inf_all_w and denom_w are the operands of T's label
- The labelled holdout is presumed to carry the same self-answering structure by construction; it was never opened by the audit and must be treated as compromisable by the same join class.
- Historical feature/label availability at the declared instant is assumed from current snapshots, not verified against release vintages.
- The current scorer passes the complete feature frame to predict_proba; prediction-time isolation is instructed but not mechanically enforced, so any framework score on these files is gameable.
- Splits use prediction week W although labels concern W+4; no label-availability embargo is enforced across train/validation or validation/holdout boundaries.
Corrections to historical documentation
- Public site page, Hub card and repository README must state the restricted suitability once rendered; no frozen package bytes change.
Reviewed 2026-10-24. Applies to this artifact; original data and tags are preserved.
WHO publishes weekly influenza laboratory surveillance by country. The contribution here is not access: it is the frozen country-week panel, the declaration that the prediction instant is the end of week W, and the derived label - will positivity in week W+4 exceed 10%.
The release ships a reproducible baseline at eval AUC 0.8700. Two independent checks are published with it, because they decide what that number means. A calendar-only model reaches 0.6269, so the task is not merely seasonal. But week-W persistence alone reaches 0.8929 - a one-line rule beats the shipped baseline. That is stated on the card, not left to be discovered.
This is a retrospective, observed-outcome benchmark. It is not a prospective forecast, it is not calibrated, and the information cutoff rests on an assumption about publication timing that the publisher does not expose. The source's licence is unresolved: no open licence is asserted, and the terms are stated plainly.