A retrospective direction task from Australia’s arrivals panel
Suitability and evaluation
Retained for retrospective model-development research with documented timing limitations. Later evaluation rows expose earlier outcomes; predictions require restricted access.
Supported uses: Retrospective train/validation model-development experiments under the documented timing and selection limitations; Final evaluation only under a protocol that isolates holdout data from development and prevents later rows influencing earlier predictions
Not supported: Claims of verified real-time or operational forecasting validity; Treating development/validation scores as untouched final-test results; Forecasting scores from unrestricted cross-row access to this artifact
- Historical feature availability
- assumed
- Training-label timing
- unverified
- Data vintage
- current snapshot
- Row selection
- unverified
- Required evaluation access
- chronological
Cross-row audit: recovery demonstrated. Independent operator-commissioned audit; predictions frozen before labels were opened; holdouts sealed; two AUC implementations in exact agreement
The labeled eval split is development/validation data. Isolate final holdout data from development and prevent later rows influencing earlier predictions. Published data is not a secure hidden test service.
- eval join (country, T) -> (country, T+1): the next row's val_m and val_m12 are the two operands of T's label
- The labelled holdout is presumed to carry the same self-answering structure by construction; it was never opened by the audit and must be treated as compromisable by the same join class.
- Historical feature/label availability at the declared instant is assumed from current snapshots, not verified against release vintages.
- The current scorer passes the complete feature frame to predict_proba; prediction-time isolation is instructed but not mechanically enforced, so any framework score on these files is gameable.
Corrections to historical documentation
- Public site page, Hub card and repository README must state the restricted suitability once rendered; no frozen package bytes change.
Reviewed 2026-10-24. Applies to this artifact; original data and tags are preserved.
The ABS releases arrivals statistics after their reference month. This task estimates the direction of the first unpublished month at the prior month’s release date. That target month may already have ended; this is not a next-calendar-month forecast issued before the month begins. Historical values come from a current revised snapshot, so exact as-of availability is assumed.
The question is a shortfall: will arrivals in the first unpublished month fall below the same month a year earlier. Positive rates run 35% / 29% / 20%, and the pandemic window is excluded rather than allowed to dominate.
Persistence is anti-correlated - repeating last year's direction scores 0.3647 - so this is mean-reversion and seasonality, not momentum. The runner-contract baseline reaches 0.6219 eval AUC, clearing the calendar-only rule at 0.5501 by 0.0718.