Dataset Factory★ EST. 2026 · BEST AT 800×600 ★
★ WELCOME TO THE DATASET FACTORY ★ PUBLIC DATA → CHECKED BUILDS → HONEST BASELINES ★ NO POP-UPS ★ NO COOKIES ★ NO TRACKERS ★ HAND-MADE HTML ★ UNDER CONSTRUCTION SINCE THE NINETIES ★
Release notes · 2026-10-20 · announcement

A citation-prediction task from Chicago's inspection record

Suitability and evaluation

Retained for retrospective model-development research with documented timing limitations. Later evaluation rows expose earlier outcomes; predictions require restricted access.

Supported uses: Retrospective train/validation model-development experiments under the documented timing and selection limitations; Final evaluation only under a protocol that isolates holdout data from development and prevents later rows influencing earlier predictions

Not supported: Claims of verified real-time or operational forecasting validity; Treating development/validation scores as untouched final-test results; Forecasting scores from unrestricted cross-row access to this artifact

Historical feature availability
assumed
Training-label timing
unverified
Data vintage
current snapshot
Row selection
unverified
Required evaluation access
chronological

Cross-row audit: recovery demonstrated. Independent operator-commissioned audit; predictions frozen before labels were opened; holdouts sealed; two AUC implementations in exact agreement

The labeled eval split is development/validation data. Isolate final holdout data from development and prevent later rows influencing earlier predictions. Published data is not a secure hidden test service.

  • reverse-history join through days_since_previous: a later inspection S with S.inspection_date - S.days_since_previous = T.inspection_date ships prev_target_cited, which is T's label
  • The labelled holdout is presumed to carry the same self-answering structure by construction; it was never opened by the audit and must be treated as compromisable by the same join class.
  • Historical feature/label availability at the declared instant is assumed from current snapshots, not verified against release vintages.
  • The current scorer passes the complete feature frame to predict_proba; prediction-time isolation is instructed but not mechanically enforced, so any framework score on these files is gameable.
  • Strictly earlier inspection dates exclude same-day history, but recorded inspection dates do not establish publication or outcome-availability times.

Corrections to historical documentation

  • Public site page, Hub card and repository README must state the restricted suitability once rendered; no frozen package bytes change.

Reviewed 2026-10-24. Applies to this artifact; original data and tags are preserved.

The City of Chicago publishes every food inspection it conducts, including the code sections cited. Access is not the contribution. The contribution is the frozen panel - 226,417 inspections split 156,841 train / 35,358 eval / 34,218 holdout - the declaration that the prediction instant is the moment the inspection begins, and features computed only from that establishment's strictly earlier history.

The release carries 40 task instances, one per cited code section, delivered as separate configs. Each config asks the same question with its own label: will this inspection cite section X? Prevalence varies enormously by section, from 0.15% to 56.1%, so accuracy is meaningless here and AUC is the only honest score.

The numbers ship with their floor. Across the 40 configs the eval AUC runs 0.5187 to 0.8292, median 0.7208. On 38 of 40 configs the model beats a persistence reference - yesterday's presence of the same code - by +0.028 to +0.247 AUC; on two it does not, and those two are recorded rather than hidden. One config, code_38, was reproduced three times independently (0.6326) including from the published bytes in a fresh environment with no factory access.

→ Open the dataset page for numbers, files and checksums

★ THANKS FOR VISITING · BOOKMARK THIS PAGE · TELL A FRIEND ABOUT THE DATASET FACTORY · SIGN THE GUESTBOOK BEFORE YOU LEAVE ★