Dataset Factory★ EST. 2026 · BEST AT 800×600 ★
★ WELCOME TO THE DATASET FACTORY ★ PUBLIC DATA → CHECKED BUILDS → HONEST BASELINES ★ NO POP-UPS ★ NO COOKIES ★ NO TRACKERS ★ HAND-MADE HTML ★ UNDER CONSTRUCTION SINCE THE NINETIES ★
Release notes · 2026-10-31 · announcement

Will a planning appeal be allowed? Validation-time features, decided cases only

Suitability and evaluation

Retained for retrospective model-development research. Each row is one decided s78_core planning-permission-family appeal whose features are its own registration attributes plus LPA prior-outcome history computed strictly from decisions earlier than the row's validation instant; the prediction instant is the validation event (Valid Date, event_time) and no point-in-time or operational claim is made. The publisher serves a rolling database (pinned here at the 2026-10-22 edition) and no historical vintages, so training-label timing and full-row historical availability are unverified and none is claimed. The evaluation contract requires isolated rows: the LPA-history features embed other rows' labels (lpa_last_outcome is the previous same-LPA decided case's label), a declared and probed cross-row exposure, while the shipped illustrative runner passes the whole eval feature table to the predictor - unrestricted-access scores overstate honest performance and the requirement is a requirement, never an enforcement claim. Measured finding carried beside every figure: the stock runner baseline (eval AUC 0.6108) exceeds every measured simple reference on the identical 28,104 eval rows - procedure-type rate 0.5551 (+0.0558, row CI95 [0.0467, 0.0626], LPA-cluster CI95 [0.0462, 0.0654]), LPA history rate 0.5414 (+0.0694, [0.0622, 0.0766] / [0.0597, 0.0792]), shipped rate feature 0.5422 (+0.0686, [0.0619, 0.0756] / [0.0596, 0.0800]) - every paired interval excluding zero under both the row-level and the LPA-clustered protocol: demonstrated headroom over these references under this protocol, with its scope recorded (one model configuration, no re-tuning; two validation years of eval coverage). Persistence is measured at 0.5045 (92%+ of eval rows' LPAs have train-era history; the -1 sentinel marks the rest). The calendar-identity control collapsed onto the LPA rate exactly (eval years have no train-era valid-year support) and is recorded as a degenerate control, never as independent confirmation.

Supported uses: Retrospective research and method comparison on the pinned 2026-10-22 edition, predicting row-wise from the shipped validation-instant columns under an isolated-rows protocol; Agent evaluation with the labeled eval split as development/validation data and the holdout kept outside agent development until a design is frozen; Study of LPA-level and procedure-level prevalence and of which validation-instant attributes carry outcome signal

Not supported: Operational or live validation-instant screening claims: no historical operational-availability evidence exists and none is claimed; Point-in-time historical reconstruction: the publisher serves a rolling current vintage; a later re-fetch is a different vintage, not what was knowable earlier; Quoting the model AUC (0.6108) without the reference comparison (procedure-type 0.5551, LPA rate 0.5414, shipped rate 0.5422, persistence 0.5045, with the paired row-level and LPA-clustered intervals) beside it; Scoring the eval split with the whole feature table and presenting the result as an isolated-rows figure: the LPA-history features make cross-row label recovery measurably possible, so unrestricted-access scores overstate honest performance; Treating the split as certified decision-time separation: the split key is the validation instant and slow cases put some training labels after the eval window's earliest instants, so label timing is unverified

Historical feature availability
unverified
Training-label timing
unverified
Data vintage
current snapshot
Row selection
outcome conditioned
Required evaluation access
isolated rows

Cross-row audit: recovery demonstrated. Gate 1.5.0's own probe over the full artifact (check contract.cross_row_probe_0, job pins-planning-002): the within-LPA donor join covers 36,643 train / 21,593 eval / 43,747 holdout rows, and on the covered rows the later row's lpa_last_outcome feature is an earlier shipped row's label verbatim exactly when that previous case was already decided before the donor's instant (measured mismatches 15,428 / 8,803 / 18,430 are rows whose previous case was still pending at the donor's instant or where another same-LPA case was decided in between; 12,705 / 6,085 / 12,143 ambiguous same-day keys). The exposure is inherent to any prior-outcome feature and unmitigated at runtime by the shipped contract, which passes the whole eval feature table to the predictor: quality.json declares isolated_rows as a REQUIREMENT for honest evaluation and claims no enforcement. Any evaluation must present one row at a time to the solver; unrestricted-access AUCs are not comparable to isolated-rows AUCs and must not be presented as such.

The labeled eval split is development/validation data. Isolate final holdout data from development and prevent later rows influencing earlier predictions. Published data is not a secure hidden test service.

  • Single current-snapshot vintage (the 2026-10-22 edition, sha256-pinned for both source files); the publisher republishes the rolling database monthly-to-quarterly and retains no historical vintages, so no historical operational-availability claim is possible
  • Training-label timing and full-row historical availability are unverified; the split key is the validation instant because the outcome date cannot ship, so the task is retrospective research only
  • All publisher-sourced columns are current-snapshot vintage: values are as revised in the pinned edition, not as first published at past instants
  • The population is the publisher's own decided-cases-only extract (in progress, withdrawn and turned away cases are absent) further restricted to binary decisions - outcome-observability conditioning recorded as a limitation, never resolved by relabelling
  • The cross-row exposure over LPA-history features is declared and requires an isolated-rows protocol; the shipped runner does not enforce it
  • The headroom finding's scope: one model configuration with no re-tuning, two validation years of eval coverage; the calendar-identity control was degenerate (collapsed onto the LPA rate), so seasonality/identity headroom is not independently established; the model's per-validation-year AUC and Brier were not computed; beating the measured references does not enumerate all simple rules and does not prove a causal signal
  • The holdout was never evaluated; no holdout figure exists and its first use must follow a frozen design

Reviewed 2026-10-31. Applies to this artifact; original data and tags are preserved.

The UK Planning Inspectorate publishes its Casework Database as open data on GOV.UK (Open Government Licence v3.0), as a rolling database republished monthly-to-quarterly since March 2018. The contribution here is not access: it is the sha256-pinned 2026-10-22 source snapshot, the derived binary label (Allowed vs Dismissed), the decided-cases eligibility frame with counted exclusions (2,757 non-binary decisions, 3,110 counted cross-file duplicates, 2 decided-before-valid anomalies), fixed disjoint temporal splits on the validation instant, and a scored, reproducible contract. 139,264 rows: train 54,834 (17,048 positives), eval 28,104 (8,469), and a labeled holdout 56,326 (17,705) that ships but is never scored in development.

The packaged baseline is illustrative, and the comparisons decide what the number means. The stock runner-contract baseline reaches eval AUC 0.6108 on the 28,104 eval rows; on identical rows the simple references reach: procedure-type historical allow rate 0.5551, LPA historical allow rate 0.5414, persistence (the last same-LPA decided outcome) 0.5045, with majority-class accuracy 0.6987 on this split. The calendar-identity control collapsed exactly onto the LPA rate (eval validation years have no train-era valid-year support) and is recorded as degenerate, never as independent confirmation; year-rate and global-rate constants carry null AUC and are recorded as degenerate too. Per-validation-year eval AUCs are reported (2020: 13,797 rows / 4,126 positives; 2021: 14,307 / 4,343) and reference differences carry an LPA-clustered bootstrap (150 resamples). Under this protocol the stock model exceeds every measured simple reference on the same rows.

Honest framing, measured rather than asserted. The prediction instant is the validation event (Valid Date, present on 100% of the 186,785 measured decided rows; the measured valid-to-decision lag is 91-365 days for 74% of cases), and every feature is recorded at or before registration. The population is decided-cases-only by publisher design - in-progress, withdrawn and turned-away cases never enter the source - so the label conditions on reaching a decision and no claim is made about appeal success before a decision exists. The publisher serves one revised-in-place current snapshot with no historical vintages, so training-label timing and historical availability are unverified and none is claimed. The split key is the validation instant because the outcome date cannot ship, so slow cases put some training labels after the eval window's earliest instants.

Cross-row outcome exposure is declared and measured: the LPA-history features embed earlier rows' labels (a later row's lpa_last_outcome is the previous same-LPA decided case's label verbatim when that case was decided in between), so honest evaluation requires isolated rows - one row per solver context. The shipped illustrative runner does not enforce that requirement; it passes the whole eval feature table to the predictor, and unrestricted-access scores overstate honest performance. The holdout is downloadable by design and stays outside every evaluated agent's workspace. Publication is gated on an independent adversarial pre-flip audit of the frozen package, per the standing practice since 2026-10-27.

→ Open the dataset page for numbers, files and checksums

★ THANKS FOR VISITING · BOOKMARK THIS PAGE · TELL A FRIEND ABOUT THE DATASET FACTORY · SIGN THE GUESTBOOK BEFORE YOU LEAVE ★