Dataset Factory★ EST. 2026 · BEST AT 800×600 ★
★ WELCOME TO THE DATASET FACTORY ★ PUBLIC DATA → CHECKED BUILDS → HONEST BASELINES ★ NO POP-UPS ★ NO COOKIES ★ NO TRACKERS ★ HAND-MADE HTML ★ UNDER CONSTRUCTION SINCE THE NINETIES ★

Wser Effluent Exceedance

At the start of a compliance year, will a federally regulated Canadian wastewater system report at least one effluent monitoring period whose average CBOD or suspended-solids concentration exceeds the limit that applies to it?

Versioned research data. Read the suitability requirements before interpreting scores.

Suitability and evaluation

Evaluation setting: retrospective research. Read the prediction setting and limitations below. Public validation is development data; downloadable holdout labels must stay outside the evaluated agent workspace.

At a glance

QuestionAt the start of a compliance year, will a federally regulated Canadian wastewater system report at least one effluent monitoring period whose average CBOD or suspended-solids concentration exceeds the limit that applies to it?
FieldEnvironment and Climate Change Canada (ECCC) / binary-classification
Rows18,126
Documented splits3
This versionv-3ceae5479cda761c
Released2026-11-15
Validation ROC AUC0.8993001552065363
Validation Brier0.0874060572590605
Artifact15ee03d514ebc5eaedefc7fba7611d2525fd4b9c828bdfb120ae55b8b8f4c1b4

Get this dataset

How to load it

from datasets import load_dataset
ds = load_dataset('earino/factory-wser-effluent-exceedance', revision='v-3ceae5479cda761c')

The version links and checksums identify this artifact. Exclude the labelled holdout from the agent's workspace when running an evaluation.

What is inside

SplitRowsPositive labels
eval5,894—
holdout3,988—
train8,244—

Files and checksums

DownloadBytesSHA-256
LICENSE-EVIDENCE.json783bb9d09df34f77705c6ccd88751b05a2208ec3d5f33dbf75d5b00f1bd1f2ed122
LICENSE.md9416206e5f78f3d15ea7e11a2dec58c05f4c6670779e90eda5bbb77617e6bf65e9d
README.md8,100e01441d09e4c6011104e0c76fe6781fa223267e9227648bba856dba01d447c9c
SOURCES.json1,83121050eedd20e2afbfb3825c5c66ee0d7e9c54042c3adb3c05e869ddd9ff5d3eb
evaluation.json3,11996cf1fc801ff7e75ac7a2d5f73abb5e88bf8f38e276d8e754abef484a277de6d
extract__meta.json2,006f1c52fe3b6d2891f4dba03417a15b1e097313452f26858af6e1bd90aee97a95a
extract__private__holdout.csv713,711696a55f28a532d8f0e24ca5912f55d5e13e87095e8fee12ff907c3d3c286060e
extract__public__eval.csv1,056,905357b0a613a3be2dc73246c189159db69b7fd5761468b73e2dbc4dd701b18cd95
extract__public__train.csv1,555,738483e3ef72c333d60b33ee9b3e90ca2417b3cb419e88c7cc01ad30ae95cbe09c7
extract__quality.json31,878579517c28f8f81aac6f8508f4ace96a0d46ff96a009cca6c57134c359c1a29ee
get_dataset.py3,680fad89222d50f9d9709225c34909be74c591df409c08b8b5951eba9d3cadfdd48
qualification.json22,1871f3daaec40b23aba9337c3bc0c83d0a5f82bb424fb2b2d4d8cedd52e2512f7a9
reproduction__LICENSE-CC0-1.0.txt7,048a2010f343487d3f7618affe54f789f5487602331c0a8d03f49e9a7c547cf0499
reproduction__LICENSE-MIT.txt1,12456b932ff064e5da113ae10175c8546325f64f26e1ece71683ffcbf8b2304fe1c
reproduction__artifact_scrub.py11,3698c663253e691a7b8751f9c853b40a3752a833360caafb1ea8e8f534e64af4b68
reproduction__build.py45,762389315df31ecdec2f8e2fd56d1aa3c3604b10ce0d0c2488d4f38aef80e31ed7f
reproduction__construction.json27,256122dc1c859ff87c810f2b66308b61e20a30db7af388c907e888fc0a29f8b43ce
reproduction__dataset_package.py7,927c2364defe58dcc99322be59a1bb04894ce00e823ce37afeb37bc31012f744327
reproduction__dataset_run.py12,3818085fae14d6e93b852e692e18c1ef72113a3601f48f78901b6d1ae6db278de57
reproduction__get_dataset.py3,680fad89222d50f9d9709225c34909be74c591df409c08b8b5951eba9d3cadfdd48
reproduction__leakage_battery.py17,379adbf7ee0aca0012b78cc1f1e4aeb968a23717cae30f66a02e080215e75f830e3
reproduction__provenance.json57311a9c4f355a6345b1a64b1bb34afd669daa68a62b53b6635c45f9861295a81a7
reproduction__qualify_dataset.py66,067bff6d43d0904182e2612c32bab702e41addbf389904a236c209e78be39e37a6b
reproduction__recovery_recipes.py17,609f64c542fcda232169c95ace17421b45275194b934938c0ea4c9b68e6f844ad0e
reproduction__requirements.txt1605c72f3268174d20364624f76680e189c408a05dba2e7b8cfe12d077b3865e7f0
reproduction__source.json24,2246538fc4104aa50598eacacb8de56369dd9f57876dea3d1e08b29375673d8c5bb
reproduction__source_io.py5,6070c60eed4968cd342148efbaac7e4fa98f00708d8e6c14c0929928687b9af1ea3
reproduction__train.py2,500a3c6bcf13735bc85c52129ded68f839090dffdc266ebc3810dc61b2e6ea5e7e8
reproduction__validate.py1,277b597e7f84fed614e64b4a86fbecbd6ec0145916a24d0d221a796586976418e7b
reproduction__validate.sh4823f06ca48f11c2e05d331b4d8394284925de660c1d4b086d2f8243959e83b7e6d
source-checks.json12,83936e6db1cb38b4f2f1ed86ba00c0dd2ec4b8ffeff064e8f6110a3535d80fc6d1a
source-snapshot.tar.gz1,795,58487198b20c5d403dbbb33faa88d421c6fd44ded04571ef635cf2cc3c84d6205e3
verify_dataset.py1,276f4a4bb7f565cdbd075c520621f881876b491eaed91a5e6e70afb31e4753f9834

Release notes · 2026-11-15

At the start of a compliance year, will a federally regulated Canadian wastewater system report at least one effluent monitoring period whose average CBOD or suspended-solids concentration exceeds the limit that applies to it?

Population: Wastewater systems filing at least one monitoring report in target year Y in which a CBOD or suspended-solids limit greater than 0 is recorded, with at least one prior year of monitoring history. The system-year must also have at least one such positive-limit row with a reported average; system-years with a positive limit but a missing average and no observed exceedance are excluded rather than labeled 0. No current-vintage operational-status snapshot is used to exclude historical system-years because the identification status field has no reporting-year stamp and is not year-versioned. Because no limits existed before 1 January 2015, target years run 2015 to 2024. Systems outside the regulations (Nunavut, Northwest Territories, north of the 54th parallel in Quebec and Newfoundland and Labrador) never appear. Quebec municipal systems appear under the equivalency agreement with provincial limit conventions carried in the limit columns. Eligibility and labels use year-Y reports, so this is a retrospective research cohort, not a deployable 1 March forecast roster.

Target: For system i and target calendar year Y, label = 1 if any monitoring row for year Y with a reported CBOD limit greater than 0 has average CBOD strictly greater than that limit, or any row with a reported suspended-solids limit greater than 0 has average suspended solids strictly greater than that limit; otherwise 0. Computed from the reported average and the reported limit, not from the pre-computed compliance column. Rows for 2013 and 2014 are excluded because no limits applied.

Prediction time: 1 March of target year Y. Every monitoring report for a completed averaging period is due within 45 days of the period end (annual and Q4 reports therefore due by 14 February), and combined-sewer-overflow reports are due by 15 February, so all year Y-1 effluent results and CSO volumes are on file at the statutory deadline. The timing metadata treats the 45-day statutory lag as a publication proxy; actual publication dates of the downloaded current-vintage CSV are not verified. Identification-report attributes are treated as a current-vintage snapshot.

Splits: Fixed chronological boundaries by target year: train 2015-2019, eval 2020-2022, holdout 2023-2024. No embargo year is needed because each label window is a single non-overlapping calendar year and every feature is dated on or before 31 December Y-1, so no row can see a later row's outcome. Target year 2025 is excluded because its reports are still maturing in the current data vintage. Reporting-completeness strata and province are kept balanced across the three periods by construction of the year windows, not by resampling.

Outcome horizon: The full calendar year Y (1 January to 31 December), observable from reports due by 14 February of Y+1.

Inputs: prior_year_exceedance: 1 if any Y-1 monitoring row with a positive applicable limit and a reported average has average strictly above limit; 0 if Y-1 has an evaluable positive-limit row and none exceed; missing if Y-1 has no evaluable positive-limit row. Limit-based history is restricted to 2015 and later, so 2015 targets are missing even if stray pre-2015 rows carry positive limits.; prior_year_worst_margin: Maximum over Y-1 CBOD and suspended-solids evaluable rows of average minus limit, floored at 0; missing when no evaluable Y-1 limit row. Limit-based history is restricted to 2015 and later.; ever_exceeded_before: 1 if any year before Y has an evaluable exceedance; 0 if at least one prior evaluable year and none; missing if no prior evaluable year. Limit-based history is restricted to 2015 and later.; history_years_reported: Count of distinct calendar years before Y with at least one monitoring row.; periods_reported_prior_year: Count of distinct year-period-month monitoring records in Y-1, a measure of sampling exposure; 0 if no Y-1 monitoring row.; reporting_frequency: Canonical annual, quarterly or monthly reporting frequency from the latest Y-1 monitoring row; inferred from period tokens if the field is blank.; system_type: Intermittent or continuous wastewater system type from the latest Y-1 monitoring row.; avg_daily_effluent_volume_m3: Mean reported average daily effluent volume in Y-1, in cubic metres per day; missing if no Y-1 value.; owner_type_operator_type: Owner type and operator type from the latest Y-1 row, joined by a separator; missing if neither is reported.; province: Province from the latest Y-1 monitoring row, falling back to the identification snapshot.; treatment_family: Mechanical or activated-sludge class, lagoon class, no treatment, or other, parsed from the identification treatment-type list.; hydraulic_retention_ge_5_days: 1 if reported hydraulic retention time is five days or more, else 0; missing if not reported.; chlorine_use: 1 if chlorine or a chlorine compound is reported in use, else 0; missing if not reported.; receiving_water_type: Reported description or name of the water frequented by fish at the discharge point, such as a watercourse, lake or fish spawning area, taken from the water-frequented-by-fish identification column; not the generic discharge-place description.; prior_year_acute_lethality_failures: Count of failed rainbow-trout acute lethality tests in Y-1; missing when no acute test row exists, which is expected for systems below the testing threshold.

Recorded structural checks and baselines

PASSED Gate 1.6.1.

Existing unmodified runner baseline on public validation only; no holdout scoring

These historical checks do not establish availability at each prediction time or protection against every shortcut. Scores below describe the recorded protocol.

Limits

Source, licences, and citation

Environment and Climate Change Canada (ECCC)

Contains information licensed under the Open Government Licence – Canada.

Source: Open Government Licence - Canada (version 2.0). Compilation: CC0-1.0. Code and documentation: MIT.

Our additional compilation rights only; upstream terms remain in force.

Suggested citation

Versions

★ THANKS FOR VISITING · BOOKMARK THIS PAGE · TELL A FRIEND ABOUT THE DATASET FACTORY · SIGN THE GUESTBOOK BEFORE YOU LEAVE ★