Phmsa Pipeline Penalty Reduction
Will a PHMSA Notice of Probable Violation with a proposed civil penalty be closed within 36 months with a lower assessed penalty than proposed?
Versioned research data. Read the suitability requirements before interpreting scores.
Suitability and evaluation
Evaluation setting: retrospective research. Read the prediction setting and limitations below. Public validation is development data; downloadable holdout labels must stay outside the evaluated agent workspace.
At a glance
| Question | Will a PHMSA Notice of Probable Violation with a proposed civil penalty be closed within 36 months with a lower assessed penalty than proposed? |
|---|---|
| Field | U.S. Department of Transportation, Pipeline and Hazardous Materials Safety Administration (PHMSA) / binary-classification |
| Rows | 958 |
| Documented splits | 3 |
| This version | v-bbc08f68dbbd24b5 |
| Released | 2026-11-13 |
| Validation ROC AUC | 0.6172268907563025 |
| Validation Brier | 0.18742755379536652 |
| Artifact | 04b70fa61c8b4b68fb11974080984ef075ff8470c5ee52379579266b5ccfac1a |
Get this dataset
How to load it
from datasets import load_dataset
ds = load_dataset('earino/factory-phmsa-pipeline-penalty-reduction', revision='v-bbc08f68dbbd24b5')
The version links and checksums identify this artifact. Exclude the labelled holdout from the agent's workspace when running an evaluation.
What is inside 
| Split | Rows | Positive labels |
|---|---|---|
| eval | 139 | — |
| holdout | 149 | — |
| train | 670 | — |
Files and checksums
| Download | Bytes | SHA-256 |
|---|---|---|
| LICENSE-EVIDENCE.json | 651 | 9d2abc5924e3955c2934d70222207498c3a1c7f792b32151503d7eb8ad6b5496 |
| LICENSE.md | 948 | 477fe699db77e1e1b4690879dfd9c51d234ba374847c45db0a5637df093bf23a |
| README.md | 5,072 | 0da2d2b1e7dc040b57918c790946fdebb141060d550d872b35328aaa28c65ffa |
| SOURCES.json | 997 | e0b6bffc8f735726fa257341fa70b577c1963633f1ece9b8d2bcfc0c5cbc07ba |
| evaluation.json | 2,280 | b1321867156ee9bc8ffd800db903b5724eca7137113e89cb307fb7dbd785fee5 |
| extract__meta.json | 1,511 | 2f925208572551b36fd013a5dafae65bb51da04f014915db22c981a0ec7a4d71 |
| extract__private__holdout.csv | 14,700 | 8c408e94871aece221d4d1ed6d420ee9d9ab6574c00e71534ba8aaf0aee5ef1e |
| extract__public__eval.csv | 12,911 | 8ee6b14d7148f2a06b622010bbf0b8bb21e647956585764486b803ff055d27dc |
| extract__public__train.csv | 64,666 | adaf6e3a33524ffe745a6d6fadd07ca3dedf2a93c8e110f450559cf56c277f74 |
| extract__quality.json | 22,738 | d291f3d5ac04d1d83676c3ed864aaf30299cb9ad69f894f4450a798f2cf5be55 |
| get_dataset.py | 3,680 | fad89222d50f9d9709225c34909be74c591df409c08b8b5951eba9d3cadfdd48 |
| qualification.json | 18,377 | 1ccb7a62e299c7a2750097d9cd8dcb1a79aab110a660ad33445c1c3171c86f10 |
| reproduction__LICENSE-CC0-1.0.txt | 7,048 | a2010f343487d3f7618affe54f789f5487602331c0a8d03f49e9a7c547cf0499 |
| reproduction__LICENSE-MIT.txt | 1,124 | 56b932ff064e5da113ae10175c8546325f64f26e1ece71683ffcbf8b2304fe1c |
| reproduction__artifact_scrub.py | 11,369 | 8c663253e691a7b8751f9c853b40a3752a833360caafb1ea8e8f534e64af4b68 |
| reproduction__build.py | 24,214 | 32b1f0e70cb58713fd3482c9938f8c8bffbf21c17ddd8e64429d05bceb67434b |
| reproduction__construction.json | 24,675 | d7b716730c2a8ea9690474ccd6591173c46ca1405e02bd30b087a129f49eed31 |
| reproduction__dataset_package.py | 7,927 | c2364defe58dcc99322be59a1bb04894ce00e823ce37afeb37bc31012f744327 |
| reproduction__dataset_run.py | 12,381 | 8085fae14d6e93b852e692e18c1ef72113a3601f48f78901b6d1ae6db278de57 |
| reproduction__get_dataset.py | 3,680 | fad89222d50f9d9709225c34909be74c591df409c08b8b5951eba9d3cadfdd48 |
| reproduction__leakage_battery.py | 17,379 | adbf7ee0aca0012b78cc1f1e4aeb968a23717cae30f66a02e080215e75f830e3 |
| reproduction__provenance.json | 573 | c4b7e3bae1aa9b5152d0080fd6ef89ce05da483deaa897e8ffa979063150217c |
| reproduction__qualify_dataset.py | 66,067 | bff6d43d0904182e2612c32bab702e41addbf389904a236c209e78be39e37a6b |
| reproduction__recovery_recipes.py | 17,609 | f64c542fcda232169c95ace17421b45275194b934938c0ea4c9b68e6f844ad0e |
| reproduction__requirements.txt | 160 | 5c72f3268174d20364624f76680e189c408a05dba2e7b8cfe12d077b3865e7f0 |
| reproduction__source.json | 16,145 | a35e9f85825c0a4417c01ed3a6e454c713c379801ce4d66a37af8e1950d2bc8d |
| reproduction__source_io.py | 5,607 | 0c60eed4968cd342148efbaac7e4fa98f00708d8e6c14c0929928687b9af1ea3 |
| reproduction__train.py | 2,500 | a3c6bcf13735bc85c52129ded68f839090dffdc266ebc3810dc61b2e6ea5e7e8 |
| reproduction__validate.py | 1,277 | b597e7f84fed614e64b4a86fbecbd6ec0145916a24d0d221a796586976418e7b |
| reproduction__validate.sh | 482 | 3f06ca48f11c2e05d331b4d8394284925de660c1d4b086d2f8243959e83b7e6d |
| source-checks.json | 15,889 | b51bbe7217be2dd9875375baa37363db35d981c425022227f3c6513d00dd2314 |
| source-snapshot.tar.gz | 524,520 | 996b9e49aefc759eafb59c3890c44842c71971b23d8188d687a3f936a7836c68 |
| verify_dataset.py | 1,276 | f4a4bb7f565cdbd075c520621f881876b491eaed91a5e6e70afb31e4753f9834 |
Release notes · 2026-11-13
Will a PHMSA Notice of Probable Violation with a proposed civil penalty be closed within 36 months with a lower assessed penalty than proposed?
Population: NOPV cases with positive Proposed_Penalties and nonmissing Notice_Sent_Date from 2002-01-01 through 2023-06-30; deduplicated to earliest NOPV per WMS_Activity_Seq. Eligibility uses only decision-time information.
Target: y=1 iff the case has a nonmissing Closed_Date no later than 36 months after Notice_Sent_Date and Assessed_Penalties is nonmissing and strictly less than Proposed_Penalties (zero counts as elimination); otherwise y=0, including cases not closed with a reduction by 36 months. Closed cases with missing Assessed_Penalties are coded 0, not imputed.
Prediction time: Notice_Sent_Date, before the operator response, hearing or any order.
Splits: Fixed chronological boundaries by Notice_Sent_Date: train 2002-01-01 to 2016-12-31; evaluation 2017-01-01 to 2019-12-31; holdout 2020-01-01 to 2023-06-30. No embargo is needed because no lagged outcomes, rolling histories, operator histories, or future case records are used; related cases are deduplicated by WMS_Activity_Seq before splitting.
Outcome horizon: 36 months after Notice_Sent_Date; holdout notices end 2023-06-30, so all outcomes are observable by 2026-06-30 against the 2026-11-03 extract
Inputs: proposed_penalty_usd: Proposed civil penalty in dollars stated in the Notice of Probable Violation.; violation_category_count: Number of nonempty comma-separated violation categories in Violation_Category.; cited_regulation_count: Number of distinct nonempty regulation codes in Cited_Regulations.; integrity_management_flag: Whether Violation_Category includes Integrity Management.; pipeline_type: Pipeline segment type associated with the enforcement case, such as INTERSTATE LIQUID or INTERSTATE GAS TRANSMISSION.; region: PHMSA region that initiated the enforcement case.; discovery_route: How the case was discovered, such as Integrated, Specialized, Failure Investigation, Construction, SRCR or PA Program.; notice_actions: Notice action codes such as CO, CP, CO/CP, WI or NOA.; proposed_compliance_order: Whether a compliance order was proposed with the case.
Recorded structural checks and baselines
PASSED Gate 1.6.1.
Existing unmodified runner baseline on public validation only; no holdout scoring
These historical checks do not establish availability at each prediction time or protection against every shortcut. Scores below describe the recorded protocol.
Limits
Current-vintage retrospective extract as of 2026-11-03; historical fields may have been updated after case closure, so this does not prove the exact feature set was operationally available to an earlier decision-maker.
Right-censoring: cases not closed with a reduction within 36 months are coded 0, so later reductions are missed; closed cases with missing Assessed_Penalties are also coded 0 and may hide administrative eliminations recorded as blanks.
Selection: only NOPVs with positive proposed penalties are eligible; warning letters, notices of amendment, corrective action orders, and compliance-only cases are excluded.
Cases cluster by operator and sometimes share an inspection; no operator history or lagged outcome features are used, but evaluation should account for operator clustering.
The data dictionary is dated 2023 and uses some field names that differ from the 2026 raw header; construction must follow the raw header and map semantics explicitly.
No causal interpretation: inspection routes, regions, and case initiation reflect PHMSA enforcement selection and legal process, not random assignment.
The U.S. public-domain basis is territorial; 17 U.S.C. § 105 does not establish a worldwide dedication, and reuse outside the United States may depend on local law. No third-party photo or document rights are included.
Current-vintage extract as of 2026-11-03; historical records may have been revised after case closure, so this is retrospective research rather than a historical forecasting claim.
Right-censoring: cases not closed with a reduction within 36 months are coded zero, so later reductions are missed.
Closed cases with missing Assessed_Penalties are coded zero per the frozen task, not imputed.
Only NOPVs with positive proposed penalties are eligible; warning letters, notices of amendment, corrective action orders and compliance-only cases are excluded.
Cases cluster by operator and sometimes share an inspection; no operator history or lagged outcome features are used, and no corporate-hierarchy reconciliation is attempted.
The data dictionary is dated 2023 and uses some field names that differ from the 2026 raw header; construction follows raw header aliases and fails with observed headers if a required field is absent.
No full-row historical availability ledger exists; feature availability at historical notice dates is assumed from current field definitions and the dictionary.
No causal interpretation: inspection routes, regions and case initiation reflect PHMSA enforcement selection and legal process, not random assignment.
The U.S. public-domain basis is territorial; 17 U.S.C. 105 does not establish a worldwide dedication, and reuse outside the United States may depend on local law.
No uncertainty interval; observed differences do not establish universal headroom
Source, licences, and citation
U.S. Department of Transportation, Pipeline and Hazardous Materials Safety Administration (PHMSA)
None required
Source: Public Domain U.S. Government (PHMSA/DOT data-inventory designation for the Pipeline Enforcement Transparency Portal); U.S. Government work, 17 U.S.C. § 105. Compilation: CC0-1.0. Code and documentation: MIT.
Our additional compilation rights only; upstream terms remain in force.