Malaysian retail price increases
Research task: predict whether an item's price at a Malaysian premise rises next week - 832,729 item-premise-weeks from the PriceCatcher daily price panel, state of Johor.
Versioned research data. Read the suitability requirements before interpreting scores.
Suitability and evaluation
Retained for retrospective model-development research with documented timing limitations. Later evaluation rows expose earlier outcomes; predictions require restricted access.
Supported uses: Retrospective train/validation model-development experiments under the documented timing and selection limitations; Final evaluation only under a protocol that isolates holdout data from development and prevents later rows influencing earlier predictions
Not supported: Claims of verified real-time or operational forecasting validity; Treating development/validation scores as untouched final-test results; Forecasting scores from unrestricted cross-row access to this artifact
- Historical feature availability
- assumed
- Training-label timing
- unverified
- Data vintage
- current snapshot
- Row selection
- unverified
- Required evaluation access
- chronological
Cross-row audit: recovery demonstrated. Independent operator-commissioned audit; predictions frozen before labels were opened; holdouts sealed; two AUC implementations in exact agreement
The labeled eval split is development/validation data. Isolate final holdout data from development and prevent later rows influencing earlier predictions. Published data is not a secure hidden test service.
- join (premise, item, T) -> (same pair, T+7 days): the next row's prices are the operands of T's label
- The labelled holdout is presumed to carry the same self-answering structure by construction; it was never opened by the audit and must be treated as compromisable by the same join class.
- Historical feature/label availability at the declared instant is assumed from current snapshots, not verified against release vintages.
- The current scorer passes the complete feature frame to predict_proba; prediction-time isolation is instructed but not mechanically enforced, so any framework score on these files is gameable.
- Observing same-day rows in one current bulk file does not verify historical weekly completeness or revision timing. Missing next-week outcomes restrict the scored population.
Corrections to historical documentation
- Public site page, Hub card and repository README must state the restricted suitability once rendered; no frozen package bytes change.
Reviewed 2026-10-24. Applies to this artifact; original data and tags are preserved.
At a glance
| Question | Research task: predict whether an item's price at a Malaysian premise rises next week - 832,729 item-premise-weeks from the PriceCatcher daily price panel, state of Johor. |
|---|---|
| Field | Retail prices and consumer economics / binary-classification |
| Rows | 832,729 |
| Documented splits | 3 |
| This version | v2026.10 |
| Released | 2026-10-22 |
| Evaluation ROC AUC | 0.752 |
| Artifact | 06c5cd6f8a74aef9d1fab048ab88f86e311385e68c0d25416b7ac5c0281a23cf |
Get this dataset
How to load it
from datasets import load_dataset
ds = load_dataset('earino/my-pricecatcher-weekly-increase', revision='v2026.10')
The version links and checksums identify this artifact. Exclude the labelled holdout from the agent's workspace when running an evaluation.
What is inside 
| Split | Rows | Positive labels |
|---|---|---|
| eval | 280,468 | 48,990 (17.47%) |
| holdout | 154,279 | 31,519 (20.43%) |
| train | 397,982 | 59,449 (14.94%) |
Files and checksums
| Download | Bytes | SHA-256 |
|---|---|---|
| meta.json | 1,240 | 81288b72d7c566b33b6ee4c56f90fa629c53df15ac58133a993f3b49e70cdb0a |
| private-holdout.csv | 20,235,520 | 2c763488dc121da07ba43bada0d827d81e646d517c4143496be749022cc39621 |
| public-eval.csv | 35,639,435 | 62b71ae37f43b05b94d4d56817fd0f2a535cba00d70dd52d21993f4e46e6b834 |
| public-train.csv | 49,509,819 | ff054994020c3fcf95c77e88d69e7fc2ac8e42ec97944e098ac9e62f3289951e |
| quality.json | 6,569 | 8ead984e53c126fbbb9b6fd03c42fb114233f23960222a30b84ee0b204695b48 |
Release notes · 2026-10-22
Malaysia's PriceCatcher programme publishes daily retail prices for hundreds of items across thousands of premises. Access is not the contribution. The contribution is the weekly panel - 397,982 train / 280,468 eval / 154,279 holdout item-premise-weeks in the state of Johor, from November 2023 to October 2026 - and the declaration that the prediction instant is the end of week W: a forecast made on Sunday night about next week's shelf price.
The label is a direction call, not a level: will the weekly modal price be higher in week W+1 than in week W. Positive rates run 14.9% / 17.5% / 20.4%, and the rise of the rate across the windows is itself a signal about the period.
The baseline is strong enough to be interesting: the harness's XGBoost pass reaches 0.7520 eval AUC, twenty-one points above the best naive reference (an item-level prior at 0.5354, with momentum and mean-reversion near 0.50). Unlike tasks where persistence wins, this one rewards learning something about the item and the week.
Recorded structural checks and baselines
PASSED Gate 1.4.0.
One baseline through the runner's training and validation contract. No agent or harness comparison was run and none is implied.
These historical checks do not establish availability at each prediction time or protection against every shortcut. Scores below describe the recorded protocol.
Limits
Derived from PriceCatcher (KPDN and DOSM, data.gov.my) under CC BY 4.0; the compilation is CC0. Not an official Malaysian government product.
State-scoped to Johor. Other states are not represented in this release.
Pairs with fewer than three observations in a week are excluded at the prediction instant; a following week that is too sparse is counted as an unobserved outcome (143,978) rather than dropped silently.
Missing lags are sentinel-coded -1.0, which is unambiguous only because prices are strictly positive.
The holdout is labelled and ships with the release; keep its labels outside an evaluated agent's workspace.
The source feed revises monthly; a later rebuild will not reproduce these bytes.
No claim of real-time or operational forecasting validity: historical feature and label availability at the prediction instant is assumed from a current source snapshot, not verified against release vintages.
The labeled eval split is development/validation. Final holdout scoring requires prediction-time access restrictions; the original scorer provided the complete feature frame. Downloadable holdout labels must stay outside model development.
Source, licences, and citation
PriceCatcher: transactional records (KPDN and DOSM, data.gov.my)
PriceCatcher, Ministry of Domestic Trade and Cost of Living (KPDN) and Department of Statistics Malaysia (DOSM), data.gov.my, CC BY 4.0.
Source: CC-BY-4.0. Compilation: CC0-1.0. Code and documentation: MIT.
The weekly modal aggregation, row selection, derived label, features, splits and packaging are our work and are dedicated to the public domain under CC0 1.0. The source rows remain under CC BY 4.0.
Suggested citation
KPDN and DOSM. PriceCatcher. data.gov.my; 2026. Derived dataset: week-ahead retail price increase, Johor.