Early test version. The cost and inflation figures are real official statistics. This page shows measured changes only; the weights that combine the cost series into a category are our own estimates.
Before we publish a number we test it. Before each experiment we write down what we expect; afterwards we publish the result as it is, including what did not work.
Experiments finished Experiments in the log below with a result written down: those that held, those that did not and those not distinguishable, together. Latest log entry October 21, 2026.
22
Running, awaiting results Experiments whose expectation was written down before the run and whose result is not in yet. The result goes into the same entry, whatever it is.
4
Gaps found and closed Experiments in which testing found a gap in the calculation or the data, and the gap was closed.
2
Missed or not distinguishable Experiments that did not meet their expectation, or whose margin over the simple rule was small enough to be explained by chance. They stay on this page.
18
Counted from the log below. “Not distinguishable” means the margin over the simple rule was small enough to be explained by chance; “running” experiments have their expectation written down and no result yet.
Coverage · United States
Markets tested against a simple rule, of 11 on the site Markets in the country switcher whose indicator has been compared with a simple rule in the "Results so far" table below. Not tested yet: Mexico.
10
Months compared with the simple rule, May 2019 to June 2026 Past months over which this market's indicator was compared with the simple rule (the results table below). Each month was recomputed with only the data published by then. Other experiments in the log may cover different periods.
84
Months of official cost history, January 2019 to September 2026 The monthly history of this market's cost index in the latest issue. It is computed from official statistical series only; it is not a shop price.
92
The rules we set ourselves
Expectation first, experiment second. What counts as success is written before the experiment starts; the bar is never moved after seeing the result.
Compete with a simple rule. The indicator does not count as accurate unless it clearly beats a rule that does no thinking, such as “always say the most frequent colour”.
The past, as seen on the day. Every past month is recalculated using only data that was published at the time. Later knowledge is not used.
Values never change afterwards. Each month's published figures and forward records are stored with a chained seal; if one changes, publication stops.
Failures are published too. Experiments that did not work are not deleted from the record.
Official decisions from official texts. The “Recent official decisions” rows on the country pages come only from official sources, and every row links to its source. A figure in a row is not called “verified” until it has been checked against the official text itself. These rows are not forecasts; they are a record of decisions taken. The direction mark follows a fixed rule: for tax, customs, minimum-wage and energy decisions, the direction of the effect on cost; for interest-rate and instalment decisions, the direction of the effect on demand. The mark does not say how much prices will change.
Test stages
The indicator passes five stages from left to right. The tags in the boxes link to the experiments below.
Stage 1 · runningIs the arithmetic rightAre code and data free of errors?Code tests · Data gate · Cross-checkE3
Stage 2 · runningIs it robustDoes corrupt data or a small assumption change the colour?Sensitivity · Corrupt-data drillE2E21
Stage 4 · in designIs it ready to publishIs the new version better than the old one?Shadow run · Parameter ledger · Text auditE20E28fE29E30
Stage 5 · in designOngoingWhat have we not thought of yet?Red team · Label accuracy
As expectedGap found, closedNot distinguishableDid not holdRunning
Experiment log
Experiments awaiting results first, then finished ones, newest first. The numbers are those of our internal log; gaps are experiments not shown here. Four questions for each: what we thought, what we expected, what came out, what we did.
Experiment E30 · October 21, 2026
Shelf state: are stock-outs, delivery times, list-versus-sale gaps and product age leading signals?
Running · pre-registered
What we thought
Stock-outs, longer delivery times, shrinking discounts and new-model launches are the retailer-side signals the literature links to coming price changes. We can test them only on the data we collect ourselves.
What we expected
Written before any shelf month existed, as a statement of intent only: these fields are recorded in our own price observations; the test runs once 12 monthly points exist; the criteria are written then, before the result is seen.
What came out
Once 12 months of data exist (end of 2027 at the earliest).
What we did
Results will be written here as they accumulate.
Experiment E29 · October 21, 2026
Do our shelf prices tell this month's price gap before TÜİK does?
Running · pre-registered
What we thought
Part of what remains forecastable sits in the months TÜİK has not yet published. Our own shelf prices see the current month before the release; their value depends on how much a one-month deviation carries into the following months, which experiment E23 measured at close to zero.
What we expected
Written before any shelf month existed. From February 2027 (with four monthly shelf points), descriptive lines: the relationship and the gap between the shelf index's monthly change and the TÜİK sub-index. No rule, no claim.
What came out
First line February 2027.
What we did
Results will be written here as they accumulate.
Experiment E28f · October 21, 2026
The old January minimum-wage term as a challenger in the forward register
Running · pre-registered
What we thought
Four to six Januaries cannot separate a wage effect from the January price-list reset under 40–80% inflation. Only forecasts written before the outcome can settle the furniture question; for appliances the term is already refuted.
What we expected
Written before the first seal. The live version (A-2026.11.1) has no wage term; the challenger is the same indicator with a fixed-size wage term in January. Expected: the challenger loses for appliances and ties for furniture.
What came out
First seal November 2026; no scored forecast yet.
What we did
Results will be written here as they accumulate.
Experiment E20 · October 20, 2026
Weights shadow register: externally sourced cost shares against the live weights
Running · pre-registered
What we thought
Turkish price data cannot choose the weights (experiment E11): evidence comes only from forecasts written down before the outcome. Cost shares derived from TÜİK's supply-use table are the only weight set with an external justification.
What we expected
Every month from November 2026, for Türkiye, at 1- and 3-month horizons: each weight set's forecast is sealed before the outcome is known; the target is the change in the TÜİK sub-index. The decision rule is written in advance; a weight change happens only if it passes in this register.
What came out
First seal November 2026; no scored forecast yet.
What we did
Results will be written here as they accumulate.
Experiment E33 · October 21, 2026
We compared the new method with the old colour indicator under rules written in advance
Passed (a replacement, not an accuracy claim)
What we thought
The parts of the live indicator (cost index, currency, January wage term, events) produced no evidence at the 3-month horizon (experiments E5, E9, E14, E28). A forecast built only from what has passed a test (the calendar rule for European appliances, the long-run average elsewhere; three probabilities instead of one colour) should be at least as good.
What we expected
Written before the run and passed an independent review: the new method scores above zero against the old indicator on the probability score, with the 90% interval above zero; in Europe in 7 of 7 countries.
What came out
The new method scored better in every market tested: Europe 2014–2026 (appliances +0.64, furniture +0.48; 7 of 7 countries), Türkiye 2021–2026 (+0.47; 90% interval +0.37 to +0.56), the US on 2022–2026 only (+0.39). Most of the difference comes from the old method giving a single colour: a hard colour pays the full penalty on every miss, and when the old method is softened the margin falls to +0.16 to +0.47. The 2019–2022 period could not be measured for the US; when the full series is loaded, that result replaces the US line here.
What we did
This is a replacement, not a forecasting claim: outside European appliances the new method is a long-run average, and there no method has beaten the simple forecast. A decision for the December 2026 issue is pending: the proposal is that the colour on the European appliances page follows the calendar rule that passed its test; the Turkish and US pages keep showing data.
Experiment E28 · October 21, 2026
We tested the effect of tax changes and the January minimum wage on prices with sizes from the literature
Did not hold; the live rule was worse
What we thought
Our target is the category price minus headline inflation. A broad tax moves that gap only as far as it passes through differently to the two sides; a category-specific tax passes through almost fully. Because the minimum wage also lifts headline inflation, its effect on the gap should be close to zero. The sizes came from the literature only, not tuned on the data.
What we expected
Written before scoring and passed an independent review: in most events the sized forecast beats the raw calendar rule and the 90% interval of the total gain stays above zero. A third criterion: the sized forecast must not be worse than the live rule, which passes the full effect through.
What came out
Tax events (3 events, 11 comparisons): the sized forecast beat the calendar rule in 4 (36%); total gain –1.48 (90% interval –3.22 to +0.27). The first two criteria failed. The third passed: the live rule on its own was worse than the calendar rule too. The only clean category tax, the 2019 expiry of the appliance special tax, moved prices far less than expected. 8 of 8 events fell inside the 90% band. The January minimum-wage term gave the wrong sign in 8 of 10 Januaries for appliances and could not be verified for furniture.
What we did
The event layer did not clear the bar written in advance. The January minimum-wage term was removed for both categories before the 4 November seal (version A-2026.11.1); the old formula is sealed as a challenger for comparison (experiment E28f). The October issue stays as published.
Experiment E23 · October 21, 2026
We tried four small additions to the calendar rule: spread, drift, rebound and one month fresher data
No evidence
What we thought
The calendar forecast's probability spread should widen when inflation rises (spread); if a category's price drifts slowly, a 10-year average lags behind (drift); temporary sales return to the old price within 1–2 months (rebound); seeing the latest published month one month earlier gives a short-memory term its last chance.
What we expected
Each written before the run and passed an independent review: on 120 origins in 2014–2023, probability score against the raw calendar rule above zero with the 90% interval above zero, and a gain of at least +0.05.
What came out
Spread +0.003 (90% interval –0.005 to +0.012). Drift –0.002 (–0.004 to 0.000). Rebound +0.002 (–0.004 to +0.005), with the coefficient of the opposite sign to the one expected. Rebound with one month fresher data +0.016 (+0.003 to +0.022), but below the +0.05 bar and –0.002 on 2014–2023. All four failed the criterion (experiments E23, E24, E34).
What we did
The raw calendar rule stays as it is. We learned that the errors change from period to period, which explains why a one-parameter correction does not carry over. The drift question is now followed only in the forward register.
Experiment E22 · October 21, 2026
We tried improving the European calendar rule by pooling across countries
No evidence
What we thought
Each cell of a 10-year calendar table rests on about 10 values; sampling noise costs score. Since the European sale calendar is largely shared across countries, pooling the same calendar month across countries should reduce the noise.
What we expected
Written before the run and passed an independent review: on April 2024 – June 2026, probability score above zero in each category with the 90% interval above zero; above zero on 2019–2023 too.
What came out
378 comparisons. Appliances +0.034 (90% interval +0.004 to +0.064); furniture –0.016 (–0.061 to +0.014); 2019–2023 –0.023. Half the checks failed: no evidence. One more line: a 50/50 pool of the calendar rule and the calendar-blind reference cannot, by the arithmetic, be worse than the average of its parts; on 2014–2023 it beat the reference with a score of +0.158 in 7 of 7 countries. That is an expected result, not evidence of skill.
What we did
The raw calendar rule stays the reference in both categories. The 50/50 pool is sealed in the forward register for European furniture as the first challenger from the October 2026 origin; no change to the site.
Experiment E21 · October 20, 2026
We corrected a drift in how the cost index is calculated
Correction applied
What we thought
In a fixed-base index the components that have risen fastest since 2018 gain weight by themselves and the slow ones lose it. A chained calculation applies the nominal weights to each month's change.
What we expected
Written before the run; this is a correction, with no pass criterion. Expected: effective weights differ from nominal by 2–4 points, the expected pressure moves by 0.1–0.5 points on average, and fewer than 5% of past Turkish forecasts change colour.
What came out
The live colours did not change in Türkiye, the US, Germany or France. In the backtest 35 of 450 Turkish forecasts (7.8%) changed colour: 7 became right, 20 became wrong; the 5% expectation was missed. Turkish accuracy went from 48.9% to 46.0%; the other countries moved within 0.5 points. The loss sits in about 10 clusters of currency-shock months; the independent review counted it as noise.
What we did
The chained index is live from the November 2026 issue (version A-2026.11). The old method is sealed in the shadow register for comparison. The October issue stays as published.
Experiment E17 · October 19, 2026
We looked at appliance prices after currency shocks, 10 countries pooled
No evidence
What we thought
After the home currency falls sharply against the euro, goods with imported inputs should run above headline inflation for a few months; the effect should grow with the size of the fall and look alike across countries. Pooling 10 countries with their own currency gives many more shocks than Türkiye alone.
What we expected
Written before the run and passed an independent review; the shock list was recorded before the price data was fetched. Criterion: probability score above zero in the shock months with the 90% interval above zero; leaving out each cluster of shocks in turn must not change the result.
What came out
Pre-registered set (6 countries, 9 shocks, 25 comparisons): score –0.062 (90% interval –1.26 to +0.25). The slope comes only from the 2008–09 shocks; the later ones alone give –0.01. Extended set (34 shocks, 98 comparisons): –0.119 (–0.26 to –0.01). In Türkiye's largest shock (December 2021) appliances ran 5.5 points below the calendar rule over the next 3 months: the rise happened in the shock month itself and did not carry over.
What we did
No currency-shock layer was added. A "shock now" warning could only be tested with its own pre-registered plan.
Experiment E16 · October 19, 2026
We checked whether housing activity signals furniture prices in advance
No evidence
What we thought
People who move buy furniture; a year-on-year rise in housing activity should push furniture prices above the calendar rule a few months later.
What we expected
Written before the run and passed an independent review: 5 European countries, April 2024 – June 2026; Türkiye on an expanding window 2019–2026. To pass: probability score above zero with the 90% interval above zero, a positive coefficient that is stable across periods. The expected outcome was written too: no evidence in either.
What came out
European furniture (building permits, 135 comparisons): score –0.017 (90% interval –0.049 to +0.009); 2 of 5 countries above zero; the coefficient's sign changes between periods. Turkish furniture (house sales, 90 comparisons): –0.032 (–0.110 to +0.039). No evidence, as expected. (Probability score: how much the forecast's error falls relative to the calendar rule; 0 equals the rule, negative is worse than the rule.)
What we did
No demand layer was added. Housing data stays a monitored series only.
Experiment E15 · October 19, 2026
We tested the calendar itself as a forecasting rule
Passed for appliances only
What we thought
In Europe the sale seasons fall in roughly the same months every year. A rule that only asks "what happened in this month in past years" might beat the indicator that looks at costs.
What we expected
Written before the run: across 7 European countries, April 2024 – June 2026, the calendar rule beats the calendar-blind simple forecasts for both furniture and major appliances by more than the lower end of the confidence interval.
What came out
Passed for major appliances: calendar rule 61%, "most frequent colour" 51%. Not distinguishable for furniture: both 61%. Adding costs to the calendar rule added nothing.
What we did
We raised the bar for appliances in Europe: from now on every indicator must first beat this calendar rule. From November 2026 the European indicator's and the calendar rule's forecasts are sealed every month before the outcome is known; no accuracy claim until enough months have accumulated. For furniture we found no evidence that the calendar works.
Appliances: calendar rule61%
Appliances: most frequent colour51%
Colour accuracy, major appliances, 7 European countries, 189 comparisons, April 2024 – June 2026. Horizontal line: 95% confidence interval; dark line: the simple rule's level.
Experiment E14 · October 19, 2026
We tried correcting the European indicator with a pass-through factor
Did not hold
What we thought
The European indicator says red far more often than furniture prices outpace inflation (experiment E9). If cost pressure passes through to prices weakly, shrinking the forecast gap by a factor learned from data should remove the red bias and beat the simple forecast.
What we expected
Written before the run: across 7 European countries, on the April 2024 – June 2026 hold-out, a gain of at least 5 points over the best simple forecast with the 90% interval of the gain above zero; no country below its own simple forecast, 4 of 7 above. The expected outcome was written too: the factor comes out small and the model moves towards "always yellow".
What came out
378 comparisons. Live model 42%, "most frequent colour" 51%: a gap of –9 points (90% interval –16.4 to –5.3), so the model is clearly worse than naming the most frequent colour. The factor model said yellow in all 378 comparisons and landed on the same 51% as the simple forecast: gain 0. The factor's sign changes from period to period. No check passed.
What we did
No new European forecast page was opened. The German and French pages, and the European countries added later, show measured official figures only, no colour. The results table on this page is the record of that decision.
Live model42%
Factor model51%
Most frequent colour51%
Colour accuracy, April 2024 – June 2026, 7 countries × 2 benchmarks = 378 comparisons, 1-point band. Horizontal line: 95% confidence interval; dark line: the simple rule's level.
Experiment E13 · October 19, 2026
We tried to measure the effect of official decisions on prices
Partly measurable
What we thought
If dated official decisions (VAT and special consumption tax, the currency regime, the minimum wage, instalment limits) move furniture and appliance prices relative to headline inflation within three months, the effect can be learned from history.
What we expected
Written before the TÜİK rerun: a decision counts as an "effect" if it pushes the 3-month relative change into the outer 5% of all 3-month windows, with the same sign, in two separate data sources (Eurostat 1996–2026, TÜİK 2018–2026).
What came out
Furniture VAT cut (November 2018): –7.9 points; its expiry (July 2019): +7.9 points; both sources agree within a point. Appliance special-tax cut: no clear effect (–2.9). Currency shocks: appliances +13.9 (2018) and +16.7 (2021). Six minimum-wage rises: mixed signs, nothing consistent. Instalment limit (2021): no effect. Furniture's relative price fell after two of three interest-rate rises; none crossed the threshold.
What we did
No accuracy claim. Decision dates were verified in the Official Gazette; with a multiple-testing correction for 17 decisions × 3 categories, whether tax decisions enter the product with a size was tested in a separate experiment (E28).
Experiment E12 · October 19, 2026
We checked whether lagged exchange-rate changes explain what the index misses
No evidence
What we thought
If a rise in the exchange rate reaches product prices with a lag, exchange-rate changes one and two quarters earlier should explain part of the index's error.
What we expected
Written before the run and passed an independent review: the error falls by at least 5% in every category, direction accuracy falls in none, the 90% interval of the gain stays above zero, and the gain survives removing the currency-shock months.
What came out
The error rose 0–2%, so no gain; the 90% interval of the gain includes zero. Without the shock months the gain is –3% to –4%. Direction accuracy equals the control in every category. The coefficients are small and change sign within the period.
What we did
The live weights are unchanged; the site makes no claim about an exchange-rate effect. We return to this only if the forward register shows a persistent miss in devaluation months.
Experiment E11 · October 19, 2026
We tried learning the cost weights from data instead of assuming them
Direction accuracy did not improve
What we thought
We assume each category's cost shares (fabric, foam, panel, metal, labour, currency). Letting the data adjust the weights, without straying far from the assumption, should lower the error outside the test period.
What we expected
Over 54 months since 2022, using only data published on the day, the learned weights lower the error by a margin clearly larger than noise and do not worsen the "will it outpace inflation" direction call.
What came out
The first run contained a leak of future data; its numbers were withdrawn and the experiment was corrected and repeated. Corrected result: the 3-month level error fell 9–16%, but direction accuracy did not rise: 48–56% with the assumed weights, 44–46% with the learned ones; "most frequent answer" 54%.
What we did
The live weights are unchanged. A lower level error with a lower direction accuracy is not a reason to change. A weight change happens only if it passes a forward test (experiment E20).
Sofa set: assumed weights56%
Sofa set: learned weights44%
Direction accuracy ("will it outpace inflation"), sofa set, 54 months, leak-corrected run. Intervals are narrower than they should be because the windows overlap. Horizontal line: 95% confidence interval; dark line: the simple rule's level.
Experiment E10 · October 19, 2026
We tested the 12-month hint on Germany and France
Did not carry over
What we thought
If the 12-month rule found on Turkish data (experiment E8) reflects a real mechanism, it should beat the simple forecast in countries that were not used to find it.
What we expected
The rule was fixed as found in Türkiye: a category that outpaced inflation over the last 12 months does so over the next 12. It counts as carried over if the lower end of the confidence interval clears the simple forecast in at least two of the countries tested.
What came out
Germany, furniture: rule 52%, best simple forecast 71%. France, furniture: 60% against 50%, not distinguishable. For major appliances the simple forecast is already at 98–99% in both countries. The rule passed in 0 of 2 countries. The US was not tested yet.
What we did
No change to the site. The hint enters the forward register, where forecasts are sealed every month before the outcome is known, as a challenger from November 2026; promotion needs 24 independent scored forecasts.
Germany, furniture: rule (12 months)52%
Germany, furniture: best simple forecast71%
Accuracy on "will it outpace inflation", 12 months, 80 overlapping months. No confidence interval shown because the windows overlap. Horizontal line: 95% confidence interval; dark line: the simple rule's level.
Experiment E9 · October 19, 2026
We tested the German and French indicator on real data
Below the simple forecast
What we thought
The cost-pressure indicator built for Türkiye may also work for Germany and France on European statistics.
What we expected
The same rule as for Türkiye: colour accuracy above the best simple forecast by more than the lower end of the confidence interval.
What came out
2019–2026, 450 comparisons per country. Germany: indicator 32% (interval 28–36), "most frequent colour" 54%. France: 36% (interval 32–40), simple forecast 58%. The indicator says red far more often than red actually happens.
What we did
No change to the indicator. The target change (experiment E8) was tried on these countries too and did not hold (experiment E10); a correction factor did not help either (experiment E14). That is why the German and French pages show measured official figures only, no colour.
Germany: indicator32%
Germany: most frequent colour54%
Colour accuracy, Germany, January 2019 – June 2026. Horizontal line: 95% confidence interval; dark line: the simple rule's level.
Experiment E8 · October 19, 2026
We changed the target and the horizon: wider bands, yes/no, 1–12 months
A hint at 12 months only
What we thought
A 3-point band may be too narrow for the monthly noise in price indices. A wider band, the plain question "will it outpace inflation" or a longer horizon may be learnable.
What we expected
As the band widens everyone's accuracy rises. The real measure stays the same: does a rule beat the simple forecast on the same target.
What came out
32 targets and horizons were tried. At 1–6 months no rule beat the simple forecast. Only at 12 months, on "will it outpace inflation", did a simple rule stand out: furniture 69% against 56%, major appliances 65% against 54%.
What we did
We count this as a hint, not evidence. Many cells were tried and 12-month windows overlap, so both can make the result look better than it is. It was tested on Germany and France the same day and did not carry over (experiment E10); from November 2026 it is tracked as a challenger in the forward register.
Furniture: rule (12 months)69%
Furniture: most frequent answer56%
Accuracy on "will it outpace inflation", 12 months. No confidence interval shown because the windows overlap. Horizontal line: 95% confidence interval; dark line: the simple rule's level.
Experiment E7 · October 19, 2026
We tried one-line simple rules
Simple rules did not beat it either
What we thought
Perhaps the problem is the weights; one-line rules that need no weights (such as "last quarter repeats") might do better.
What we expected
At least one rule beats the simple forecast by more than its confidence interval.
What came out
Furniture: best rule 36%, "most frequent colour" 42%. Major appliances: best rule 49%, "most frequent colour" 54%. None came close.
What we did
We saw that the 3-point colour band itself may be too noisy. The next experiment changes the target.
Furniture: best rule36%
Furniture: most frequent colour42%
Colour accuracy, furniture, the same 90 months. Horizontal line: 95% confidence interval; dark line: the simple rule's level.
Experiment E6 · October 19, 2026
We tried adding world raw-material prices
Rejected
What we thought
World prices such as oil, cotton and metals might signal Turkish furniture costs a few months in advance.
What we expected
With a single weight fixed in advance, accuracy rises by at least 5 points. Tuning the weight after seeing the result is not allowed.
What came out
Accuracy fell from 49% to 45%.
What we did
The raw-material layer did not enter the indicator. We did not tune the weight afterwards to make it fit; that would produce a model that fits the past and knows nothing about the future.
Without raw materials49%
With raw materials45%
Colour accuracy, the same 90 months. Horizontal line: 95% confidence interval; dark line: the simple rule's level.
Experiment E5 · October 19, 2026
We tested the indicator on real data from 2019 to today
Not distinguishable
What we thought
Product prices follow costs. The pressure computed from the cost series could tell whether a category will rise at least 3 points faster or slower than inflation over the next 3 months.
What we expected
Colour accuracy above the best simple rule, with even the lower end of the 95% confidence interval above that rule.
What came out
450 comparisons over 90 months: indicator 49% (interval 44–54), "always say the most frequent colour" 45%. The gap is not large enough to rule out chance. It gets the direction of change right close to two thirds of the time but misses about half of the big rises.
What we did
We started presenting the indicator as a "price-pressure summary" rather than a forecast, left confidence at "low" and removed the accuracy claim. The same test was repeated after the calculation method was corrected (experiment E21).
Indicator49%
Most frequent colour45%
Colour accuracy, January 2019 – June 2026, TÜİK consumer price sub-indices. Horizontal line: 95% confidence interval; dark line: the simple rule's level.
Experiment E4 · October 19, 2026
We screened 16 new data series that wanted to join the indicator
1 of 16 passed
What we thought
Official statistics, spending and expectation data and world raw-material prices might capture something the cost series we use miss.
What we expected
Some series pass four gates: a stable relationship in changes, a lead declared in advance, not measuring the same thing as an existing input, and a clearly lower error outside the test period.
What came out
Only one series passed all four gates. The others looked related only because everything rises together in high inflation, measured the same thing as an input we already use, or did not reduce the error.
What we did
Only the passing series stays as a candidate. A rejected series can be tried again only with a new reason written down in advance, not by changing its lag after seeing the result.
Experiment E3 · October 17, 2026
We turned the known weaknesses of the calculation into tests
Partly closed
What we thought
Every risk we can describe should exist as a test that fails today and passes by itself on the day it is fixed. That way no fix goes unnoticed.
What we expected
A handful of weaknesses are found and each is tracked by its own test.
What came out
We found five. The most important: a past value could be overwritten later, which would have meant the track record could be corrected after the fact. Three are closed, two are still open.
What we did
Each month's value is now stored with a chained seal; if it is changed later the system stops. The two open weaknesses are tracked by automatic tests that are expected to fail until they are closed.
Experiment E2 · October 17, 2026
We tried to fool the indicator with corrupt data
Gap found, closed
What we thought
Wrong or deliberately corrupted price data must either be caught or leave the colour unchanged. A colour that changes silently is not acceptable.
What we expected
Every corrupt-data scenario we prepared is caught or stays harmless.
What came out
In the first run one scenario passed the checks and silently changed one category's colour. It was caught during the test, before publication.
What we did
We added a check that closes this gap. Every scenario is now caught or leaves the colour unchanged. A new scenario is added every quarter.
Experiment E1 · October 17, 2026
We rehearsed the test rig on synthetic data
Rig works
What we thought
Before testing on real data we had to trust the test itself: for every past month it must use only data published on that day, and it must race the indicator against simple forecasting rules.
What we expected
The rig runs end to end, uses no data from the future, and flags an indicator that fails to beat the simple rule as "did not beat".
What came out
The rig ran. On synthetic data the indicator scored 54% and the "always say the most frequent colour" rule 68%, and the report correctly wrote this up as "did not beat".
What we did
No change to the indicator; this experiment only shows that the test is trustworthy. Until real price indices were loaded we published no accuracy claim.
Indicator54%
Most frequent colour68%
Colour accuracy, synthetic data. Horizontal line: 95% confidence interval; dark line: the simple rule's level.
Where we stand today
The stages that check the arithmetic and resistance to corrupt data are running, and caught two real gaps before publication. The indicator's ability to predict the future is not proven: in every test on real data its margin over a simple rule was small enough to be explained by chance, or it was below the rule (experiments E5, E9, E14). The one positive finding is the calendar rule for major appliances in Europe (E15); that is the calendar's success, not the indicator's.
That is why the US pages show measured official figures and no colour: in our tests the cost-based colour was not distinguishable from the simple rule (table below). There is no forward record for the US yet; nothing forward-looking is claimed until one exists and has been scored.
Results so far
This tests the cost-based colour we used until October 2026. No country page shows that colour today; the pages show measured official figures instead, and this table is the reason.
How often the indicator's colour matched what actually happened over the next 3 months (every month since 2019), next to the best simple rule (usually “always say the most frequent colour”). The range is the 95% confidence interval. “Better” or “below” means the whole range lies on one side of the rule; otherwise the result is “not distinguishable”.
Country
Indicator
Simple rule
Result
Belgium
33% (29–38)
51%
Below the rule
Canada
40% (36–45)
42%
Not distinguishable from the rule
Germany
32% (28–36)
54%
Below the rule
Spain
33% (28–37)
63%
Below the rule
France
36% (32–40)
58%
Below the rule
Italy
28% (24–33)
59%
Below the rule
Netherlands
40% (35–44)
39%
Not distinguishable from the rule
Austria
22% (18–26)
52%
Below the rule
Türkiye
49% (44–54)
45%
Not distinguishable from the rule
United States
40% (36–45)
43%
Not distinguishable from the rule
Tests cover January 2019 to June 2026; backtests run on October 19, 2026.
Countries without a row have not been tested yet.
In none of these tests could we show that the indicator predicts better than the simple rule.