Data basis: 123,193 episodes, 10 markets, 23 setups, 05 Jan 2015 – 05 Jun 2026. Exit: trailing stop BE 0.5 / TS 1.0 / step 0.5, net of spread and slippage. The subject is not the trade results but the context features of our episode base — the columns a model is allowed to see at the moment of entry. No trading recommendation.
Lookahead bias is the mistake everyone warns about and everyone eventually builds anyway. Not because you read in tomorrow's close — it is never that crude. It happens because a context variable that is defined for the day gets queried at a point where the day has not got that far yet.
That is exactly what happened to us. Four columns in our episode base were a glimpse into their own future. This document describes how the leak arose, what gave it away, how we proved and closed it — and what could not be repaired. We are writing it up because the class of error is general and the diagnostic signature transfers to any backtest.
1. The error
Alongside the trade result, every episode in our base carries a set of static context features describing the day: the previous day's high and low, a compression measure from the previous day's OHLC, the distance from the entry to the boundaries of the 30-minute opening range, and a directional bias relative to the cash-open price.
The features were pulled from a daily row — without checking whether each value was already known at trigger time. For previous-day values that is harmless; they have been fixed since the prior evening. For two pairs of features it was a leak:
| Feature | defined by | unknown for | affected |
|---|---|---|---|
| Distance to upper / lower opening-range boundary | 30-minute opening range | triggers inside the first 30 minutes | 60,129 episodes (48.8%) |
| Directional bias (absolute and direction-signed) | cash-open price | triggers before the cash open | 8,479 episodes (an FX morning setup 93.5%, a gold setup in the Asian session 100%) |
A setup that triggers after ten minutes was handed the opening-range boundaries that only settle twenty minutes later. A pre-open setup was handed an open price that did not exist yet. For half of all episodes, the state therefore contained information about the price path after entry.
We had fixed this same class of error once before. On 11 Jun 2026 we added a time check for the features derived from the first 5 and 15 minutes. That fix was correct — but it only covered the two columns we happened to be looking at. The other four went unchecked.
2. What gave it away
The leak did not surface during construction. It surfaced as a by-product of our edge persistence study, where a simple neighbourhood model (kNN, walk-forward) on the context features produced a separation that was too good to be true.
Even then, the first suspicion did not come immediately. The directional bias on its own looked unremarkable: correlation with the trade result +0.011. Only the direction-signed variant — bias times trade direction — showed +0.173. The effect averages out across long and short, because a positive bias helps longs and hurts shorts. Checking a direction-neutral feature in isolation is not enough. You have to multiply it by the trade direction, or the leak hides in the symmetry.
3. The proof
The diagnosis that proves the bug beyond doubt is simple: split the episodes by whether the trigger fell before or after the moment the feature becomes genuinely known, and measure the correlation with the trade result separately.
| Feature | trigger before the moment | trigger after | after the fix |
|---|---|---|---|
| Distance to upper OR boundary | +0.198 | +0.007 | constant 0 before OR end |
| Distance to lower OR boundary | +0.213 | +0.003 | constant 0 before OR end |
| Directional bias × direction | +0.173 | +0.010 | constant 0 before open |
That is the signature of a leak, not an edge. A genuine feature would predict at least as well after it becomes known as before. Here it is the reverse: as soon as the feature is legitimately available, its predictive power is gone. The +0.2 comes exclusively from episodes for which the feature should not have existed at all.
At model level, the difference looked like this: the kNN tercile spread — expected value of the best minus the worst third, walk-forward, within cell — fell from +0.726 R to +0.191 R. Three quarters of the apparent predictive power was future.
The sharpest diagnostic detail was calibration. Before the fix, the model under-predicted: where it forecast +0.316 R, +0.493 R materialised. After the fix it over-predicts, as it should: forecast +0.241 R, realised +0.197 R. A clean, noisy predictor must over-predict — regression to the mean. A predictor that systematically under-predicts knows something it should not. This rule is the cheapest lookahead test we know, and it requires no look at the code.
4. Fix and verification
The fix is a time check: every context feature is set to zero if the trigger precedes the moment the value genuinely settles. The base was rebuilt from scratch.
Three controls had to hold afterwards:
- All four validation anchors unchanged. The reference values against which we check every rebuild of the base landed exactly on the old numbers.
- R values across all 123,193 episodes bit-identical (max |ΔR| = 0). The fix changes features, never results.
- The old state is archived, so every before-and-after claim remains reproducible.
The second control is the important one. Because no trade result was touched, all previous setup verdicts and the persistence study remain valid. They rest on results, not on features. The leak corrupted the explanations, not the measurements.
5. What could not be saved
Before the fix we had trained several reinforcement-learning models on this base. The observation such a model sees at every step included the context features wholesale — and with them the four oracle columns.
These models are contaminated and cannot be cleaned retroactively. A neural network that learned during training to respond to a column containing the future has built its weights around that signal. There is no way to subtract the signal from the weights. Their evaluation numbers are too optimistic, and most so exactly where the leak was largest: on setups that trigger early in the session. The only consequence is to retrain, and no result from the old generation counts.
That is the expensive part of a lookahead bug. The base can be rebuilt in an hour. Everything that was learned on the base has to be thrown away.
6. What this means
Three rules we have treated as mandatory since:
Every context feature carries a timestamp from which it is known. Not the day — the time of day. The time check lives in the construction of the base, not in the model; otherwise someone forgets it.
Direction-neutral features are checked multiplied by trade direction. Otherwise the leak cancels across long and short and stays invisible.
Calibration is a lookahead test. A model whose forecasts come out too small is suspect. A clean model over-predicts.
That we had to fix the same class of error twice is the real lesson. The first fix was local — two columns that happened to be conspicuous. The second was structural — a time check for every column, with a search script that queries the error class across all features. Only the second one is a fix.
7. Limits
- This is a post-mortem, not a backtest. The numbers describe the size of a leak, not an edge.
- The correlations are in-sample diagnostics. They prove the leak; they do not measure the remaining predictive power of the clean features — at +0.191 R in the kNN tercile spread it is well below the contaminated value and is itself not yet confirmed out of sample.
- Not tested: whether further, subtler leaks exist that trigger none of the three signatures (before/after split, direction-signed correlation, under-prediction). The search script covers the class "daily value queried before its moment", not every conceivable class.
- The contaminated RL models were not retrained. All statements about the size of their optimism are qualitative until clean comparison runs exist.