Data basis: 15 markets (DAX, FTSE, CAC, SMI, Dow, Nasdaq, S&P 500, Nikkei, Hang Seng, ASX 200, gold, EURUSD, GBPUSD, USDJPY, bitcoin), Dukascopy minute data. Search period 5 January 2015 to 30 December 2022, physically separate holdout 1 January 2023 to 5 June 2026, exactly one holdout run per candidate. Costs per market as spread plus slippage, raised outside cash hours. Every calculation chain also ran on random markets with identical code. Global Benjamini-Hochberg correction (q = 0.10) over all holdout tests. No trading recommendation.
The question was simple: what do you find if you forget everything you think you know and search the data from scratch? No own setups, no favourite ideas, no weighting by gut feeling. Instead, every pattern family that books, forums and papers call profitable, on the same data and under the same rules.
The rules were strict, because a pattern search always finds something otherwise. Test 900,000 variants at a 5% error rate and you get about 45,000 "significant" hits without any of them being real. So every variant was logged, every calculation chain was checked against a random market, and only a small number of candidates named in advance were allowed into the holdout at all.
Number of logged tests in the 16 groups, log scale. Every variant counts in the multiple-testing correction, including those that looked bad.
1. The protocol
| Component | Implementation |
|---|---|
| Search period | 2015 to 2022, the only place where searching and optimising was allowed |
| Holdout | 2023 to June 2026, stored physically apart, exactly one run per candidate |
| Random market | every minute candle mirrored around its open with probability ½: same volatility, same daily profile, no direction |
| Ledger | every tested variant with sample, mean and t-value |
| Benchmarks | random walk, placebo days, placebo levels, entry one or two minutes later, costs times 1.5 |
| Verification | every candidate reproduced by an independent re-implementation before it could enter the holdout |
| Correction | Benjamini-Hochberg over all holdout tests together |
The random market was the most important tool. It has the same volatility, the same quiet and hectic hours, the same jumps, but no direction. Any rule that makes money there measures the calculation chain, not the market. How effective this protection is shows in the study on the random market as a ruler: our own older backtest chain earned up to 0.5 R per trade there.
2. The families and their results
| Family | What was tested | Result | Study |
|---|---|---|---|
| Candlestick patterns | engulfing, pin bar, doji, inside bar, NR7 and relatives | random level | Candlestick patterns |
| Chart formations | head and shoulders, double tops, triangles, flags | target rate as in a random walk (46.5 vs 46.8%) | Formations, measured moves |
| VWAP and indicators | VWAP bands, RSI, MACD, Bollinger, confluences | outside bitcoin no variant net t ≥ 3 | VWAP and indicators |
| Al Brooks | always-in, H2/L2, wedges, failed breakouts | "80% of breakouts fail" = random rate of 80.3% | Al Brooks |
| Hougaard | 3,780 management policies | management alone on the random market = minus costs | Hougaard |
| Market Wizards | Raschke, Williams, Crabel, Sperandeo, Fisher, Schwartz | 54 hits at p < 0.05 where about 51 are expected | Market Wizards |
| Market structure | retests, sweeps, FVG, order blocks, OTE, Silver Bullet | real levels get retested no more often than invented ones; 0 of 3,308 rules pass | Smart money concepts |
| Price dynamics | gaps, shocks, opening range, levels, round numbers | random walk; round numbers break more often than placebo levels | Gaps and levels |
| Turning points | overnight range and prior close | clustering = dwell-time artefact | Overnight range |
| Time and calendar | drift map, weekday, turn of month, expiry | no index window positive net | Time and calendar |
| First vs last hour | "the close follows the open" | agreement 48 to 51% | First and last hour |
| Macro events | FOMC, NFP, CPI, ECB, earnings | intraday chance level | Macro events |
| Markets among each other | lead-lag, handovers, relative value | real, but at most 0.58 times the costs | Lead-lag |
| Psychology and risks | fear spikes, sentiment, geopolitics, moon | random markets produce more strong results | Psychology |
| Single idea | Nasdaq falls at 8 am, short the retest | negative in every variant | Nasdaq retest |
| Literature | seven published anomalies rebuilt, pre-registered | gross 40 to 80% of the paper size, net zero | Literature |
| Machine learning | 98 features, ridge and XGBoost, walk-forward | out-of-sample IC +0.004 to +0.006 | Machine learning |
| Brute force | 682,000 rules, 20 random-market searches | best finding worse than the typical random finding | Brute force |
| Fixings | London, gold, bitcoin, yuan, ECB | no edge | Fixings |
| Further forced flows | volatility-target funds, Japanese fiscal year-end | no edge | Forced flows |
| Index rebalancing | closing auction on event days | FTSE confirmed globally | Index event days |
| Tokyo fix | Japanese companies buying dollars at 09:55 | passed locally, not globally, forward test | Tokyo fix |
3. The holdout
Out of all families, 19 candidates made it into the holdout. In the search period all of them looked plausible, with t-values between 2.2 and 4.9. That is exactly what you expect when you pick the best out of hundreds of thousands of variants. The holdout shows how much of that was selection.
t-value per candidate in the untouched holdout from 2023 to June 2026, one run each. Orange: confirmed after the global correction. Light: passed locally, not globally. Grey: same direction but not significant. Dark: rejected.
| Outcome | Count | Examples |
|---|---|---|
| confirmed globally (q = 0.0008) | 1 | FTSE on index event days, +11.5 bps per event day, t 3.9 |
| passed locally, not globally (q 0.13 to 0.15) | 3 | two Tokyo fix rules, one financial stress rule suspected of revision lookahead |
| same direction, not significant | 6 | US reversal after a large prior day (65% from a single day) |
| rejected | 9 | including a close-momentum rule from the literature that ran significantly reversed in the holdout (t −3.7) |
The most interesting case is the pair of index event markets. Sydney was stronger in the search period (t 3.4) and died in the holdout. London was weaker (t 2.3) and survived. That is a reason to treat even the one finding with care.
4. What remained
Two things have a counterparty that must trade at a fixed moment, and both point the same way in every control:
- Index event days in the FTSE: passive funds trade index changes in the closing auction. Holdout +11.5 bps per event day, +11.9 bps on real broker ticks, about eight dates a year. To the study.
- The Tokyo fix in USDJPY: Japanese companies buy dollars at 09:55. +4.5 to +5.6 bps per trade on real ticks, not confirmed globally, pre-registered forward test until September 2027. To the study.
Then there are structures that are real but not tradable: from a move of 0.3 to 0.5 ATR onwards, trends continue in 52 to 54% of cases instead of 50%. Markets measurably lead each other. Round numbers get broken more often than invented levels. All of it is smaller than the spread of a CFD.
5. What it means
Directional patterns on CFD minute data are empty after costs. That holds for the classic chart patterns as much as for the methods of well-known traders and for machine learning. What remains has a mechanism: someone who must trade at a fixed moment, whatever the price. These edges are small and rare, but they do not disappear because someone else knows them too.
The second lesson concerns the tool. The biggest source of error was not the market but the calculation chain. Without checking a backtest against a random market, you cannot tell whether you are measuring the market or your simulation.
6. Limits
- Mechanical rules on CFD minute data with fixed costs. Discretion, order book data, futures costs and multi-day holding periods were outside the scope.
- Power of the holdout. 3.4 years are not enough to confirm or rule out real effects of 1 to 2 basis points. "Not found" means "not demonstrable as a rule with a positive expectation".
- Patterns that only emerged from 2023 (for example with the boom in zero-day options) cannot be found by this design. That was the deliberate price of a clean holdout.
- BID data create false effects around the rollover (17:00 New York) and the gold reopening. These windows were blocked or computed with real spreads.
- Revised macro data create lookahead unless real-time vintages are used. The one affected candidate is marked accordingly.
Addendum, 6 October 2026
Eleven follow-up measurements have been added since the scan: our own rules on the corrected chain and on real ticks, news trading, higher timeframes after CFD financing, data beyond price, more forced flows. The count now stands at 41 holdout tests of trading rules, plus four tests of filters that only predict the range. After the global multiple-testing correction over all 41 rules, two pass: the index event day in the FTSE 100 (q 0.002) and RSI(2) dip buying on US index CFDs (q 0.003, with caveats: selected among related rules, small in risk units). Everything else sits at q ≥ 0.14. The four filters (VIX term structure, VVIX, low dealer gamma) pass as predictors of the next session's range, not its direction.
The follow-up measurements: Our own breakout rules, re-measured, Night-session breakouts in Asian indices and gold, News trading in gold and bitcoin, Swing patterns after CFD financing, VIX term structure, VVIX and dealer gamma, The overnight effect with CFDs, RSI(2) dip buying on index CFDs, Forced flows, second round, Gold trend following is market return, The first 5-minute candle and the +982%, The day after large US moves.
Addendum, 8 October 2026
Four more measurements have been added: widely shared trader ideas about charm, double tops with a liquidity sweep, Fibonacci targets and the 15:50 rule, bitcoin over the weekend, in the order book and at liquidation levels, the size of the move after the open, and the expected move from option prices. None of them yields a tradable rule.
Correction to the count of 6 October: there we reported 41 holdout tests of trading rules. The 17 holdout runs of the bitcoin study were missing from that number. With them and one observation from the expected-move study the count is now 59. After the global multiple-testing correction exactly two still pass: the index event day in the FTSE 100 at q 0.002 and RSI(2) dip buying on US index CFDs at q 0.005.
The new measurements: Charm, double-top sweeps, Fibonacci targets and 15:50, Bitcoin: weekend, order book and liquidation levels, A bigger open after a strong pre-open hour, The expected move from option prices.
Addendum, 20 October 2026
Nine more measurements have been added: the published imbalances of the opening and closing auctions (SPY, QQQ and 20 Nasdaq 100 stocks, data from Databento), three freely available alternative data sets (FINRA short-volume ratio, SPY fund flows, Wikipedia views) and the cross of EMA(9) and VWAP. All were pre-registered, and none yields a tradable rule. A single one, the closing imbalance of the Nasdaq 100 stocks, passed the search period (t 2.6) and fell to zero in the holdout.
On the count: since 8 October, 19 holdout tests of trading rules have been added, six from further forced flows (tax dates, spot bitcoin ETF flows, Stock Connect, CAC 40 and SMI on index event days), twelve checks of filters and exits on our own day rules, and the Nasdaq imbalance of this round. That makes 78 rule holdouts, plus four filter tests. After the global multiple-testing correction across all 82, exactly two rules still pass: the index event day in the FTSE 100 (q 0.001) and RSI(2) dip buying on US index CFDs (q 0.003). The next best rule is at q 0.14.
The new measurements: SPY closing auction, SPY opening auction, closing imbalance of the 20 largest Nasdaq 100 stocks, QQQ closing auction via Nasdaq, QQQ opening auction via Nasdaq, FINRA short-volume ratio, SPY fund flows, Wikipedia views of crash and recession, EMA9 crossing the VWAP.
Addendum, 21 October 2026
Two more measurements: the RSI(2) pullback family on twelve markets outside the US indices and the US Treasury auctions. The pullback rules passed the search period with 23 candidates, and none held in the holdout. The auctions already missed the gate in the search period. Before the holdout we now check every candidate on real CFD minute data. This exposed a data error in free EURUSD daily candles (t 16, zero on CFD minutes).
On the count: with the 23 holdouts of the pullback study there are 101 rule holdouts, plus four filter tests. After the global multiple-testing correction across all 105, exactly two rules still pass: the index event day in the FTSE 100 (q 0.001) and RSI(2) dip buying on US index CFDs (q 0.004). The next best rule is at q 0.17.
The new pages: pullback rules on 15 markets, US Treasury auctions, our workflow since October 2026.
Addendum, 26 October 2026
Nine more studies since 21 October, plus an addendum. Calm uptrends that accelerate, the climax fade, the EIA inventory report and the weak week before Friday missed the gate in the search period, the volume profile showed no difference from a random level. For four rules we ran a single holdout anyway on our decision, as a labelled exception without a passed gate: the reversal after intraday acceleration, EMA9 × VWAP only at a high VIX, and the S&P 500 additions from the effective date and mirrored from the announcement. None held. RSI(2) dip buying on S&P 500 stocks, by contrast, passed the gate and the holdout (t 2.00) but is at q 0.22 after the global correction.
On the count: as of 23 October 2026 there are 106 rule holdouts, plus four filter tests. After the global multiple-testing correction across all 110, exactly two rules still pass: the index event day in the FTSE 100 (q 0.0015) and RSI(2) dip buying on US index CFDs (q 0.0044). The next best rule is at q 0.18.
The new pages: calm uptrends that accelerate, acceleration intraday, the climax fade, the EIA inventory report, the weak week before Friday, the volume profile (POC), S&P 500 additions, single-stock events after earnings, RSI(2) dip buying on S&P 500 stocks, EMA9 × VWAP with the high-volatility test.
Addendum, 11 November 2026
Thirteen more studies since 26 October. Ten are pre-registered rule tests, none gives an edge after costs. Missing the gate in the search period were Fibonacci confluence, overnight calendar effects, the TWAP on trend days in both directions, after-hours earnings breakouts, alternative data before earnings, liquidity withdrawal on gold ticks, false breakouts of the overnight range, the channel break, and insider purchases on a CFD basis. For insider purchases the holdout confirmed only the first day's excess before costs (t 5.03); net, nothing is left. In the study of position size and exit by volatility one candidate passed the search period and sat at t 0.09 in the holdout. Two studies are descriptive: bottoms after sell-offs and statistical arbitrage with S&P 500 stocks; the double bottom is a structure test against the random market.
On the count: as of 11 November 2026 there are 108 rule holdouts, plus four filter tests. This includes a re-check of 27 October that we had not counted here so far, and the volatility candidate. After the global multiple-testing correction across all 112, exactly two rules still pass: the index event day in the FTSE 100 (q 0.0015) and RSI(2) dip buying on US index CFDs (q 0.0045). The next best rule is at q 0.19.
The new pages: Fibonacci confluence, overnight calendar effects and 02:00 to 06:00, the TWAP on trend days, after-hours earnings breakouts, insider purchases, alternative data before earnings, bottoms after sell-offs, liquidity withdrawal on gold ticks, position size by volatility, the double bottom measured mechanically, statistical arbitrage, the false breakout of the overnight range, the channel break.
Addendum, 14 November 2026
One more study, this time on purchased order-book data: market maker traces in the ES order book, 40 days, every single order. The market makers' retreat after losses is visible but misses the gate in the search half (t −2.7 instead of −3). After one-sided flow bursts the direction is a coin flip. Because no candidate passed the gate, no rule went into a holdout; the count stays at 108 rule holdouts.
📄 Full study as PDF: 10 pages incl. all tables, figures and limits (in German).
Disclaimer: Historical statistics are no guarantee of future market behaviour. This study is not investment advice. Trading carries a risk of loss up to total loss.