Data basis: 17 breakout rules on DAX, FTSE 100, Dow and Nasdaq 100, 15-minute candles, corrected calculation chain. Training 2015–2019 (37,540 trades), test 2020–2022 (22,636 trades). The holdout from 2023 was not loaded. The same procedure on two random markets. Costs per round trip as in every study of this series, standard errors clustered by day. Agent, benchmark rule and gate were fixed before the first run (registered on 11 November 2026, SHA-256 5ab9e565…, correction run 524969f5…). No trading recommendation.
Reinforcement learning promises what is hard to capture with fixed rules: an agent that decides at every moment of a running trade whether to hold, trail the stop, close partially or exit entirely, and that learns these decisions by itself from tens of thousands of trades. We had trained such an agent in the summer of 2026. In a walk-forward test it was ahead in all 20 windows. Then we found four features that looked into their own future, and later a second error in the calculation chain that showed profits even on a random market. Both errors are fixed. What was missing was the cross-check: the same agent, retrained on the clean basis.

Left the distance of the agent to the best fixed rule in the test 2020–2022, per trade in R, with one standard error: as trained in the summer (orange) and in the correction run with uncapped reward (blue-grey). Right the same distance in the correction run, training against test.
1. The setup
| Building block | Definition |
|---|---|
| Agent | PPO with a network of two layers of 128 neurons each, 3 million training steps, unchanged from the summer, no tuning |
| Actions | seven per candle: hold, stop to break-even, tight trailing stop, wide trailing stop, close, add, close partially |
| Features | only information known at the time of the decision |
| Benchmark rule | the best of four fixed rules (hold, wide trailing stop, tight trailing stop, stop to break-even), chosen in training |
| Variant A (primary) | every trade ends at the end of the session at the latest |
| Variant B | trades run for up to five trading days, as in the summer |
| Random market | the same procedure on two artificial markets without direction |
The gate for variant A: the agent has to beat the best fixed rule in the test with t ≥ 2, this lead has to be at least two standard errors above that on both random markets, and the agent itself has to be positive after costs. On a random market every exit rule has the same expected value, namely minus costs. A lead there would point to a further error.
2. Result: gate missed on all three points
| Test 2020–2022, variant A | best fixed rule | agent | agent minus rule |
|---|---|---|---|
| real, seed 42 | wide trailing stop, +0.001 R (t 0.0) | −0.062 R (t −8.1) | −0.063 R (t −2.9) |
| real, seed 43 | wide trailing stop, +0.001 R | −0.035 R (t −2.2) | −0.036 R (t −1.9) |
| random market 1 | hold, −0.085 R | −0.081 R | +0.004 R (t 0.2) |
| random market 2 | wide trailing stop, −0.073 R | −0.084 R | −0.011 R (t −0.6) |
22,636 trades per run on the real markets. The seed is the random number with which the training starts; the second run shows how strongly the result depends on it.
The agent is not better than the rule but worse, and clearly negative after costs. On the random markets the distance is at zero, as it has to be, so the calculation is clean. Measured against the random markets the distance is −0.067 R (t −2.0) and −0.052 R (t −1.8). In variant B with a holding period of up to five days the agent sits at −0.044 R (t −3.0) and 0.056 R behind the best rule (t −1.4).
In passing, the table shows what the rules themselves are worth: with the best fixed exit, the 17 breakout rules sit at zero after costs in the test.
3. What the agent learned
The agent holds for 2.7 candles on average, so about 40 minutes. It closes early. In doing so it cuts off the part of the distribution a trailing stop lives on: with the wide trailing stop 3.9 % of trades end above 5 R, with the agent 0.2 %. It still pays the costs on every trade.
In training things look different. There the agent beats the rule by 0.055 R per trade (t 3.5), with the second seed by 0.114 R (t 7.4). On the first random market, where there is nothing to learn, it beats the best rule there by 0.085 R in training (t 4.7). A lead that arises just the same on a market without structure is memorised noise. In the test nothing of it is left.
4. The objective function was wrong, the correction does not help
The follow-up check found an error of our own in the training setup that already existed in the summer. The reward was normalised during training and capped in the process, at about 5 R when converted. About 4 % of the trades with the wide trailing stop lie above that and contribute roughly 0.10 R per trade to the mean. So the agent never saw what the big runners are worth. Calculated with capped results it was even better than the rule in the test (−0.066 against −0.098 R). It learned what the objective function demanded.
That is why there was exactly one correction run with uncapped reward, pre-registered, otherwise identical:
| Test 2020–2022, uncapped | agent | agent minus rule | in training |
|---|---|---|---|
| real, seed 42 | −0.033 R (t −2.6) | −0.034 R (t −1.8) | +0.073 R (t 4.6) |
| real, seed 43 | −0.060 R (t −5.8) | −0.061 R (t −3.1) | +0.073 R (t 4.8) |
| random market 1 | −0.085 R | 0.000 R (t 0.0) | +0.111 R (t 6.1) |
| random market 2 | −0.079 R | −0.006 R (t −0.3) | +0.131 R (t 9.3) |
By the reading fixed in advance, this is a tie with the first seed and a worse result than the rule with the second. Without the cap, too, the agent cuts off the big runners (0.8 to 1.2 % of trades above 5 R instead of 3.9 %), and without the cap, too, it learns noise in training, most strongly on the random markets. There is no third run; that was fixed in advance.
What this means
The finding from the summer was an artefact of future information and a faulty calculation chain. Without both, the same agent learns to close early and loses to a rule that can be described in one line. We take three things from this:
- The lead in training says nothing. Only the random market shows that it arises without any structure as well.
- Check the objective function. A common default setting of the library made exactly the trades invisible that matter.
- More than one seed. Two training runs on the same data are 0.03 R apart in the test. A single run is one draw.
The result matches our measurement of machine learning in intraday trading. We will not train further reinforcement learning on these features.
Limits
- One agent, one setting. No tuning, no other architecture, on purpose. This does not show that reinforcement learning for exits cannot work in principle, only that this agent finds nothing on these features.
- Features from candles. The agent does not see order book, options or news data.
- Three test years, four markets. 15-minute candles, without financing costs and without slippage on the exit.
- Second look at the test. The correction run uses the same test period with a change found afterwards. It was pre-registered but is not an independent test.
- Seeds. Two training runs on the same data arrive at noticeably different results. The statement rests on the fact that neither beats the rule.
All pattern families and holdouts in the overview. Related: brute-force search against random markets.
📄 Full study as PDF: 7 pages incl. all tables, figures and limits (in German).
Disclaimer: Historical statistics are no guarantee of future market behaviour. This study is not investment advice. Trading carries a risk of loss up to total loss.