Every verdict in BacktestAtlas is made with five safeguards: out-of-sample test, every try counted, luck baseline, costs included, frozen verdict.
1. Out-of-sample test
Every idea is developed on data up to 2014 and judged on 2015–2026. Classic technical rules are additionally reported as in-sample upper bounds.
2. Every try counted
Every variant is counted, including failures. A result is judged against how many attempts it took.
3. Luck baseline
Random placebo signals run on the same markets. A real signal family has to beat that range to count. For a single backtest, the expected best result of pure chance after the same number of tries is the bar.
4. Costs included
Spreads, slippage and financing are included in every out-of-sample test.
5. Frozen verdict
See the public chain. From November 2026 every verdict is timestamped and hashed when it is made, so the track record cannot be edited later.
What we do not claim
The test window has been viewed many times, so it is labelled and not called a clean holdout. Many tests use US-listed ETFs. The strategy library of 542,100 backtests is in-sample: it shows how rules did over the full history, not on data they were not built on, and is labelled that way in every answer. We publish aggregated statistics only, no prices or trades.
Model knowledge check
On 2026-11 we asked three Claude models 22 questions about register results and 4 about well-known papers. Mean error on the papers: haiku 4.5 pp, opus 0.0 pp, sonnet 6.8 pp; on the register: haiku 18.2 pp, opus 16.7 pp, sonnet 19.7 pp.
Let your AI check how ideas like yours did before it runs a single test. In our pilot, agents that checked first spent 1% of their backtests on known dead ends instead of 15%, and every backtest not wasted is time and tokens you keep. Cite this entry as ER-METHOD, as of 2026-11-01.
Questions
How is the evidence tested?
Development up to 2014, test 2015–2026, costs included, every attempt counted, signals compared with placebos.