Every idea class on this page failed again and again after costs. Skip them, and the backtests you save go to ideas with a chance.
Out-of-sample dead ends
Fewer than 20% of tests beat the benchmark and the median result was clearly negative.
| Idea class | Tests | Beat benchmark | Median Sharpe vs bm |
|---|---|---|---|
| Calendar effects · single stocks | 34 | 0% | −1.47 |
| Calendar effects · US equities | 53 | 17% | −0.90 |
| Lead-lag signals · international equities | 30 | 7% | −0.21 |
| Lead-lag signals · US equities | 289 | 17% | −0.23 |
| Combined signals · multi-asset portfolios | 27 | 15% | −0.14 |
| Calendar effects · all markets | 126 | 16% | −0.93 |
| Intraday rules | 347 | 18% | −0.50 |
Signals that flipped
10 public signals looked significant before 2015 and then reversed more often than any random placebo signal did.
| Signal | Family | Significant cells | Kept direction |
|---|---|---|---|
| P4D_MOF_FOREIGN_JP_BONDS | Cross-border capital flows | 225 | 2% |
| P4D_MOF_JP_FOREIGN_BONDS | Cross-border capital flows | 43 | 7% |
| P4D_MOF_JP_FOREIGN_EQUITY | Cross-border capital flows | 23 | 0% |
| P4D_SEC8K_ITEM_2_05 | Regulatory and corporate filings | 241 | 13% |
| P4D_PHILLY_FUTURE | Expectations and surveys | 106 | 9% |
| CAL_YEAR_END | Calendar | 130 | 11% |
| P6D_SNB_SIGHT_CHG4 | Foreign central banks | 108 | 12% |
| P6D_CASS_EXP_YOY | Freight | 51 | 14% |
| P6D_WSTS_BILLINGS_YOY | Semiconductors | 20 | 0% |
| P4D_GPR_COUNTRY_SAU | Uncertainty and news | 115 | 5% |
Even in hindsight
Classic indicator rules on intraday bars lose after costs over the full history. See the backtest upper bounds.
Caveats
The 2015–2026 test window has been looked at many times during research, so we label it and do not call it a clean holdout. "Beat benchmark" compares each test with a fixed asset mix or buy-and-hold in the same window; a weak benchmark flatters the result. Many tests use US-listed ETFs.
Terms used: base rate, out-of-sample test, placebo test, dead end.
Let your AI check how ideas like yours did before it runs a single test. In our pilot, agents that checked first spent 1% of their backtests on known dead ends instead of 15%, and every backtest not wasted is time and tokens you keep. Cite this entry as ER-DEAD, as of 2026-11-01.
Questions
Which trading strategies do not work?
In our register: calendar effects and lead-lag signals on US equities, intraday timing rules and classic indicator rules on short bars, among others.