Your Backtest Is Lying to You
$4.99
Buy on Amazon View on Apple BooksPrefer it in print? See every format on Amazon →
Price on Apple Books. Amazon’s price may differ.
- Published
Description
One of our backtests showed a 191% return. Fixing one line of bookkeeping code turned it into 8%. Nobody had cheated. The backtest had simply been too kind, in a way none of us noticed at first.
This book comes from more than 400 numbered experiments on crypto perpetual futures, run by an individual trader with an AI research assistant, with every cost counted and every plan written down before the results. It collects the fifteen traps that made our tests look better than they were, and the checks that would have caught each one earlier.
Some of the traps inside:
• The return you booked is not the return you got. One line of bookkeeping turned 191% into 8%.
• Funding you never received. A multi-asset trend test reached a t-statistic of 3.67 only because funding had been filled in for years that had none; on price alone it was 1.17.
• Choosing the days after you have seen them. Picking events by the day's full range earned +31.0 basis points an event; picking with only what was known beforehand gave -18.4.
• One seed is a coin toss. Six runs of the same model gave Sharpe ratios from 0.14 to 1.30.
• The more you try, the luckier you look. Across hundreds of search rounds, genuinely new survivors came up in about 0.6% of them.
• Coins that are no longer in your data. Survivorship quietly removes the losers before you start.
Every chapter opens with a true story from the research records, explains what went wrong in plain English, gives a test you can run on your own backtest, and ends with a five-minute checklist. The afterword turns the fifteen checks into one page.
This book does not tell you what to trade. There are no signals and no strategy to copy. A few ideas are still being tested forward, and we do not count any of them as proven; this book does not publish them.
For anyone who builds backtests, reads other people's results, or does research with an AI assistant.
For education only. Not investment, legal or tax advice. A backtest is not a live trading record. Trading leveraged derivatives can lose all the money you put in. Not endorsed by Binance, Anthropic or OpenAI.
AI-generated content: the text of this book was written by an AI research assistant, Anthropic's Claude, which also ran most of the research. Some research tasks were done with OpenAI's Codex. The author set the project's direction and made every decision involving money, accounts or live trading.