Friday, October 30, 2026.
This week’s letters covered walk-forward testing and the three months of data I keep off-limits to myself. Both exist because backtests are easy to fool. These are the four questions I now ask of any backtest before I believe it, my own included.
Was every number graded on days the strategy never saw? If the rules were tuned on the same days they were scored on, the score measures the tuning. Walk-forward testing (Tuesday’s letter) is the fix.
How many versions were tried before this one? Test twenty worthless ideas and about one will pass by chance. A result means little without the count of attempts behind it. In July I tested nine ideas and one produced a clean win, and I print the nine.
What does it cost to trade? In September I measured my own paper bot. A round trip cost about 142 basis points of the money traded, against an edge of about 16 per signal. A backtest that ignores costs can show an edge that trading never delivers.
What does something simple score on the same days? If a plain baseline gets most of the way there, the complicated strategy has to earn its complexity. A plain logistic regression got 96 percent of the way to my deep model’s score.
None of these needs special software. They need the discipline to ask them before you see a result you like.
The Tape
The Graded Tape grades every signal my model fires, every morning, wins and misses, at gradedtape.com. Its numbers change nightly, so I link to it here instead of printing a figure that would be out of date by the time you read this.
From the archive
Tuesday’s letter was “Walk-forward testing, explained for people who lost money after a good backtest”, and Thursday’s was “Why I froze three months of data I’m not allowed to look at”. The September research log is free to read in full and shows these questions applied to a month of real tests.
The note goes out free every Friday.
Starting today, a paid subscription comes with a 14-day free trial. Paid subscribers get the build notes at the end of each letter, the working papers that close each arc and a monthly research log with each study’s full tables.
Lab notebook, education and research purposes only. Not investment advice, not a solicitation; no advisory relationship is created by reading this. I trade a small account and describe my own process, errors included.



