Learn / Learn by Doing · Lesson 10 of 10
Backtesting without misleading results
Prerequisites: How to use Replay the Market · Bid–ask spreads and liquidity
01 · Overview & Worked Setup
Point-in-time means the rule can see only what was published as of that date. Survivorship bias is keeping only the funds that still exist. Selection bias is trying ten rules and reporting the one that won. Costs are spreads and fees, not zero.
Walk-forward locks a rule on a training window, then scores a later window that was not used to fit. Replay on this site already hides future sessions until you advance the clock. That is the same idea in a blotter.
Teaching lab
Walk-forward windows
Twelve classroom sessions. Training never includes a later test day. Survivorship, costs, and point-in-time membership are still outside this slider.
- Train 1–4, then test 5–6
- Train 5–8, then test 9–10
- Windows
- 2
- Look-ahead
- Blocked
Each test block starts after its own training block.
What this scenario means
A backtest that trains on the same days it scores is not a test. Dropping funds that later died is survivorship bias. Ignoring spreads and fees overstates results.
Guided challenge
Why is a single in-sample fit on all 12 days not in this list?
Educational simulation only. Hypothetical results are not forecasts or trade recommendations. Hypothetical classroom numbers. One standard contract is 100 shares unless labeled otherwise. Not a live quote, not a fill, and not a probability forecast.
02 · Worked example: 12 sessions, train 4, test 2
Windows: train 1–4 test 5–6; train 5–8 test 9–10. Day 12 never becomes training data for an earlier test. A single fit on days 1–12 has no held-out block.
If you drop a fund that liquidated in month 6 from the month-1 universe, you have survivorship bias. If you ignore the $20 round-trip spread from the liquidity lesson, you overstate P/L (profit and loss).
03 · Assumptions and limits
Integer classroom days. No claim that a walk-forward proves a live edge. Monte Carlo path sketches are not backtests of a fund book.
Checking for a quiz…
Teaching numbers only — not a live quote, not a fill, not financial advice. All lessons · Back to Practice · Replay · Journal.
