Learn / Learn by Doing · Lesson 10 of 10

Intermediate

Backtesting without misleading results

Prerequisites: How to use Replay the Market · Bid–ask spreads and liquidity

01 · Overview & Worked Setup

Point-in-time means the rule can see only what was published as of that date. Survivorship bias is keeping only the funds that still exist. Selection bias is trying ten rules and reporting the one that won. Costs are spreads and fees, not zero.

Walk-forward locks a rule on a training window, then scores a later window that was not used to fit. Replay on this site already hides future sessions until you advance the clock. That is the same idea in a blotter.

Teaching lab

Walk-forward windows

Twelve classroom sessions. Training never includes a later test day. Survivorship, costs, and point-in-time membership are still outside this slider.

2 – 6
1 – 4
  1. Train 1–4, then test 5–6
  2. Train 5–8, then test 9–10
Windows
2
Look-ahead
Blocked

Each test block starts after its own training block.

What this scenario means

A backtest that trains on the same days it scores is not a test. Dropping funds that later died is survivorship bias. Ignoring spreads and fees overstates results.

Guided challenge

Why is a single in-sample fit on all 12 days not in this list?

Educational simulation only. Hypothetical results are not forecasts or trade recommendations. Hypothetical classroom numbers. One standard contract is 100 shares unless labeled otherwise. Not a live quote, not a fill, and not a probability forecast.

Related: Replay the Market →

02 · Worked example: 12 sessions, train 4, test 2

Windows: train 1–4 test 5–6; train 5–8 test 9–10. Day 12 never becomes training data for an earlier test. A single fit on days 1–12 has no held-out block.

If you drop a fund that liquidated in month 6 from the month-1 universe, you have survivorship bias. If you ignore the $20 round-trip spread from the liquidity lesson, you overstate P/L (profit and loss).

03 · Assumptions and limits

Integer classroom days. No claim that a walk-forward proves a live edge. Monte Carlo path sketches are not backtests of a fund book.

Checking for a quiz…

Teaching numbers only — not a live quote, not a fill, not financial advice. All lessons · Back to Practice · Replay · Journal.

Trades | Learn. Practice. Analyze. Improve.