A properly designed backtest simulates the actual live trading experience as faithfully as possible. Key requirements: use only information that would have been available at the time of each trade (no lookahead bias); adjust for corporate events (stock splits, dividends, mergers, delistings) correctly; include transaction costs (commissions, bid-ask spread, market impact for larger positions); include realistic execution assumptions (market-on-close prices rather than idealized intraday fills for end-of-day strategies); exclude survivorship bias (include all stocks that existed in each period, not just those that survived to today).

Walk-forward testing is the gold standard validation approach. Split the historical data into in-sample (optimization period) and out-of-sample (validation period) segments. Optimize strategy parameters on in-sample data, then evaluate on the untouched out-of-sample period. Repeat this process across multiple non-overlapping windows. If the strategy degrades severely out-of-sample relative to in-sample performance, it is likely overfit. Genuine strategies show modest but consistent degradation out-of-sample; overfit strategies show dramatic collapse.

Backtest metrics: annualized return (the compound annual growth rate of the simulated portfolio), Sharpe ratio (risk-adjusted return), maximum drawdown (worst peak-to-trough decline), win rate (percentage of trades with positive return), average win vs. average loss, and Calmar ratio (annual return divided by maximum drawdown). No single metric is sufficient — a strategy with high Sharpe ratio and a maximum drawdown of 50% requires very different risk management than one with modest Sharpe and a 15% drawdown.

Statistical significance is critically important and routinely ignored in backtesting. A strategy that outperforms the benchmark by 2% annually over a 5-year backtest — with an information ratio of 0.5 — has an extremely wide confidence interval: the true out-of-sample performance could easily be zero or negative given only 60 monthly data points. The t-statistic for alpha significance requires many years of data to be confident that outperformance is not luck. Harvey, Liu, and Zhu's research suggests that a t-statistic of at least 3.0 (not the typical 2.0) is needed for strategy discovery after adjusting for multiple testing.

Survivorship bias is the most common error: testing on an index's current constituents excludes companies that went bankrupt, were acquired, or were removed from the index during the test period. This inflates backtested returns because failed companies are excluded retroactively. A proper backtest uses a 'point-in-time' universe that includes all stocks that existed at each historical date, including those that subsequently failed.

Overfitting (discussed in the next section) is the second most common error — adding parameters and rules until the strategy fits the specific historical period perfectly but generalizes poorly. Other errors: ignoring market impact (assuming trades execute at last price without moving it), ignoring dividends (which significantly affect total return for longer holding periods), using price data without adjusting for splits (creating false signals from artificial price discontinuities), and testing on too short a period (failing to include multiple market cycles, which are necessary to assess strategy behavior across different regimes).