Backtesting

The most dangerous mistakes are look-ahead bias (using data that wouldn't have been available at trade time), survivorship bias (only testing on stocks that still exist today), and overfitting (tuning parameters to match historical noise rather than real patterns). Always use out-of-sample data for validation.

A backtest generates many numbers — knowing which to trust and which to be skeptical of separates professional quants from retail over-fitters.

Total return: The raw profit of the strategy over the backtest period. Meaningless without context — a 50% return over 10 years is mediocre; over 1 year is strong. Always compare against a benchmark (S&P 500 buy-and-hold).

Maximum drawdown (max DD): The largest peak-to-trough decline in equity during the backtest. This is the most important risk metric. A strategy with a 40% max drawdown may be psychologically impossible to trade live — most people abandon a system after a 20-25% drawdown regardless of long-term expectations.

Sharpe ratio: Excess return divided by standard deviation of returns. Measures return per unit of risk. Above 1.0 is acceptable; above 1.5 is good; above 2.0 is excellent. A Sharpe below 0.5 suggests the strategy barely compensates for its volatility.

Win rate vs. profit factor: Win rate alone is meaningless without knowing the average winner and loser size. A 40% win rate strategy can be highly profitable if winners average 3x the size of losers. Profit factor (gross profit / gross loss) above 1.5 is the minimum threshold for a viable strategy.

Number of trades: Statistical validity requires enough trades to distinguish signal from noise. Fewer than 100 trades produces unreliable statistics. Fewer than 30 is statistically meaningless. For high-frequency day trading strategies, aim for 500+ simulated trades minimum.

Avoiding common pitfalls requires deliberate methodology:

Model transaction costs realistically: Include commissions, exchange fees, and slippage. For stocks, model slippage as half the bid-ask spread plus market impact. For small-cap stocks or large orders, market impact can exceed the spread. Underestimating transaction costs is one of the most common reasons backtests look better than live trading.

Use adjusted price data: Historical prices must be adjusted for stock splits and dividends. Using unadjusted data creates phantom price jumps that appear as trading signals. Most data providers (Polygon.io, Yahoo Finance) provide adjusted close prices — verify before backtesting.

Simulate realistic execution: Assume fills at the open or next bar after signal generation — not on the same bar (which would require predicting the close). Assuming you fill at the signal bar's close is a form of look-ahead bias.

Apply position sizing consistently: Backtest with the same position sizing rules you intend to use live. Fixed fractional (risk 1% per trade) produces very different equity curves than fixed dollar amounts. The position sizing model has a larger impact on outcomes than most traders realize.

Out-of-sample testing: Reserve a portion of historical data (20-30%) that the strategy never sees during optimization. Only test the final, locked strategy parameters on this out-of-sample period. If performance degrades significantly on out-of-sample data, the strategy is overfit.

Cross-market validation: Test the strategy on different markets, asset classes, or timeframes. A momentum strategy that works on US large-cap stocks should also work (to some degree) on European stocks, ETFs, or even commodities. Strategies that only work on a narrow slice of history and assets are likely overfit.

Backtests are approximations of reality, not reality itself. Several structural limitations always exist:

Market impact is underestimated: A backtest assumes you can buy and sell at modeled prices without moving the market. In reality, large positions in illiquid stocks move prices against you. The more capital you trade and the less liquid the market, the worse this becomes.

Regime non-stationarity: A strategy optimized on 2010-2020 data includes the longest bull market in history. That regime may not recur. Backtests trained on a single regime often fail when conditions change — low-volatility strategies from 2017 collapsed in 2018 and 2022.

Data quality issues: Incorrect historical prices (due to data vendor errors, unadjusted splits, erroneous ticks) create phantom profits in backtests. Always validate historical data from multiple sources before trusting results.

Psychology and execution risk: Backtests assume perfect execution and no psychological interference. In live trading, you hesitate on entries, move stops, take profits early, and make dozens of discretionary deviations that erode returns. The gap between backtested and live returns is partly a discipline and execution gap, not just a statistical one.