Overfitting

Overfitting is often invisible until you trade live. These warning signs help catch it during development.

Suspiciously high backtest returns: A strategy returning 200% annually with a Sharpe ratio above 5.0 on a single market is almost certainly overfit. Real edge in markets is modest — even top hedge funds target 15-25% annual returns.

Large parameter count: A strategy with 15+ tunable parameters can fit almost any historical data, including noise. Compare the number of parameters to the number of trades — the ratio of trades to parameters should be at least 50:1 (ideally 100:1).

In-sample vs out-of-sample gap: Split your data into training (in-sample) and test (out-of-sample) periods. If the strategy returns 40% in-sample but 5% out-of-sample, the difference is likely noise that was captured by overfitting.

Sensitivity analysis: Change each parameter by 10-20% and re-run the backtest. A robust strategy performs similarly across nearby parameter values. An overfit strategy collapses when any parameter shifts — it found a narrow "sweet spot" in the noise.

Cross-market validation: Test the strategy on correlated but different instruments (e.g., develop on SPY, test on QQQ and IWM). If it only works on the exact instrument it was developed on, it is likely overfit to that instrument's specific history.

Visual inspection: Plot the equity curve. A smooth, monotonically increasing curve with no drawdowns is a red flag. Real strategies have losing periods. An equity curve that looks "too good" is probably overfit.

Preventing overfitting requires discipline in strategy development.

Keep it simple: Start with 2-3 parameters maximum. A moving average crossover with a stop-loss (3 parameters: fast period, slow period, stop multiplier) is far less likely to overfit than a strategy with 12 custom filters. Add complexity only when each additional parameter demonstrably improves out-of-sample performance.

Walk-forward analysis: Instead of a single in-sample/out-of-sample split, use rolling windows. Train on months 1-12, test on month 13. Train on months 2-13, test on month 14. Continue until you've tested across the entire dataset. This simulates real-world conditions where you periodically retrain on recent data.

Multiple market test: Develop on one market, validate on 2-3 others. A momentum strategy should work on SPY, QQQ, and individual stocks — the underlying behavioral principle (trend persistence) should generalize. Strategies that only work on one ticker are almost always overfit.

Avoid data snooping: Do not test hundreds of parameter combinations and keep the best one. Each test you run increases the chance of finding a spurious result. Use theory-driven parameter choices (e.g., 20-day moving average because it represents one trading month) rather than optimization-driven choices.

Realistic transaction costs: Include commissions, slippage, bid-ask spread, and market impact in backtests. Many strategies that look profitable in frictionless backtests become unprofitable with realistic costs — this is a natural filter against overfit strategies that require frequent trading.