Revised Price Data: Audit Backtest Snapshot Changes

Revised price data: Compare saved price snapshots before attributing a changed backtest to strategy logic.

The six-fold equities discussion raised a reproducibility problem: refreshed adjusted prices differed from an earlier download. Its reported portfolio variants also shared much of their historical exposure. Those observations motivate a dataset audit before any comparison of strategy quality.

The source discussion is a starting question, not verified strategy evidence. Accessible comments were reviewed incompletely; some replies remained unavailable. Author results are unverified. The exercise below is original educational reasoning with hypothetical inputs, not a reproduction of those results.

Revised price data: revised-price backtest audit: freeze two inputs

Save the old and refreshed files separately. Record retrieval time, instrument identifiers, adjustment settings, date coverage and a hash of each file. A hash identifies bytes; it does not certify historical correctness. Keep the strategy code and configuration fixed while comparing the snapshots. Otherwise a changed result has several possible explanations and no clean attribution.

Start with one instrument and a small date range. Compare raw columns before comparing summary performance. Classify changes as added rows, deleted rows, changed prices, changed volume or changed identifiers. A new corporate action can alter an adjusted history without establishing that a trading rule used future information. The actual feature formula and availability timeline still need inspection.

Worked example: scale changes and thresholds

Imagine yesterday's close changes from 100 to 50 and today's close changes from 102 to 51. The close-to-close return remains 2%. A rule based only on that ratio may produce the same decision. A rule requiring price above 75 produces a different decision. These are invented numbers, used to expose two different dependencies on the same revision.

Now change only one observation. A rolling return, moving average or stop threshold may change differently. Recompute each intermediate feature and record the first decision that differs. Do not assume every adjustment behaves like a uniform scale factor. Also separate research prices from executable prices when reconstructing orders and cash.

Compare variants without hiding their shared history

Suppose three portfolio constructions use substantially the same signals. Keep synchronized date rows and compare their net equity paths on identical windows. Three charts are not three independent replications. Record all variants considered, the selection rule and any changes made after viewing fold results. A refreshed snapshot should trigger a reproducibility comparison, not an opportunity to quietly replace an unfavorable run.

Make the exercise reviewable

Produce a small artifact with snapshot identifiers, changed rows, changed decisions and changed portfolio returns. Include unaffected examples as controls. Ask another reviewer to rebuild the first divergent calculation directly from the saved rows. If the old input cannot be recovered, label that comparison unavailable rather than reconstructing a convenient approximation.

The useful output is an explanation of which dependency changed. It may reveal a harmless rescaling, an unstable absolute threshold, a data error or a timing issue. Each requires a different response. None can be diagnosed reliably from the final Sharpe ratio alone.

Reference and first useful step

TimeSeriesSplit provides chronological train/test indices and an optional gap. TimeSeriesSplit documentation.

Open Advisory Research to explore a local sample report and inspect its assumptions before comparing your own saved evidence. Choose a synthetic sample before importing offline JSON. Keep outputs advisory; this exercise does not place trades. For background, read backtesting fundamentals and paper trading.

Continue the research

Walk-Forward Testing: Log Every Holdout Access and Backtest History: Measure Market Regime Coverage develop adjacent checks. Use a small synthetic fixture first; these educational diagnostics do not validate a trading edge or change account settings.