Verification Harness

A backtest that produces an encouraging number has told you almost nothing. The value is in the machinery that tries to make the number go away — and in whether it survives.

This harness exists because quantitative research fails quietly. Look-ahead leaks in through a revision that was not available on the day. A model appears to predict direction when it is really nowcasting volatility. An execution assumption quietly awards fills the queue would never have given you. None of these announce themselves, and all of them survive an ordinary review.

What it does

Evaluation is purged and embargoed, so training and testing windows cannot bleed into one another. Fills are simulated against queue position with realistic costs, rebates, latency and tick constraints rather than assumed at mid. Standardisation is causal by construction, so the statistics used at serving time are the same ones available during training — parity holds structurally, not by discipline.

Data panels are assembled with publication-lag-aware point-in-time joins, then tested by rebuilding a truncated panel and asserting it matches the corresponding slice of the full one. That is the strongest available proof that no future information leaked backwards, and it is the test most pipelines skip.

The evidence is what it rejects

This harness has repeatedly falsified strategies it was pointed at, including ones we expected to work. Statistically real predictability that proved economically unexploitable once spread costs were applied. A statistically significant model rejected after its edge was traced to a volatility shortcut rather than directional skill.

We publish those results. A validation harness that has never rejected anything is not evidence of good strategies — it is evidence of a weak harness, and it is the reason most internal validation is worth less than the confidence it produces.

Available as an independent validation engagement, or built into your own research stack.