Causal64 Send a brief

Research note, November 2026

Point-in-time data: how we avoid look-ahead bias

A result is only as good as the data under it, so we check where every number comes from, when it was known, and whether a second source agrees.

By , published

0.986
correlation between two price feeds of one instrument that still gave success rates of 0.67 and 0.51
0.92 to 0.38
a backtested success rate before and after stops were checked on hourly lows instead of hourly closes
79,383
values served before their publication date, found by one point-in-time audit
4
places where one stale number kept living after it was corrected

The premise

Most failed models we have seen did not fail on the model. They failed on the table: a column known only after the fact (look-ahead bias), a feed that was not the one the system would run on, a calendar with days that never happened. The error does not show in the code, and the result looks fine until the system meets the world.

So before we fit anything we ask three questions about each input. Where does it come from, and is that the source the system will use in production? When was each value known? Does a second, independent source agree, and on what? The stories below are from our own research since June 2026, most of it on the evaluation of trading strategies for a funded-trader programme, where an account passes if it reaches a profit target before a loss limit. That setting is unforgiving: the outcome is a race between two barriers, and small data errors decide who wins the race.

Two feeds that agree on everything except the answer

In July 2026 we compared two price feeds of the same equity index over the same period (1,402 trading days from 2021), with the same strategy and the same costs. One was the feed of the venue where the strategy would trade. The other was a second price feed with a longer history. On everything we could measure precisely, they were the same instrument: daily returns correlated at 0.986, with a bias of 0.17 basis points a day (t = 0.27); the adverse intraday move that triggers a stop correlated at 0.986 with a median ratio of 1.000; and the share of days that hit the stop was 37.4 % on one feed and 37.7 % on the other.

The share of simulated programmes passed was 0.67 on the venue feed and 0.51 on the second feed.

Nothing was broken. A pass-or-fail race against two barriers amplifies tiny differences: the 2 % of variance the feeds do not share is about 24 basis points a day of noise, compounded over 100 to 250 days. One day that crosses a barrier on one feed and not on the other changes the outcome of the whole window. Figure 1 reproduces the effect in a simulation.

Left: line chart falling from 17 percent of windows ending differently at a correlation of 0.90 to 2 percent at 0.999, with 7 percent marked at 0.986. Right: bar chart of the gap between two feeds' success rates read on six windows; 74 percent of reads show no gap, 24 percent a gap of one window in six.
Figure 1. Simulation. Two return series with the same drift (0.05 % a day) and volatility (1 % a day), correlated at the value on the horizontal axis; target +10 %, loss limit -10 %, 250 days, 20,000 pairs per point. Left: at a correlation of 0.986, 7 % of windows end differently on the two feeds. Right: when each success rate is read on six independent windows, as ours was, 26 % of reads show a gap of 0.16 or more.

This had a practical consequence. Our strategy register carried a success rate of 0.83 reached in a median of 73 days. The evaluation engine had loaded the second feed, so that number was a level on the wrong feed. On the venue feed the same system gave 0.67, a ruin rate of 0.09 and a median of 141 days. We now hold two rules. The production feed is the judge; other feeds can tell us whether a difference between two systems holds, never what level to expect. And a success rate is reported as an interval tied to its feed, never as a number that travels.

When the data resolution invents the result

The worst error we found in 2026 was a resolution error. Our engine checked whether a stop-loss had been hit by looking at hourly closing prices. A price that dips below the stop and recovers within the hour never shows in a close. On 22 July we rebuilt the engine and checked stops against hourly lows, which we verified against tick data: over 897 days the hourly low matched the tick minimum at the median, to 0.0 basis points.

The share of days that hit the stop went from 38.8 % to 49.8 %. The median stop sat 48 basis points away, on a market whose median hourly range in the busiest session is 28 to 29 basis points: the stop was inside ordinary hourly noise. We applied the corrections one at a time, each written down before the run.

Bar chart of the share of programmes passed after each correction: 0.92, 0.91, 0.88, 0.88, then 0.38 once stops are checked on hourly lows. Ruin rises from 0.08 to 0.56.
Figure 2. Real numbers, rebuild of 22 July 2026. Each bar adds one correction to the previous one. Making the slope signal causal cost one point, a causal implied-volatility input three more, a static daily loss rule nothing. Checking stops on hourly lows took the pass rate from 0.88 to 0.38 and the ruin rate from 0.12 to 0.56.

A whole line of results between 0.92 and 0.95 had never existed. To make sure the new engine was right rather than differently wrong, we wrote a second one from scratch: a plain event loop that re-parses the raw files and shares no code with the first. It reproduced 335 out of 335 programme outcomes and durations exactly. We now treat the granularity at which a barrier is checked as a free parameter that can overfit like any other.

Calendars, clocks and days that never happened

Three audits from the same summer show how quiet these errors are.

  • Padded days. A long intraday history had 237 filler days out of 1,497 (15.8 %): 212 weekends and 25 holidays, 99.9 % of their bars with zero range. Bars per day and hours covered both looked perfect. The damage spread forward: an expanding quantile used as a volatility threshold saw the zeros and stayed low for years, and the pass rate for 2021 onward fell to 0.37 instead of 0.51. The cheap detector is to count weekends in any new feed.
  • Phantom Saturdays. On another feed, a time-zone defect created 262 Saturdays that do not exist and inflated the day count by 17.9 %, so medians in days had to be converted back: 123 index days were about 146 calendar days.
  • The wrong clock. We assumed a server clock followed a European daylight-saving calendar. During the weeks when the two sides of the Atlantic change clocks on different dates, the opening spike of the equity exchange stayed at the same server time (27.6 against 25.9 basis points in a normal week). The server followed the exchange's own calendar at a fixed offset. Announcement times in our news filter were re-dated as absolute instants.

Coverage matters too. Hourly bars for several equity indices were in fact daily bars before 2021, so 38 % of the simulated programmes on one index never traded. That index came 84th in a cross-market ranking, then 58th, for reasons unrelated to any strategy. When a result stays identical under several ablations, we now look at the data before the strategy.

Point in time, or what was known when

A forecast made on a given day may only use what was published by that day. Two things break this. The first is revision: many official series are rewritten after release. The public vintage archive we use for macroeconomic series records first releases only from 2010 to 2012, so a series with 55 years of history has about 14 usable years if it is revised. One computed financial-conditions index was revised by 0.70 to 2.05 standard deviations and none of its values stayed as first published; a balance-sheet series from the same release kept 87.8 % of its first values. The question we now ask of every series is whether it is observed or computed. A lag does not fix a revision.

The second is publication delay. In October 2026 an audit of a business-survival bench found 79,383 values served before their publication date: yearly statistics published in September two years after the year they describe were used from January of that second year. One sealed read had used a month of data published two months after it ended. We re-dated every input. A development gain fell from +0.468 to +0.401, a leave-one-country-out count from 28 of 34 to 25 of 34, and the sealed read was declared contaminated and no longer counts.

Language models bring a third kind of leak: they may know the answer. We tested one used as a forecaster on 774 market questions, with no search and no market price, round by round. If it had learned nothing about the outcomes, its accuracy would be at chance. It was not.

Line chart of leak AUC by question round in 2026: 0.90 on 19 July, 0.83, 0.71, 0.63 on 30 August, 0.65 on 13 September and 0.49 on 27 September. The last two rounds are shaded as used for development.
Figure 3. Real numbers, leak test of October 2026 on 774 questions. An AUC of 0.5 means the model cannot tell outcomes apart without information. On the 19 July round, the model alone matched the market price (Brier score 0.108 against 0.104). Only rounds from 13 September were kept for development, labelled “contamination not detected” rather than clean; confirmation waits for a future round.

The announced training cutoff was June 2026; the model knew events from August. We no longer take a cutoff date on trust. We measure the leak, with a planted case it must detect, and keep the confirmatory read for questions that resolve after the test.

Inside our own pipelines the check is mechanical. Every feature uses data up to the previous bar, and a test corrupts all future prices and verifies that past decisions do not move by more than 10-12. It fails the run, not a report.

The register of stale facts

Data goes stale, and so do the facts we write about it. Since August 2026 every number or rule we find to be wrong gets an entry in a register with four fields: what we believed, why it fell, what replaces it, and where it was corrected. Fixing it everywhere comes before any other work.

The “where” field taught us the most. The success rate of a strategy with no edge, the baseline everything is compared to, was written as 0.37 in our validation master file in July. On 7 August we measured it with the current engine: 0.269, because the old figure came from a calculation that did not replay the engine. We corrected two notes.

Timeline from 24 July to 6 October 2026: 0.37 written in the master file; measured 0.269 on 7 August; still in the judge's docstring on 8 August; still a constant in code on 6 October, removed and replaced by a failing test.
Figure 4. Real dates. One stale number and the four places it lived. The list of places corrected was a guess, not an inventory, until a test made the old value fail the build.

The next day the old value turned up in the documentation string of the evaluation tool itself, the first thing anyone reads before using it. Two months later it was still a constant in code, in a separate working copy. That time we removed it and added a test that fails if a retired value comes back. A similar case: the operator file of a deployed system promised a pass rate of 0.85 in about 70 days, a level from the second feed. Someone reading that file would have concluded after three months that the system was broken. A stale expectation produces a false alarm on a predictable date.

What this means for a project

On a data or forecasting project with us, you receive:

  • a data audit before any model: source of every input, publication dates, revisions, coverage, filler days and time zones, with the checks written as tests that run on every refresh;
  • results computed on the data your system will use in production, with any other source labelled as such;
  • a point-in-time history where the data allows one, and a statement of how far back that is possible when it does not;
  • a register of corrections: when a figure we gave you changes, you get the old value, the reason and the new value.

What we refuse to claim: a level measured on one source as if it held on another, a backtest on series that were revised after the fact, or an accuracy figure for a language model on questions it may already know.

References

  1. Kaufman, S., Rosset, S., Perlich, C. and Stitelman, O. (2012). Leakage in data mining: formulation, detection, and avoidance. ACM Transactions on Knowledge Discovery from Data 6(4). doi:10.1145/2382577.2382579
  2. Bailey, D. H. and López de Prado, M. (2014). The deflated Sharpe ratio: correcting for selection bias, backtest overfitting and non-normality. The Journal of Portfolio Management 40(5). doi:10.3905/jpm.2014.40.5.094

Have a question like this one? Tell us what you need in a few lines.

Send us the brief

Also in this section

  • Our gatesA claim reaches a client only after it survives a fixed sequence of tests, and the sequence is written in code, not in a document.
  • Detecting overfittingSince June 2026 we have spent much of our time catching results that looked good only because we had searched for them; these are the cases, with their numbers.
Back to home