Causal Market Lab

The question: can a causal Transformer learn useful market representations from unlabeled sequences, and do those representations improve delayed-entry return prediction after costs?

The answer, on synthetic data, is no. The repository is built so that answer can be trusted.

What’s built

An audited quote pipeline with explicit event, availability, decision, entry, exit and target-known timestamps. Chronological splits with purging and embargoes, training-only normalization, and a physically separate locked holdout that remains unopened. Seven baselines from zero through histogram boosting and an MLP. A masked causal Transformer with a next-step self-supervised objective, then matched pretrained-versus-random fine-tuning and frozen-probe experiments across three seeds. Deterministic training with atomic checkpoints and exact CPU resume. A quote-based simulator with spread, fees, slippage, turnover, latency, participation limits and an impact proxy.

What it found

Optimization is established: three seeds train stably, and an interrupted run resumes to match the uninterrupted one exactly. Everything downstream is weak or absent. Ensemble MSE over zero-forecast MSE is 0.9957. Frozen pretrained probes beat matched random probes by about 0.39% MSE, and the paired block-resampling interval on that improvement crosses zero. Gross +34.0 bps becomes net −112.2 bps at 59.1× turnover.

A 32-bin return-quantile tokenizer, fitted on training rows only, loses to the empirical unigram baseline across all three seeds. It’s retained as a negative result rather than selected against. A registered scaling grid over data, context and model size finds that more data helps slightly, and longer context and the largest model do not.

All seeds and negative runs remain in the repository. No seed was selected after observing validation performance.

The information clock

For decision time d, entry time e and exit time x: available_ts(feature) <= d < e < x, and the target is log(mid[x] / mid[e]). A quote observed at the decision timestamp cannot also be the fill. Windows crossing gaps are rejected rather than forward-filled. The order is enforced in data construction and by adversarial tests.

The model has 18,533 parameters and is described as a small research Transformer. Not a foundation model, and no real-market alpha claim.