execution-microstructure-lab

How do execution algorithms, liquidity conditions, order aggressiveness and participation limits change what a trade really costs?

This repository is a small execution-research lab. It contains an event-driven limit-order-book simulator, five execution strategies (immediate market, TWAP, VWAP, POV and passive limit with cancel/replace) and a transaction-cost-analysis (TCA) layer. The TCA layer splits implementation shortfall into spread, depth, impact, timing/adverse-selection, opportunity-cost and fee components. Every number below was produced by python -m executionlab.run_all on checked-in, seeded synthetic data. No credentials or network are needed.

Scope, stated plainly: all market data here is synthetic, produced by a seeded order-flow model. The fills are simulated. This is not professional institutional order flow, and the dollar and bp magnitudes depend on the model's calibration. What the lab is good for is controlled, paired comparisons: every strategy runs on the same tape, and a no-trade counterfactual is replayed on that tape too.

Headline results (measured)

From 38,220 simulated strategy executions (plus a no-trade counterfactual for each tape). The main comparison uses 1,500 parent orders, each 5% of expected 20-minute volume, across 5 liquidity regimes. Costs are in basis points of arrival notional. Brackets are 95% CIs on tape-level means.

1. Crossing the whole book at once is the most expensive choice, and the penalty is mostly book depth, not spread. Pooled over regimes, an immediate market order cost 9.15 bp against 5.02 bp for TWAP: paired difference +4.13 bp [3.89, 4.36]. In the low-depth regime the gap widens to +10.87 bp, and 12.03 bp of the market order's cost is walking through the book.
2. Passive limit orders save the spread in calm markets and lose it to adverse selection in volatile ones. In the normal regime the passive strategy beat TWAP by -2.95 bp [-3.05, -2.86] and was cheaper on 99% of orders. In the high-volatility regime the same logic cost +4.95 bp more than TWAP, because 30-second adverse selection on its fills rose from 1.92 to 7.40 bp.
3. Lower participation only pays when the order carries little urgency. For a 5%-of-volume order, cutting the POV cap from 10% to 3% lowered shortfall by -1.34 bp with no adverse drift, while filling only 60% of the order. With 10 bp of adverse drift over the horizon, the same cut raised shortfall by +3.83 bp: 4.73 bp of extra opportunity cost against 0.90 bp of execution savings.
4. Larger orders push the break-even urgency higher. For a 15%-of-volume order, the 3% cap still won at 10 bp of drift (-1.29 bp) and only lost at 20 bp (+1.31 bp).
5. VWAP and TWAP were economically indistinguishable (normal regime difference -0.05 bp). The largest gap was -0.42 bp during liquidity shocks. Slow schedules carry a different cost: their order-level shortfall had a standard deviation of 24.8 bp for TWAP, against 1.7 bp for an immediate market order.

---

1. Why execution is harder than a backtest

A backtest usually assumes you trade at the close or the mid, in any size, instantly. Real execution breaks every part of that assumption:

Backtest assumption	What actually happens	Where it shows up here
Trade at the mid	Buyers pay the ask and sellers hit the bid	half-spread component
Unlimited size at one price	A large order eats several price levels	book walk component
The market ignores you	Your buying pushes the price up, and some of that push persists	market impact (measured against a no-trade replay)
Fills are certain	Limit orders may never fill, and they fill most often when the price is about to move against you	fill rate, adverse selection, opportunity cost
Free trading	Taker fees, maker rebates	fees

Slowing down reduces impact but leaves you exposed to the market drifting away. That is the central trade-off this lab measures.

One synthetic tape, one 6,000-share buy order. Left: the order book at arrival, with five displayed price levels per side. Right: the immediate market order fills at once at a worse average price (large blue dot). TWAP buys a slice every 30 seconds. The passive strategy rests at the bid and is filled when sellers arrive.

2. Methodology

Market model (src/executionlab/market.py). One step is one second. The latent "fundamental" price F follows a random walk. Background market makers post limit orders around F at geometric distances from an anchor floor(F - h) / ceil(F + h). Resting orders cancel at a constant hazard. Background market orders arrive as a Poisson process, and their direction is partly informed: P(buy) = logistic(beta * (F - mid)). Passive orders therefore tend to be hit just before the mid moves through them, so adverse selection emerges from the model and is not a penalty added afterwards. Three channels produce impact:

• mechanical: our market orders consume displayed depth, which refills over time because new liquidity is anchored on F and not on the mid;
• transient: market makers lean their anchor against recent net aggressive flow, D <- D*exp(-1/120s) + 0.75 bp per 1,000 shares, so faster trading accumulates more displacement before it decays;
• permanent: each 1,000 shares of our aggressive flow moves F by 0.5 bp (linear, Kyle-style).

The book (book.py) aggregates displayed size per price level and tracks queue position for our own resting orders. New orders join the back of the queue. Cancellations ahead of us advance our position pro rata. A market order fills only against displayed liquidity and never through its limit price.

Exogenous tape and counterfactual. All randomness for a session is drawn up front into a Scenario. Every strategy runs on the same tape, which gives common random numbers, and a no-trade run of the same tape provides the counterfactual mid m0(t). Impact is then measured directly as mid with our order - mid without it, so it does not have to be inferred from a regression.

No look-ahead. Strategies receive only a MarketView (current quotes, top-5 depth, past printed volume, own fills and open orders) and never the scenario, the fundamental price or the simulator. A test swaps every random draw after second k for an unrelated tape and asserts that every decision before k is identical. The VWAP volume profile is estimated from 60 earlier, disjoint sessions (data/sample/historical_volume_profile.csv), never from the session being traded.

Variance reduction and inference. Parent orders come in antithetic pairs: a buy and a sell of the same size on the same tape. Market drift largely cancels within a pair. All CIs and tests are computed on tape-level averages (cluster = tape), because orders sharing a tape are not independent. Strategy comparisons are paired (strategy minus TWAP on the same order), with t-tests, Wilcoxon tests, bootstrap CIs and Holm correction (results/main_paired_vs_twap*.csv). Order-level standard deviations are reported separately as the risk of each strategy.

3. Execution algorithms (strategies.py)

Strategy	Logic	Child orders
Immediate market	Send everything on arrival. Any remainder the book cannot absorb is resent each second as depth refills.	market
TWAP	Every 30 s, trade up to a linear schedule (front-loaded per bucket); always finishes.	market
VWAP	As TWAP, but the schedule follows the historical U-shaped volume profile.	market
POV (participation)	Every 5 s, top up to rate/(1-rate) × other participants' observed volume. No end-of-horizon clean-up, so a low cap leaves shares unfilled.	market
Passive limit	Rests one order at the near touch, sized to the next minute of schedule. Cancels and replaces when the touch moves, which loses queue priority. Crosses the spread if more than 2 minutes behind schedule; finishes with a market order in the final minute.	limit, cancel/replace, market

An optional Alpaca paper adapter (adapters/alpaca_paper.py) turns child orders into Alpaca REST payloads. It is hard-wired to https://paper-api.alpaca.markets: any other host, including the live endpoint, raises LiveTradingRefused. Its default is dry-run, and credentials are read from environment variables only at send time. It is not part of the experiment and has not been run against the network. python scripts/alpaca_paper_dry_run.py SPY buy 1000 10 prints the payloads.

4. Cost decomposition (metrics.py)

For side s (+1 buy, −1 sell), parent quantity Q, fills (q_i, p_i), unfilled U, arrival mid P0, end-of-horizon mid P_T:

IS ($) =  s·Σ q_i (p_i − P0)  +  s·U (P_T − P0)  +  fees
       =  s·Σ q_i (touch_i − m_i)        half-spread      (negative = earned by passive fills)
       +  s·Σ q_i (p_i − touch_i)        book walk        (depth consumed beyond the touch)
       +  s·Σ q_i (m_t − m0_t)           market impact    (our footprint vs the no-trade replay)
       +  s·Σ q_i (m0_t − P0)
       +  s·Σ q_i (m_i − m_t)            timing / adverse selection (market drift + intra-second pick-off)
       +  s·U (P_T − P0_T) + s·U (P0_T − P0)   opportunity cost (own impact + market drift)
       +  fees                            taker $0.0030/sh, maker rebate −$0.0020/sh

m_i is the mid just before fill i, and m_t / m0_t are start-of-second mids of the trading and no-trade runs. The identity reconciles to 1e-9 bp in tests. Fill-quality measures (fills_frame) follow standard definitions: effective spread 2s(p−m)/m, realized spread 2s(p−m_{t+h})/m, price impact 2s(m_{t+h}−m)/m, markout s(m_{t+h}−p)/p, with h ∈ {1, 5, 30, 60, 300} s. Passive-fill adverse selection is −s(m_{t+h}−m)/m, where positive means the price moved against the position just acquired. Slippage against the interval VWAP of all prints is also reported.

5. Experiment design

Experiment	Design	Size
Main comparison	150 tapes × {buy, sell} × 5 regimes × 5 strategies; order = 5% of expected 20-min volume (6,000 sh at $50); 20-min horizon	7,500 executions
Participation × urgency	POV caps 2–30% (+TWAP), 2 order sizes (5%, 15%), adverse drift ("alpha") of 0/5/10/20 bp over the horizon; normal regime	11,520 executions
Sensitivity	One-at-a-time sweeps of order size, volatility, spread, depth and horizon (urgency) around the normal regime; 80 tapes × 2 sides per point	19,200 executions

Liquidity regimes (config.py): normal (≈1.3-tick spread, 1.3 bp/√s vol); wide spread (market makers quote 3 ticks from fair value); low depth (40% of normal limit-order flow); high volatility (3× vol); liquidity shock (at a random time in the first half of the horizon, 80% of depth is pulled and replenishment slows for 2 minutes, then recovers over 2 minutes).

6. Results

Mean implementation shortfall (bp). Full CIs are in results/summary.md.

Regime	Market	TWAP	VWAP	POV 10%	Passive
Normal	5.67	3.70	3.65	4.04	0.74
Wide spread	10.40	7.39	7.32	8.00	-3.29
Low depth	16.22	5.35	5.23	6.37	-0.62
High volatility	7.83	4.00	3.92	4.51	8.96
Liquidity shock	5.63	4.68	4.26	5.16	0.11

Reading the decomposition (normal regime).

• The market order's cost is 3.89 bp of book walk on top of 1.18 bp of half-spread.
• TWAP pays the half-spread on every slice (1.27 bp) and 1.61 bp of impact, mostly the permanent component that later slices pay on earlier slices' footprint.
• Passive earns -1.12 bp of spread plus the rebate. It gives 1.96 bp back through timing, because it is filled when the price is about to move against it.

The order's footprint on the mid relative to the no-trade replay. The market order's spike decays within about 3 minutes to the permanent level (about 3 bp for 6,000 shares). POV 10% builds the same permanent level faster than TWAP. In this model passive fills leave almost no footprint, a modelling choice discussed under Limitations.

Participation versus shortfall: the research question. Does reducing participation improve implementation shortfall enough to offset the higher opportunity cost?

The answer depends on the parent order's urgency. With no adverse drift, a lower cap is always cheaper on average, partly because unfilled shares cost nothing. Each step of adverse drift makes the unfilled remainder more expensive. For a 5% order the cheapest cap moves from 2% (alpha 0) to 20% (alpha 10 bp) and 30% (alpha 20 bp). Low caps also carry far more risk. The cost–risk frontier below shows that, for a risk-averse desk, faster participation dominates in every case tested.

Adverse selection. Passive fills are systematically followed by moves against the position, and the effect grows with horizon and with volatility. Aggressive fills show the opposite pattern: a transient push that reverts as depth refills, visible as negative markouts at 5 minutes.

7. Robustness and sensitivity

• Volatility: passive beats TWAP at the base volatility (0.74 vs 3.70 bp). It is already worse at 2× (4.55 vs 3.84) and far worse at 4× (12.72 vs 4.14). Aggressive schedules barely change.
• Depth: with ¼ of normal depth the market order costs 18.64 bp. With 4× depth it costs 3.01 bp and beats TWAP (3.39): when the book can absorb the order, waiting only adds spread and impact on later slices. Deep books also hurt the passive strategy (3.05 bp), because longer queues mean slower, more adversely selected fills.
• Order size: market-order cost grows roughly linearly in size, reaching 21.71 bp at 20% of volume, against 9.17 bp for TWAP. POV 10% looks cheapest at that size only because it fills about half the order; its unfilled shares carry no drift cost in this sweep (alpha = 0).
• Spread: spread is a near-linear pass-through for every aggressive strategy and a near-linear gain for the passive one.
• Urgency (horizon): compressing TWAP from 20 to 5 minutes costs 4.17 vs 3.70 bp. With linear permanent impact, schedule speed mainly affects risk, not mean cost.

8. Validation

pytest runs 116 tests (1-2.5 minutes). Each item in the original specification maps to a test:

Requirement	Test(s)
Zero-spread / no-impact market → ~zero cost	test_metrics.py::test_frictionless_market_has_near_zero_cost (all strategies, both sides, |IS| < 0.01 bp)
Market orders never fill outside the book	test_book.py::test_market_order_walks_book_and_never_exceeds_displayed_liquidity, ..._fill_prices_match_levels_consumed, test_marketable_limit_never_fills_through_limit
Partial-fill accounting conserves quantity	test_book.py::test_queue_priority_and_partial_fill_conservation
Parent = filled + unfilled	test_metrics.py::test_quantity_is_conserved (6 strategies × 5 regimes); DuckDB reconciliation of all 7,500 main executions (results/fill_reconciliation.csv)
VWAP vs a hand-computed fixture	test_metrics.py::test_vwap_and_shortfall_on_manual_fixture
IS decomposition reconciles	test_metrics.py::test_is_decomposition_reconciles (incl. manual IS from raw fills)
Deterministic seeds	test_simulator.py::test_same_seed_is_bit_identical, test_checked_in_sample_data_is_reproducible
No future information	test_simulator.py::test_no_future_information_reaches_decisions (tape splice), test_market_view_has_no_handle_on_scenario_or_simulator
Fees reconcile to fills	test_metrics.py::test_fees_reconcile_to_fills
Paper-only broker adapter	test_stats_storage_adapter.py::test_alpaca_adapter_refuses_*

python scripts/check_readme_numbers.py checks every number tagged in this README against results/headline_metrics.json. An independent review pass found two bugs, both fixed before the final run: a pooled paired comparison that silently used only one regime, and an impact/timing split misaligned by up to one second.

9. Architecture

flowchart LR
    subgraph data["data/sample (checked in, synthetic)"]
        PO[parent_orders.csv<br/>antithetic buy/sell pairs]
        VP[historical_volume_profile.csv<br/>60 disjoint sessions]
        TP[sample_tape_normal.csv]
    end
    CFG[config.py<br/>regimes, fees, grids] --> SC
    PO --> SC[market.make_scenario<br/>pre-drawn exogenous tape]
    SC --> SIM[market.Simulator<br/>event loop, 1 step = 1 s]
    SIM <--> BOOK[book.OrderBook<br/>levels + own queue position]
    SIM -- MarketView only --> STRAT[strategies<br/>market / TWAP / VWAP / POV / passive]
    VP --> STRAT
    STRAT -- Market / Limit / Cancel --> SIM
    SC --> CF[no-trade counterfactual run]
    SIM --> TCA[metrics.tca<br/>IS decomposition, markouts]
    CF --> TCA
    TCA --> DB[(DuckDB<br/>execution_records)]
    TCA --> AN[analysis<br/>tape-clustered CIs, paired tests]
    AN --> CSV[results/*.csv<br/>summary.md, headline_metrics.json]
    AN --> FIG[figures/*.png]
    STRAT -. optional, dry-run by default .-> ALP[adapters.alpaca_paper<br/>PAPER endpoint only]

Loading

Module	Role
book.py	price-level book; queue priority, pro-rata cancellation, limit-respecting market orders
market.py	scenario generation, event loop, MarketView, action handling, counterfactual
strategies.py	five strategies + factory
metrics.py	per-fill and per-parent TCA
engine.py / experiments.py	parallel paired runs of one tape across strategies
analysis.py / stats.py	clustered CIs, paired tests, Holm, bootstrap
storage.py	DuckDB execution records + reconciliation SQL
plots.py / report.py / run_all.py	figures, machine-generated summary, one-command pipeline

10. Reproduce it

git clone https://github.com/crollila/execution-microstructure-lab
cd execution-microstructure-lab
python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
pip install -r requirements.txt && pip install -e . --no-deps
python -m executionlab.run_all          # full run: 5-8 min on 16 cores; writes results/ and figures/
pytest -q                               # 116 tests
python scripts/check_readme_numbers.py  # README numbers == generated results

• python -m executionlab.run_all --quick runs a small smoke configuration (used in CI).
• --from-records re-analyses results/execution_records.duckdb without re-simulating.
• --rebuild-data regenerates data/sample/ bit-for-bit.

Results are deterministic for a given platform and library versions (pinned in requirements.txt).

11. Limitations

• Synthetic market. The order-flow model is stylised: Poisson arrivals, geometric depth profile and constant cancellation hazard. Impact parameters are chosen to be plausible for a liquid ~$50 stock, not estimated from data. Conclusions are about mechanisms and relative rankings under stated assumptions, not about any real venue.
• Passive fills carry no permanent or transient impact. Only our aggressive flow moves the fundamental and the quote anchor. Real resting orders leak information (displayed size, repeated re-pegging), so the passive strategy's advantage in calm regimes is likely overstated.
• Single venue, one-second resolution. There are no hidden orders, no venue fragmentation and no sub-second latency race. A strategy acts on the state at the start of each second, and its orders arrive before that second's background flow. Background limit orders never cross the spread, so stale quotes are picked off only by market orders.
• Simple strategies. TWAP and VWAP are front-loaded per bucket. There is no optimal-control schedule (Almgren–Chriss) and no limit-order placement deeper in the book.
• Alpha is an exogenous drift. The "urgency" study imposes a constant expected adverse move. Real alpha decays non-linearly and is uncertain.
• Antithetic pairs reduce, not eliminate, drift noise. The buy and sell runs perturb the book differently. CIs are computed on pair means, which is valid, but they are tight because exogenous drift largely cancels. Order-level dispersion (the is_bp_order_std column) is the relevant risk measure.

---

MIT License. Synthetic data only. No live order routing exists in this code base.