What data is needed to assess Execution Algorithms?

Execution algorithms are decision-and-ordering methods that control how orders are created and split into smaller actions over time. To assess them, you need data that describes (1) the intent and rules of the algorithm, (2) the orders it sends and when, and (3) the observed outcomes after realistic transaction costs.

Use a structured checklist of inputs, provenance, timeliness, and quality checks:

Algorithm specification inputs: the execution objective (for example, minimizing market impact vs. meeting a timing target), order lifecycle rules (when to place, cancel, replace), and any constraints (limits on participation, maximum order size, or trading hours).
Order and execution data: timestamps, order IDs, side (buy/sell), order type, size, price logic used, cancellations, and partial fills.
Market context data (or a documented substitute): reference quotes used by the algorithm, liquidity measures if available, and any mapping from market data to decisions.
Cost data: spreads/fees/commissions and any additional costs that affect “net” execution quality.
Provenance and timeliness: where each dataset came from, how it was captured, clock synchronization approach, and whether data aligns to the same time windows.
Quality checks: completeness of logs, consistency of identifiers, missing-data rates, and whether the recorded data matches the algorithm version that produced it.

How does the data work in practice?

Assessment is usually about comparing what the algorithm did to what would be feasible under the same constraints. That requires separating stable mechanics from variable conditions.

Start with stable mechanics Collect the algorithm’s defined rules: how it splits orders, how it responds to fills and rejections, and what triggers it uses. If there are parameter values (thresholds, weights, or limits), record them together with the exact algorithm version.
Add the executed order trail Build an order-level event table from the execution logs: place time, cancel/replace times, fill reports (including partial fills), and the final executed quantities. Without this trail, you cannot evaluate latency sensitivity or cancellation behavior.
Attach the market context used for decisions Even if you do not use real-time market data in your study, you need a consistent description of the environment. For example, if the algorithm uses reference quotes or last known prices, you need the same reference series (or a documented proxy) aligned to decision timestamps.
Compute net execution quality with explicit assumptions Any calculation must state its assumptions: which costs are included, how spreads are represented (e.g., mid-price vs. bid/ask), and how you handle timestamps when events and prices don’t line up exactly. Relationships observed historically do not guarantee future behavior, so calculations should be treated as evidence about that period.

Example evidence structure: you would group executions by similar conditions (liquidity regime or volatility proxy, if you have it), then compare net outcomes and failure counts (e.g., excessive cancellations, poor fill ratios). If you do not have market context data, you can still assess internal consistency and robustness, but you should avoid claims that require external comparability.

What are the relevant limitations and risks?

Several limitations affect how confidently you can interpret the results:

Hidden dependencies: algorithms may rely on external feeds, risk checks, or broker/platform behaviors not present in your dataset. Missing or simplified assumptions can make the assessment invalid.
Timing and synchronization errors: even small timestamp misalignment between order events and market reference data can distort cost and performance calculations.
Incomplete cost modeling: “gross” execution looks better than “net” execution if fees, commissions, or spread effects are omitted. If cost inputs change, comparisons across periods become misleading.
Failure modes that are hard to predict: algorithms can behave differently during rapid market changes, when liquidity thins out, when orders are rejected, or when cancellations dominate. You need to look for these events explicitly in the logs.
Non-stationarity: historical relationships between execution metrics and market conditions may not hold later. Outcomes vary with market conditions, costs, execution mechanics, and jurisdiction.

Material limitation to plan for: if you lack either (a) the decision context series the algorithm used or (b) complete event logs, you cannot reliably separate whether poor results come from the algorithm’s rules or from missing/incorrect inputs.