What “execution quality” means for execution problems
Execution problems are situations where an order’s real outcome differs from what you would reasonably expect given the stated order intent and the observed market at the time. Execution quality describes how closely the realized outcome matches that expectation.
A useful way to assess execution quality is to treat it as a measurement problem: you compare an order’s realized results to an expectation built from transparent assumptions (for example, expected price reference, spread assumptions, and whether fees are included). If the gap is consistently large or happens for specific failure modes, execution quality is weak.
Core mechanics: what to measure
To assess execution quality, collect the same categories of evidence for each case:
-
Timing: the time from order submission to final fill (including any delay before confirmation). Timing matters because market movement can change the price reference before execution occurs.
-
Price realization: the difference between a defined reference price and the actual fill price. In practice, the reference might be the best available price at decision time, a quoted bid/ask snapshot, or an internal “expected fill” calculation. Slippage is then the realized minus reference move (with sign).
-
Effective cost: realized costs beyond the fill price, such as spread crossing effects, commissions, and any other disclosed transaction charges. Two executions can have the same slippage but different total effective cost.
-
Order behavior details: whether the order was filled fully or partially, whether the system re-quoted, and whether fills arrived in multiple parts. Execution quality often degrades when the path from intent to fill changes.
-
Context separation: record market conditions that can affect outcomes independently of execution, such as rapid price changes. This helps you distinguish execution-process issues from normal market volatility.
Evidence and a realistic evaluation example
Consider a single order where your expectation is based on a specific reference price at decision time, plus an assumed spread and known fees. To keep the example measurable, state assumptions explicitly:
- Reference price: the chosen mid or bid/ask level at decision time.
- Spread assumption: the half-spread used to translate that reference into an expected executable price.
- Fees: either included or excluded consistently.
Then compute:
- Price deviation = (fill price − expected executable price)
- Effective cost deviation = (fill price + assumed costs − expected total cost)
Repeat across multiple orders under similar conditions. If deviations track market movement even when execution timing is stable, the issue may be market-driven. If deviations cluster around specific conditions like delays, partial fills, or re-quotes, the likely cause is closer to the execution process.
Limitations and failure modes you must account for
Execution quality assessment has material limitations:
-
No real-time verification: without reliable time-stamped quotes and complete order records, “expected” becomes ambiguous. Different reasonable reference definitions can change the calculated slippage.
-
Variable costs and liquidity: effective cost can widen due to liquidity shifts, even if execution mechanics are unchanged.
-
Historical non-predictiveness: past deviations do not establish future performance. Relationships can break when market regimes change.
Common failure modes include:
- Latency and delayed processing: timing gaps allow the market reference to move.
- Partial fills: multiple fills can average to a worse effective cost than a single expected fill.
- Re-quotes or refusal to execute at the intended price: the realized price updates after the market moves or after system checks.
- Widened effective spread: even with small stated slippage, the total cost can be higher if crossing costs increase.
Verification checklist and a next question to ask
To verify your conclusions independently, check whether your method is reproducible:
- Are expected price references and spread assumptions stated and consistent?
- Are order timestamps and fill timestamps available to measure timing effects?
- Do you separate market volatility from execution-process delays?
- Do you include all relevant cost components in “effective cost” consistently?
- Do failure-mode patterns (partial fills, re-quotes, delays) explain the largest deviations?
A strong next question is: Which failure mode best matches the biggest deviations in your dataset, after controlling for market speed? This focuses attention on mechanisms rather than on single-number performance claims.