What “Execution Comparison” means
Execution comparison is a structured way of comparing how two ways of trading—or two providers’ order-handling approaches—translate an order into executed results. In practice, it focuses on measurable execution outcomes such as realized price, timing (how quickly execution occurs after an order is sent), and total trading costs (spreads, commissions, and other fees).
Because the word “comparison” can hide different definitions, you should first pin down the exact metric being compared. Ask what the comparison is using as inputs (order type, size, timing, market liquidity assumptions), what it is using as outputs (fill price, slippage, rejection/partial-fill rates), and whether the numbers reflect gross price movement or total cost.
How the comparison should work (mechanics to look for)
A useful execution comparison explains the mechanism that connects an order to an outcome. Use the checklist below to separate stable mechanics from variable conditions:
-
Order handling and matching approach Check what happens from order submission to execution: whether the process allows partial fills, how it handles requotes or refusals, and how it treats different order types. If the comparison mentions “improved execution,” you still need the underlying steps that produce those differences.
-
Cost model completeness Execution outcomes should be reported using a complete cost model. That means including commissions and any other explicit charges, not only the spread. If the comparison reports only spread-like numbers, it may understate real total cost.
-
Assumptions used in calculations Any example should state assumptions. For instance: the assumed execution timing, whether orders are treated as marketable at the quoted moment, and how slippage is defined. Without these assumptions, two comparisons can both be “correct” yet answer different questions.
-
Scope of the test Execution comparisons can be limited to specific hours, instruments, or volatility regimes. Confirm what period is covered and whether the comparison averages across conditions or shows sensitivity (e.g., high vs. low volatility).
-
“Both sides” per criterion For each criterion you care about, require both options to be described the same way. If one side is evaluated using one definition and the other side using a different definition, the comparison is not apples-to-apples.
Evidence and example checks (what can go wrong)
When you see an execution comparison with numbers or illustrative charts, treat it as a model that depends on assumptions. At least one material limitation or failure mode is common:
- Definition mismatch: slippage can mean different things (difference from quote, difference from mid-price, or difference from an assumed ideal price). If the definition is not explicit, the comparison can be misleading.
- Timing bias: reported execution can implicitly assume favorable timing or instant fills. Real execution depends on how quickly orders are processed and how liquidity changes.
- Selection bias: results can be cherry-picked to periods where one approach looks better. A credible comparison should explain selection rules.
- Hidden costs: comparisons that focus on price improvement while ignoring commissions, fees, or operational costs can distort “total execution quality.”
If the comparison uses historical relationships, remember that historical relationships do not establish future results. Market microstructure changes, and execution behavior can shift when liquidity, volatility, or participant behavior changes.
Limitations and how to verify independently
Execution comparison is inherently uncertain because outcomes vary with market conditions, costs, and the exact execution path. To verify independently, use these next questions:
-
What is the test’s metric and denominator? Clarify whether results are measured per order, per filled order, or averaged in another way. Partial fills and rejections can change the denominator and alter interpretations.
-
Are the definitions reproducible? Ask whether you could apply the same slippage and timing definitions to raw order/execution data.
-
Do you have primary documentation? Look for official or primary materials that describe order handling and cost components (for example, terms describing execution behavior and fee schedules). Comparisons built without these inputs are harder to verify.
-
Can you detect model fragility? Ask what would need to be true for the results to change sign (for example: if spreads widen, if volatility increases, or if liquidity decreases). If the comparison cannot explain sensitivity to conditions, treat it as less reliable.