Define “execution quality” in a demo context
Execution quality describes how closely an order’s real outcome matches what a trader would expect from the order terms and market reality. In a demo account, “execution” is often an emulation: it may reproduce some mechanics (order matching, quoting, latency simulation) but not necessarily the same plumbing as live trading. So, the goal is not to guess “which broker is best,” but to assess which execution properties are measurable and reproducible in the demo.
A helpful split is:
- Order-handling mechanics (how the platform treats market orders, limit orders, partial fills, and time-in-force).
- Execution price formation (how quoted prices translate into fills).
- Cost and constraint modeling (spreads, commissions, margin rules, and any restrictions that affect order acceptance).
Measure what you can: concrete execution factors
Start with factors that can be measured from logs (or from a reproducible test harness) without needing real-time market data.
- Order type fidelity Test both market and limit orders in controlled scenarios.
- For a market order, define the expected behavior as “fills occur near the last available quote,” then record the deviation.
- For a limit order, define the expected behavior as “fills occur only when the modeled price crosses the limit (or is treated as crossed).”
- Slippage behavior under stated assumptions Slippage is the difference between a reference price and the executed fill price. In a demo, you must state your reference clearly (for example, “last displayed mid price at the moment the order is accepted” as recorded by the platform). Then compute:
- Absolute slippage = |fill price − reference price|
- Directional slippage = fill price − reference price
Assumption: the reference timestamp and the platform’s displayed quote are time-aligned to your measurement method. If you cannot verify alignment, treat results as internal consistency checks, not truth about market impact.
- Partial fills and order lifecycle Execution quality includes whether orders are:
- partially filled and how the system decides the fill quantities,
- canceled/expired correctly for time-in-force rules,
- rejected for the same reasons that would apply in live mode (if those rules are modeled).
Record the full lifecycle events: submitted → accepted → filled (possibly in parts) → closed/canceled.
Evidence limits: what demo results can’t prove
Demo execution quality has material limitations and failure modes.
- Different market data model: A demo may use a different feed or simplified matching logic. That means “good demo fills” do not guarantee live execution quality.
- Different latency and queueing: If the demo does not simulate delays, order book depth, or queue position, slippage statistics may look better than they would be under real conditions.
- Cost representation mismatch: Spreads, commissions, or fees may be simplified. Even if fills look reasonable, total cost can be understated.
- Overfitting to one scenario: A single test window (one day, one session, one volatility regime) may produce misleading conclusions.
- Non-stationary mechanics: Demo systems can be updated. A measurement that was stable last month may not remain stable later.
A concrete failure mode to watch for: the demo might allow executions “too easily” (for example, filling limit orders immediately when a crossing is only hypothetical), or it might apply fill logic that is independent of realistic order matching. In both cases, your metrics become artifacts of the simulator rather than indicators of live execution.
A verification checklist you can run independently
Use controlled, repeatable checks rather than single-shot judgments.
- Create a measurement protocol Write down:
- order types used,
- sizes used,
- the reference price definition,
- how you capture timestamps and fill prices.
-
Repeat across conditions Run tests across different times and volatility regimes (for example, calm vs active periods) and with multiple order sizes. Assumption: the demo’s behavior is consistent enough that repetition meaningfully reveals variation.
-
Compare internal consistency, not predictions If you cannot access authoritative live benchmarks, do not interpret demo metrics as forecasts. Instead, ask whether the demo behaves consistently with its own described order rules (for example, limit-order crossing logic) and whether cost and slippage patterns change in plausible ways.