Define what “trading signal” means
A trading signal is a rule or output that tells you when to take an action based on some information source. Verification starts by separating:
- The signal generator (the rule that produces the signal)
- The execution plan (how the action would be carried out)
- The evaluation method (how outcomes are measured) If any of these parts are changed after the fact, verification becomes weak.
How verification works: from rule to evidence
Verification is mostly about showing that the signal can be reproduced and fairly evaluated. Use three layers of checks.
1) Evidence of the rules (proof of document)
Look for a clear, written description of:
- What data is used
- When it is sampled (timing)
- The exact conditions that trigger a signal
- Any parameter values and whether they can change
- How orders are mapped (entry/exit logic, stop/limit logic if any) A signal is easier to verify when the rules are complete and deterministic (or at least parameterized with clear inputs).
2) Reproducibility checks (assumptions and ready calculations)
Choose explicit assumptions and keep them consistent:
- What is assumed for transaction costs (spreads/fees) and how they enter the calculation
- What is assumed for execution timing (e.g., signal generated at time T, order filled at time T+? )
- Whether the evaluation uses the same timeframe and data source throughout Then perform a controlled backtest or audit using those assumptions. If results depend heavily on unstated choices (for example, optimistic fill timing), the signal is not reliably verified.
3) Evidence of outcomes (measured under consistent criteria)
Even if rules are clear, outcomes must be measured with transparent methodology:
- Use consistent definitions of success/failure (what counts as a win)
- Include all relevant costs and execution frictions assumed in the rules
- Report uncertainty across time (results should not hinge on a single favorable window)
Evidence to request: document, data, and change history
To verify a third-party or vendor-described signal, you generally need three kinds of material:
- Evidence of document: the exact rules and parameters as originally proposed
- Evidence of data: the data used for generation and evaluation, including source and timeframe
- Evidence of change history: what changed over time (rules, parameters, execution assumptions) Without these, “verification” often becomes trust-based rather than evidence-based.
Red flags and limitations (what can fail)
Common material limitations and failure modes include:
- Look-ahead bias: using information that would not have been available when the signal was generated
- Cherry-picking: showing only periods with favorable results
- Survivorship bias: evaluating only cases that remained available (e.g., excluding failed scenarios)
- Unclear execution: ignoring slippage, latency, partial fills, or order constraints
- Overfitting: tuning parameters so closely to historical data that performance does not generalize
A key point: historical relationships do not establish future results. Outcomes vary with market conditions, costs, execution quality, and jurisdictional and operational constraints of the platform used.
A practical “ready” verification checklist (and a clear criterion)
Use a “klaarcriterium” style test: the signal is considered verifiable when you can reproduce the same outputs from the documented rules and then evaluate performance using a fixed, transparent methodology with clearly stated assumptions.
Control-checklist
- AFVINKPUNTEN: Are the rules complete, dated (if applicable), and parameterized?
- Bewijs of document: Can you map each signal to an unambiguous execution action?
- Rode vlaggen: Are costs, timing, and fill assumptions missing or conveniently favorable?
- Klaarcriterium: Does the evaluation remain consistent when you apply reasonable, pre-stated assumptions?
If you cannot obtain the rules, data provenance, or methodology clarity, the strongest conclusion is that the signal has not been independently verified—not that it is true or false.