What are trading signals, and what do you need to evaluate
Trading signals are structured messages that claim a potential market action or direction for a financial instrument over a defined period. In practice, they usually combine a market observation (an “input”), a rule or model (the “logic”), and a forecast about price movement or a decision boundary (the “output”).
When you evaluate trading signals, treat them as a proposition that must be validated. You are not validating whether a person “sounds right”; you are validating whether the signal’s logic is clear enough to reproduce, and whether its claimed results can survive realistic trading frictions.
Mechanics checklist: inputs, rules, and claims you can verify
Use this due-diligence checklist to separate stable mechanics from changeable conditions:
- Define the signal specification: What instrument, what timeframe, and what holding period does it refer to? A signal without a specific timeframe is hard to test because outcomes depend on when you measure them.
- Identify the inputs: Are inputs observable (e.g., price levels) or model-based (e.g., calculated signals)? If inputs rely on data you cannot access or verify, you may not be able to reproduce results.
- Inspect the rule logic: Look for explicit, testable rules—conditions that trigger “buy/sell/hold” or “entry/exit.” Vague descriptions (“momentum will continue”) do not specify when the logic actually fires.
- Separate signal output from execution: A signal’s paper performance can differ from real trading due to spreads, commissions, order timing, and execution delay. Ask whether the signal evaluation includes these frictions.
- Check the evidence type: Prefer documented backtests or forward tests that show how the signal was generated and evaluated. Performance that appears only as summaries, screenshots, or untraceable numbers is difficult to audit.
- Require repeatability: The same rules applied to the same data should lead to the same actions. If small implementation choices change results dramatically, the signal may be brittle.
Evidence and examples: how to test a signal claim without guessing
A simple, assumption-explicit testing approach (without real-time data) is to define a controlled evaluation method:
- Assumption you state: Decide the evaluation horizon (e.g., “success” means the price moves in the intended direction by at least X within Y time units). If you do not define this, “accuracy” is ambiguous.
- Assumption you justify: Choose X and Y based on something consistent—ideally linked to realistic decision-making needs rather than fitted after seeing results.
- Cost model: Include a basic cost assumption such as a fixed transaction cost per trade and a conservative execution delay. Even if exact costs are unknown, you can test sensitivity.
- Holdout period: Use a period not involved in creating the logic (a “holdout”) to reduce overfitting. If a signal only works during the period it was tuned, its future reliability is uncertain.
The goal is not to predict future profits; it is to determine whether the signal’s logic is testable and whether its claimed edge survives reasonable changes in assumptions.
Limitations and failure modes to watch for
At least one material limitation applies to most signal evaluations:
- Non-stationarity: Markets change. A rule that matched historical patterns can stop working when volatility, liquidity, or participant behavior shifts.
- Look-ahead and data leakage: Signals can accidentally use information that would not have been available at the time of decision. This inflates results and fails in live use.
- Survivorship bias: Performance summaries may include only the signals/providers that “worked,” excluding those that failed.
- Overfitting: Complex logic can fit historical noise. Even if historical accuracy is high, out-of-sample performance may degrade.
- Hidden execution assumptions: Backtests may assume perfect fills or ignore the bid-ask spread, which can make results materially different from real execution.
- Ambiguous measurement: If the definition of success or timeframe is unclear, the same performance record can be interpreted in multiple ways.
Because these failure modes are common, a sound evaluation focuses on documentation and reproducible testing, not on persuasive narratives.
Verification: the “ready” criteria before you rely on any signal
Use a clear “done” or “ready” checklist (a “klaarcriterium”):
- Documentation completeness: You can write down the rules and reproduce the decisions using the stated inputs.