Define Ulcer Index before you test it
Ulcer Index is a drawdown-based statistic. Conceptually, it summarizes how far and how long a price series stays below its recent peak. For backtesting, the key point is not the label but the exact definition you implement: which “peak” you reference, how often you sample data, the window or period used, and how the drawdown series is converted into the final number.
A responsible approach starts by writing down the calculation you will reproduce. If you cannot state the full procedure precisely, you cannot reliably test it. Treat the indicator as a measurement tool that produces a value from inputs, not as a standalone trading signal.
Separate stable mechanics from variable conditions
Backtesting accuracy often fails when people mix the indicator’s mechanics with changing market and execution conditions.
Keep these concerns distinct:
- Indicator inputs: the price series used to compute drawdowns, the resampling frequency, and the Ulcer Index window.
- Backtest environment: assumptions about when values are “known” (timing), how orders are filled, and what frictions apply.
- Decision rule (if any): how you would transform indicator outputs into an evaluation metric.
Even if you are not building a trade rule, you still need to define an objective. For example, you might evaluate whether periods with higher Ulcer Index coincide with larger drawdowns in the underlying series. That is a relationship check, not a forecast.
State assumptions for data and costs
Responsible backtesting documents assumptions. Common assumption categories include:
1) Data scope and continuity
- Use a single, consistent historical series.
- Decide what happens around missing data and corporate or structural changes (for FX this is mainly about data continuity and how the series is constructed).
2) Sampling and timing
- Define whether you compute Ulcer Index at the bar close, intrabar, or using some other convention.
- Ensure your “decision time” cannot use information from after it would be made.
3) Costs and frictions If your evaluation includes any action taken from the indicator, include realistic frictions in the evaluation metric. Typical material frictions are spreads, commissions, and slippage. Even if your goal is educational rather than performance-focused, including costs helps prevent results that look strong only because costs were ignored.
Control bias with explicit failure checks
Bias is a major failure mode in indicator backtests. Include checks that specifically attempt to break the test.
Look-ahead bias Make sure the Ulcer Index value used at time t is computed only from data available up to time t. A frequent mistake is computing using the entire bar’s price path when the decision would only be possible at the bar close.
Overfitting (multiple testing) bias If you try many parameter settings (window lengths, thresholds, rules), you can end up selecting a setup that matches noise. Limit the search space, or separate parameter selection from final evaluation.
Selection bias Avoid evaluating only the periods that “worked” well. Use predefined evaluation periods or a walk-forward procedure.
A practical way to do this responsibly is to create a checklist of what could go wrong (timing errors, parameter leakage, and cost omission) and verify each one before interpreting any results.
Use out-of-sample checks to test robustness
Historical relationships do not establish future results. To reduce the chance that your findings are an artifact of one period, evaluate on data not used for parameter selection.
Common evergreen structures include:
- Train/validation/test splits: pick parameters using training and confirm only on validation, then finalize on test.
- Walk-forward (rolling) evaluation: repeatedly re-estimate parameters using only earlier data and assess on later periods.
Define a single performance metric for the evaluation. For example, if you are only studying drawdowns, you might compare realized drawdown depth relative to Ulcer Index levels. Keep the metric stable across experiments.
Know at least one material limitation
A clear limitation is that Ulcer Index compresses drawdown information into one number. That can hide distribution details. Two periods can share similar Ulcer Index values while differing in rebound speed, volatility around peaks, or clustering of extreme moves.
Also, indicator behavior can vary by market regime. If your backtest includes only one regime (for example, mostly calm conditions), the relationship might not hold when conditions change.