You can always find crazy ads promising trading systems with high returns and 100% success rates picking tops and bottoms. These so-called systems achieve their results by using many rules and many exceptions. They were perfectly developed or curve-fit. The rules are always over-optimized. Looking good on paper only is all you get. They won’t last or hold up to the real world.
A good trend following trading system must be robust. There are five general criteria for system robustness:
- Sensitivity analysis on system rules (parameters).
- Testing on many markets.
- System-wide risk analysis.
- System consistency.
- Can trend following be described in simple and logical terms?
Sensitivity Analysis
A trading system with no more than three to five parameters to optimize is ideal. Parameters are the quantitative component of the rules or conditions that must be met.
Test in Many Markets
A significant indication of robustness is to use a system optimized for one market on many different markets without changing any of parameters. If a system optimized on the S&P 500 can trade a Japan fund, a small-cap fund, and an emerging markets fund, the confidence in that system is increased.
System-wide Risk Analysis
System-wide risk analysis imagines all ways system may under-perform its objectives. Think through the options.
Consistency
Consistent returns show a system, over many trades, is taking advantage of an edge. The word edge is used the same way a casino has an edge at roulette, over a large number of trades, a system with an edge makes money.
Can trend following be described in simple and logical terms?
A system must be explained in simple and logical terms. If a system depends on the phase of the moon or on the exponential moving average of the Fibonacci oscillator, then reject the system. You must understand the basis for a system’s success.
Why Robustness Matters More Than Optimization
The systems advertised with 100% success rates and extraordinary backtested returns all share the same structural flaw: they were optimized until they fit the historical data perfectly. A system with enough parameters can always be tuned to produce impressive past performance. The question is never how well the system performed in the past. The question is whether the past performance reflects a genuine structural feature of markets or the artifact of tuning the rules to that specific dataset.
Sensitivity analysis answers this question. If a system that produces exceptional returns with a 20-day lookback period produces nothing at 18 days or 22 days, the 20-day parameter is not capturing a genuine market feature. It is a precisely tuned setting that happened to work on the historical sample. A robust system produces reasonable returns across a range of parameter values, not just at the exact optimized setting. The trader who can change the lookback from 20 to 25 days and still have a profitable system has a robust system. The trader who must use exactly 20 days has a curve-fitted system that will fail on new data.
Multi-market testing is the robustness criterion that is hardest to fake. A system that works only on the specific market it was tested on reflects characteristics of that market’s history, not a genuine principle about how markets behave. A system that works on bonds, currencies, equity indices, agricultural commodities, and energy futures across multiple decades has been exposed to enough diverse conditions to provide meaningful evidence of genuine structural edge. The Turtle system’s documented performance across dozens of markets over decades is the most extensive robustness test available for any systematic trend following approach.
The simplicity requirement is Occam’s razor applied to trading systems. A system that can be described in a paragraph is testable, understandable, and diagnosable when it fails. A system that depends on the phase of the moon or on a cascade of interacting indicators cannot be understood, and therefore cannot be evaluated or improved. You cannot know why it worked or why it stopped working. Simplicity is not a limitation. It is the prerequisite for genuine understanding of why a system produces its results.
Frequently Asked Questions
What is curve-fitting and why is it dangerous?
Curve-fitting is the process of adjusting a system’s rules and parameters until they produce impressive historical performance on a specific dataset. The resulting system has been optimized to the characteristics of that dataset, including its noise. When new data arrives, the noise characteristics are different, and the precisely tuned parameters no longer work. Curve-fitted systems produce excellent backtests and poor live trading results because they were built to explain the past rather than to capture a genuine recurring market feature.
How does sensitivity analysis test system robustness?
By checking whether the system’s performance is stable across a range of parameter values rather than dependent on a single precisely tuned setting. A robust system produces reasonable results with a 15-day lookback, a 20-day lookback, and a 25-day lookback. A curve-fitted system produces exceptional results at exactly 20 days and poor results at any other setting. Stability across parameter ranges is evidence that the system is capturing a genuine market feature rather than a noise characteristic of the training dataset.
Why must a trading system be explainable in simple terms?
Because if you cannot explain why a system works, you cannot evaluate whether it will continue to work, diagnose why it fails during a drawdown, or distinguish between a system that has genuinely stopped working and one that is experiencing normal variance. A system you understand is a system you can follow with conviction through difficult periods. A system you do not understand is a black box that you will abandon at the first extended losing streak, which is precisely when following it is most important.
Trend Following Systems
Want to learn more and start trading trend following systems? Start here.
