One group that tests trading systems is Futures Truth. It is run by long time industry veteran John Hill:
John R. Hill is President and founder of Futures Truth Magazine. Futures Truth Company was established to bring truth to the world of publicly offered trading systems. The public is literally flooded with trading systems which carry promise of great wealth. If one only has the discipline to “follow the system.” Many traders have purchased systems for large sums of money only to find out later that the system is worthless. Our purpose is to provide information to assist you in both the purchasing of and the actual decision to trade a system. It is up to you to reach your own conclusion from the facts we present.
A trading system must answer the following 5 questions (from book Trend Following):
- How do you determine what market to buy or sell at any time?
- How much of a market do you buy or sell at any time?
- How do you determine when you buy or sell a market?
- How do you determine when you get out of a losing position?
- How do you determine when you get out of a winning position?
Using money management in system testing is critical. More on Indicators.
Why These Five Questions Define a Complete Trading System
Futures Truth’s mission is to evaluate whether publicly offered trading systems actually work in live trading conditions rather than just in hypothetical backtests. The five questions from Trend Following define the standard against which any system should be evaluated, because a system that cannot answer all five is incomplete and cannot be honestly evaluated for real-world performance.
The first question, which market to buy or sell, determines the system’s universe. A system that can trade any market but has no defined selection criteria is not a system. It is a set of entry and exit rules waiting for a human judgment to decide where to apply them. Market selection can be as simple as “all liquid global futures markets” or as specific as “equity index futures in G10 countries,” but it must be defined.
The second question, how much to buy or sell, is position sizing. This is the component that Ginyard’s research and the PhD experiment both confirm is the most neglected and most important. A system with correct entry and exit signals but no defined position sizing rule will produce radically different results depending on the arbitrary sizing decisions made at each trade. Position sizing must be defined, and it must be defined based on account equity and market volatility rather than on conviction or intuition.
The third question, when to buy or sell, is the entry signal. This is the component that most trading system marketing focuses on almost exclusively. It is the least important of the five. The entry signal determines when to start a trade. Whether the trade is ultimately profitable depends on the exit rules and position sizing at least as much as on the entry timing.
The fourth and fifth questions, when to exit losers and when to exit winners, are the exit rules. These are the most psychologically demanding components of the system because they define the actions that loss aversion and the desire to lock in gains both push the trader to violate. The stop loss defines the maximum acceptable loss on any trade. The trailing stop or breakout exit defines when the winning position should be closed. Both must be defined before entry, when the emotional relationship with the position has not yet formed.
Futures Truth’s testing service is valuable precisely because it evaluates systems against live trading results rather than hypothetical backtests. The gap between backtest and live performance is where most commercially sold trading systems fail. Curve-fitted systems produce impressive historical results and poor live results. A system that holds up under Futures Truth’s live performance evaluation has passed the test that hypothetical backtests cannot replicate.
Frequently Asked Questions
What does Futures Truth test and why does it matter?
Futures Truth evaluates publicly offered trading systems against live market performance rather than hypothetical backtests. This matters because backtests can be manipulated through curve fitting to show impressive historical results that fail in live trading. A system that performs well in Futures Truth’s ongoing evaluation has demonstrated that its results are not purely historical artifacts. The evaluation provides the independent verification that commercial trading system marketing cannot credibly provide.
Why is position sizing the most critical of the five questions?
Because the same entry and exit rules produce radically different performance depending on how positions are sized. A system that risks 25% of equity per trade will produce catastrophic drawdowns from the same signal sequence that a system risking 1% per trade handles comfortably. The edge in the entry and exit rules is only realized if the position sizing allows the account to survive the losing trades long enough to capture the winning ones. Without defined volatility-adjusted position sizing, the system’s theoretical performance cannot be replicated in practice.
Why must exit rules be defined before entry?
Because the act of entering a position creates an emotional relationship with it that makes the exit decision subject to loss aversion, the desire to lock in profits, and all the other behavioral biases that systematic exit rules are designed to override. A stop loss defined before entry is a rule executed in the presence of emotion. A stop loss decided after entry is a decision made under the influence of emotion. Rules defined before entry produce consistent execution. Decisions made under emotion produce the disposition effect.
Trend Following Systems
Want to learn more and start trading trend following systems? Start here.
