Optimization: Teaching an old dog new tricks
Trading system optimization has been a hot topic of discussion since the first commercial trading system programs appeared in the 1980s. Optimization has always been a double-edged sword because while it discovers the best inputs for a specific trading system logic, it also defines what’s best by the past, which may or may not repeat itself. However, in light of new technologies, optimization is finding new roles in a system trader’s toolbox.
The basic idea of classic optimization is simple. In developing a trading system the first step is to develop a valid premise. Then, a set of rules is developed to exploit that premise. Most rules, of course, need inputs (look-back periods, range or volatility multipliers, etc.). Optimization is the process of determining the values to use for the inputs.
Early on — and even today for some — a popular method of optimization was to blindly test every conceivable combination of input values within a defined range, then trade in real time using the set that returned the best hypothetical results in the past. This is a recipe for disaster. Even if a system is based on a sound premise, the best set of parameters over the past 20 years is rarely the best set for any given day over the next 20 years.
IS IT ROBUST?
A better way to use optimization, when it comes to individual system performance, is to use it to judge a trading system’s stability. Consider the results of a basic triple moving average crossover system on a diversified basket of 15 markets tested from Jan. 3, 1980, to Aug. 9, 2007, with $100 deducted per trade for slippage and commission.
For this system, the following inputs will be optimized:
• Short average (SA): lengths from five to 30, in steps of five
• Middle average (MA): 30 to 60, steps of five
• Long average (LA): 60 to 100, steps of five
These input ranges create 378 possible combinations. Let’s look for the valid cases where SA < MA < LA and use a statistical test to judge the robustness of the optimization. This test is simple. All combinations are tested and the average and standard deviation of the summary statistics are calculated:
Average profit: $864,377.95
Drawdown (DD): ($151,278.47)
Standard deviation (SD): $33,175.56
DD + 1 x SD: $750,791.11 ($184,454.03)
DD + 2 x SD: $637,204.27 ($217,629.59)
DD + 3 x SD: $523,617.42 ($250,805.16)
Then, as a further test of the system’s validity, one, two and three times the standard deviation of the different profit values is subtracted from the average profit.
At a minimum, the system should be profitable at twice the standard deviation. Ideally, it should be profitable at three times. This triple moving average crossover system still makes more than $520,000 at 3 x SD, which represents the 99% confidence area.
One way to examine the performance of different input values is to use a surface chart. “On the surface” (below) shows what happens to net profit as we move across different input values for the moving averages. The first chart shows the net profit figures for different values of the short and medium moving averages and the second chart shows the net profit figures for different values of the medium and long moving averages.
The parameter set that produced the most money was SA = 25, MA = 55 and LA = 75. This combination made $1,088,191 with a drawdown of $105,888 in the backtest. There were 28 combinations that produced more than $1 million in the backtest and 165 combinations produced more than $900,000, representing about 43% of the combinations. As a summary, an SA between 20 and 30, an MA between 40 and 50 and an LA between 80 and 95 produces a stable area of the curve based on net profit.
If you were considering this trading system, another set worth a closer look would be 25, 45 and 90. This combination produced $1,013,964 with a drawdown of $124,764. There are other combinations with better profit/drawdown ratios, but the optimization surface is not as smooth. Further, this set of parameters is in the center of our proposed range.