Can your system do the walk?
Next consider the raw trade report (see “Trade by trade”). The raw trade report shows each trade for both the training and run periods from Jan. 3, 1991, to Feb. 2, 1995. The runs with “out” in the last column are in the training set. The “trans” trade is in both the training and out-of-sample set. The last two trades are in the “run” or out-of-sample set.
The next way to handle transition trades is to exit all trades, exit only if direction changes or exit and reenter. Exiting all trades is simple. At the end of a boundary, all open trades are exited and a new signal is used to get back into the market. While this is a simple approach, it can have a significant effect on system performance. Exiting only if the trade direction shifts produces more realistic results and, in the case of longer-term systems, does not disrupt a long-term trade that may be responsible for significant profits.
Our final transition-trade solution is to exit and reenter in the current direction. This option is similar to only exiting if the direction changes with one exception. If the direction changes, a new position is opened on the first day of the new out-of-sample period. This concept is often best for long-term trend-following systems.
CASE STUDY: PORTFOLIO EXAMPLE
Most traders trade a collection, or portfolio, of markets. This complicates the procedure, but it is not necessarily complex. This is the portfolio we’ll review:
• Natural gas
• Japanese yen
• 30-year Treasury bonds
• Crude oil
We will use overlapping dates from Jan. 3, 1991, to Nov. 25, 2009. We will also deduct $100 per trade for slippage and commission. We will use the triple moving average crossover system and again select our parameters based on the net profit/drawdown ratio. Our optimization scan will be as follows:
Short moving average = 5 to 20, steps of 5
Medium moving average = 20 to 60, steps of 10
Long moving average = 60 to 120, steps of 20
Our training period will include 1,250 data points, and the out-of-sample period will have 300.
The walk-forward results are solid (see “Portfolio report,” right). Our data period includes 12 testing windows. Out of these 12 windows, 10 were profitable. We also can see that our parameters are stable. The last three windows use the same set of parameters, and two other times we had the same set of parameters in two adjacent windows. These last three windows all used 5, 60 and 80. Our walk forward analysis produced $321,953.75 using our exit all trades on window boundary mode.
This system is a trend-following system, so exiting trades on each boundary is not optimal. If we try the second approach that exits only if direction changes, we make $408,957.50. Exiting and reentering each trade makes $406,648.75. Walk-forward analysis is a type of adaptive system technology, and most trend following systems did badly in 2009, but our walk-forward analysis here was profitable in 2009 because parameters were allowed to change over time.
While we used the built-in ratio of net profit/drawdown to determine our optimal parameter settings, there are times we may want to program custom functions. For example, if we want to add a simple function that weights recent results more, we could add the below code fragment to the measure:
If BarNumber = LastBar and drawdown <> 0 Then
setoptimizefactor((NetProfit + NetProfit - NetProfit[BarSize - 1]) / Drawdown)
We can then just select “optimize factor” in TradersStudio and use this custom function. Custom functions can be as complex as needed. You could even use a function such as optimal f to evaluate the system or product scoring based on objective rules.
One goal with parameter selection is to select a set of parameters where neighboring sets produce similar results. When developing a system, we can judge this robustness by producing three dimensional charts of parameters instead of just a raw performance score. We can see robust areas between two parameters and performance using charts, but the situation becomes significantly more complicated if we have five parameters.
Recent research has delved into ways to find robust parameter sets across many variables so walk-forward testing can be automated. The solution is in using an n-dimensional map of all the parameters and tying the score to each set of parameters. Next, different parameter neighbors are formed and robust areas are found. Fuzzy logic can be employed to address the issue of oscillation between two sections of the space with similar scores but different trades. This oscillation problem often can make walk-forward optimized perform badly.
Although walk-forward testing has been with us for a couple decades now, the area has fully automated processes, from parameter selection to system modification, have only become possible with recent technological developments. As traders become more familiar and comfortable with these tools, a new era of hands-off trading may emerge where systems are viable for years, not months, and profits can be reliably captured across a decade.
Murray A. Ruggiero Jr. is a consultant. His firm, Ruggiero Associates, develops market timing systems. He is the author of “Cybernetic Trading Strategies” (Wiley). E-mail him at email@example.com.