In the first installment of this series, we introduced a random entry system that based its entries on a virtual coin flip (see “Guide to trading system development,” September 2012). The base system was backtested across four years of euro forex data to gather trade data for statistical analysis. As we saw, the base system was unprofitable. In mathematical terms, the base system has a negative expectancy of -0.81.
Expectancy is an important metric. It is the amount you can expect to win or lose for every dollar risked. It is calculated with the following formula:
(Winning percentage x Average win) – (Losing percentage x Average loss)
That is, the system loses 81¢ for every dollar risked. The astute reader likely recognized this aspect of the system as a stumbling block that often trips up the beginning system trader: It is unlikely, if not impossible, that a system with a negative expectancy can be made profitable through trade management. As it turns out, this system is unprofitable regardless of any stop loss or profit target that may be added.
This does not mean the system necessarily should be discarded. It often is the case that, though the first version fails, an adjustment to some aspect of it may result in a profitable system. In this case, there is another parameter of our demonstration system that can be adjusted: The entry time.
The 8 a.m. (EST) entry was chosen arbitrarily. This may be the critical variable that can salvage our model and allow system development to proceed. Indeed, this turns out to be the case. A simple optimization of the entry time (stepping from 3 a.m., EST, the London open, through 8 a.m., EST, in one-hour increments) creates a profitable system. When backtested across the same data, an entry time of 4 a.m. produces a net profit of $2,853. Expectancy is a positive 2.75. Now we can apply trade management techniques to see if we can improve the results.
Walk-forward analysis
If a system is backtested across an entire historical data series, the results of such tests will indicate only how the system would have fared had it been operational during the time period covered by the historical data. The results tell us nothing about how the system will fare going forward in real time.
One way to address this limitation is walk-forward analysis and optimization, popularized within the trading community by Robert Pardo. With this approach, the historical data are broken up into smaller periods of time. The system then can be backtested and optimized against just a portion of data, called the in-sample data. After the system parameters have been optimized, it then can be tested on the portion of the data that immediately follows, or the out-of-sample data.
The out-of-sample data period usually is a percentage of the in-sample data period. The results of the out-of-sample tests are recorded. The process then is repeated by “sliding forward” the in-sample data period and then re-optimizing the parameters. The amount by which the in-sample data is slid forward is equal to the length of the initial out-of-sample data period (see “Walk-forward testing,” below).
In the chart, the in-sample data period is six months and the out-of-sample data period is two months. The first optimization is run on the data from January through June. The test is run on the data from July through August. The out-of-sample results are recorded. The in-sample data period then slides forward by two months. A new optimization is run and the parameters are re-set accordingly. Another out-of-sample test is run September through October.
This walk forward through the data removes some of the limitations of basic backtesting. Profitable results from this method of testing should provide a higher level of confidence in the system when traded in real time in a live account.
Setting our stop
The next phase of development of the trading system is to add risk management in the form of the initial stop loss (ISL). It provides a defined exit point should the market move against the position.
We use Maximum Adverse Excursion (MAE) analysis to help set this stop. MAE is the greatest distance, in points or ticks (depending on the contract), that the market moves against a trade when it is open. In the case of a losing trade, MAE can occur at the exit price. Recall that we collected MAE data from the basic system and performed a simple statistical analysis of that data: We measured the mean, median and standard deviation of the MAE data. That analysis will form the basis of our walk-forward optimization analysis to find the optimal stop loss.
The MAE data gathered from the backtest of the 4 a.m. entry time produced the following results:
Mean: Wins: 0.00334 (33.4 pips); losses: 0.01016 (101.6 pips)
Median: Wins: 0.00290 (29.0 pips); losses: 0.00900 (90.0 pips)
Std. dev.: Wins: 0.00269 (26.9 pips); losses: 0.00605 (60.5 pips)
It’s likely that the optimal initial stop loss will lie somewhere between the mean MAE values of winning and losing trades. Ideally, the optimal value will cut short the losses on the losing trades without prematurely exiting the winning trades. Of course, this is not going to be the case: Some trades that would have been profitable in the absence of an initial stop loss will be exited early; some losing trades that would have rebounded to a smaller loss will do the same. Trading is a series of compromises; our hope is to find the best one.
If we subtract one standard deviation from the mean loss MAE, the result is about 41 pips. This is near the mean win MAE, and is thus an acceptable lower boundary of our test range. If we add two standard deviations to the mean win MAE, the result is about 87 pips, near the median loss MAE. This is an acceptable upper boundary of our test range. Rounding to the nearest multiple of two, we have:
Lower boundary of test range: 40 pips
Upper boundary of test range: 88 pips
This range should encompass all practical values of the initial stop loss. It should leave enough room for winning trades to develop while also reducing the losses on the losing trades. If we optimize in increments of two pips, the optimizer will test 25 possible values for the ISL. Given the power of today’s computers, this should take only a minute per optimization run.
With our range of MAE values set, we’ll commence the walk-forward optimization of the ISL. To ensure that the test data’s characteristics more closely align with the current state of the market, we’ll use a date range of January 2010 through December 2011 for the walk-forward analysis. Because this is a day-trading system, we’ll use a small optimization period of six months with an out-of-sample period of two months. This creates nine passes through the data. Starting with the January 2010 data, the sequence of optimization and walk-forward runs follows the pattern shown in “Walk-forward testing.”
To simplify things, and to ensure consistency across all the optimization runs, we’ll choose the stop-loss value that returns the highest profit. Choosing our optimized parameter this way may reduce robustness, but will suffice for demonstration purposes. The results of the nine passes are shown in “Loss analysis” (below). We can compare the final result with the base system by backtesting the base system on the same data. Here are the results:
Profit without ISL: $1,428.52
Profit with ISL: $775.92 (expectancy: 1.19)
Adding the initial stop loss did not improve profitability. Instead, it cut the profits nearly in half. This does not imply, however, that there is no advantage to having the ISL in place. In fact, the opposite is true: It allows us to quantify the risk of each trade. Without the initial stop loss, the risk is open-ended and dependent upon price at the time of the forced trade exit at the end of the trading session.
The largest loss incurred by the base system was 448 pips. Several consecutive losses of this magnitude could inflict serious damage to the trading account, and such occurrences are to be expected. In contrast, the largest loss incurred with the initial stop loss system was 97 pips. Note also that the expectancy is positive. For these reasons, we’ll continue using the initial stop loss as we seek the optimum profit target.
Profit points
Maximum Favorable Excursion (MFE) is the greatest distance that the market moves for a trade during the time the trade is open. Our strategy for optimizing the profit target is similar to that used for optimizing the ISL. In this case, however, we are conducting the profit target analysis with the stop loss already in place.
Here are the MFE data gathered from the initial backtest:
Mean: Wins: 0.01055 (105.5 pips); losses: 0.00355 (35.5 pips)
Median: Wins: 0.00973 (97.3 pips); losses: 0.00303 (30.3 pips)
Std. dev.: Wins: 0.00563 (56.3 pips); losses: 0.00307 (30.7 pips)
The average winning trade amount was $70.98, or about 71 pips. It appears that the average winning trade is giving back about 34 pips before exiting at the end of the session. Because our goal is to try to capture those pips, 71 pips is a logical value for the lower boundary of the test range. For the upper boundary, we’ll add two standard deviations to the mean win MFE; the sum is about 218 pips. Though it may seem like wishful thinking to imagine profits this large, note that the largest winning trade with the ISL was 293 pips. Rounding to the nearest multiple of five, we have:
Lower boundary of test range: 70 pips
Upper boundary of test range: 220 pips
If we optimize in increments of five pips, the optimizer will test 31 possible values for the profit target. We’ll reuse our boundaries for optimizing the initial stop loss, keeping the increment at two pips. The total number of combinations is the product of the possible values for each variable: 25 x 31 = 775 possible combinations of initial stop loss and profit target values. This time, each optimization run will take a bit of time, perhaps several minutes or more, depending upon the speed of your computer. The optimization runs will be performed over the same time periods as before. The results are shown in “Merged results” (below).
Adding a profit target to the stop-equipped system caused profitability to drop from $775.92 to $103.05; expectancy remained positive, but dropped to 0.264. The results from the walk-forward period of November through December 2011 wiped out nearly all the profit that had accumulated during the prior walk-forward periods. Such a result calls into question the robustness of the system.
Although these results may not appear promising, they have a purpose. This is precisely why testing is so important before trading a system with real money. In the last part of this series, we’ll address this as we apply money management to the system and perform Monte Carlo analysis as the final test of our system.
Neil Rosenthal is a retired dentist who trades his own account. He also is an experienced computer programmer. He can be reached at rightedgetrading@gmx.com.
