Many traders these days want to manage their speculative accounts with fully automated trading systems. The basic goal is to find one or more reliable systems to employ. Accomplishing that simple task, however, is not so simple. Reliability is hard to define, particularly when profits are part of the equation. Even if you knew for certain a particular trading system would provide 25% compound growth over the next five years, success would not be guaranteed.
There are several factors that can contribute to unexpected failure. First, circumstances could require you to take money out in too short a time frame, robbing the system of its ability to profit. Another reason is your personal risk tolerance (or drive for reward) may not mesh with the system’s ups and downs. Finally, the system itself may not have been robust from the beginning, and it may fail quickly when faced with changing market environments.
Here, we will examine all three factors and use them to construct a framework for better analyzing trading systems for real-world trading.
Time and reliability
One of the biggest issues with successful trading is knowing your time frame. You must plan both when you will need your money and how much you will need. The time frame and required system growth date are critical.
Another issue is loss penalty. To demonstrate this, we’ll use a simple triple moving average system. It trades a small basket of futures: Cotton, mini natural gas, euro, copper, Treasury bonds, mini crude oil. We will use $100 for slippage and commissions, trading one-lots exclusively. We will test from Jan. 4, 1991, to April 16, 2013. We are using 12, 60 and 70 as our three moving average lengths.
The system wins only 38% of its trades. The average winning trade lasts 153 days, whereas the average loser lasts only 55 days. If we look at monthly returns, however, six out of 10 months make money. Our winning months average $7,028, and our average losing month is –$5,805. At first glance, this may seem like a paradox—how can only 38% of the trades make money, while 60% of the months are winners? The answer lies in the role of time. This is a trend-following system; as such, a large percentage of trades shows an interim profit at some point even though (per the rules of the system) only 38% are closed out as winners.
Because the average winning trade lasts 153 days, we can assume we must trade for at least 600 days before we make money. That would be just four winning trade periods. Even though it may seem like a long time (almost two years), it is still a short period measured in terms of trade time frame.
We can find the probability of being profitable N periods into the future using Ralph Vince’s concept of Leverage Space Theory. In Vince’s new book “Risk Opportunity Analysis,” he describes a technique that joins probability tables, allowing you to calculate the probability of being profitable after N time frames. Using a leverage space library created by Joshua Ulrich, we calculated the probability of profit from one to 24 months. We use the robust method to calculate f, which is P/2=0.30 in our case because we have 60% winning monthly periods. Our results are shown in “Monthly profit probability” (below).
On the first month, the probability of being profitable matches the winning percentage in the results of our simulation. This rises slowly over 23 months, when it breaks the 75% threshold. Keep in mind that this still means that after two years, 25% of the time you would be losing money. If we look at the monthly returns more closely, we can see that, indeed, we have two losing years in a row.
The win percentage by trade and period return do not give us the same view of chance of future profitability over different periods. We could look at daily returns and apply a Monte Carlo simulation. To do this, we would take sample windows of daily returns over 10, 20, 40, 80, 100, 150, 200, 300, 400, 500 and 600 days. The idea is to take different sampling periods and then calculate the probability of a given window length being profitable after, for example, 100 trials. This will provide a different view of profitability over a given time horizon.
Further research in profitability and time horizons is an important and exciting area. It also is a burgeoning field, as viable applications of the relevant math just are being developed.
Each of us has different financial goals, if even because those goals are set in the context of our individual financial situations. Because of this, no trading system can meet the needs of all traders. Even experienced traders have difficulty following systems that don’t share their expectations. Traders typically encounter two different issues that pressure them to break the rules.
One reason traders abandon a system is a lack of trust. A trader can begin to question whether or not the system is based on a valid, sound and mathematical premise. The best fix for this is to fully understand the logic behind a system. You can trade commercial systems, but you must understand the premise behind them.
The second problem is poor assessment. Most traders skim the numbers of a standard system performance report and think they’ve learned all they need to know. This is not good enough, and sometimes it requires just a small peek beyond the numbers to glean significant insight. For example, if a system wins 40% of its trades, that the longest losing streak is 10 trades, and the average trade lasts 60 days, it means a system could go two years without a winner. Ask yourself: Could you go two years without a winning trade and still believe in a trading system, regardless of how much money it made over a 20-year backtest?
Unfortunately, it’s difficult to know if you have the psychological fortitude to keep trading such a system until you live through it. There is substantial risk at play. Not following the complete system by taking only certain trades could leave you on the sidelines when one or two extremely profitable trades present themselves. Often, regardless of a system’s rate of profitability, 80% of a system’s money is made from about 20% of its best trades.
All systems have bad years, and some will have several. Some systems will come back and become profitable, while others will not. It’s important to have a plan and a strategy that lets you sleep at night. Timing is important. For most, it’s easier to take a drawdown after you have made money. For example, if you start with a $200,000 account and take a 30% drawdown immediately to $140,000, you are far more likely to pull the plug than if you first made $100,000 before the 30% drawdown, which then would have dropped you back to $240,000.
Some traders also are known to be adrenaline junkies. It’s a thrill to trade and win. Many traders have found themselves making consistent profits with a reliable system and then risk more on non-system trades because they craved more action. One possible solution is to let your broker do your trading for you. Such programs are popular, and many brokers offer this service.
The role of data
Data-based system reliability compares a system’s recent statistical performance to earlier periods. It also can use the results of a given system’s optimization. This type of reliability analysis does not require knowing the rules of the system. However, you must be able to run backtests and trade the system in real time. There are limitations to this approach, the most important of which is that we don’t know if the system is based on anything valid or logical.
One such test is called optimization space robustness analysis. We use the optimization grid over valid sets of parameters and take average and standard deviation of important performance columns such as net profit, drawdown and profit factor. We then analyze the average across the optimization space and examine these values minus one and two standard deviations.
Consider the results in “Space invaders” (right) from the same triple moving average crossover system discussed earlier. This analysis confirms likely expectations. Even though initially it looks good, the system is not viable. Granted, the average is profitable, but this is a minimum requirement. We also would like the average minus one standard deviation to be profitable and, ideally, the average minus two standard deviations also should make money.
Net profit is only one statistic that should be examined using this analysis. It also should be performed across other measures, such as drawdown, long-side profit, short-side profit and profit factor.
Born not made
A trading system must be built on a strong premise. If we build a given system based on a theory, then as long as our assumptions are true, we can expect the system to be reliable. The premise for a system must be based on strong scientific method.
In the case of intermarket-type systems, premise development can be at a rather high level. That is to say, do the other markets have direct influence on the market we are trading? In the case of a trend-following system, the premise that “markets trend” is not enough. Different markets have different structural price movements. It is imperative that these trends are a part of your understanding because these differences determine whether a channel breakout or moving-average approach works best.
Let’s consider an intermarket model based on the well-proven concept of intermarket divergence. Here are the rules:
Calculate price relative to a simple moving average
Let InterInd = Close of Intermarket – Average (Close of Intermarket,N)
Let MarkInd = Close Traded Market – Average(Close of Traded Market,M)
Assess positive correlation
If InterInd > 0 and Markind < 0 then buy at next bars open
If InterInd< 0 and MarkInd >0 then sell at next bars open
Assess negative correlation
If InterInd< 0 and MarkInd< 0 then at buy at next bars open
If InterInd > 0 and MarkInd >0 then sell at next bars open
This simple concept has proven to be a robust methodology for predicting future price action using intermarket analysis.
To many, that may seem like a bold claim, so let’s consider why intermarket divergence works. Intermarket analysis fundamentally is an arbitrage methodology. Because we are using different markets on difference scales, we can’t tell when they are out of sync except in the case of divergence. In this case, we know we have a miss in pricing that we can exploit. If we find a market that leads other markets (which it does not have to do all the time), then we can predict when the market we are trading will reverse and adjust to correct the incorrect pricing.
Let’s look at trading T-bonds using utility stocks. The original model, published in 1998, used New York Stock Exchange utility stocks as the independent variable. Using the same parameters as when we developed the system in 1998, we can revisit the consistency of the system at the time (see “Profit map,” below). We have a broad and wide profitable area. As profit slips, it does so slowly. This is an example of a robust optimization space.
Now consider the period after the system was published. (Unfortunately, the original series of utility stocks was discontinued. We replaced it with the Philadelphia Electrical utility average.) This period — Dec. 31, 1997 to April 11, 2013 — is shown in “Rougher waters” (below). The curve is not as flat and wide as the in-sample optimization period, but we still have a broad area of profitable performance. Parameters from four to 14 for T-bonds and 14 to 22 for the utility stocks produce between $100,000 and $150,000 over this period. The area of $50,000 to $100,000 (or more) profit represents a large part of this curve.
Based on this study, we decide to take a closer look at a system that uses an eight-period moving average for T-bonds and an 18-period moving average for the utility stock index. Deducting $75 for slippage and commission, we make $79,275 on 57% profitable trades from Sept. 22, 1987 through Dec. 31, 1997. Our average winning trade is $2,017, and our average losing trade is $554. Our profit factor was 1.92. Out-of-sample results are represented by the period following when the system was published. From Dec. 31, 1997 to April 11, 2013, the system made $122,331 on 60% winning trades. The average winner was $1,913, while the average loser was $1,882. The profit factor over this period was 1.56.
Now that you have seen that this system has done well over time, you should ask about the premise. Utility stock prices are based on perceived future borrowing interest costs. Normally, this logic is sound, but during deregulation of California power, this relationship had problems. This system’s only two consecutive losing years were 1999 and 2000, during the California energy crisis.
Intermarket analysis is a powerful tool, but a deep understanding of the markets is required to ensure that you accurately can spot what will and will not work. This becomes especially critical when correlations are broken by other forces. Auto-detection of these times is a key area of research for intermarket-based trading systems.
It’s not easy answering these two simple questions. Is a system reliable? Can I trade it and make money? Often, these two questions lead to more questions than answers. In any case, a key for both system traders and discretionary traders is to know yourself. If you don’t understand your own risk tolerances and your hunger for profit, you don’t know if you’re capable of following any given system. Understand yourself, understand the premises of your trading system and use the data analysis tools covered in this article, and you’ll be well on your way to long-term profitability in the markets.
Murray A. Ruggiero Jr. is the author of “Cybernetic Trading Strategies” (Wiley). E-mail him at firstname.lastname@example.org.