Many of today’s market analysis platforms enable traders to evaluate both actual and theoretical trading activity by generating a strategy performance report — an objective evaluation of a trading strategy’s performance. A strategy is a set of precise rules used to enter and exit the market, and strategy development involves defining those rules by using historical backtesting, optimization and forward testing to determine the most effective and potentially profitable parameters for the system. The report estimates how well a strategy should work.
A strategy performance report is a vital component of the process that helps traders identify the strengths and weaknesses of a system, and quantify the system’s viability using a series of mathematically based performance metrics.
Quite often, even experienced traders fall into an analysis rut, relying on the same numbers and interpretations to evaluate their trading systems. For all of us, beginning and experienced traders alike, it’s helpful to step back and reconsider what our system reports are telling us.
Natural selection
There are dozens of performance metrics that can be used to evaluate a strategy, and traders often develop a preference for the metrics each finds most useful to his or her own trading style and goals. “Performance snapshot” (below) shows an example of the summary page of a strategy performance report, which includes an overview of all trading activity for the specified period.
Using a performance report effectively is not always straightforward. While it is exciting to zoom in on metrics such as total net profit or percent profitable, each metric alone does not provide a comprehensive view of the strategy. Consider, for example, a strategy that boasts a 90% profitable statistic, meaning that 90% of the completed trades during the specified period made money. While this appears to be good news at first, it does not provide a clear picture of the strategy’s performance.
A deeper analysis would consider factors such as the size of each winning and losing trade. In an extreme case, if 90% of the trades had an average winning trade of $5, and the remaining 10% of trades had an average losing trade of $500, it would be a losing system.
As another example, a strategy performance report could indicate that a system has a total net profit of $800,000. That might be an encouraging figure, but traders also must consider the maximum drawdown or the account size required values to determine how much money is needed to make that $800,000. It could be that the trader would need a much larger account, or would need to sustain greater losses than is comfortable, to trade the strategy.
The metrics have to be considered separately as well as collectively to produce an accurate assessment of the system’s performance.
Key metrics
“Common measures” (click here) shows a list of commonly used performance metrics. Seven metrics in particular provide a good starting point for testing a potential trading system or evaluating a live trading system:
1) Total net profit
The total net profit typically is the metric that appears at the top of a strategy performance report, and represents the bottom line for a trading system during a specified period. It is calculated by subtracting the gross loss of all losing trades (including commissions) from the gross profit of all winning trades:
Total net profit = Gross profit – Gross loss
While it is common to focus backtesting efforts on increasing this number, the metric alone can be deceptive. The total net profit provides no indication that a system is performing efficiently, and it does not represent the amount of risk to which a trader would be exposed through implementing the system. Other metrics should be viewed along with the total net profit value to evaluate the strategy accurately.
2) Profit factor
Another statistic that garners lots of attention is the profit factor. This value relates the amount of profit per unit of risk, where values greater than one indicate a profitable system. It is calculated by dividing the gross profit by the gross loss:
Profit factor = Gross profit / Gross loss
In theory, this value should be as high as possible. In reality, however, extremely high profit factors during testing rarely correlate in live trading. Many successful trading plans have profit factors that come in over 1.5 and perhaps even as high as 5.0 in extreme cases.
Rather than an outright high number, traders should look for consistency over time, regardless of the trading period, when comparing profit factor values. In other words, a system has a better chance of performing well during live trading if the profit factor values show positive correlation over different time periods. The profit factor should not change dramatically from one time period to another. If it does, it’s a sign the trading system is not reliable.
3) Percent profitable
The percent profitable metric also is known as the probability of winning, and is calculated by dividing the number of winning trades by the total number of trades for a specified period:
Percent profitable = Number of winning trades / Number of total trades
Again, in theory, this value should be as high as possible. Trading style and strategy type, however, are factors that affect the ideal percent profitable value. Trend-following strategies, for example, tend to have lower percent profitable values — perhaps even as low as 40% —because the trades that do win are typically quite large, and any losing trades are usually closed for a small loss.
Scalping strategies, by comparison, typically rely on winning small amounts of money while risking similar amounts on any one trade. A scalping strategy may generate dozens of trades each trading session. Because each win is relatively small, and each loss tends to be close in value to the winners, the system requires a higher percent profitable value to make money over time. Like the profit factor, percent profitable values should remain fairly consistent across various periods.
4) Average trade net profit
The average trade net profit is the expectancy of the system, and represents the average amount of money that was won or lost on each trade during the specified period. It is calculated by dividing the total net profit by the total number of trades:
Average trade net profit = Total net profit / Total number of trades
This metric shows the average profit the trader can expect each trade to make over time. Position sizing will affect this value because larger positions will multiply the value. Using consistent position sizing during testing will help traders accurately compare systems and/or optimizations.
Another consideration is the outlier — a trade that creates a profit or loss that falls outside the statistical average or norm for the system. An outlier can inflate the average trade net profit falsely, making the system appear far more profitable than it really is. Outliers can be removed to show more realistic results. Trading plans that rely on one or two unusually large wins may prove to be unreliable or unprofitable in live trading.
5) Maximum drawdown
The maximum drawdown is the worst-case scenario for a trading period, and measures the greatest loss from a previous equity peak. The metric helps measure risk, and determines the account size that would be necessary to trade a specific strategy. Ideally, the maximum drawdown would be as small as possible; however, all trading systems have losing trades and there will be the inevitable drawdown. The maximum drawdown should never be greater than the amount of money that a trader is willing to risk.
Maximum drawdown is one of the most important metrics because it estimates immediately whether a trading system is appropriate or even possible for a trader (based on risk and account size).
6) Buy-and-hold return
The buy-and-hold return displays the percentage return of holding the commodity or security in a long position for the entire testing period. This metric should be compared with the annual rate of return for the strategy. It helps traders evaluate if a system outperforms a buy-and-hold strategy. Additional effort and costs are associated with executing multiple trades (dozens, hundreds or even thousands) over a specified period vs. a single buy-and-hold trade. The strategy’s annual rate of return should more than justify the extra time and costs (fees and commissions).
However, this statistic should be evaluated in the context of the time the strategy spends in the market. The reason is while a strategy is out of the market, with funds invested in Treasury bills or cash, downside risk effectively is eliminated. All else equal, less time in the market is preferred. So, for some traders, a strategy that makes 80% of buy-and-hold but spends 60% of its time in T-bills may be the better answer for a specific trader on a risk-adjusted basis.
7) Equity curve
Strategy performance reports include a performance graph, or equity curve, that provides a visual representation of the system’s gains and losses. Traders can tell at a glance if the system is performing well, or if its performance has changed significantly over time. Ideally, an equity curve is as straight as possible while consistently climbing over time (see “Up and steady,” below).
With all of these performance measures, it is vital to test both in- and out-of-sample data to ensure that the system is not a victim of curve-fitting, or over-fitting as trader Bill Eckhardt calls it. Curve-fitting involves tweaking a system to the point that it is extremely profitable on the set of historical price data you are testing rather than optimizing the strategy as a whole. These systems typically fall apart during live trading. Performance measures and equity curves that remain consistent throughout in- and out-of-sample testing help verify a system’s potential.
Traders today can take advantage of the powerful tools offered by many market analysis platforms. When developing a trading strategy, or evaluating live performance, a strategy performance report provides important data regarding the system’s efficiency and potential profitability, but like all tools it must be used correctly.
While traders will gravitate toward specific metrics within the strategy performance report, it is important to view metrics collectively because the real meaning of each is shown only when compared with other measures. In addition, comparing the data over different trading periods (using in-sample and out-of-sample data, for example) can provide a more accurate assessment of the strategy’s potential.
Jean Folger is the co-founder of, and system researcher with, PowerZone Trading, LLC. She can be reached at www.powerzonetrading.com.
