Building robust strategies

January 25, 2016 01:00 PM

Backtesting a quantitative trading strategy is essential in measuring its potential, but as everyone knows, past performance is not necessarily indicative of future results. 

The key to determining the performance of successful —on paper—strategies, is its robustness. Robustness in trading is the property of a trading strategy either to adapt to or to withstand changes in the market, and therefore maintain positive gains. The ultimate check of a trading strategy’s robustness is live trading. This takes time. Is there A method to test the robustness of a strategy based only on past data? Perhaps. 

Quantitative traders frequently use statistical methods to assess the robustness of a trading strategy. 
If the strategy meets certain statistical criteria then it should maintain its performance. Typically, these criteria include requirements for the number of trades, distribution of returns on a trade-by-trade basis and over a long period of time. Various integral metrics like a strong Sharpe and Sortino ratio also help. 

But, regardless of the complexity of these metrics, most often the key robustness assessment criterion can be reduced to a relatively simple requirement: That the distribution of returns over time is as linear as possible. This means that a strategy delivers a similar return pattern during a given period, and therefore because the strategy performs equally well during different market environments, it can be expected to perform well when the market environment changes again in the future.

The opponents of this approach always point to examples when a strategy that performed perfectly on past data fails. Unfortunately, these examples are numerous, and this adds to the opinion that no past performance data can guarantee the future performance.

While past data cannot guarantee future results, there are better and worse ways to measure simulated strategies. What we need to add to the analysis of robustness, and what most pure statistical methods lack, is the analysis of the past performance relative to market processes exploited by this strategy, and changes in performance relative to changes in these market processes.

For example, let’s consider a strategy that utilizes inefficiencies in the spot forex market, which regularly occur at certain times thanks to specific bank operations (around opening, settlement, clearing and so on). If the market environment is stable, a simple time-based strategy can work well. However, if it changes, the strategy performance will suffer. To avoid significant drawdowns, we should know what factors could change the environment. For forex, this could be banking rules that alter established banking operations. In late 2012-early 2013, strategies built prior to rule changes may no longer work, and not because of backtest statistics.

Even in development it is possible to test a strategy given sufficient historical data that refers to a similar market environment in history. Although there was no strict similarity between any past change in regulations and today’s processes, during 2006-2007 similarly significant changes could be observed. Therefore testing a strategy during this period would have provided an idea about possible performance when the regulations undergo new changes.

A weak equity curve during 2006-2007 would prove the concept of the strategy, rather than signify a flaw in its logic. And an attempt to fit the strategy to all known historical data, including the period when it should not work by design, will weaken it due to curve fitting.

Statistical methods of robustness assessment are not sufficient to produce any more or less useful prognosis of the future performance. It is essential to start with understanding the reasons why a particular strategy makes money; the real, physical market processes that are exploited by this strategy. Then, knowing the factors that may affect these market processes, and knowing how similar factors affected the market in the past, we will have a betterassessment of the strategy’s robustness. Understanding why the strategy makes money allows us to stop trading that strategy at early stages when the market environment that the strategy was successful in is likely to change, and therefore will not produce positive results. 

From this standpoint an acceptable strategy may exhibit an unpleasant equity curve, but at the same time may be quite robust in terms of its capability of working under the right market conditions in the future. And smoothness of returns within this model is achieved by identifying the best market conditions it works under and employing it at those times. 

By combining quantitative and qualitative analysis you can build and execute robust, long-lasting models.

About the Author

Alex Krishtop is an expert in systematic and automated trading. He is the director of education at Algorithmic Traders Association and runs an exclusive educational course in systematic trading. @atassn