From the January 01, 2012 issue of Futures Magazine • Subscribe!

Back- vs. forward testing: Test twice (or more), trade once

Traders eager to begin trading in a live market frequently make the mistake of relying exclusively on backtesting results to evaluate a system’s potential. Backtesting, which refers to the testing of a trading idea on historical data to verify how a system would have performed during a particular time period, can produce misleading results. It’s important to have a more complete approach to trading system evaluation.

Because backtesting is only part of a proper evaluation process, focusing on backtesting results alone can lead a trader to believe he or she has a rock-star trading system when, in fact, the system may perform poorly in other phases of testing and, eventually, during live trading. Finding positive correlation between backtesting results and other phases of testing, including out-of-sample and forward performance testing, is vital in accurately assessing the viability of a trading system.

Backtesting basics

Backtesting allows traders to apply trading ideas to historical data to see how the system would have performed. Many of today’s trading platforms offer the ability to backtest, and provide efficient and easy-to-use methods of testing ideas on past market data.

Without putting real cash on the line, traders can evaluate the effectiveness of a trading idea with a few simple keystrokes. As long as an idea can be quantified, it can be backtested — from simple moving average crossovers, to complex systems that incorporate multiple trade filters and triggers. "Trading the Russell" (below) shows a strategy that is being tested on the mini Russell 2000 contract.

Some trading platforms have strategy "wizards" or "builders" that allow analysts to select from a field of variables to create a custom strategy. Traders can use these strategy building tools, write their own code (typically using the platform’s proprietary language), or work with a qualified programmer to develop a trading idea into a testable form. Frequently, a trading system will incorporate user-defined input variables, such as the moving average length or the number of standard deviations, which allow the trader to tweak — or make small changes — to the system. These subtle modifications often can lead to dramatic changes in backtesting results.

Optimization studies are another feature that many trading platforms offer in conjunction with backtesting capabilities. Optimization entails entering a range for a specified input — a moving average length, for example — and letting the computer perform the calculations to determine the input that has the best performance. For example, you can optimize a strategy to find the best profit target. You can set the optimization study to test values between $200 and $600 in $20 increments.

A multi-variable optimization analyzes two or more variables in conjunction to establish what combination leads to the most favorable results. A multi-variable optimization could determine, for example, which moving average length and relative strength index (RSI) level, when combined, would yield the most favorable results.

Page 1 of 4 >>
Comments
comments powered by Disqus
Check out Futures Magazine - Polls on LockerDome on LockerDome