There used to be a time when technical analysts believed that all data providers were created equal. There also was a time when they believed that all technical analysis software packages provided the same predefined indicators. That time has past.
Back in the day when there were very few data vendors from which to choose, technical analysts and traders simply accepted what was given to them in terms of market data and technical analysis tools. But as competition grew and computer power increased, the need for "clean" data as well as the ability to build signals, test systems and manipulate data became more important.
Today, speed of execution has been thrown into the mix, but the question remains, are technical analysts and traders getting the same price information or are certain data vendors more reliable than others?
A complete comparison study of data vendors and software creators is beyond the scope of this article. However, it still is important to know some of the issues facing analysts and traders so that they can consider their options carefully before spending their money on something that may not suit their needs.
The basis for all trading systems is the raw price data that makes up all past market activity. But the way data is delivered is different, as is the quality of that data. This is particularly true with futures data that needs to be tied together from a mosaic of different expiry months. This article will update you on the data game, as well as provide insight into how to create continuous data.
For technical analysts, professional traders and retail traders, the data-feed is the lifeblood of their technical analysis software, so choosing a high quality data provider is a good way to ensure that their investment in the software program has been money well spent.
When live data-feeds of futures data began to be readily available to the analyst and trader, it tended to be unreliable because the emphasis at the time was to deliver the product and worry about the quality later. The expression, "garbage in, garbage out" was sounded by many at the time, not just the fundamental traders. There wasn’t a lot of complaining from users because open outcry floor trading was where the action was and the trading edge remained there.
As computers improved and high-speed internet connections began to replace the old phone line hook-ups, the need for better, faster data-feeds as well as improved technical analysis software and order entry platforms increased. Electronic trading began to grow in popularity and traders began to take notice of the accuracy of their data-feeds. Now more than ever with the advent of Application Programming Interface (API) trading, traders have to be certain that the data they are using to build indicators and create systems is the same data being used to execute their trades.
Despite all of these changes, the need for quality data remains. The process used to be simple, no matter the data source you bought the data that fit your budget to ensure you had enough money left over to purchase your technical analysis software. Because data costs often eclipse the initial purchase price of the program, the need for reliable data remains foremost.
Whether evaluating technical analysis software or seeking a compatible and reliable data vendor, users must consider some important issues. The first key issue is the debate between using end-of-the-day vs. intraday data. End-of-the-day data summarizes the daily trading activity and delivers it to the user so that he can manipulate it to fit his needs. Intraday data is delivered to the user throughout the day either in real-time or delayed.
Those using end-of-the-day data have to be sure that it is compatible with their technical analysis software. In addition, they must have access to a historical database so that they can develop and test their trading systems. This means that the user has to be sure the data has been cleaned to get correct backtesting results.
Intraday data users have the same issues with data, but because the data they are looking at is live, they will experience the inevitable "bad tick" from time to time. These happen; however, the key is how the data vendor handles their occurrence. To provide the user with more accurate data, some vendors, like TradeStation, now process real-time insertion, deletion and correction messages sent by the market centers. When such events occur, they automatically are applied to the data stored on their servers. This updated information is sent to the various TradeStation windows so that any such messages that the exchanges send can take effect quickly.
Another event that can occur with live data is that market centers send data that sometimes does not coincide with current trading activity. It is important for intraday users that the data vendor has filters to remove trades of this type so that such trades, which may be misleading, do not alter your quotes.
Compatability
Besides working with clean data, a trader must make sure that his data works seamlessly with the technical analysis program he is using. One thing to consider is purchasing a software program that is compatible with many data providers. This will ensure that if there is anything wrong with the data-feed of one vendor, he can move quickly to another vendor without having to buy new software.
For those traders using historical data, it is important that the data has been filtered after the end of the trading day and before it is distributed. This is important especially to those who run end-of-the-day signals prior to the next day’s opening. There is no worse feeling than to create a list of buy and sell orders to execute, only to find the data you were working with was wrong and corrected the next day.
Traders using historical data also should look for vendors that offer data exporting. This is important if you are working with a program that includes portfolio tracking or management software. Often a trader will have two software programs, one that analyzes the markets and another that manages the trades. One of these programs may be incompatible with the data-feed. Rather than spend the money on two feeds, verify that you can export the data and the problem will be solved.
The bottom line is, verify that your data vendor is providing you with clean data that is 100% compatible with your analysis software package. In addition, become familiar with your vendor’s process for cleaning up bad data points and the timeliness of such corrections.
Besides working with an analysis software program’s technical indicators, some traders may want to utilize the program’s backtesting and system optimization capabilities. The success of this trading process relies completely on the accuracy of the data as well as the ability to manipulate it to fit your trading style or trading system requirements.
One key element that a software package or data provider must give you is the ability to create continuous contracts (see "Customize"). Futures markets are specific in the way they trade in a series of short-term contracts that are active only as the near-term contract for a short period of time. This is different than stocks or forex markets that offer a continuous stream of prices. To backtest a system on historical futures data, you need to patch together the futures prices to generate a similar continuous stream of prices.
Contango, backwardation and other factors (crop seasons, etc.) generate a difference in the price of different expiration date contracts. In effect, when it comes time to move from one contract to the next, there likely will be a gap between the old contract price and the new contract price. These gaps can be substantial and make your data appear disjointed or incomplete.
Although some traders believe that valuable information may be lost if data is smoothed or manipulated for back-testing purposes, algorithmic or non-discretionary system developers feel the need to splice together continuous contracts.
There may not be a single or correct way to compute a continuous contract for futures markets, but there are three popular ways to do it. One method is the spot month continuous contract, the second is to roll on volume or open interest and the third is to back-adjust continuous contracts.
Each methodology has its positives and negatives. For example, the spot month rollover may not be useful for certain agricultural contracts because one month may be the old crop and the next the new crop. This is likely to produce a price gap (see "Old crop/new crop").
Not all contracts have the greatest volume or open interest in the spot month, so rolling on volume or open interest may produce problems with the back-tested results. With this methodology the trader is making the assumption that liquidity is the only appropriate criterion for selecting delivery months.
The "back-adjusted" continuous contract seems to be the most preferred method because it actually measures and removes gaps caused by the rollover from one contract to the next.
Two other ways to build continuous contracts may not be familiar to many traders. These are the Panama Canal method described by Ed Seykota and the proportional back-adjusted methods. The Panama Canal method suggests raising or lowering each price series until the gap is filled. The problem with this method is it creates a trade bias depending on what parts of the data stream you move up or down. The proportional back-adjusted method appears to be the right approach to this problem because it ensures a constant relative relationship between prices across the trading history.
As mentioned earlier, each method has its drawbacks and benefits. However,to find the process that matches a trader’s preference, a good data vendor must provide the trader with the ability to manipulate the data in the way that he sees fit.
Data needs have changed over the course of several years. The virtual elimination of pit trading and the tremendous growth of electronic trading have increased the importance of having reliable intraday and end-of-the-day price and volume data. Traders have to be aware that data vendors vary in the products they offer. Good, clean, reliable data is a must for analysis software and trading algorithms to perform as designed. In addition, data providers must furnish traders with the ability to manipulate data to suit their preferences.
James A. Hyerczyk is a Gann technician and trading educator who has been analyzing markets since 1982. He authored "Pattern, Price & Time," and writes a futures, forex, ETF and equities advisory newsletter for traders and institutions at www.patternpricetime.com. He can be reached at jhyerczyk@yahoo.com.
