When Bill Eckhardt left the University of Chicago and foresook his nearly completed PhD in mathematical logic in 1973, he did not abandon his educational pursuits; rather, he focused them on a myriad of disciplines that supported his research in creating trading systems. Eckhardt joined high school friend Richard Dennis as a trader on the Mid-America Exchange. The two would later become partners in C&D Commodities, where they created technical trading systems and launched the famous turtle trading experiment. Details of the turtle experiment have become legendary, but the proof of its value can be seen in the many highly successful trading businesses it launched, as is demonstrated by our class of Top Traders of 2010. Eckhardt launched his own commodity trading advisor (CTA) in 1991, which has produced a compound annual return of 17.35% over 20 years and earned 21.09% in 2010. In addition to building trading systems, Eckhardt has developed a science of trading and written academic papers on the philosophy of science. Here, we discuss his scientific approach to trading.
Futures Magazine: Many in the traditional investment world cling to the notion that markets are efficient. Trend-following is not valid under the efficient market hypothesis (EMH) but here you are 30 years later. Why has the EMH persisted? Talk about this anomaly.
Bill Eckhardt: The random walk model of price change has been so durable because it’s nearly correct. The difference between futures prices and certain random walks is too small to detect using traditional time series analysis. Incredibly, this difference is detectable using trading systems.
FM: How do trading systems do this?
BE: Trading systems can be highly sensitive to non-linear relations in price series. So why doesn’t this revolutionize the modeling of price series? Statistical estimators probe particular features of the price series; they are equipped with confidence levels, give information about possible models, and are useful for prediction. From the point of view of the modeler, trading systems do not locate specific features of the price series; they have no confidence levels and are useless for prediction. Worst of all, they say little about any possible model. Trading systems can be highly remunerative, but they don’t tell the modeler what he or she needs to know. In the same way models, although valuable in other respects, do not help in designing trading systems.
FM: You have spent a lifetime in trading and in research. Name a couple of simple truths that you have discovered.
BE: Improve your trading or it will degrade; there’s no coasting in this game. You can be creative in research but don’t trade creatively; in other words, stick to your systems. Trading is a little like morality in that it’s a lot easier to know what you should do than to do it. Finally, use only robust estimators and very large samples, not dozens, but thousands.
FM: It seems there is a constant battle with systematic traders between sticking with your models and constantly improving and evolving. How difficult is it to balance these elements that at times seem conflicting?
BE: If your trading system is inadequate, you shouldn’t use it. If your system is good, then stick to it faithfully. In the meantime search vigorously for improvement. When the new system is ready you can change to it – you are not thereby failing to stick to your system. So there need be no conflict between persistence and change.
BE: By trying to improve your system you can make it worse. You can over-fit to past data or maybe just do something that is statistically invalid. There is an idea, though it is not universally subscribed to, that you should not optimize your systems. That you should just figure out what are reasonable numbers and go with that. I don’t believe in that; we optimize all the time, but there is some truth to it in the sense that if you over-fit, you are going to hurt yourself. Optimizing is a somewhat hazardous procedure, as is trading. And it has to be done with carefulness and deliberateness, and you have to make sure that you are not over-fitting to past data.
FM: How do you ward off curve-fitting?
BE: What most people use to ward it off is the in-sample/out-of-sample technique where they keep half their data for optimization and half their data for testing. That is an industry standard. We don’t do that; it wastes half of the data. We have our own proprietary techniques for over-fitting that we actually just improved on a year ago. It is important to test for over-fitting; if you don’t have your own test use the in-sample/out-of-sample [technique].
I can talk a little more about over-fitting, if not my personal proprietary techniques. First of all I like the [term] over-fitting rather than curve-fitting because curve-fitting is a term from non-linear regression analysis. It is where you have a lot of data and you are fitting the data points to some curve. Well, you are not doing that with futures. Technically there is no curve-fitting here; the term does not apply. But what you can do is you can over-fit. The reason I like the term over-fit rather than curve-fit is that over-fit shows that you also can under-fit. The people who do not optimize are under-fitting.
Now the two numbers that most determine if you are over-fitting are the number of degrees of freedom in the system. Every time you need a number to define the system, like a certain number of days back, a certain distance in price, a certain threshold, anything like that is a degree of freedom. The more degrees of freedom that you have the more likely that you are to over-fit. Now the other side of it is the number of trades you have. The more trades you have, the less you tend to over-fit, so you can afford slightly more degrees of freedom. We don’t allow more than 12 degrees of freedom in any system. If you put more bells and whistles on your system it is easy to get 40 degrees of freedom but we hold it to 12. On the other side of that, for us to make a trade we have to have a sample of at least 1,800; we won’t make a trade unless we have 1,800 examples. That is our absolute minimum. Typically we would have 15,000 trades of a certain kind before we would make an inference as to whether we want to do it.
The reason you need so many is the heavy tail phenomena. It is not only that heavy tails cause extreme events, which can mess up your life, the real problem with the heavy tails is that they can weaken your ability to make proper inferences. Normal distribution people say that large samples kick in around 35. In other words, if you have a normal distribution and you are trying to estimate a mean, if you have more than 35 you’ve got a good estimate. [In] contrast, with the kind of distributions we have with futures trading you can have hundreds of samples and they could still be inadequate; that is why we go for 1,800 as a minimum. That is strictly a function of the fatness of tails of the distribution. You have to use robust statistical techniques and these robust statistical techniques are blunt instruments. [They] are data hogs, so both seem to be disadvantages but they have the advantages of tending to be correct.
FM: What is it about your systems that have persisted so long and what adjustments, if any, have you made over the years?
BE: Aristotle’s Ship of Theseus gradually had every one of its parts replaced. Was it still the Ship of Theseus? Over the decades every part of our systems has been modified. It’s our scientific approach that persists and supports the whole structure.
FM: There seems to be a growing understanding of tail risk and the value of strategies that can take advantage of it instead of simply hedging it. As a result, managed futures are becoming more popular. Do you agree and, more importantly, do you see this as a permanent shift or a temporary reaction to the recent financial crisis?
BE: I’d like to distinguish diversifying from hedging. Futures trading diversifies risk and can improve returns, but it falls short of a hedge. Hedging requires anti-correlation to the risk that is hedged. Futures returns mostly are uncorrelated to those of other assets. This makes them an excellent vehicle for diversification. The large-tail phenomenon means that most statistical tests overestimate reliability and underestimate risk. I don’t know if it’s possible to take advantage of this, but it’s important to protect yourself from it.
FM: You make an important distinction between diversifying and hedging, but many equity-based managers are looking at tail risk insurance strategies. Wouldn’t an allocation to managed futures provide better insurance as it has been shown to be negatively correlated in bear equity markets?
BE: If you select unique periods such as when stocks are extremely weak, it creates a powerful selection bias. The anti-correlation found in these studies may have been a selection artifact. Futures trading has its own heavy tail to add to the mix. The key is independence, which makes efficient diversification possible.
FM: It has been shown that normal distribution curves under-estimate tail risk in equities. Does your approach give you a better picture of the tail risk in your trend-following approach?
BE: Tail risk is hard to estimate but we spent over 25 years on this project. We have worked on it really hard and we do have various techniques to deal with the fact that the tails are so heavy. It is absolutely crucial because the tail risk changes everything that we do. Every single part of designing and implementing the system is affected by the fact that you have more extreme values than you expect under any kind of normal model.
FM: Are you better prepared because trend-following as a model attempts to take advantage of this tail phenomenon in the general market?
BE: Yes, but it has its own tail risk. I have a little bit of trouble with the idea that the tail risk in futures trading is what is helping because I see it strictly as a hindrance, strictly as a problem to be overcome. You may have a point because I guess it helps to have these really big outsized moves. It is only going to help you if you treat it like a wild tiger. …Trend-following doesn’t work only because of the tail risk but tail risk turns up the volume.
FM: We spoke to Richard Dennis a couple of years ago and he said that trend-following is getting more difficult and systems don’t hold their edge as long. Do you agree?
BE: In general I agree that it is getting harder and you have to improve, though I don’t know that you have to improve faster. Hastiness can be costly in this game. When you are doing your research and you are improving, it should be with the confidence that the system you are currently trading is good. The value of the current system you are trading gives you the time to make a deliberate and cautious improvement.
FM: Some people claim managed futures’ ability to be negatively correlated to equities in poor equity markets has to do with the long-term bull market in bonds and that may not be the case in a rising interest rate environment. Is this something investors should be weary of?
BE: It sounds to me that what they are doing is they’re looking at the fact that trend-following has been profitable, which probably irritates them anyway — [Trend-following] really only has a record of 30 or 40 years— then they are looking at what the biggest economic factor [was] in that time? Well we have had these declining interest rates and they say one has to cause the other. I just don’t see it. In analyzing my own performance I can’t find a pattern where I am making an unusual amount of money in the interest rates and not the other stuff. So I would be very skeptical of that idea that all the money in trend-following comes from the behavior in one group of markets.
I want to mention that this whole question of risk control it is not a completely objective question. I wanted to invent a science of trading that I at least partially completed; I certainly haven't done everything that I set out to do. When I was a young man I wanted to devise objective risk systems. In other words, once you have a system, what is the right size to trade, period. After years of working on this I convinced myself that it did not have a unique answer. You need at least one subjective piece of the puzzle to put it together, and that is an individual’s risk aversion. Now that is subjective. There is no rule that says how averse you should be to risk, that is an integral element of your personality. But unless you know how averse to risk you are or unless you can impute risk aversion to your clients, you really can’t settle the question of how big you should trade.
FM: Is there a risk that managers end up measuring themselves instead of the markets?
BE: Every strategy has a finite capacity. It is possible for a trading strategy to get oversubscribed and no longer work. That hasn’t happened with trend-following but the game has gotten harder.
FM: Every few years after a rough period someone says trend-following is dead.
BE: I lived through the death of trend-following a half dozen times and, like Mark Twain’s death, it was highly exaggerated.
FM: You have a fascinating background. Tell us how you went from a PHD candidate at the University of Chicago to trading futures.
BE: My interest in futures trading dates back to high school, as does my first collaboration with Rich Dennis. At the University of Chicago I specialized in mathematical logic and had a great opportunity to pursue my interests in the philosophy of science and the history of ideas. The latter were influential in shaping my approach to trading. In 1973 my advisor and I were having disagreements about the direction my dissertation was taking, and Rich suggested I take a little time off and come down to the floor to trade. I didn’t go back [to school].
FM: Many of the turtles have gone on to great success as well as some of the C&D traders, even though their strategies have evolved into something quite different than the original. Was there something much more basic in the lessons that allowed so many to succeed and create new strategies?
BE: The turtles were stringently selected and highly talented. They also received training, practice, and guidance. They got a good start from us, but they deserve most of the credit for their ongoing success. The successful turtles have branched out widely, but my trading has also changed; it’s evolve or perish....
As I recall more than half the course revolved around developing the right attitude, guarding against debilitating emotions, how to think about risk, and how to handle success and failure. Teaching the turtle system itself doesn’t take very long. I was saying you need less than 12 degrees of freedom in a system; versions of the turtle system had three or four. We spent a lot of time talking about our theories on how to control risk; that was actually the bulk of the course. Attitude, emotional control, discipline; those things are harder to teach. All the turtles learned the system and learned the strategy; that was the easy part, but some of them brought the right attitude and right mental set to it and they prospered and became very rich. Others had a more halting career and did not succeed as well. They had the same training, but maybe they did not have the same emotional make-up.
FM: Talk a little bit about position sizing and risk management and the role they play in long-term success.
BE: When and where you initiate a trade is a lot less important than how large you trade and how you liquidate. Unfortunately, traders tend to put a great effort into trade initiation and let risk management stagnate. Small improvements in risk or volatility assessment may not be exciting, but they are among the most lasting and beneficial changes. One approach to avoid is to design the system first, then to tack on risk management. I compare this to Wells’ “The Island of Doctor Moreau,” because it’s like grafting the head of one creature onto the body of another. System and risk management should be developed together; the connection should be seamless.
FM: You and Richard Dennis made a bet on whether you could teach trading; can you teach pension managers and financial academics that markets aren’t always efficient?
BE: Various eyewitnesses to this episode disagree, a good example of the unreliability of this kind of testimony. I don’t believe I ever advocated that trading cannot be taught, but if my memory is faulty on this point, I freely admit that, like virtually any human activity, trading can be taught and practice can improve performance. All parties agree, however, there was no bet. Possibly an argument among hundreds of others, but no bet.
FM: That question was supposed to be funny. Basically I am curious about your opinion of why a strategy — trend-following — that has been utilized successfully by many managers over many years is still questioned, even dismissed, by many academics.
BE: For the last 20 years many academics have been critical of the random walk model of price change, although suggested changes are usually minor. A few commentators have even pointed out that the existence of profitable trading systems conflicts with the random walk model. However these same commentators take this as a sign that there is something wrong with their models; they do not analyze and improve trading systems which to them are only inconvenient details about price series.
FM: Some academics still do not believe trend-following works.
BE: They are flying in the face of a lot of data. Forget me; there are dozens of managers with a long track record. A few people — a very few out of a large sample — can profit even in a random market, but that is very few; 1 in 500, 1 in 1,000. Trend-following has had a much better track record than that. But whenever they are confronted with this, the track records of trend followers, they say these people are making money some other way. That is not impossible. When I was in the pit, there were good traders who used these numbers based on three-day moving averages, they were totally pointless numbers and I knew it back then, but they were given out as support and resistance levels and these traders were among the best traders I knew who used these numbers and they would swear by them. I watched their trading and after a while I noticed that the numbers didn’t make any difference, maybe they were trading on the numbers but they were always buying on the bid and selling on the offer. So I guess it is possible that somebody could make money and not know how they are doing it, but that was on the floor where you are getting edges. Trend followers, like myself, are trading off the floor where we are giving up edges so our expected performances should be negative in a random walk. In the face of all that empirical evidence they should be reassessing their position but I have discovered that a lot of academics never change their position and the only way [it will change] is when the older academics die.
Part of the problem is that estimators don’t show every divergence from randomness. Let’s say you have a system and it goes long corn. Let’s say it is a model that says corn goes up 2¢. If corn goes up 8¢, by the normal way of measuring, it was off by 6¢ so it was a bad prediction. It was a good trade but a bad prediction. You simply can’t treat trades and prediction the same way and the academics all are focused on prediction. Now superficially it seems like trading is a form of prediction but it really isn’t. If you design your system where you are trying to predict the market, then it doesn’t work. You have to concentrate on projecting losses, risk management and finding something that works, but if you are directly looking for prediction that tends to be self-stultifying.
FM: But isn’t that the point of trend-following — that you are not predicting, just defining a trend and taking advantage of it?
BE: That’s right. And if you look at me as a predictor instead of as a trader—as a trader I am way ahead, as a predictor I am scoring about 35%, so I am not very good as a predictor. Those are different skills. But still even with trend followers you will hear people say, “Where do you think the market is going?” It is just human nature to try and approach this in terms of making a prediction.
FM: After watching managers for more than a decade, it seems to me that there is more diversity within what is loosely defined as the medium- to long-term trend-following space than most people give it credit for.
BE: That is right. People look too much at the correlation between traders. The fact of the matter is [that] correlations tend to overstate the relationship. The reason being, whatever kind of a trader you are — let’s say you are a medium- to long-term trader — if you are a fundamentalist, if you are countertrend—you know you try and buy the bottom and sell the top — or if you are a trend follower, at that time frame you are going to make money and everyone makes money in the trends. They do it in different ways, they get in at different places, they have different approaches but they make money in the trends. So when you look at the performance they correlate more than they should. So now if you narrow your scope to just trend followers, they are really going to correlate more than they should. So that gives the impression that they are really all just doing the same thing, which they're not. So there is more diversification among trend followers than one would expect. There really are different varieties of trend-following and they really do have different properties.
FM: There are signs this industry that you have been involved in for more than 30 years is moving into the mainstream. What do you see as the future?
BE: Well, it is well overdue. I don’t have to tell you that they were trading futures in the 19th century and this is the 21st. People talk about futures as derivatives and that might be technically true but the fact is these have been around for a long time. For all that time futures have had the reputation of being really risky. It took people to lose several fortunes over and over again in the stock market to finally figure out that it is the stock market that is really risky. Futures are only as risky as you want [them] to be because you can leverage [them] down so easily. I don’t understand why futures had the bad reputation for all of those years, and if it is coming into the mainstream it is long overdue.
FM: We know you are more of a technician than a fundamentalist but give us your outlook on the investing landscape. Is it a good time to be in managed futures? Is it a good time to be a trend follower?
BE: I can’t predict the direction of the economy, but it’s a safe bet that it’s going to be a roller coaster ride. In the past such periods have been good for the trend follower. This is, however, a flimsy argument – it’s impressionistic and rests on a small sample. The real reason to participate is that futures trading has been beneficial in general, and now is likely to be as good a time as any.