Divergences: S&P 500 and the Dow

November 29, 2015 09:00 AM

It is a staple of drive-time radio news reporting, “In business today, the Dow did X, the S&P 500 did Y and the Nasdaq, bless its little heart, did Z.” There you have it, the largest economy in the world reduced to an index of 30 stocks, a broad-based index and an index based on the firms’ choice of their listing exchange.

While you might excuse the Nasdaq from your expectations—these indexes should move in the same direction on the grounds of sector dependence—should you afford the Dow Jones Industrial Average and S&P 500 the same leeway? After all, the Dow was or is reputed to be the home of large blue-chip American firms, and the SPX is nothing other than a repository of large-capitalization firms with a few inclusion restrictions tossed into the mix. Of course, this was before the Dow found itself absent industrial stalwarts of yesteryear such as General Motors (GM), U.S. Steel and Alcoa and found itself with credit-card processor Visa (V), fast-food king McDonald’s (MCD) and sneaker manufacturer Nike (NKE). Whatever happened to the industry component? 

The Dow and SPX are constructed differently. The Dow is a price-weighted index constructed by summing the prices of its 30 constituents and adjusting by a divisor, 0.14968 at the end of July 2015, to account for the effects of stock splits and stock dividends:

The SPX is a capitalization-weighted index calculated by multiplying the number of outstanding shares for each of the 500 stocks by its price, multiplying by 10 and then dividing by the 1941-43 baseline value:

The largest difference between the two indexes is how they are selected for inclusion. The Dow’s components are selected by a committee at S&P Dow Jones Indexes while the SPX’s components roughly cleave to the largest 500 stocks with restrictions on trading volume.

The Dow has tended to have a higher dividend yield over time than the SPX. At the end of July 2015, the DJIA yielded 2.41% vs. 2.04% for the SPX. This small but persistent difference gives the Dow a tortoise-and-hare advantage over the SPX over long periods.

Different pathways

One of the more odd and more enduring aspects of the relationship between the Dow and the SPX is that the two indexes’ total returns have had a very high r2 (or percentage of variance explained) of 0.983 since January 1988. Like the statistician who drowned in a river averaging three feet in depth, this close relationship can and does obscure some important differences.

Let’s re-index the total return series and display them on a common logarithmic scale along with a rolling three-month correlation of returns. There are several periods, such as early 1994 and late 2000, of marked decline in correlation of returns. Both of these periods correspond to periods when the Dow started to outperform the SPX (see “Not constant,” below).

We can rearrange the data to display the Dow/SPX relationship as a one-year rolling beta of the Dow to the SPX and map it against the cumulative excess return of the Dow. The Dow’s beta to the SPX is a measure of its relative volatility; standard portfolio theory holds investors should be compensated for assuming more risk. This is a nice theory, but the history of the past quarter-century has been the Dow has outperformed while having a beta less than 1.00 continuously since mid-2002.

The Dow has had a higher average daily return, 0.040% vs. 0.038%, and a lower standard deviation of those returns, 0.011% vs. 0.012%, since January 1988 (see “Dow out-performance,” below). This is in spite of decisions such as adding Microsoft months before it lost its antitrust case, and adding Intel just as the dot-com bubble approached its peak. This idea of having your cake and eating it too is an artifact of the DJIA’s non-random selection process. The index is not selected via a rules-driven process but rather by a committee seeking to replicate the broad market with a small basket of stocks.

Weighty issues

A snapshot of the Dow’s non-representative quality can be seen in the weights of its 30 stocks in July 2015 mapped against the weights of those same 30 stocks in the SPX (see “Weights and measures,” below). The largest stock by weight in the Dow, at 7.91%, was Goldman Sachs, a stock accounting for a mere 0.45% of the SPX. It does not take a large stretch of the imagination to see a very good or very bad day for one of the Dow stocks can produce an inordinate change in the Dow while having a far more moderate effect on the SPX.

As a case in point, Dow members IBM and United Technologies both had bad days on July 21, 2015 (declining 5.88% and 7.06% respectively). This helped push the Dow down by 1% while the broader-based SPX declined by only 0.43% that day.

While we might think the narrower base of the Dow and the propensity of individual stocks to have singularly bad days might skew the Dow’s distribution of returns negatively in comparison the SPX, this has not been the case over the long term. Moreover, the SPX’ distribution of returns has a more positive kurtosis — or a more peaked distribution of returns. The broader-based SPX has fatter tails on its distribution of returns and a greater propensity to large declines (see “Skewed performance,” below).

Different signs

How unusual is it for the DJIA and SPX to have different colored arrows at the end of the trading day? While it always feels a bit odd when it happens, the answer is the two indexes diverge frequently. The early 1994 and late 2000 periods noted above, when correlation of returns declined and the Dow’s relative performance rose, were characterized by a high frequency of divergent direction, as was the 2013-14 period after Goldman Sachs, Visa and Nike were added to the index.

The long-term average for the number of opposite signs within a rolling three-month period is 6.82, or about 10.8% (see “Opposing forces,” below). Restated, this is not at all an unusual phenomenon.

Beware of spreads

A number of researchers have discovered all primates have a hard-wired fear of snakes; this apparently was of great use to our arboreal ancestors. Had the same researchers dug a little deeper, they might have discovered an equally ancient mental code convincing us spreads are less risky than outright positions. While it is true the long-term risk of a spread position--measured as the sum of the Dow and SPX variances minus twice their covariance--is lower than either index’s alone, we are back to that same statistician drowning in the river.

Traders often trade much larger spread positions than outright positions and are charged with lower futures margins when doing so. This exposes spread traders to those many days with opposite signs when they could end up being either right or wrong on both legs of the trade.

The different construction and weighting methodologies for the two indexes make the occasional surprise gain or loss in this trade a fact of life. No matter how finely you tune your hedge ratios or how frequently you recalculate them, you are exposed to the risk of disconnected direction. You also are exposed to the odd conundrum that the more narrowly based and price-weighted Dow has been able to deliver higher returns at a lower volatility than the SPX for a very long time for reasons almost wholly dependent on the ex-post skill of the index selection committee. Until this changes, you will keep on hearing those same short radio reports for the different indexes for a very good reason: They are different.

About the Author

Howard L. Simons is president of Rosewood Trading. @simonsresearch