Average correlation and stock market returns Joshua M. Pollet∗ [email protected]
Mungo Wilson† [email protected]
This version: October 2007 Abstract If the Roll critique is important, changes in the variance of the stock market may be only weakly related to changes in aggregate risk and subsequent stock market excess returns. However, since individual stock returns share a common sensitivity to true market return shocks, higher aggregate risk can be revealed by higher correlation between stocks. In addition, a change in stock market variance that leaves aggregate risk unchanged can have a zero or even negative eﬀect on the stock market risk premium. We show that the average correlation between daily stock returns predicts subsequent quarterly stock market excess returns. We also show that changes in stock market risk holding average correlation constant can be interpreted as changes in the average variance of individual stocks. Such changes have a negative relation with future stock market excess returns. JEL Classification: G12 Keywords: correlation and Roll critique
University of Illinois at Urbana-Champaign Hong Kong University of Science & Technology We thank Klaas Baks, Michael Brennan, John Campbell, Timothy Johnson, Jun Pan, Neil Pearson, Allen Poteshman, Bruno Solnik and Joshua White as well as participants in the HKUST finance symposium and seminar participants at MIT Sloan and UIUC for their comments. We are grateful to Qin Lei for providing total payout ratio data for the United States. †
The trade-oﬀ between risk and expected return is at the center of any equilibrium theory of finance. From an aggregate perspective, as systematic risk increases, risk-averse investors require a higher risk premium to hold the market, and the equilibrium expected return must rise. The following equation for excess log market returns is consistent with this intuition: rm,t+1 − rf,t+1 = β 0 + β 1 V art [rm,t+1 ] + λxt + εt+1
where β 1 is positive, V art [rm,t+1 ] is the conditional variance of market returns, xt represents other potential sources of variation in expected returns, and Et [εt+1 ] = 0. Campbell (1993) derives this relationship for a representative agent with Epstein-Zin preferences under fairly general conditions. It is often assumed that the market portfolio satisfies a variance-in-mean relationship for excess log market returns where λ is zero in equation (1). A large literature starting with Merton (1980) and including among many others Campbell (1987), French, Schwert, and Stambaugh (1987), and Bollerslev, Engle, and Wooldridge (1988) has attempted to identify a variance-in-mean relationship in the data with mixed results. (See Campbell, Lo and MacKinlay (1997) Chapter 12 for a survey and for more recent references see Harvey (2001) and Lettau and Ludvigson (2002).) The evidence that innovations to market risk are negatively correlated with contemporaneous returns is much stronger than the evidence that conditional variance predicts future returns. The direct evidence of an aggregate risk market return trade-oﬀ is usually weak. Since Roll (1977), a seminal critique of empirical studies regarding the validity of the CAPM, researchers have recognized that market portfolio returns are not directly observable. Although the Roll critique is usually stated in the context of cross-sectional tests of the CAPM, it is also relevant for time-series tests of a variance-in-mean relation for the stock market because the variance of returns for the true market portfolio may be only weakly related to stock market variance. We find that changes in true market risk may nevertheless reveal themselves through changes in the correlation between observable stock returns. Since the return to the true
market portfolio is a common component for most asset returns (on average positively related to individual asset returns), an increase in aggregate risk, other things equal, is associated with an increased tendency of stock prices to move together. As a result, such increases in correlation reveal increases in true market risk. If the stock market risk premium depends positively on aggregate market risk, then the average correlation between stock returns should forecast future stock market excess returns. In addition, measures of stock market variance may themselves be poor forecasts of future stock market returns even in the presence of a strong variance-in-mean relationship for the true market portfolio. Indeed, if most changes in stock market variance are due to changes in the variance of components orthogonal to the variance of the true market portfolio (i.e. holding true market risk constant), then such changes in stock market variance must be accompanied by oﬀsetting changes in the covariance of the stock market with the rest of the true market portfolio. We show that, empirically, changes in the sample variance of daily US stock market excess returns are almost completely captured by changes in the product of the average variance for the largest 500 individual stocks and the average correlation between all pairs of stocks for the largest 500 stocks. Of these two, the average correlation strongly predicts future excess stock market returns while the average variance has no discernible forecasting power. Thus, the weak ability of stock market variance to forecast stock market expected returns is due to the contamination of average correlation by average variance. Average correlation also forecasts market returns with an out-of-sample R2 of more than 3%, and therefore passes the test proposed by Goyal and Welch (2006). While there is a negative contemporaneous relationship between the innovations in realized returns and average correlation, the associated bias in the OLS estimator for the predictive regressions is not the source of the findings. The large negative correlation between shocks to average correlation and shocks to realized returns that we find may explain the phenomenon of asymmetric correlation (see for example Longin and Solnik (2001), Ang and Chen (2002), and Hong, Tu and Zhou (2003)) in which stock returns are more highly correlated when the market declines. If an increase in average
correlation is due to an increase in aggregate risk, then the discount rate for future expected cashflows on most assets, including the stock market, should increase as well. Holding expected future cashflows constant, higher future expected returns induce an immediate fall in price (volatility feedback, as described by Campbell and Hentschel (1992)). Therefore, our findings suggest that asymmetric correlation is caused by a form of volatility feedback induced by fluctuations in the volatility of the true market portfolio rather than changes in stock market volatility. Our results are consistent with the analysis of asymmetric correlation across countries by Forbes and Rigobon (2002). Their study shows that an increase in the variance of a heteroskedastic common component of stock market returns across diﬀerent countries (“interdependence”) induces an increase in the correlation between stock markets across countries without any change in the sensitivity of the return for each specific stock market to the common shock (“contagion”).1 Goyal and Santa-Clara (2003) find a positive relationship between (equal weighted) average variance and the CRSP value-weighted market return from 1963 until 1999. Their results use the average variance of all stocks in the CRSP universe rather than the largest 500 stocks. We find a negative relationship between average variance and the subsequent market return after controlling for average correlation. In this respect, our findings are similar to those of Bali, Cakici, Yan and Zhang (2005). They show that the forecasting power of average variance disappears in a more recent sample (through 2001), disappears if NASDAQ and AMEX stocks are excluded from market returns and disappears if value-weighted average variance or median variance is used in place of equal-weighted average variance. In general, the equal-weighted average variance for all stocks in the CRSP universe is probably dominated by the statistical behavior of extremely small firms. Since Brown and Kapadia (forthcoming) find that patterns in average individual stock variance are driven by equity issuance patterns, this measure of average variance will also be aﬀected by the fraction of companies that are recent issues. Ghysels, Santa-Clara and Valkanov (2005) claim to have identified a variance-in-mean re1
Failing to control for the heteroskedasticity of the common component results in tests for contagion that are biased in favor of rejecting the hypothesis of zero contagion. However, although the test is biased, the correlation estimates themselves are consistent.
lationship using mixed data sampling to estimate conditional market variance with higher accuracy than conventional methods. Our results are not necessarily inconsistent with theirs but we focus our analysis on the diﬀerent components of aggregate risk. We also demonstrate high out-of-sample forecasting power for average correlation. Yu and Yuan (2006) suggest that the inability to identify a strong variance-in-mean relationship in the time series of market returns is due to a failure to account for sentiment-driven returns. They find a marginally significant coeﬃcient on conditional variance when an empirical proxy for sentiment, and its interaction with conditional variance, are included on the right-hand side of (1).2 Driessen, Maenhout and Vilkov (2006) claim that the high average risk premium earned on stock index option writing can be reconciled with the low or zero average risk premium earned on individual stock option writing if the price of risk for bearing correlation risk is much higher than the price of risk for bearing individual stock volatility risk. Their paper infers “implied correlation” from the prices of options on the S&P 100 index and on its 100 constituents and shows that implied correlation is highly correlated with subsequent realized correlation, but that implied correlation is considerably higher on average. While these results are likely to be generated by the same underlying phenomena, they do not present evidence regarding the predictability of stock market returns using correlation. In addition, this options-based evidence does not require that average volatility risk is unpriced. Since individual option risk is largely diversifiable using a portfolio of individual options, the diﬀerence between implied and realized volatility, as a proportion of implied volatility, could be approximately zero even if the small undiversifiable component of individual stock option risk has a large positive risk premium. By contrast, our results are not consistent with a large positive individual variance premium. The next section summarizes the assumptions under which the market risk premium exhibits a variance-in-mean relationship and presents a stylized model demonstrating how correlation between stock returns is positively related to the stock market risk premium. We also show that the average variance of stock returns can be unrelated or even negatively related 2
In unreported results we find that the inclusion of the same sentiment proxy, together with its interaction with average correlation, does not noticeably change the predictive relationship between average correlation and subsequent market returns.
to the stock market risk premium and negatively related to the covariance of stock market returns with the return of the non-stock component of the true market portfolio in this situation. We present the main empirical results in section 3. Section 4 considers long horizon forecasting power, out-of-sample forecasting power, horse races with other candidate predictors and presents some evidence that average variance is negatively related to the covariance of bond and stock market returns. Section 5 presents evidence from a simulation indicating that our results cannot be explained by measurement or estimation error in the presence of the standard variance-in-mean relationship for stock market returns. Section 6 concludes.
The importance of correlation
When the portfolio of assets whose returns are observable (e.g. stocks) is the true market portfolio, aggregate risk is identical to stock market risk. Hence, the variance of the return to the market portfolio is directly observable and, under typical assumptions (for example, Merton (1973) and Campbell (1993)), the market risk premium is related to this observable variable. However, when the stock market portfolio is only a subset of the true market portfolio, the inability to observe the variance of the market portfolio interferes with the empirical analysis of the aggregate risk-return relationship. In certain circumstances, the correlation between stocks rather than stock market variance may better reveal changes in true aggregate risk, and hence, the stock market risk premium. We present one model in which correlation between stocks is more strongly related to the stock market risk premium than stock market variance. We start by summarizing several special cases where the discrete-time version of the ICAPM simplifies to the conditional CAPM, and consequently, the risk premium is linearly related to market variance. We also consider a more plausible setting where the ICAPM still exhibits a variance-in-mean relationship even though the conditional CAPM does not hold.
ICAPM special cases
Campbell (1993) assumes that there exists a representative agent with Epstein-Zin preferences, that asset returns and consumption are conditionally (jointly) lognormally distributed, and
that all wealth is tradeable. If the representative agent has an elasticity of intertemporal substitution of one, Campbell provides an expression for the expected excess log return for portfolio p. ⎡ ⎤ ∞ X σ 2p,t = γCovt [rp,t+1 , rm,t+1 ] + (γ − 1)Covt ⎣rp,t+1 , ∆Et+1 η j rm,t+1+j ⎦ Et [rp,t+1 ] − rf,t+1 + 2 j=1
In this equation, rp,t+1 is the log return for a portfolio p, rm,t+1 is the log return on the market portfolio, rf,t+1 is the log return on the riskless asset (known at time t), η is a constant of loglinearization, γ is the representative investor’s coeﬃcient of relative risk aversion, and the time t subscripts indicate that the conditional variances and covariances are possibly time-varying.3 Holding all else constant, the first term of the left hand side of this equation indicates that the expected excess log return for portfolio p (adjusted for Jensen’s inequality) is positively related to the conditional covariance of the return for portfolio p with the market return. The second term on the left hand side indicates that the risk premium is also positively related to the covariance of the return for portfolio p with news about future market returns whenever γ > 1. There are several special cases where this equation simplifies to the conditional CAPM.4 The first and perhaps most obvious special case is log utility where γ = 1 and the second term vanishes altogether. Log utility removes all intertemporal hedging considerations and is quite restrictive. In the second special case the investment opportunity set is constant, and therefore, the second term of expression (2) vanishes because there is no news about future expected returns. In such a setting, any attempt to identify a trade-oﬀ between aggregate risk and expected returns would fail because both the expected market return and market variance are constant. Next, we could assume that the market return follows an AR(1) process and the second term would be proportional to the covariance of the return for portfolio p with the 3
This relationship is based on equation (25) from Campbell (1993) and is appropriately restated for time varying variances and covariances. If the elasticity of intertemporal substitution is not equal to one, the risk premium for the market is also linearly related to the conditional covariance of the market return with the future variances of consumption growth minus scaled market returns. 4 Campbell (1993) explicitly discusses many of these special cases.
market return. If market returns are observable in this setting, then lagged market returns provide the best forecast of subsequent expected returns. Finally, we consider a more plausible special case where the ICAPM exhibits a variancein-mean relation for the market risk premium even though the conditional CAPM does not hold. Assume that the covariance of rp,t+1 with the market return as well as the covariance between rp,t+1 and news about subsequent market returns are both linearly related to the same underlying state variable yt . Covt [rp,t+1 , rm,t+1 ] = δ p,0 + δ p,1 yt ⎤ ∞ X Covt ⎣rp,t+1 , ∆Et+1 η j rm,t+1+j ⎦ = δ h,0 + δ h,1 yt ⎡
Rearranging these two expressions, ⎡
Covt ⎣rp,t+1 , ∆Et+1
∞ X j=1
η j rm,t+1+j ⎦ = δ h,0 +
δ h,1 (Covt [rp,t+1 , rm,t+1 ] − δ p,0 ) δ p,1
Substituting into equation (2), µ ¶ µ ¶ σ 2p,t δ h,1 δ h,1 δ p,0 = (γ − 1) δ h,0 − + γ + (γ − 1) σ pm,t . Et [rp,t+1 ] − rf,t+1 + 2 δ p,1 δ p,1 The expected excess log return for portfolio p is linearly related to the covariance of the return for portfolio p with the market return.5 In all of these special cases, the risk premium is linearly related to market variance σ 2m,t = ζ 0 + ζ 1 σ 2m,t Et [rm,t+1 ] − rf,t+1 + 2
where ζ 1 > 0. As noted by Campbell, the coeﬃcient for market variance is not a direct estimate of risk aversion nor is the intercept necessarily equal to zero. In equation (3) the diﬀerent components of true market variance (for example, average 5
The stochastic volatility model of Heston (1993) has similar properties.
correlation and average variance) enter symmetrically. For intertemporal considerations to explain why average variance fails to predict returns in a more general setting, it would have to be the case that an increase in average variance simultaneously raises market variance and lowers the covariance between the market return and news about subsequent returns. Meanwhile, the predictive power of average correlation indicates that an increase in average correlation raises market variance but is not accompanied by an oﬀsetting decrease in the covariance between the market return and news. In the next subsection we ignore intertemporal considerations and focus on the alternative possibility that the variance-in-mean relationship is valid, but because the variance of the market portfolio is actually unobservable, average correlation is a better measure of aggregate risk than stock market variance.
ICAPM with an unobservable market portfolio
In the special cases described above the conditional risk premium for the stock market (asset s) is given by the following equation where the true (but unobservable) market portfolio is asset m.
Et [rs,t+1 ] − rf,t+1 +
σ 2s,t = φ0 + φ1 Covt (rs,t+1 , rm,t+1 ) 2
If the conditional CAPM holds, then φ0 equals zero and φ1 equals the coeﬃcient of relative risk aversion γ. We replace the conditional covariance of rm and rs with the analogous covariance in levels6 and then decompose this covariance into the weighted sum of the variance of the stock market and the covariance of the stock market with the unobservable component of the true market portfolio (asset u). Et [rs,t+1 ] − rf t+1 +
σ 2s,t 2
∼ = φ0 + φ1 Covt (Rs,t+1 , Rm,t+1 ) = φ0 + φ1 Covt (Rs,t+1 , ws Rs,t+1 + (1 − ws ) Ru,t+1 ) = φ0 + φ1 (ws V art (Rs,t+1 ) + (1 − ws ) Covt (Rs,t+1 , Ru,t+1 ))
In a continuous-time setting these two quantities are identical.
To analyze the relationship between aggregate risk and correlation between stocks we consider a simple setting. Assume there are N symmetric stocks (where N is large) in the stock P Covt (Ri,t+1 ,Rm,t+1 ) for all i, and σ 2s,t ∼ market such that Rs,t+1 = N1 i Ri,t+1 , β t = = ρt σ 2t where V art (Rm,t+1 ) ρt (average correlation) is the pairwise correlation between any two stocks and σ 2t is the vari-
ance of any stock (average variance). We make additional assumptions regarding the structure of risk by assuming that the aggregate shock has variance σ 2m,t , while shocks that aﬀect stock returns contain a common stock market component with variance θt σ 2z,t and an orthogonal idiosyncratic component with variance (1 − θt ) σ 2z,t . The aggregate shock is independent from both components of the stock-specific shocks, and hence σ mz,t = 0. Hence, the return for the stock market can be written in terms of the true market return Rm,t+1 and the common stock market component of the stock-specific shocks ¯εz,t+1 . Rs,t+1 = β s,0 + β t Rm,t+1 + ¯εz,t+1 The variance for each stock and pairwise covariance between any two stocks is given below. σ 2t = V art (Ri,t+1 ) = β 2t σ 2m,t + σ 2z,t σ 2s,t = ρt σ 2t = Covt (Ri,t+1 , Rj,t+1 ) = β 2t σ 2m,t + θt σ 2z,t Rearranging these two equations, σ 2z,t
σ 2m,t =
¶ 1 − ρt σ 2t 1 − θt µ ¶ σ 2t ρt − θt β 2t 1 − θt
Since the return for the true market portfolio is the weighted combination of the return to the stock market (asset s) and the unobserved portfolio (asset u), the return to the unobserved portfolio can also be specified in terms of the true market return Rm,t+1 and the common stock market component of stock-specific shocks ¯εz,t+1 . Ru,t+1
ws =− β + 1 − ws s,0
1 − ws β t 1 − ws
ws ¯εz,t+1 1 − ws
Thus, the covariance between the return of the stock market and the return of the unobserved component of the true market portfolio can be expressed in terms of average correlation and average variance. Covt (Rs,t+1 , Ru,t+1 ) =
1 − ws β t 1 − ws
σ 2t βt
ρt − θt 1 − θt
ws θt − 1 − ws
1 − ρt 1 − θt
Substituting this expression into equation (4), the risk premium for the stock market is then given by Et [rs,t+1 ] − rf,t+1 +
σ 2s,t φ1 φ1 = φ0 + ρ σ2 − θt σ 2t 2 β t (1 − θt ) t t β t (1 − θt )
Note that ws disappears completely from this expression. This expression states that the risk premium for the stock market is linearly related to stock market variance, minus a correction for that part of stock market variance which is unrelated to aggregate risk θt σ 2t . Holding all else constant, a change in average variance aﬀects the first term and the second term in opposite directions with similar magnitudes. Holding all else constant an increase in average correlation only aﬀects the first term. Consequently, when changes in stock market variance are mainly due to changes in average variance, correlation is more strongly related to subsequent market returns than stock market variance. To examine the empirical implications of this equation more concretely, we assume that the unobservable moments are parameters (β t = β and θt = θ) and consider a linear approximation of expression (5) around the unconditional expectation of average correlation E [ρt ] and average £ ¤ variance E σ 2t . We obtain the following expression for the expected excess log return Et [rs,t+1 ] − rf,t+1 ≈ c0 +
£ ¤! £ ¤ ¶ µ φ1 E σ 2t E σ 2t φ1 (E [ρt ] − θ) 1 − − σ 2t ρt + β (1 − θ) 2 β (1 − θ) 2
where c0 is the constant of linearization. Both of the unobservable parameters, β and θ, aﬀect the magnitudes of the coeﬃcient estimates for average correlation and average variance. For plausible parameter values, the coeﬃcient for average correlation is positive while the coeﬃcient for average variance is actually negative. Since average variance is the dominant component of stock market variance, its negative risk premium explains the anaemic relationship between
stock market variance and subsequent market returns. This linear approximation underpins our empirical analysis in the next section. Under these assumptions, the risk premium for the stock market is positively related to a weighted average of stock market variance and the covariance of the stock market with the unobserved portfolio return. If average variance is not positively related to the risk premium for the stock market, then average variance must be negatively related to Covt (Rs,t+1 , Ru,t+1 ) because an increase in average variance increases stock market variance. To the extent that we can observe a portion of Covt (Rs,t+1 , Ru,t+1 ) we can construct an additional test of the model’s implications.
An approximation for stock market variance
In the previous subsection we assume that stock market variance is identical to the product of correlation ρt and individual stock variance σ 2t . In our empirical work we use an approximation. The stock market portfolio s is the value-weighted portfolio of all stocks where wj,t is the market capitalization of stock j divided by the market capitalization of the entire stock market. The variance of the stock market return is given by σ 2s,t =
N N X X
wj,t wk,t ρjk,t σ j,t σ k,t
A useful approximation for stock market variance is to express it as the product of average correlation between all pairs of stocks and the average variance of all individual stocks. We define σ 2t to be the equal-weighted cross-sectional average variance for the N stocks. σ 2t =
N 1 X 2 σ j,t N j=1
We let ξ jkt denote the pairwise stock-specific deviations from the cross-sectional average for variance ξ jk,t = σ j,t σ k,t − σ 2t
and rewrite the expression for stock market variance. σ 2s,t
N N X X j=1 k=1
¡ ¢ wj,t wk,t ρjk,t σ 2t + ξ jk,t
N X N X
wj,t wk,t ρjk,t +
N N X X
wj,t wk,t ρjk,t ξ jk,t
Stock market variance is the sum of two terms. The first term is the product of the equal-weighted average of individual stock return variances and the value-weighted average of return correlations across all pairs of stocks in the portfolio. The second term depends on the cross-sectional relationships between weights, pairwise correlations, and the cross-products of standard deviations. When all assets have the same individual variance, the second term is equal to zero and the expression can be simplified accordingly. σ 2s,t = σ 2t
N N X X
wj,t wk,t ρjk,t = σ 2t ρt
This expression has two components: equal-weighted average variance and value-weighted average correlation. We approximate stock market variance with the right-hand side of equation (8). In the next section we show that this approximation captures almost all of the time-series variation in stock market variance.
Predicting stock market returns with correlation Proxies for the components of market variance
Every quarter from 1963 until 2004 we estimate the two components on the right-hand side of expression (8) directly from daily stock returns for a value-weighted portfolio of the 500 largest stocks, by market capitalization, in the CRSP universe.
Given Dt trading days in quarter t, the sample variance of daily returns for stock j is
!2 ⎞ Ã Dt Dt X X 1 1 =⎝ (1 + Rj,t−d ) ⎠ (1 + Rj,t−d ) − Dt − 1 Dt ⎛
and our empirical proxy for σ 2t , average variance, is 500
1 X 2 σ bj,t . 500
Although we estimate the conditional covariances and variances using daily returns, we are interested in the relationship between the quarterly stock market excess return and these moments for the same time interval. Therefore, we multiply our daily variance and covariance estimates by sixty-three, the average number of trading days in a quarter. The sample correlation for stocks j and k, denoted b ρjkt , is calculated in the typical fashion
given the definition of σ b2jt .
b ρjk,t =
σ bjk,t σ bj,t σ bk,t
We define wj,t as stock j’s market capitalization divided by the sum of all 500 market capiP P talizations observed at the end of quarter t. Our proxy for N k6=j wj,t wk,t ρjk,t , average j=1 correlation, is
N X X j=1 k6=j
wj,t wk,tb ρjk,t .
We also estimate the conditional stock market variance, σ b2s,t , from the daily index returns
of the CRSP value-weighted index. Hence, equation (7) implies that: σ b2s,t = b0 + b1 (AVt ∗ ACt ) + υt
where E [υt |AVt ∗ ACt ] = 0. The coeﬃcient b1 may not equal one because (i) we exclude the sum of squared weights from ACt , (ii) there is measurement error in AVt and ACt , and (iii) there are diﬀerences in the composition between the CRSP market index and the valueweighted portfolio of the 500 largest stocks.
Other data and summary statistics
In our predictive regressions, we use the log stock market return minus the log three-month T-bill return based on the CRSP value-weighted portfolio (the best available proxy for the portfolio of all stocks) and the S&P 500 (which is arguably untainted by the illiquidity that might aﬀect returns of small stocks in the broader CRSP index). We also use: cay from Lettau and Ludvigson (2001) (from Martin Lettau’s website); tp, the log ratio of total market cash flows (including dividends and net share repurchases) to shareholders, the second total payout ratio measure compiled by Lei (2006); and the relative bill rate rb, equal to the log yield on the three month T-bill minus a triangular moving average of the three month T-bill yields during the previous four quarters. The total payout ratio measure, tp, has significant predictive power at a quarterly frequency in our sample, whereas the log dividend-price ratio and smoothed price-earnings ratio do not. Finally, in section 4 we use the daily holding period return on 10-year US government treasuries from the CRSP daily bond file. Table 1 reports summary statistics for all of the relevant time-series. All data are observed quarterly from the first quarter of 1963 to the end of 2004, except for the two excess log returns and the covariances between stock market and bond returns, which are observed until the end of the first quarter of 2005. However, we drop this last quarter when calculating the summary statistics. Average correlation between all pairs of the 500 largest stocks in the market averaged 23.6% in our sample period, with a standard deviation of 9.4%. The mean for average variance of the 500 largest stocks is 2.796% while the mean for stock market variance is, not surprisingly, much lower at 0.492%. Average variance is also 3.7 times as volatile as stock market variance and strongly positively correlated with stock market variance. Average variance has a smaller but positive correlation with average correlation of 25.7%. The average covariance between daily stock market and 10-year bond returns estimated quarterly (scaled to match the quarterly time interval) is 3.299%, but this covariance is volatile, with a standard deviation of 10.86%, a maximum of 35.63% and minimum of -54.43%. This covariance is negatively correlated with stock market and average variance (around -30% to -32% for each) but only weakly negatively correlated with average correlation (around -5%).
The first, second and third autocorrelations of each time series are also reported in Table 1. Average correlation, average variance and the stock-bond covariance are all fairly persistent, with first autocorrelations of respectively of 58.6%, 73.9%, and 60.7%. Stock market variance is much less persistent, even though the correlation between average variance and average correlation is 25%. Stock market variance, average correlation, and average variance are negatively correlated with stock market returns as are cay and rb. tp is mildly positively correlated with returns, making it an unusual forecasting variable. The stock-bond covariance is mildly positively correlated with returns and negatively correlated with cay, rb, and tp. Figure 1 plots the time series for average correlation and average variance. Our measure of average variance has peaks and troughs that parallel the behavior of the standard deviation measure considered by Goyal and Santa-Clara. However, our measure does not exhibit a pronounced upward time trend. While average variance and average correlation move together to some extent, this positive association disappears from 1999 until the end of 2001. In addition, the distribution for average variance is more positively skewed than the distribution of average correlation. The shaded regions represent periods of recession according to the NBER. There does not appear to be an obvious pattern linking recessions to the behavior of either measure. Both measures peaked in the last quarter of 1987 due to the ‘Black Monday’ stock market crash.
Validity of the stock market variance decomposition
Table 2 reports OLS regressions of σ b2s,t on combinations of average variance and average cor-
relation. The t-statistics use Newey-West standard errors with six lags. The results do not
qualitatively change if we use OLS standard errors or change the lag length for the Newey-West estimator. Column 1 reports direct estimates of (12). The R2 of the regression is 98.7%, which indicates that the variation in σ b2s,t is captured by average variance and average correlation. Columns
2 through 4 present estimates of the relative importance of average variance and average
correlation in changes in stock market variance. Column 2 shows that average correlation accounts for 38.52% of variation in stock market variance and column 3 that average variance
accounts for a much larger 60.19%. Not surprisingly, since estimates of average variance and average correlation are positively correlated, the additional explanatory power provided by including average variance in a linear regression is substantial, as shown by the R2 of 79.19% in column 4. Column 5 uses the variance of the S&P 500 daily excess return instead of the excess return for the CRSP value-weighted index in column 1. The R2 for this specification is only slightly lower than the analogous specification for the CRSP index at 95.99%. While both correlation and average variance are important components of stock market variance, table 2 indicates that average variance is the dominant component.
In Table 3, we present quarterly stock market return forecasting regressions using the conditional market variance σ b2s,t and its two major components, average variance AVt and average correlation ACt . Usually, the dependent variable is the excess log return of the CRSP value-
weighted index. In the last column the excess log return for the CRSP index is replaced by the excess log return for the S&P 500 index. Column 1 shows that average correlation is a strong and highly economically significant forecaster of excess log stock market returns. A one-standard deviation increase in the average correlation of the daily returns of the 500 largest stocks in the market forecasts a 1.88% additional stock market excess return over the following quarter. The estimate is highly statistically significant with a robust t-statistic of 3.09. Variation in average correlation between stock returns accounts for 4.8% of variation in quarterly stock market return, comparing favorably with any other forecasting variable proposed to date. Column 2 indicates that average variance has no forecasting power. The estimate is not significant and has the ‘wrong’ sign: an increase in stock market risk due to higher average individual variance forecasts a low future return. Column 3 shows that the sample variance of the stock market return, σ b2s,t , as we estimate
it, has only a weak relationship with subsequent returns, with a t-statistic of 1.53 and an R2 of 1.23%. Since stock market variance is approximately equal to the product of average correlation and average variance, the relatively poor forecasting power of stock market variance
is due to the lack of forecasting power associated with average variance. While not statistically significant, these estimates could be economically important: a one-standard deviation increase in the estimate of stock market variance forecasts an additional 1% of stock market return during the following quarter. In column 4, including both measures of risk in a linear regression further reduces the sign of the coeﬃcient on average variance and boosts the forecasting power and significance of average correlation. As discussed in section 2, it is the component of average correlation that is unrelated to average individual variance that predicts future stock market returns. The coeﬃcient estimates in this regression indicate that average correlation has a positive risk premium while average variance has a negative risk premium. Equation (6) allows us to interpret the coeﬃcients in column 4 in terms of the parameters of our model with symmetric stocks. For example, if the beta for each stock with the true market portfolio equals one (β = 1), and the conditional CAPM holds (φ1 = γ), then these estimates imply that the relative risk aversion coeﬃcient γ is equal to 6.6. Column 5 uses the ratio of stock market variance to average variance in place of average correlation to forecast returns. Scaling stock market variance by the component uncorrelated with future returns produces a somewhat more powerful forecasting variable than average correlation with an R2 of 5.2%. Since the decomposition for stock market variance is not perfect, stock market variance divided by average variance is only approximately equal to the estimate of average correlation. Since stock market variance, average variance, and average correlation are each estimated with error and this error almost certainly contains a common component, stock market variance divided by average variance could be a better forecasting variable than average correlation. The specification in column 6 replaces the excess log return of the CRSP index with the excess log return of the S&P 500 index, but is otherwise identical to the specification in column 1. The stocks in the S&P 500 index are all heavily traded and any return predictability that we observe is unlikely to be due to illiquid assets. Average correlation forecasts S&P 500 returns almost as well as it forecasts the broader CRSP index, with an R2 of 4.3% and a t-statistic of 2.86. We conclude that average correlation is not merely a proxy for the changing liquidity of
lightly traded assets in the broader stock market index.
Estimation bias in small samples
We estimate a restricted VAR(1) for excess log stock market returns and average correlation. First, we are interested in the persistence of average correlation shocks. Second, we are interested in the volatility of shocks to stock market returns, the volatility of shocks to average correlation, and the correlation of these shocks. Third, we need to estimate the bias in the OLS estimate from the forecasting regressions due to persistence in average correlation and the correlation between the shocks to average correlation and realized returns. The first equation in the VAR is rs,t+1 − rf,t+1 = a + bACt + ut+1 and the second equation is ACt+1 = c + dACt + vt+1 where the variances and covariance of the residuals are written σ 2u , σ 2v , and σ uv . The coeﬃcient b is our estimate of the ability of average correlation to forecast returns, but, as Stambaugh (1999) discusses, if σ uv is negative then the estimate of b is biased upwards in small samples. If d, the persistence of shocks to average correlation, is large (close to 1), this bias can be severe. If average correlation forecasts returns, there are strong theoretical reasons to believe that σ uv should be negative. Campbell (1991) shows that shocks to log returns are approximately equal to shocks to the present value of expected future log dividend growth minus shocks to future expected returns: rt+1 − Et [rt+1 ] ≈ ∆Et+1
∞ X s=0
η s ∆dt+1+s − ∆Et+1
η s rt+1+s
where the log-linearization constant η is slightly less than one. This approximate accounting identity shows that, holding expected future cash flows constant, a positive and persistent shock to expected returns must result in a negative shock to current realized returns, since the same
expected future cash flows are discounted at higher rates. Expected returns whose innovations are negatively correlated with shocks to realized returns are said to be mean-reverting. When expected returns are positively related to systematic risk, as captured by market variance or average correlation, persistent increases in risk cause increases in required or expected returns, inducing negative realized returns. Campbell and Hentschel refer to this special case of mean reversion as volatility feedback. The more persistent the shock to risk, the larger the volatility feedback. Therefore, volatility feedback generates an estimation bias of the form described by Stambaugh. Table 4 reports estimates of the restricted VAR. The parameter estimates for a and b in the top panel are familiar from Table 1 and Table 3. Newey-West t-statistics with the usual 6 lags are reported in square brackets. The estimate of d in Table 4 is 0.585, the standard deviation of average correlation shocks σ v is 7.6%, the standard deviation of the stock market return shocks σ u is slightly more volatile than average correlation shocks and is 8.4%, and the correlation between the shocks ρuv is -54%. To estimate the bias under the null hypothesis of no predictive power, we carry out a simulation exercise following that in Baker, Taliafero and Wurgler (2006). The OLS estimate of d is biased downwards by approximately (1 + 3d)/T , where T is the sample length. Our bias-adjusted estimate of d in the full sample is therefore 0.602. We use our estimates of the constant terms in each equation and of the residual covariance matrix but also impose diﬀerent values of ρuv (the correlation of the residuals) and adjust σ uv accordingly. We then simulate data for average correlation and for the excess log return for the stock market under the assumption that they are generated by the restricted VAR with b equal to zero. We use the actual starting variables and discard the first 100 simulated observations, then calculate OLS estimates of b for the remaining sample. The lower panel of Table 4 reports choices of d, ρuv and sample size T for the simulation, together with the mean implied estimate of b, its standard deviation, and the probability that the simulated estimate is greater than the actual estimate. This probability is our Monte Carlo estimate of the correct small-sample bias-corrected p-value for our estimate of b under the null that true b is zero.
For the full sample, using d equal to 0.602 we should estimate b of 0.200 less than 0.5% of the time according to the simulation using the sample estimate of ρuv . In this case the bias in the estimator for b is about 0.01. For the worst-case scenario of ρuv = −1, we estimate a slightly higher p-value of 0.8% but we still reject the null at conventional levels of significance. Increasing d to 0.756, three standard errors above its OLS estimate, and setting ρuv = −1 does not materially increase the p-value. Although the bias in bb increases to 0.022 because average
correlation is more persistent, the dispersion of the simulated estimates is lower. Similarly, choosing a high persistence of 0.95, while increasing the bias to 2.8%, actually reduces the p-value to 0.1%. These results indicate that small sample bias in the predictive regression is not a serious problem for the estimate of b in the full sample of 168 quarters.
Analysis of sub-samples and diﬀerent horizons
In Table 5 we estimate predictive regressions for the excess log return for the CRSP valueweighted index on average correlation in diﬀerent subsamples. We report estimates for the full sample (from the end of the second quarter of 1963 to the end of the first quarter 2005), both halves of the sample, all four quarters of the sample and from 1976 until 2005. The last sample begins after the first oil price shock in the 1970s. Goyal and Welch find that many candidate forecasting variables have very little forecasting power in this sub-sample and they conclude that these candidates rely heavily on the oil price shock of the early 1970s for in-sample forecasting power. In the periods 1963-1973, 1973-1984, and 1994-2005 average correlation demonstrates the ability to forecast stock market returns. For the 30 years since 1976 it is also strongly economically and statistically significant, with an R2 of 5.05% for the stock market return. Average correlation is less powerful in the second half of the sample, from 1984 with a t-statistic of 1.8. Column 6 shows that from 1984-1994 average correlation has no ability to forecast market returns. This is not due to the stock market crash of 1987: in unreported results we estimate the same equation excluding the months around the crash and the results remain unchanged.
In figure 1 it can be seen that average correlation fell to very low levels in the late 1980s even though returns remained high. With the exception of the decade 1984-1994, the coeﬃcients on average correlation are fairly stable, between 0.2 and 0.3. Table 6 reports predictive regressions of CRSP market excess log returns for return intervals of 1 month, 1 quarter, 6 months, 1 year and 2 years. The observations of the dependent variable in each regression are non-overlapping. Average correlation forecasts monthly excess log returns on the broad index with a coeﬃcient of 0.054, a t-statistic of 2.55 and an R2 of 1.31%. The coeﬃcient and R2 increase with the horizon from the monthly return regression to the semiannual return regression and then slightly decrease at annual horizons. At an annual horizon the predictive coeﬃcient is 0.372 and the R2 is 6.94%. Hence, a 1-standard deviation increase (annualized) in average correlation suggests that the annual expected return is approximately 7% percentage points higher (holding all else constant). In unreported results, we do not find qualitatively diﬀerent estimates if we replace the valueweighted average correlation measure with an equal-weighted version; use the largest 50 or 100 stocks to construct average correlation instead of the largest 500; use log average correlation instead of average correlation; or replace the dependent variable with the excess return in levels. We also estimated the average correlation and average variance of daily returns of the 48 Fama-French industry portfolios (equal-weighted across industries) and the 25 Fama-French size and book-to-market sorted portfolios (equal-weighted across portfolios). Both measures of average correlation predict excess stock market returns in both logs and levels with similar magnitudes and statistical significance. None of the average variance measures demonstrate any predictive power.
Comparisons with other predictors
Table 7 presents a number of forecasting regressions. Our objective is to control for alternative predictors and ensure that average correlation is not simply replicating other well-known predictability results. We consider cay, tp, rb, and lagged excess log stock market return as potential forecasting variables. The first three columns estimate predictive regressions for each of the other forecasting
variables. The total payout ratio tp and the relative bill rate rb are individually significant in this sample period with t-statistics of 2.06 and 2.08 and R2 of 2.11% and 2.00%, respectively. The variable cay is strongly significant and has an R2 of 4.7% (slightly lower than that of average correlation). Putting all the controls and other forecasting variables together in column 4 gives an R2 of 6.89%. Average correlation remains statistically significant, and its coeﬃcient estimate is stable, when the other forecasting variables are included. Comparing column 4 (lagged return, cay, tp and rb only) with column 5, average correlation increases the R2 from 6.89% to 10.63%. The t-statistics for average correlation, cay and rb indicate that all three variables contain distinct information about future returns, albeit at reduced levels of significance for cay and rb when average correlation is included. However, tp is not individually significant in either regression with cay.
Out-of-sample forecasting ability
Goyal and Welch suggest that most forecasting variables with in-sample forecasting power do not demonstrate ability to forecast returns out of sample. They propose a test in which forecasting regressions use only data available at t and the one-period-ahead squared forecast error is compared to the squared diﬀerence between year t + 1 realized return and the sample mean return for time t. They then compare either the diﬀerence in root mean squared error (“∆RM SE”) or out-of-sample R2 (one minus the ratio of the sum of squared forecast errors over the sum of squared diﬀerences from the time t sample mean). Both these measures will be positive if the proposed variable has superior forecasting ability out of sample relative to an average of past returns, zero if there is no diﬀerence and negative otherwise. With the exception of cay, they fail to find superior out-of-sample performance for any variable. This failure suggests that the estimates from forecasting regressions are unstable and are quite sensitive to sample period choice. Since the cointegrating vector used to construct cay is estimated over the entire sample, this method does not really test the out-of-sample performance of cay.7 By contrast, average correlation exhibits out-of-sample forecasting power. 7
A number of studies discuss the out-of-sample forecasting properties of cay. See Brennan and Xia (2001), Hahn and Lee (2002), Ang and Liu (2004), as well as Goyal and Welch and Campbell and
In Table 8 we report the two out-of-sample performance measures for two diﬀerent measures of average correlation (value-weighted, following the rest of the paper and equal-weighted), together with average individual variance, stock market variance, tp and cay. Since Campbell and Thompson (2005) have suggested that short in-sample estimation periods can be responsible for poor out-of-sample performance, we allow all variables at least 20 years to "tune up", and make the first out-of-sample forecast starting in 1983. This approach ensures that the period of assessment is the same across all predictive variables.8 Average variance, tp and cay all have negative ∆RM SE and out-of-sample R2 during the out-of-sample assessment period. Surprisingly, stock market variance has a slightly positive out-of-sample R2 of 0.112% and ∆RM SE of 0.5%. Value-weighted and equal-weighted correlation have similar out-of-sample R2 of 3.26% and 2.58% respectively and ∆RM SE of approximately 14% and 11%.
Predicting stock market return and bond return covariance
Changes in average variance are the most important source of changes in stock market variance according to Table 2. Therefore, the failure to forecast returns with average variance is a puzzle. If changes in average variance are often due to changes in the risk of this common stock-specific component, then section 2 suggests that increases in average variance increase stock market risk but reduce the covariance of stock market with non-stock returns. The resulting net eﬀect on the stock market risk premium may therefore be small or even negative. If this argument is correct, there should be at least one important class of non-stock assets whose return covariance with the stock market is negatively related to the average variance of stocks. We present evidence that government bonds satisfy this requirement, implying that an investor holding long-term bonds and stocks in her portfolio will not necessarily be exposed to greater portfolio risk when there is an increase in average variance for stocks. We estimate the required covariance with the within-quarter sample covariance of daily returns on the CRSP value-weight index and the 10-year US Treasury bond, from the first Thompson (2005). 8 Average correlation has a positive out-of-sample R2 even if the tune up period is set to one quarter.
quarter of 1963 to the first quarter of 2005. We term this variable the stock-bond covariance and attempt to forecast it one quarter ahead with average correlation, average variance, stock market variance and current stock-bond covariance. Table 9 reports our results. Columns (1) to (4) present estimates of univariate predictive regressions. Average correlation is unrelated to subsequent stock-bond covariance. Average variance, by contrast, is significantly negatively related, with a t-statistic of 2.4 and an R2 of 9.8%. Stock market variance is also negatively related, but not significantly. (This is not surprising, since average variance is the largest component of stock market variance.) Stock-bond covariance is also highly persistent, as shown in column (4), with an AR(1) persistence coeﬃcient of 0.61 and R2 of 37%. Controlling for average correlation, column (5) shows average variance is more closely related to subsequent stock-bond covariance, with the coeﬃcient increasing in magnitude, the t-statistic rising to 3.1 and the R2 to 9.99%. Finally, even controlling for the stock-bond covariance at date t and average correlation in column (6), average variance is a marginally significant negative predictor of stock-bond covariance in quarter t + 1, with a t-statistic of 2.2. This evidence is consistent with the hypothesis that most of the variation in average variance, and therefore stock market variance, is due to changes in the stock-specific risk common to stocks and unrelated to changes in aggregate risk. In unreported work, we have also found suggestive evidence that average variance is negatively related to the subsequent covariance of stock market returns with corporate bonds and labor income growth, while average correlation is unrelated to the covariance with corporate bonds and positively related to the covariance with labor income growth. However, the data quality for these asset classes is much poorer, and hence, the statistical tests are not particularly informative.
In this section we present evidence that our results are not easily explained as being due to measurement error in a stock market variance-in-mean world. Instead, the results are consistent with a model of stock market returns in which average correlation risk is priced but average variance risk is not. We proceed by simulating the distributions of returns under the
assumption that returns are generated under a variance-in-mean relation between risk premia and stock market variance. Our simulations assume a symmetric cross-section of N stocks (same expected return and variance of return and identical correlation across all pairs of stocks). The optimal portfolio for such stocks is equal-weighted and so we regard this as our proxy for the stock market portfolio where the true market portfolio and the stock market portfolio are the same by assumption. For each day in quarter t + 1, the vector of daily returns rd for individual stocks is drawn from a joint normal distribution. The ‘true’ variance of the daily market return is: σ 2m,t =
1 0 ι Σt ι N2
where ι is a conformable vector of ones and the covariance matrix for the N stock returns is Σt . Given a vector of daily stock returns rd , the simulated market daily return is rm,d =
1 0 ι rd N
The simulated quarterly log market return is the sum of the 63 daily log market returns simulated for each quarter. The riskless rate is normalized to be zero so that excess returns are equal to raw returns. We scale the variance measures by 63 to express the measures of risk in units reflecting the same time interval as quarterly excess returns. The within-quarter estimates of market variance, average variance, and average correlation are obtained using the estimators described in Section 2. Under the assumptions of this model, the estimate of stock market variance is the estimate of true market variance. The data generating process for estimating measurement error is the typical variance-inmean model: ¡ ¢ rd ∼ N (ι ψ 0 + ψ 1 σ 2m,t , σ 2t Ωt )
where Ωt is a matrix with ones on its diagonal and ρt everywhere else. We choose to set σ 2t to equal average variance and ρt to equal average correlation from our actual estimated time series for each. Since all stocks are symmetric in the simulation the specification of the covariance matrix as σ 2t Ωt is not an approximation. However, our estimates of average correlation for this
part are equal-weighted, as opposed to the value-weighted estimates we use in most of the rest of the paper. Although our results are clearly not easy to reconcile with model (14), simulating returns under (14) is useful both to estimate the measurement error of our actual estimates and provides a baseline of anticipated results if (14) is correct. The parameters ψ 0 and ψ 1 for our simulations are chosen to match the unconditional sample mean and standard deviation of excess log stock market returns in Table 1. These parameters are the respective analogs of ζ 0 and ζ 1 −
from equation (3) where the stock market is the true market portfolio.
The simulations indicate that model (14) cannot account for all of our results. Table 10 presents a selection of results from our simulations. We set N = 500 stocks and simulate 250 time series of 168 quarters each. Each quarter contains 63 trading days, so for each simulated time series there are 63 x 168 x 500 daily returns. For every quarter we estimate average correlation, average variance and market variance from the simulated daily returns. To avoid confusion we will use these phrases to refer to our estimates from the simulated data, and the notation ρt , σ 2t and σ 2m,t to refer to their true or assigned values. We report the means, across all simulations, of the mean, standard deviation and first autocorrelation of the simulated quarterly excess log market returns, average correlation and average variance in the first three rows of Table 10. The autocorrelation of returns arises from the simulation. This autocorrelation is 0.073 and is over three times higher than the analogous estimate for the CRSP index and about 1.5 times higher than the estimate for the S&P500 index. The simulated autocorrelation may be somewhat high due to our failure to impose volatility feedback in the simulation, whereby shocks to variance are negatively correlated with shocks to returns. The means, standard deviations and first autocorrelations of the simulated estimates of average correlation and average variance are all close to their sample estimates (in the simulations the average correlation estimates are constructed on an equal-weighted basis rather than on a value-weighted basis). The next row of numbers in Table 10 presents estimates of mean proportional deviation from true values. For example, the mean proportional deviation for estimated market variance
|b σ 2m,t − σ 2m,t | σ 2m,t
averaged across all time-periods and simulations. The table shows that the mean error for estimated market variance is about 14.2% of true value, 11.8% for average correlation and lowest for average variance at 2.65%. We also estimate a subset of the regressions presented in Tables 2 and 3. We report means of the simulated coeﬃcient estimates, t-statistics, and also power estimates (the percentage of simulations in which the t-statistics are greater than 1.96) and average R2 . We present results for regressions of contemporaneous estimated market variance on each of average variance, average correlation and their product. The results are very close to those in Table 2: average variance explains an average of 62% of the variation in market variance, average correlation about 48.8% of the variation in market variance and their product more than 99.99% of the variation in market variance. The bottom right-hand corner of Table 10 reports results for regressions of excess log market returns on average variance, average correlation and market variance. The main finding from these regressions is that model (14) is inconsistent with the observed poor predictive power of average variance compared to the predictive power of average correlation. According to the results in Table 10, average variance should forecast stock market returns reasonably well and better than both average correlation and market variance. The mean R2 of the average variance predictive regression in Table 10 is 5.5% and we reject the null in 52% of the simulations versus 1.9% and 32% for average correlation. Market variance should also be a better forecaster than average correlation, with an average R2 of 3.7% and rejection rate of 38% according to the model. It is interesting that even when stock market returns satisfy a variance-in-mean model by construction, average variance should be a superior predictor than stock market variance. The puzzle that is diﬃcult to reconcile with a variance-in-mean model of stock market returns is that both average variance and average correlation are persistent components of stock market variance, but average variance does not forecast stock market returns and average correlation does forecast returns. Our simulation results suggest that there is no simple explanation based on measurement error. First, these errors seem to be small, and smaller
for average variance than average correlation. Second, to fit our findings the measurement errors would have to be persistent in a way that does not confer forecasting power on average variance. One alternative possibility is that average variance is forecasted by, but does not forecast, average correlation, so that it is not a truly separate source of market risk. In unreported results we find little evidence of such cross-predictability in either direction.
The absence of an easily detectable relationship between stock market risk and subsequent stock market returns poses diﬃculties for mainstream asset pricing models. We claim that the lack of evidence of such a relationship is a manifestation of the Roll critique: if the market portfolio contains important non-stock assets, then changes in stock market variance are not necessarily closely related to changes in aggregate risk. Therefore, even if changes in aggregate risk drive changes in risk premia, changes in stock market risk need not be related. We propose a solution based on the observation that true market risk should be a source of interdependence among observable stock returns. Hence, an increase in aggregate risk is partially reflected in an increase in average correlation between stock returns whenever the stock market as a whole has a positive sensitivity to aggregate market shocks. Therefore, average correlation should forecast stock market returns whenever there is an aggregate riskexpected return trade-oﬀ. Furthermore, changes in stock market variance that are unrelated to changes in average correlation could have a zero or even negative relationship with future stock market returns. We show that such unpriced stock market risk is, for practical purposes, captured by the average variance of individual stocks. We present new results showing that average correlation and average variance together account for almost all variation in stock market variance and that average variance is the dominant component. We show that average correlation forecasts future stock market returns while average variance is negatively related to future returns. Average variance is also negatively related to the covariance between stock market and bond returns, consistent with the hypothesis that changes in average stock variance unrelated to average correlation do not necessarily
reflect changes in true aggregate risk. The previous literature on the empirical relation between the time series of stock market risk and return is focused on the econometric diﬃculties in identifying such a relationship even when truly present (see for example Ghysels, Santa-Clara and Valkanov for an important recent contribution). The main alternative explanation in the existing literature assumes some degree of deviation from a standard representative agent rational-choice framework under which linear or loglinear aggregate risk-risk premium relations do not hold. By contrast, our paper assumes both that aggregate risk is related in a straightforward way to the market risk premium and that failure to detect such a relation for the stock market is not due to measuring stock market risk with error. Instead, we apply the Roll critique to the risk-return trade-oﬀ and exploit the common sensitivity of observable asset returns to aggregate risk.
References  Ang, Andrew and Joseph Chen, 2002. “Asymmetric Correlation of Equity Portfolios”, Journal of Financial Economics, Vol. 63, 443-494.  Ang, Andrew, Joseph Chen and Yuhang Xing, 2005. “Downside Risk”, Review of Financial Studies, forthcoming.  Ang, Andrew and Jun Liu, 2004. “How to Discount Cashflows with Time-Varying Expected Returns”, Journal of Finance, Vol. 59, 2745-2782.  Baker, Malcolm, Ryan Taliafero and Jeﬀrey Wurgler, 2006. “Predictive Regressions Based on Managerial Decision Variables: Is There a Small-sample Bias?”, Journal of Finance, forthcoming.  Bali, Turan G., Nusret Cakici, Xuemin Yan and Zhe Zhang, 2005. “Does Idiosyncratic Risk Really Matter?”, Journal of Finance, Vol. 60, 905-929.  Bollerslev, T., Robert Engle and J. Wooldridge, 1988. “A Capital Asset Pricing Model with Time Varying Covariances”, Journal of Political Economy, Vol. 96, 116-131.  Brennan, Michael J. and Yihong Xia, 2001. “TAY’s as Good as CAY”, mimeo.  Brown, Gregory and Nishad Kapadia, 2006. “Firm-Specific Risk and Equity Market Development”, Journal of Financial Economics, forthcoming.  Campbell, John Y., 1987. “Stock Returns and the Term Structure”, Journal of Financial Economics, Vol. 18, 373-399.  Campbell, John Y., 1991. “A Variance Decomposition for Stock Returns”, Economic Journal, Vol. 101, 157-179.  Campbell, John Y. 1993. “Intertemporal Asset Pricing Without Consumption Data”, American Economic Review, Vol. 83, 487-512.  Campbell, John Y. and L. Hentschel, 1992. “No News is Good News: An Asymmetric Model of Changing Volatility in Stock Returns”, Journal of Financial Economics, Vol. 31, 281-318.  Campbell, John Y., Martin Lettau, Burton G. Malkiel and Y. Xu, 2001. “Have Individual Stocks Become More Volatile? An Empirical Exploration of Idiosyncratic Risk”, Journal of Finance Vol. 61, 1-43.  Campbell, John Y., Andrew W. Lo and A. Craig MacKinlay, 1997. The Econometrics of Financial Markets, Princeton University Press.  Campbell, John Y. and Samuel Thompson, 2005. “Predicting the Equity Premium Out of Sample: Can Anything Beat the Historical Average?”, NBER working paper 11468.
 Cochrane, John H., 2006. “The Dog that Did Not Bark: A Defense of Return Predictability”, mimeo.  Cox, John C., Jonathan E. Ingersoll and Stephen A. Ross, 1985. “A Theory of the Term Structure of Interest Rates”, Econometrica, Vol. 53, 385-408.  Driessen, Joost, Pascal J. Maenhout and Gregory Vilkov, 2006. “Option-Implied Correlations and the Price of Correlation Risk”, mimeo.  Engle, Robert, D. Lilien and R. Robbins, 1987. “Estimating Time-Varying Risk Premia in the Term Structure: The ARCH-M Model”, Econometrica, Vol. 55, 391-407.  Forbes, Kristin J. and Roberto Rigobon, 2002, “No Contagion, Only Interdependence: Measuring Stock Market Comovements”, Journal of Finance , Vol. 57, 2223-2261.  French, Kenneth R., G. William Schwert and Robert F. Stambaugh, 1987. “Expected Stock Returns and Volatility”, Journal of Financial Economics, Vol. 19, 3-29.  Ghysels, Eric, Pedro Santa-Clara and Rossen Valkanov, 2005. “There is a risk-return trade-oﬀ after all”, Journal of Financial Economics, Vol. 76, 509-548.  Glosten, Lawrence, Ravi Jagannathan and David Runkle, 1993. “On the Relation Between the Expected Value and the Volatility of the Nominal Excess Return on Stocks”, Journal of Finance, Vol. 48, 1779-1801.  Goyal, Amit and Pedro Santa-Clara, 2003. “Idiosyncratic Risk Matters!”, Journal of Finance, Vol. 58, 975-1008.  Goyal, Amit and Ivo Welch, 2006. “A Comprehensive Look at the Empirical Performance of Equity Premium Prediction”, mimeo.  Hahn, Jaehoon and Hayong Lee, 2002. “On the Estimation of the Consumption-Wealth Ratio: Cointegrating Parameter Instability and Its Implications for Stock Return Forecasting”, mimeo.  Harvey, Campbell, 1991. “The World Price of Covariance Risk”, Journal of Finance, Vol. 46, 111-157.  Harvey, Campbell, 2001. “The Specification of Conditional Expectation”, Journal of Empirical Finance, Vol. 8, 573-638.  Heston, Steven, 1993. “A Closed-Form Solution for Options with Stochastic Volatility with Applications to Bond and Currency Options”, Review of Financial Studies, Vol. 6, 327-343.  Hong, Yongmiao, Jun Tu and Guofu Zhou, 2003. “Asymmetric Correlation of Stock Returns: Statistical Tests and Economic Evaluation”, mimeo.  Lei, Qin, 2006. “Cash Distributions and Returns”, mimeo.
 Lettau, Martin and Sydney C. Ludvigson, 2001. “Consumption, Aggregate Wealth and Expected Stock Returns”, Journal of Finance, Vol. 56, 815-849.  Lettau, Martin and Sydney C. Ludvigson, 2002. “Measuring and Modelling Variation in the Risk-Return Tradeoﬀ”, in Ait-Sahalia and Hansen (Eds.) Handbook of Financial Econometrics, North-Holland, Holland.  Longin, F. and Bruno Solnik, 2001. “Extreme Correlation of International Equity Markets”, Journal of Finance, Vol. 56, 649-676.  Merton, Robert C., 1980. “On Estimating the Expected Return on the Market: An Exploratory Study”, Journal of Financial Economics, Vol. 8, 323-361.  Roll, Richard, 1977, “A Critique of the Asset Pricing Theory’s Tests: Part I”, Journal of Financial Economics, Vol. 4, 129-176.  Shiller, Robert S., 1981. “Do Stock Prices Move Too Much to Be Justified by Subsequent Changes in Dividends?”, American Economic Review, Vol. 71, 421-436.  Stambaugh, Robert F., 1999. “Predictive Regressions”, Journal of Financial Economics, Vol. 54, 375-421.  Yu, Jianfeng and Yu Yuan, 2006. “Investor Sentiment and the Mean-Variance Relation”, mimeo.
Figure 1: Average variance and average correlation 16
12 0.5 10 0.4 8 0.3 6
Average variance (multiplied by 100)
Year Notes: Average correlation is the cross-sectional average of the pairwise correlation of daily returns during each quarter for all pairs of the 500 largest (by market capitalization) exchange-traded stocks in the United States at the end of that quarter. This cross-sectional average is value-weighted based on the end-of-quarter market capitalization. Average variance is the cross-sectional average of the quarterly variance of daily returns for the same 500 large stocks (equal-weighted). The shaded regions represent periods of recession according to the NBER. The variances of daily returns are multiplied by 63, the average number of trading days in one quarter, in order to express these sample moments in terms of the same time interval as quarterly excess returns.
Table 1: Summary statistics
Mean Median Min Max St dev Auto(1) Auto(2) Auto(3)
Stock market variance (%)
Average variance (%)
Stock-bond covariance (%)
0.011 0.025 -0.300 0.202 0.086 0.022 -0.059 -0.032
0.012 0.021 -0.309 0.188 0.081 0.049 -0.051 -0.024
0.492 0.325 0.029 6.297 0.621 0.314 0.231 0.187
0.236 0.237 0.034 0.065 0.094 0.586 0.512 0.427
2.796 2.133 0.926 14.921 2.292 0.739 0.630 0.604
3.299 2.523 -54.434 35.628 10.862 0.607 0.409 0.430
0.000 0.000 -0.031 0.036 0.013 0.846 0.745 0.675
0.012 0.016 -0.066 0.068 0.022 0.955 0.887 0.806
0.000 0.000 -0.042 0.036 0.012 0.601 0.311 0.228
-0.307 -0.303 0.776 0.257 1.000
0.103 0.119 -0.303 -0.051 -0.323 1.000
-0.087 -0.074 -0.064 0.112 -0.239 0.187 1.000
0.060 0.056 -0.262 0.175 -0.604 0.320 0.407 1.000
-0.230 -0.222 -0.071 -0.045 -0.076 0.150 -0.065 0.063 1.000
Correlation Matrix recrsp res&p500 Stock market variance Average correlation Average variance Stock-bond covariance cay tp rb
-0.421 -0.421 1.000
-0.298 -0.287 0.621 1.000
Notes: Average correlation is the cross-sectional average of the pairwise correlation of daily returns during each quarter for all pairs of the 500 largest (by market capitalization) exchange-traded stocks in the United States at the end of that quarter. This cross-sectional average is value-weighted based on the end-of-quarter market capitalization. Average variance is the cross-sectional average of the quarterly variance of daily returns for the same 500 large stocks (equal-weighted). We define reCRSP as the quarterly log return for the CRSP value-weighted index minus the 3-month T-bill log return and reS&P500 is the corresponding excess return for the S&P500 index. rb is the relative bill rate (the 1-month T-bill log yield minus a 12-month triangular weighted average of its past values). cay is from Lettau and Ludvigson (2001) and is estimated using their full sample from 1951 Q4 to 2005 Q4. tp is the log ratio of total stock market cashflows (dividends plus share repurchases minus seasoned equity offerings) divided by the stock market price from Lei (2006). Stock market variance is the variance of the CRSP value-weighted index return estimated quarterly. Stock-bond covariance is the covariance of the CRSP value-weighted index return and the daily holding period return of the 10 US Treasury bond, from the CRSP daily bond returns file, estimated quarterly. The variables are all observed from 1963 Q1 to 2004 Q4. The variances and covariances of daily returns are multiplied by 63, the average number of trading days in one quarter, in order to express these sample moments in terms of the same time interval as quarterly excess returns.
Table 2: Decomposing stock market variance Dependent variable: Variance of daily stock market returns estimated at t
(Average variance)* (Average correlation) R2 (%) N
0.030 [4.398] 0.179 [4.723]
*The dependent variable is the sample variance of the S&P 500 index daily return instead of the sample variance of the CRSP value-weighted index. Notes: Unless otherwise stated, the dependent variable is the variance of the CRSP value-weighted daily index return estimated quarterly from 1963 Q1 to 2004 Q4. Average correlation is the cross-sectional average of the pairwise correlation of daily returns during each quarter for all pairs of the 500 largest (by market capitalization) exchange-traded stocks in the United States at the end of that quarter. This cross-sectional average is value-weighted based on the end-of-quarter market capitalization. Average variance is the crosssectional average of the quarterly variance of daily returns for the same 500 large stocks (equal-weighted). Newey-West t-statistics with 6 lags are reported in brackets beneath the coefficient estimates. The variances of daily returns are multiplied by 63, the average number of trading days in one quarter, in order to express these sample moments in terms of the same time interval as quarterly excess returns.
Table 3: Predicting stock market excess returns Dependent variable: Quarterly excess log return for the stock market index at t+1 (1)
0.226 [3.631] -0.175 [0.512]
-0.413 [2.130] 1.534 [1.527]
Stock market variance
(Stock market variance)/ (Average variance) 2 R (%) N
*The dependent variable is the excess log return on the S&P 500 index instead of the excess log return for the CRSP value-weighted index. Notes: Unless otherwise stated, the dependent variable is the log return for the CRSP value-weighted index minus the log return for the three month T-bill from 1963 Q2 to 2005 Q1. Average correlation is the cross-sectional average of the pairwise correlation of daily returns during each quarter for all pairs of the 500 largest (by market capitalization) exchange-traded stocks in the United States at the end of that quarter. This cross-sectional average is value-weighted based on the end-of-quarter market capitalization. Average variance is the crosssectional average of the quarterly variance of daily returns for the same 500 large stocks (equal-weighted). Stock market variance is the variance of the CRSP value-weighted daily index return estimated quarterly. Newey-West t-statistics with 6 lags are reported in brackets beneath the coefficient estimates. The variances of daily returns are multiplied by 63, the average number of trading days in one quarter, in order to express these sample moments in terms of the same time interval as quarterly excess returns.
Table 4: Bias corrections for predictive regressions with small sample Excess log stock market return at t+1 = a + b*(Average correlation at t) + u(t+1) Average correlation at t+1 = c + d*(Average correlation at t) + v(t+1) Original OLS coefficient estimates
a -0.036 [2.156]
Covariance and correlation estimates u v Bias simulations
d 0.602 0.602 0.756 0.950
Covariance matrix u 0.007
ρuv -0.538 -1.000 -1.000 -1.000
v -0.003 0.006 T 168 168 168 168
b 0.200 [3.092]
c 0.098 [7.093]
d 0.585 [11.558]
Correlation (standard deviation) u v u (0.084) -0.538 v (0.076) Mean(bsim|b=0) Std(bsim|b=0) P(bsim>0.200|b=0) 0.010 0.070 0.005 0.018 0.069 0.008 0.022 0.059 0.008 0.028 0.037 0.001
Notes: The table reports results from simulations designed to address the potential for small sample bias in predictive regressions (Stambaugh 1999). ρuv is the correlation of the residuals from the two regressions. Following Baker, Taliaferro and Wurgler (2006) we select the following parameters for the simulation: 1) d=0.602 is the bias-corrected estimate of d based on proposition 4 in Stambaugh (1999), and 2) d=0.756 is the bias-corrected estimate plus three standard errors. We generate pseudodata under the null that b=0 using the relevant OLS estimates and the assumed values for d, ρuv and sample length T recorded for each of the bias simulations. We create 50,000 simulated estimates of b. The mean and standard deviation of the resulting distribution are reported in columns 4 and 5 of the bias simulations. Column 6 reports the probability that a simulated estimate is larger than the original OLS for each set of simulation parameter values under the null hypothesis that b=0. Since the simulated distribution is slightly positively skewed, these probabilities are larger than those for a normal distribution. Newey-West t-statistics with 6 lags are reported in brackets beneath the coefficient estimates.
Table 5: Predicting stock market excess returns during different sample periods Dependent variable: Quarterly excess log return for the CRSP value-weighted index at t+1 Sample period:
1963 Q2 1984 Q1
1984 Q2 2005 Q1
1963 Q2 1973 Q3
1973 Q4 1984 Q1
1984 Q2 1994 Q3
1994 Q4 2005 Q1
1976 Q1 2005 Q1
2 R (%) N
Notes: The dependent variable is the log return for the CRSP value-weighted index minus the log return for the three month T-bill for each particular sample period. Average correlation is the cross-sectional average of the pairwise correlation of daily returns during each quarter for all pairs of the 500 largest (by market capitalization) exchange-traded stocks in the United States at the end of that quarter. This cross-sectional average is value-weighted based on the end-of-quarter market capitalization. The last column excludes all returns up to and including the first oil price shock. Newey-West t-statistics with 6 lags are reported in brackets beneath the coefficient
Table 6: Predicting stock market excess returns at different frequencies Dependent variable: Excess log return for the CRSP value-weighted index at t+1 Monthly
R2 (%) N
Notes: The dependent variable is the log return for the CRSP value-weighted index minus the log return for the riskless asset of the maturity matching the stock market return interval for each particular frequency. Average correlation is the cross-sectional average of the pairwise correlation of daily returns during each quarter for all pairs of the 500 largest (by market capitalization) exchange-traded stocks in the United States at the end of that quarter. This cross-sectional average is value-weighted based on the end-of-quarter market capitalization. For each frequency the regressions use non-overlapping observations of the dependent variable. The 6-monthly regression is from 1963 Q3 to 2005 Q3. The biennial regression is from 1966 Q4 to 2004 Q4. Newey-West t-statistics with 6 lags are reported in brackets beneath the coefficient estimates.
Table 7: Predicting stock market excess returns with controls Dependent variable: Quarterly excess log return for the CRSP value-weighted index at t+1
(1) 0.010 [1.681]
(2) 0.004 [0.550]
(3) 0.011 [1.778]
(4) 0.007 [0.787]
0.328 [0.891] -0.949 [2.332]
0.154 [0.412] -0.747 [1.837]
1.468 [2.714] 0.574 [2.058]
R2 (%) N
(5) -0.037 [2.284]
Notes: The dependent variable is the log return for the CRSP value-weighted index minus the log return for the three month T-bill from 1963 Q2 to 2005 Q1. Average correlation is the cross-sectional average of the pairwise correlation of daily returns during each quarter for all pairs of the 500 largest (by market capitalization) exchange-traded stocks in the United States at the end of that quarter. This cross-sectional average is value-weighted based on the end-of-quarter market capitalization. rem is the dependent variable lagged one quarter. cay is from Lettau and Ludvigson (2001) and is estimated using their full sample from 1951 Q4 to 2005 Q4. tp is the log ratio of total stock market cashflows (dividends plus share repurchases minus seasoned equity offerings) divided by the stock market price from Lei (2006). rb is the relative bill rate, the log 1 month bill yield minus a 12-month triangular weighted average of its previous values. The variables are all observed from 1963 Q1 to 2004 Q4. Newey-West t-statistics with 6 lags are reported in brackets beneath the coefficient estimates.
Table 8: Out-of-sample forecasts for the quarterly excess log stock market return Forcasting Variable:
Sample period: Out-of-sample R
∆RMSE Number of forecasts
Value-weighed average correlation
Equal-weighted average correlation
Stock market Variance
1963 Q2 to 2005 Q1
1963 Q2 to 2005 Q1
1963 Q2 to 2005 Q1
1926 Q2 to 2005 Q1
1927 Q4 to 2005 Q1
1952 Q1 to 2005 Q1
Notes: Each column above tests the ability of a particular variable to forecast the quarterly log excess return on the CRSP value weighted index in the subsequent period. Following the procedure in Goyal and Welch (2006), the forecasting power of each variable is compared to the forecasting power of the historical sample mean of the return using only information available at the date of the forecast. A positive out-of-sample R2 and difference in root mean squared forecast error (∆RMSE) indicate superior forecasting ability relative to the historical sample mean of stock market returns. The first forecast is made in 1983 (20 years after the beginning of the sample for average correlation) and we allow each variable to estimate the relevant predictive relationship for the longest sample period possible before making forecasts. The value-weighted average correlation is the cross-sectional average of the pairwise correlation of daily returns during each quarter for all pairs of the 500 largest (by market capitalization) exchange-traded stocks in the United States at the end of that quarter. This cross-sectional average is value-weighted based on the end-of-quarter market capitalization. The equal-weighted average correlation is an analogous cross-sectional average constructed using equal weights. Average variance is the cross-sectional average of the quarterly variance of daily returns for the same 500 large stocks (equal-weighted). cay is from Lettau and Ludvigson (2001) and is estimated using their full sample from 1951 Q4 to 2005 Q4. tp is the log ratio of total stock market cashflows (dividends plus share repurchases minus seasoned equity offerings) divided by the stock market price from Lei (2006).
Table 9: Predicting stock market return and bond return covariance Dependent variable: Sample covariance of daily stock market returns and daily 10 year bond returns at t+1 (1)
Stock market variance Stock-bond covariance
2 R (%) N
Notes: The dependent variable is the sample covariance of CRSP value-weighted daily index return with the daily holding period return on 10year US government bonds ("bond covariance"), estimated quarterly from 1963 Q1 to 2005 Q1. Average correlation is the cross-sectional average of the pairwise correlation of daily returns during each quarter for all pairs of the 500 largest (by market capitalization) exchangetraded stocks in the United States at the end of that quarter. This cross-sectional average is value-weighted based on the end-of-quarter market capitalization. Average variance is the cross-sectional average of the quarterly variance of daily returns for the same 500 large stocks (equal-weighted). Newey-West t-statistics with 6 lags are reported in brackets beneath the coefficient estimates. Stock market variance is the sample variance of the CRSP daily index return. The variances and covariances of daily returns are multiplied by 63, the average number of trading days in one quarter, in order to express these sample moments in terms of the same time interval as quarterly excess returns.
Table 10: Simulated variance-in-mean regressions For each day in quarter t+1, the vector of daily returns rd is distributed N(ψ0+ψ1σ tρt, σ tΩt) where ψ0=-0.00033 and ψ1=6 2
Moments of simulated estimates
Average relative deviation from assigned value (%)
Dependent variable: market variance at t Regressor
Dependent variable: quarterly excess log return at t+1
Pr(tstat > 1.96)
Mean(R ) (%)
(Average correlation)* (Average variance)
Pr(tstat > 1.96)
Mean(R ) (%)
Notes: Table 10 presents results from 250 simulations of the time-series behavior of stock returns. For 500 symmetric stocks (identical means, variances, and pairwise correlations) we draw 63 daily returns for each stock under the assumption that the cross-section of returns are conditionally normally distributed as described in this table's heading. For each day the market return is the equal-weighted average of the 500 individual stock returns and the quarterly log stock market return is the sum of the log daily market returns within the quarter. The riskless rate is normalized to be zero so that excess returns are equal to raw returns. The parameters σ2t and ρt are the observed estimates of daily average return variance and average correlation respectively for quarter t. The correlation matrix for daily returns Ωt has the observed average correlation ρt on all of its off-diagonal elements. We follow this procedure for 168 quarters for each quarterly return time-series. The variances and covariances of daily returns are multiplied by 63, the average number of trading days in one quarter, in order to express these sample moments in terms of the same time interval as quarterly returns. We obtain quarterly simulated estimates of average correlation, average variance, and market variance as well as a time series of assigned "true" values for each of these parameters. We choose ψ0 and ψ1 to match the observed unconditional sample mean and standard deviation of the quarterly stock market log excess return in Table 1. We report the mean of the average simulated log excess return, standard deviation and first autocorrelations of quarterly market returns together with the average relative deviation, defined as the mean abs(estimate - true value)/(true value), for market variance, average correlation and average variance. We estimate OLS regressions, for each simulation, of contemporaneous estimated market variance on each of estimated average variance, average correlation and their product separately. We report the means of the resulting simulated coefficient estimates, the t-statistics using White standard errors, and the R2 of the regressions. Similarly, we regress the simulated log market excess return on each of estimated average variance, average correlation and market variance and report the same statistics.