A Vector Error-Correction Forecasting Model of the U.S. Economy: Comment

William D. Lastrapes Department of Economics Terry College of Business University of Georgia Athens, Georgia 30602

June 12, 2001

The presence of cointegration in a system of non-stationary variables clearly has the potential to improve unconditional macroeconomic forecasts – the tendency for drifting variables to gravitate toward a long-run equilibrium is information that should help track the future paths of these variables. But does such information help in practice? This is an empirical question that, despite previous work in the area, has not been completely answered. Anderson, Hoffman and Rasche (2002), AHR hereafter, have conducted a wellmotivated and interesting study of this issue. They compare the forecasting performance of a vector error correction model (VECM) that imposes theoretically reasonable cointegration restrictions, to naive random walks and independent published forecasts. AHR conclude that, for the handful of macro variables they consider and the particular forecasting period, the VECM “performs well relative to the published forecasts ... and is superior to the naive random walk.” Before I comment on the validity of this conclusion, let me briefly summarize their approach. The VECM contains proxies for output (real GDP), inflation (based on the CPI and the GDP deflator), real money balances, and interest rates (short- and long-term). AHR presumably choose these variables because of their relevance to policy analysis and to the state of the macro economy, but also because they allow for four reasonable (and empirically supported) cointegrating vectors (assuming I(1) univariate processes): a real money demand relationship, stationarity of ex post real interest rates (using both measures of inflation and the long-term interest rate), and stationarity of the term structure spread. AHR estimate the long interest rate elasticity of money demand only; all other coefficients in the cointegrating vectors are imposed a priori. The authors recursively estimate the VECM (so that the forecasts are basically ex ante) and compute out-of-sample 1-quarter to 16-quarter ahead forecast errors over the period 1989 to 1997 and over various subperiods. They assess forecast performance of the VECM, relative to the alternatives, by comparing root mean squared errors (RMSE) for

individual variables and the generalized forecast error second moment (GFESM), a systemwide statistic proposed by Clements and Hendry (1993). The VECM outperforms the random walk models according to the system-wide criterion, but generally does no better (though no worse) than any of the alternative forecasting methods considered according to the equation-by-equation RMSE comparisons. This finding is used to support the authors’ (imprecise) claim that the VECM “perhaps offers just the right balance as an econometric model for business forecasting.” In making this claim, AHR place a lot of weight on the GFESM as an appropriate criterion for judging relative forecast accuracy. But what loss function justifies using a system-wide criterion like GFESM to assess the usefulness of the forecasts? My guess is that consumers of these forecasts (policy-makers?) would place greater weight on output and inflation errors in their loss functions, in which case it is not clear that a properly weighted system-wide criterion would come down on the side of the VECM. Since at least some of the variables are presumably included simply as information variables not of independent forecasting interest (e.g. the money supply), reliance on the GFESM criterion for many forecasting purposes may be inappropriate. Note, for example, that the forecasting improvement in the VECM as measured by the GFESM comparison could be due in large part to its RMSE effectiveness in forecasting real money balances at short horizons (see Tables 6, 7 and 8). But if the forecaster’s main concern is with predicting output or output growth, the apparent system-wide gain from using the VECM may be practically irrelevant in light of its poor individual performance with respect to these two variables. Perhaps there is value in knowing how the forecast errors are correlated across variables, which is accounted for by the GFESM, but a careful specification of the loss function (or at least a check of the sensitivity of the comparisons to different loss functions) would provide a more useful assessment of forecast performance.1 1

Most non-Bayesian forecasting studies ignore loss functions; typically, they implicitly assume quadratic loss. –2–

If we then discount comparisons and assessment using the system-wide criterion and focus on the effectiveness of individual variable forecasts, the authors’ claim of VECM superiority relative to the random walk is substantially weakened. As with most forecasting studies, AHR conduct a horse-race, and make judgements of superiority based on the “winner” (the forecast that is symmetrically “least-wrong” on average for a given variable and forecast horizon). But is the horse-race strategy itself superior to its alternatives? Suppose, for example, that we rephrase the empirical question from “Does the VECM win?” to “Does the VECM contain additional information (over its competitors) that can improve forecasts?” Consider the following regression, proposed by Fair and Shiller (1990): yt+k − yt = α0 + α1 (ˆ y1,t+k − yt ) + α2 (ˆ y2,t+k − yt ) + t+k , where yˆi,t+k is the point forecast of model i, based on information known at time t, for the scalar random variable yt+k . For example, forecast 1 might rely on the VECM, while forecast 2 might be a published forecast. For any given forecast horizon k, this regression can be run over the out-of-sample period, using (if desired) updated estimates of the forecast models to compute the forecasts. If, say, model 1 contains information useful for forecasting future values of y that is not contained in model 2, then α1 should be significantly different from zero, even if model 2 has a lower RMSE than model 1. Such an empirical strategy is better suited to answer the latter question above than is the RMSE horse-race.2 Tables 1 and 2 give some illustrative results based on the Fair-Shiller approach when we consider forecasts of real output. I have used essentially the same data as AHR (real GDP, M1, CPI, GDP deflator, the fed funds rate, and the constant maturity 10-year t-bill 2

As noted by Fair and Shiller (1990), this approach to evaluating forecasts is related to the literature on optimal combination of forecasts, and the encompassing literature. Lamoureux and Lastrapes (1993) use a similar strategy to evaluate forecasts of stock return volatility. –3–

rate, all from FRED), and have estimated five different time-series models: an unrestricted VAR, a first-differenced VAR, a VECM in which four cointegrating vectors are estimated, a VECM in which the four cointegrating vectors are imposed a priori (including the interest elasticity of money demand, which I set to 0.085, the overall estimate over the full period from AHR), and a simple AR(1) with constant for each variable in the system (essentially a random walk). The initial estimation period is 1957:I to 1988:IV; one-step and four-step ahead forecasts are generated over the out-of-sample period from 1989:I to 2000:IV by recursively updating the estimates each period.3 Table 1 contains the RMSE’s for the ex ante, out-of-sample forecasts. The firstdifferenced VAR wins the one-step race, but the univariate AR wins for the four-step horizon by a wide margin. In each case, the AR model has a lower RMSE than the VECM’s. (How “significant” or “important” this difference is cannot be determined, which is another of the obvious drawbacks of the RMSE strategy.) But now consider Table 2, in which I report results from regressions similar to the specification above. To clarify the role of each multivariate model, I have run simple regressions (i.e. yt+k −yt = α0 +α1 (ˆ y1,t+k −yt )+t+k ). In effect, the regressions allow a test of the hypothesis that a particular multivariate model contains no additional forecasting information (out-of-sample) over the simple random walk. In all cases, α1 is significantly positive, indicating that the vector models contain information for forecasting GDP not contained in the random walk, even though the latter might win the RMSE horse race (as it does for the four-step horizon).4 It is of interest to run similar regressions for the other variables deemed important for forecasting, especially inflation, and to compare the multivariate models among themselves and to the published forecasts using this approach. But Table 2 does suggest that the 3

The systems include 6 common lags and a constant. I did not include the 1979 dummy used by AHR. 4 I used robust standard errors to correct for the implied moving average process in the four-step regressions and potential heteroskedasticity. –4–

VECM could have forecasting value over the random walk despite not winning the RMSE comparisons. While this result could potentially support the VECM as a useful forecasting tool, the author’s might need to modify their claim of superiority of the VECM. There are other ways in which the paper could better improve our understanding of the forecasting benefits of vector error correction models. 1) It is puzzling to me why AHR have not compared the VECM to other multivariate models, since the focus of the paper is on the value of imposing cointegration restrictions. Without considering other multivariate models, forecasting gains could be due solely to the information brought in by including the other variables in the system. 2) AHR follow most other studies of this type by focusing on only one moment of the probability distribution describing the random variable being forecast – the mean as “point forecast.” At least, comparisons of confidence intervals across forecast models would be of interest. 3) Finally, I would like to see more consideration of why the forecast horizon matters. For example, why does the VECM forecast inflation at short-horizons better than the random walk, but the opposite is true for long-horizons. Why does the VECM not yield better performance at long-horizons, given that long-run restrictions are imposed? In summary, Anderson, Hoffman and Rasche have written a paper that sheds some light on the practical benefits of using VECM models for macroeconomic forecasting. However, I am not convinced that they have made a sufficient case for their conclusion that the VECM is superior to the alternatives they consider, and that it provides the right balance as an econometric model for forecasting. More likely, further research will show that the VECM will be one tool among many that can help in forecasting the economy.

–5–

References Anderson, Richard G., Dennis L. Hoffman and Robert H. Rasche, “A Vector ErrorCorrection Forecasting Model of the U.S. Economy” Journal of Macroeconomics Clements, M.P. and David F. Hendry, “On the Limitations of Comparing Mean Square Forecast Errors” Journal of Forecasting 1993, 617-37. Fair, Ray C. and Robert J. Shiller, “Comparing Information in Forecasts from Econometric Models” American Economic Review 80, June 1990, 375-89. Lamoureux, Christopher G. and William D. Lastrapes, “Forecasting Stock Return Variance: Toward an Understanding of Stochastic Implied Volatilities” Review of Financial Studies 1993, 293-326.

–6–

Table 1. RMSE over out-of-sample period 1989:I to 2000:IV

Model

1-step ahead

4-steps ahead

VAR

0.00897

0.02960

D-VAR

0.00805

0.03012

VECM1

0.00876

0.02998

VECM2

0.00923

0.03486

AR

0.00832

0.02497

Notes: VAR is an unrestricted VAR, D-VAR is a first-differenced VAR, VECM1 is the estimated VECM, VECM2 restricts cointegrating vectors by theory, and AR is the univariate autoregressive model.

–7–

Table 2. Out-of-sample forecast regressions yt+k − yt = α0 + α1 (ˆ y1,t+k − yt )

One-step ahead yˆ VAR

D-VAR

VECM1

VECM2

coefficient estimate t-statistic α0

0.005

3.47

α1

0.401

2.49

α0

0.003

1.87

α1

0.538

2.95

α0

0.004

3.11

α1

0.423

2.67

α0

0.005

4.35

α1

0.438

2.93

Four-step ahead yˆ VAR

D-VAR

VECM1

VECM2

coefficient estimate t-statistic α0

0.021

4.97

α1

0.375

2.81

α0

-0.001

-0.28

α1

0.584

8.88

α0

0.020

4.62

α1

0.334

2.59

α0

0.025

6.62

α1

0.327

2.85

–8–