Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Applied time-series analysis Part II Robert M. Kunst
[email protected] University of Vienna and Institute for Advanced Studies Vienna
November 29, 2011
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Outline
Introduction and overview ARMA processes Time series with a trend Cointegration
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Trending time series Many economic time-series variables do not look stationary, as they indicate monotonic changes in their mean, so-called ‘trends’. Such variables should be transformed, before one may treat them as ‘stationary’. Two classes of transformations are often considered: 1. Fitting simple (for example, linear) functions of time τ (t) to the data and subtracting them. This may yield stationary ˜t = Xt − τ (t); X 2. First differences ∆Xt = Xt − Xt−1 may also be stationary. Transformations of the first kind reportedly dominated the literature before the days of Box and Jenkins who preferred transformations of the second kind. Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Trend stationarity and difference stationarity
If Xt is not stationary but Xt − τ (t) is stationary for a smooth and monotonic trend function τ (t), Xt is called trend stationary. If Xt is not stationary but ∆Xt = Xt − Xt−1 is stationary, Xt is called ‘difference stationary’, first-order integrated, or I(1). If ∆Xt is a stable ARMA(p, q) process, Xt can also be called ARIMA(p, 1, q).
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
0
5
10
15
20
Trend stationarity
0
50
100
150
200
10 trajectories from the trend-stationary process Xt = 0.1t + Yt , Yt = 0.5Yt−1 + εt , with (εt ) iid N(0,1). Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
0
50
100
Difference stationarity
0
50
100
150
200
Time
10 trajectories from the integrated process ∆Xt = 0.5 + 0.5∆Xt−1 + εt , with (εt ) iid N(0,1). Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Discrimination of trend and difference stationarity Wrong transformations are no good. First differences of trend-stationary processes are stationary but often non-invertible. Fitting trends to difference-stationary processes does not yield stationary residuals. How distinguish the two generating types of model? 1. Visual analysis (Box and Jenkins): compare residuals from trend fitting and differencing, prefer the transformation with simpler ACF; 2. Unit-root tests (since 1979). Warning: usual information criteria do not work for this decision problem. Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Dickey-Fuller test DF-0: the idea Assume the AR(1) model Xt = φXt−1 + εt . For the test problem with the null and alternative hypotheses H0 : φ = 1,
HA : φ ∈ (−1, 1),
one may consider the test statistic DF0 =
φˆ − 1 , ˆ σ ˆ (φ)
ˆ is the usual standard error of the coefficient from a where σ ˆ (φ) regression output. Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Properties of the DF-0 statistic Under the null hypothesis φ = 1, the random walk, will the coefficient estimate φˆ converge with great speed to the true value φ = 1: R1 B(ω)dB(ω) ˆ T (φ − 1) → 0R 1 2 0 B (ω)dω
in distribution, with a non-standard limit law expressed in the form of integrals over functions of Brownian motion. Also the test statistic DF0 has a related limit under H0 : R1 B(ω)dB(ω) DF0 → R01 ( 0 B 2 (ω)dω)0.5 Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Brownian motion Definition Brownian motion or a Wiener process is a real-valued stochastic process (Xt ) defined on T = [0, 1] satisfying 1. X0 = 0; 2. for any s1 ≤ t1 ≤ s2 ≤ t2 ≤ . . . ≤ tn , the random variables Xt1 − Xs1 , . . . , Xtn − Xsn are independent; 3. for any s < t, the r.v. Xt − Xs has a normal distribution with mean 0 and variance (t − s)σ 2 ; 4. the paths (trajectories) are continuous.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
10
x[1:1000]
0
−5
2
0
4
5
6
x[1:100]
8
15
10
20
12
14
The shape of Brownian motion
0
20
40
60
80
100
0
200
400
600
800
1000
Time
0
x[1:1e+05]
x[1:10000] 0
−300
−200
50
−100
100
100
200
150
Time
0
2000
4000
6000 Time
8000
10000
0
e+00
2
e+04
4
e+04
6
e+04
8
e+04
1
e+05
Time
Identifying the sample end with ‘1’ and the start with ‘0’, generated random walks with increasing T give an impression of Brownian motion that evolves for T → ∞. Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Problems with the DF-0 test
◮
The distributions of T (φˆ − 1) and of DF0 are non-standard under the null: tabulated significance points must be used;
◮
neither the null of a pure random walk nor the alternative of an AR(1) variable with zero mean are very interesting in practice: deterministic terms must be added, and these modify the distributions;
◮
the null distribution is very sensitive to autocorrelation: higher-order models should be considered.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Dickey-Fuller test DF–µ Assume the AR(1) model with a constant Xt = µ + φXt−1 + εt . For the test problem with the null and alternative hypotheses H0 : φ = 1,
HA : φ ∈ (−1, 1),
one may consider the test statistic DFµ =
φˆ − 1 , ˆ σ ˆ (φ)
ˆ is the usual standard error of the coefficient from a where σ ˆ (φ) regression output. Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Properties of the DF–µ statistic Under the null hypothesis φ = 1 and assuming µ = 0, the pure random walk, will the coefficient estimate φˆ converge with great speed to the true value φ = 1: R1 B1 (ω)dB(ω) ˆ T (φ − 1) → 0R 1 2 0 B1 (ω)dω
in distribution, with a non-standard limit law expressed in the form of integrals over functions of mean-corrected Brownian motion B1 . Even the test statistic DFµ has a related limit under H0 : R1
B1 (ω)dB(ω) DFµ → R01 ( 0 B12 (ω)dω)0.5 Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Usage of the DF–µ test
The null hypothesis of a pure random walk and the alternative of a stable AR(1) model with general mean can be interesting for non-trending variables. They are still not representative of the difference-stationary and trend-stationary models.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Dickey-Fuller test DF–τ Assume the AR(1) model with a constant Xt = µ + τ t + φXt−1 + εt . This model is equivalent to Xt = a + bt + Yt , Yt = φYt−1 + εt . For the test problem with the null and alternative hypotheses H0 : φ = 1,
HA : φ ∈ (−1, 1),
one may consider the test statistic DFτ =
φˆ − 1 , ˆ σ ˆ (φ)
ˆ is the usual standard error of the coefficient from a where σ ˆ (φ) regression output. Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Properties of the DF–τ statistic Under the null hypothesis φ = 1 and assuming τ = 0, the potentially drifting random walk, will the coefficient estimate φˆ converge with great speed to the true value φ = 1: R1 B2 (ω)dB(ω) ˆ T (φ − 1) → 0R 1 2 0 B2 (ω)dω
in distribution, with a non-standard limit law expressed in the form of integrals over functions of trend-corrected Brownian motion B2 . Even the test statistic DFτ has a related limit under H0 : R1 B2 (ω)dB(ω) DFτ → R01 ( 0 B22 (ω)dω)0.5
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
−30
−2
−20
0
−10
0
2
10
20
4
Trend-corrected Brownian motion
20
40
60
80
100
4000
6000
8000
10000
0
200
400
600
800
1000
0e+00
2e+04
4e+04
6e+04
8e+04
1e+05
−200
−40
−100
−20
0
0
20
100
40
200
0
0
2000
Approximating de-trended Brownian motion by random walks with extracted linear time trends and increasing T . Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Handling autocorrelation by augmenting All DF tests (DF–µ and DF–τ ) are sensitive to autocorrelation of ut in the basic regression Xt = dt + φXt−1 + ut or its equivalent form ∆Xt = dt + ϕXt−1 + ut . This autocorrelation invalidates the asymptotic distribution. A remedy is adding lags of ∆Xt as regressors, such that errors are now approximately white noise: ∆Xt = dt + ϕXt−1 +
p−1 X
ψj ∆Xt−j + ut .
j=1
Statistics such as tϕ will again have the DF distributions. Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Why use lagged differences for augmentation?
◮ ◮
◮
Lagged differences are stationary both under H0 and HA ; P any autoregression Xt = pj=1 φj Xt−j + εt can be transformed P to the model ∆Xt = ϕXt−1 + p−1 j=1 ψj ∆Xt−j + εt ; ϕ = 0 iff there is a unit root in the characteristic polynomial Φ(z).
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
How find the augmenting lag order?
Two recommended suggestions: 1. Fit autoregressive models with appropriate deterministic terms to the data, choose the one (p) with lowest AIC (or BIC). Use p − 1 lags for the differences ∆Xt ; 2. determine p as a (slowly increasing) function of the sample size. It is not recommended to use different lag selection procedures, such as portmanteau (Q) tests on residuals.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Alternative unit-root tests Three types of alternative tests deserve attention: 1. Phillips-Perron test is a variant of the DF test, with the same asymptotic distribution and no recognizable differences in general performance; 2. various modifications of the DF test are available, some of them with power advantages (Elliot, Rothenberg, Stock); 3. the KPSS (Kwiatkowski, Phillips, Schmidt, Shin) test uses I(0) as the null and I(1) as the alternative: generally less reliable than DF test, interesting additional tool.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Cointegration: the idea Time-series variables are called ‘cointegrated’ if they are I(1) (and thus non-stationary) but linear combinations are stationary (I(0)). For example, the difference X − Y may be stationary, even though both X and Y are I(1). This feature is important, as: ◮
Technically, using first differences only of cointegrated variables in a joint model is inefficient and yields bad longer-run predictions, error-correction models must be constructed;
◮
cointegrating relations indicate economically interesting (static) ‘equilibrium’ conditions in dynamic systems.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
4.4
4.6
4.8
5.0
A prototypical example: consumption and income
1975
1980
1985
1990
1995
2000
2005
Austrian private consumption and disposable income for the years 1976–2009, in logarithms. Series are trending but difference remains roughly constant. Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
A definition for cointegration of two series Definition If two processes Xt , Yt are each ARIMA(p, 1, q) and if there exists a linear combination Zt = β1 Xt + β2 Yt with β1 6= 0 and β2 6= 0 such that Zt is stationary, then Xt and Yt are called cointegrated, and (β1 , β2 )′ is called the cointegrating vector. Remark. The definition can be extended to I(1) processes, which are slightly more general than ARIMA(p, 1, q). Example. For consumption and income in logarithms, β = (1, −1)′ could be the cointegrating vector. For a version without logarithms, β = (1, −0.9)′ may be the cointegrating vector.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
The general definition for n variables Definition Given an n–dimensional vector process Xt = (X1t , . . . , Xnt )′ , suppose that ∆Xt = (∆X1t , . . . , ∆Xnt )′ is stationary, whereas at least one component Xjt is I (1), and that a linear combination with at least one βk 6= 0 Zt = β1 X1t + . . . + βn Xnt = (β1 , . . . , βn )′ Xt is stationary (or I (0)). Then, Xt is called cointegrated, in symbols CI (1, 1), and the vector β is a cointegrating vector.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Another empirical example: interest rates
Clive Granger and Heather Anderson considered 11 interest rates with terms to maturity of one to eleven months. They showed these are all cointegrated and individually I(1). X (j) − X (k) are stationary for any pairs of maturities j, k. All vectors of the form (0, 0, 1, 0, 0, −1, 0, . . .)′ are cointegrating vectors. Remark. A deeper problem is whether we can accept that interest rates are I(1).
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Procedures for estimating and testing cointegration For the two main tasks, i.e. for statistical decisions on cointegration and estimating cointegrating vectors, there exist two classes of methods: 1. Methods based on the cointegrating regression (simple and inefficient EG–2step, and other techniques, some of them efficient); 2. Methods based on system estimation (VAR-based method of Johansen and others: efficient procedures). EG–2 and Johansen’s VAR method are by far the most popular estimation and inference procedures.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Engle-Granger two-step method The EG–2 method is the most popular single-equation method for estimating and testing cointegration. It involves inefficient and efficient components. It is called EG–2step, as it has two steps: 1. A cointegrating regression: an I(1) variable Y is regressed on other I(1) variables X1 , . . . , Xk . If Y and some of the Xj cointegrate, residuals should be nearly I(0). Unit-root tests on residuals should reject unit roots. 2. An error-correction model: first differences ∆Yt are regressed on lags of first differences ∆Yt−l and ∆Xj,t−l and on the residuals from step 1.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
The cointegrating regression
If (Y , X1 , . . . , Xk )′ cointegrate with cointegrating vector (β0∗ , β1∗ , . . . , βk∗ )′ and β0∗ 6= 0 and at least one of the βj∗ 6= 0 for j = 1, . . . , k, then the coefficient estimates in the regression Yt = µ + β1 X1,t + . . . + βk Xk,t + ut converge to βj = −βj∗ /β0∗ fast, in the sense that T (βˆj − βj ) → D for some distribution D. Even for the true values, the residuals are typically not white noise.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Testing for cointegration using residuals If ut is I(1), the ‘cointegrating’ regression is not balanced. The most customary test is an (augmented) DF test on the residuals uˆt . ◮
If the DF test on uˆt accepts its null, Y and (X1 , . . . , Xk )′ do not cointegrate. Abandon the project.
◮
If the DF test on uˆt rejects, Y and (X1 , . . . , Xk )′ cointegrate. Proceed to next step.
Warning: The null distribution of this DF test is not the usual tabulated DF distribution, automatic significance points are invalid. Quantiles were tabulated first by Phillips and Ouliaris.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Modelling following the cointegrating regression If ut really cointegrates, uˆt = Yt − µ ˆ − βˆ1 X1,t − . . . − βˆk Xk,t estimates the disequilibrium term. The full description of the dynamic behavior of Y follows from the error-correction model ∆Yt = µ0 + αˆ ut−1 +
p X j=1
γ0,j ∆Yt−j +
pl k X X
γl,j ∆Xl,t−j + εt ,
l=1 j=1
where lag orders p,pl are set such that εt is white noise.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Example: consumption and income Suppose we think that consumption C and income Y are cointegrated. 1. Run DF tests on C and Y : both should be I(1); 2. Regress C on Y : yields residuals uˆ; 3. Run DF tests on uˆ: use quantiles from Phillips and Ouliaris: should be I(0); 4. Regress ∆Ct on uˆt−1 and on lags of ∆Ct and of ∆Yt . 5. Complete the model by regressing ∆Yt on uˆt−1 and on lags of ∆Ct , ∆Yt .
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Problems of the EG–2 procedure 1. EG–2 is not efficient in the statistical sense; 2. EG–2 requires that the dependent variable Y and at least some of the explanatory variables are I(1): requires pre-testing; 3. EG–2 fails if Y is not cointegrated with X1 , . . . , Xn but the explanatory variables are cointegrated among themselves. 4. By construction, EG–2 expresses a causal direction from X1 , . . . , Xn to Y . This does not correspond to the cointegration concept, where all variables are jointly endogenous.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Johansen’s system estimation
Johansen’s procedure is the most popular efficient procedure for estimating and testing in cointegrating models. It builds on vector autoregressions (VAR). The Johansen procedure may be interpreted as a multivariate Dickey-Fuller test.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Multivariate Dickey-Fuller models Any vector autoregression Xt = µ + Φ1 Xt−1 + . . . + Φp Xt−p + εt can be re-written as ∆Xt = µ + ΠXt−1 + Ψ1 ∆Xt−1 + . . . + Ψp−1 ∆Xt−p+1 + εt , with identical errors and a one-to-one correspondence between parameters in both forms.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
The long-run impact matrix Π O.c.s. that, if X = (X1 , . . . , Xn )′ is cointegrated, then Π in ∆Xt = µ + ΠXt−1 + Ψ1 ∆Xt−1 + . . . + Ψp−1 ∆Xt−p+1 + εt , is singular and can be represented as Π = αβ ′ , with (n × r )–matrices α, β of rank r . The r columns of β are cointegrating vectors. These are not unique, any linear combination will do.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Loading coefficients α
The cointegrating vectors in β describe equilibrium relations. The coefficients in α describe how the variables react to deviations from equilibrium. Typically, deviations tend to be corrected: error correction. β sets the target, α does the work. The coefficients in α are called loading coefficients or adjustment coefficients.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Johansen procedure step by step The procedure uses the following steps: 1. All variables Xj , j = 1, . . . , n should be either I(1) or I(0); 2. Determine the VAR lag order p: multivariate information criteria; 3. Determine the cointegrating rank r by sequences of hypothesis tests: estimate β; 4. Estimate the full EC-VAR model given p and r to estimate α and all Ψj .
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
The test sequence In order to determine the ‘cointegrating’ rank of Π, use the sequence: 1. Test for H00 : r = 0 versus H0A : r > 0; if accepted stop: no cointegration; 2. Test for H10 : r ≤ 1 versus H1A : r > 1; if accepted stop: r = 1; 3. etc. until Hn−1,0 : r ≤ n − 1 versus Hn−1,A : r = n; if rejected be surprised: the system is stationary. Most authors prefer this testing ‘up’ to testing ‘down’.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
The mathematical principle behind the scene
The Johansen procedure builds on canonical correlations between the n variables in X and in ∆X . If a linear relationship between the stationary ∆X and a linear combination of the integrated X is strong, this combination is a cointegrating vector. Canonical correlation problems are solved by eigenvector and eigenvalue analysis. Eigenvectors corresponding to non-zero eigenvalues are cointegrating vectors.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Deterministic terms in the Johansen procedure Multivariate unit-root tests are affected by the presence of intercepts, trends. There are many combinations, only two are really relevant: 1. If some variables are trending, use the standard model with an unrestricted intercept: do not include trends; 2. If no variable is trending, use the model with restricted intercept ∆Xt = α(µ∗ + βXt−1 ) + Ψ1 ∆Xt−1 + . . . + Ψp−1 ∆Xt−p+1 + εt . The two versions have separate tables of significance points.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
Introduction and overview
ARMA processes
Time series with a trend
Cointegration
Example: consumption and income If C and Y cointegrate, the rank r should be 1. The system can be written as ∆Ct µ1 α1 Ct−1 = + [1, −1] ∆Yt µ2 α2 Yt−1 Ψ11 Ψ12 ∆Ct−1 ε + + 1,t Ψ21 Ψ22 ∆Yt−1 ε2,t α1 < 0 corrects previous over- or under-consumption by changing C . α2 is often small. Ψ describes short-run fluctuations and inertia of consumer behavior.
Applied time-series analysis Part II
University of Vienna and Institute for Advanced Studies Vienna
