Springer Texts in Business and Economics

More information about this series at http://​www.​springer.​com/​series/​10099

Klaus Neusser

Time Series Econometrics

Klaus Neusser

Bern, Switzerland

ISSN 2192-4333e-ISSN 2192-4341

ISBN 978-3-319-32861-4e-ISBN 978-3-319-32862-1

DOI 10.1007/978-3-319-32862-1

Library of Congress Control Number: 2016938514

© Springer International Publishing Switzerland 2016

Springer Texts in Business and Economics

This Springer imprint is published by Springer Nature

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made.

Printed on acid-free paper

The registered company is Springer International Publishing AG Switzerland

Preface

Over the past decades, time series analysis has experienced a proliferous increase of applications in economics, especially in macroeconomics and finance. Today these tools have become indispensable to any empirically working economist. Whereas in the beginning the transfer of knowledge essentially flowed from the natural sciences, especially statistics and engineering, to economics, over the years theoretical and applied techniques specifically designed for the nature of economic time series and models have been developed. Thereby, the estimation and identification of structural vector autoregressive models, the analysis of integrated and cointegrated time series, and models of volatility have been extremely fruitful and far-reaching areas of research. With the award of the Nobel Prizes to Clive W. J. Granger and Robert F. Engle III in 2003 and to Thomas J. Sargent and Christopher A. Sims in 2011, the field has reached a certain degree of maturity. Thus, the idea suggests itself to assemble the vast amount of material scattered over many papers into a comprehensive textbook.

The book is self-contained and addresses economics students who have already some prerequisite knowledge in econometrics. It is thus suited for advanced bachelor, master’s, or beginning PhD students but also for applied researchers. The book tries to bring them in a position to be able to follow the rapidly growing research literature and to implement these techniques on their own. Although the book is trying to be rigorous in terms of concepts, definitions, and statements of theorems, not all proofs are carried out. This is especially true for the more technically and lengthy proofs for which the reader is referred to the pertinent literature.

The book covers approximately a two-semester course in time series analysis and is divided in two parts. The first part treats univariate time series, in particular autoregressive moving-average processes. Most of the topics are standard and can form the basis for a one-semester introductory time series course. This part also contains a chapter on integrated processes and on models of volatility. The latter topics could be included in a more advanced course. The second part is devoted to multivariate time series analysis and in particular to vector autoregressive processes. It can be taught independently of the first part. The identification, modeling, and estimation of these processes form the core of the second part. A special chapter treats the estimation, testing, and interpretation of cointegrated systems. The book also contains a chapter with an introduction to state space models and the Kalman filter. Whereas the books is almost exclusively concerned with linear systems, the last chapter gives a perspective on some more recent developments in the context of nonlinear models. I have included exercises and worked out examples to deepen the teaching and learning content. Finally, I have produced five appendices which summarize important topics such as complex numbers, linear difference equations, and stochastic convergence.

As time series analysis has become a tremendously growing field with an active research in many directions, it goes without saying that not all topics received the attention they deserved and that there are areas not covered at all. This is especially true for the recent advances made in nonlinear time series analysis and in the application of Bayesian techniques. These two topics alone would justify an extra book.

The data manipulations and computations have been performed using the software packages EVIEWS and MATLAB. ¹ Of course, there are other excellent packages available. The data for the examples and additional information can be downloaded from my home page www.neusser.ch. To maximize the learning success, it is advised to replicate the examples and to perform similar exercises with alternative data. Interesting macroeconomic time series can, for example, be downloaded from the following home pages:

Germany: www.bundesbank.de

Switzerland: www.snb.ch

United Kingdom: www.statistics.gov.uk

United States: research.stlouisfed.org/fred2

The book grew out of lectures which I had the occasion to give over the years in Bern and other universities. Thus, it is a concern to thank the many students, in particular Philip Letsch, who had to work through the manuscript and who called my attention to obscurities and typos. I also want to thank my colleagues and teaching assistants Andreas Bachmann, Gregor Bäurle, Fabrice Collard, Sarah Fischer, Stephan Leist, Senada Nukic, Kurt Schmidheiny, Reto Tanner, and Martin Wagner for reading the manuscript or part of it and for making many valuable criticisms and comments. Special thanks go to my former colleague and coauthor Robert Kunst who meticulously read and commented on the manuscript. It goes without saying that all errors and shortcomings go to my expense.

Klaus Neusser

Bern, Switzerland/Eggenburg, Austria

February 2016

Notation and Symbols

r

number of linearly independent cointegration vectors

α

n × r loading matrix

β

n × r matrix of linearly independent cointegration vectors

convergence in distribution

convergence in mean square

convergence in probability

corr(X,Y)

correlation coefficient beween random variables X and Y

γ X ,  γ

covariance function of process { X t }, covariance function

ρ X ,  ρ

correlation function of process { X t }, correlation function

ACF

autocorrelation function

J

long-run variance

α X ,  α

partial autocorrelation function of process { X t }

PACF

partial autocorrelation function

n

dimension of stochastic process, respectively dimension of state space

 ∼ 

is distributed as

sgn

sign function

tr

trace of a matrix

det

determinant of a matrix

 ∥ ∥ 

norm of a matrix

⊗

Kronecker product

⊙

Hadamard product

vec( A )

stakes the columns of A into a vector

vech( A )

stakes the lower triangular part of a symmetric matrix A into a vector

GL( n )

general linear group of n × n matrices

group of orthogonal n × n matrices

L

lag operator

Φ (L)

autoregressive polynomial

Θ (L)

moving-average polynomial

Ψ (L)

causal representation, MA( ∞ ) polynomial

Δ

difference operator, Δ = 1 − L

p

order of autoregressive polynomial

q

order of moving-average polynomial

ARMA(p,q)

autoregressive moving-average process of order ( p ,  q )

ARIMA(p,d,q)

autoregressive integrated moving-average process of order ( p ,  d ,  q )

d

order of integration

I(d)

integrated process of order d

VAR(p)

vector autoregressive process of order p

integer numbers

real numbers

complex numbers

set of n-dimensional vectors

imaginary unit

cov(X,Y)

covariance beween random variables X and Y

expectation operator

variance operator

Ψ (1)

persistence

linear least-squares predictor of X T + h given information from period 1 up to period T

linear least-squares predictor of X T + h using the infinite remote past up to period T

P

Probability

{ X t }

stochastic process

WN(0,  σ ² )

white noise process with mean zero and variance σ ²

WN(0,  Σ )

multivariate white noise process with mean zero and covariance matrix Σ ²

IID(0,  σ ² )

identically and independently distributed random variables with mean zero and variance σ ²

IID N(0,  σ ² )

identically and independently normally distributed random variables with mean zero and variance σ ²

X t

time indexed random variable

x t

realization of random variable X t

f ( λ )

spectral density

F ( λ )

spectral distribution function

I T

periodogram

transfer function of filter Ψ

VaR

value at risk

Contents

Part I Univariate Time Series Analysis

1 Introduction 3

1.​1 Some Examples 4

1.​2 Formal Definitions 7

1.​3 Stationarity 11

1.​4 Construction of Stochastic Processes 15

1.​4.​1 White Noise 15

1.​4.​2 Construction of Stochastic Processes:​ Some Examples 16

1.​4.​3 Moving-Average Process of Order One 17

1.​4.​4 RandomWalk 19

1.​4.​5 Changing Mean 20

1.​5 Properties of the Autocovariance Function 20

1.​5.​1 Autocovariance Function of MA(1) Processes 21

1.​6 Exercises 22

2 ARMA Models 25

2.​1 The Lag Operator 26

2.​2 Some Important Special Cases 27

2.2.1 Moving-Average Process of Order q 27

2.​2.​2 First Order Autoregressive Process 29

2.​3 Causality and Invertibility 32

2.​4 Computation of Autocovariance Function 38

2.​4.​1 First Procedure 39

2.​4.​2 Second Procedure 41

2.​4.​3 Third Procedure 43

2.​5 Exercises 44

3 Forecasting Stationary Processes 45

3.​1 Linear Least-Squares Forecasts 45

3.​1.​1 Forecasting with an AR(p) Process 48

3.​1.​2 Forecasting with MA(q) Processes 50

3.​1.​3 Forecasting from the Infinite Past 53

3.​2 The Wold Decomposition Theorem 54

3.​3 Exponential Smoothing 58

3.​4 Exercises 60

3.​5 Partial Autocorrelation 61

3.​5.​1 Definition 62

3.​5.​2 Interpretation of ACF and PACF 64

3.​6 Exercises 65

4 Estimation of Mean and ACF 67

4.​1 Estimation of the Mean 67

4.​2 Estimation of ACF 73

4.​3 Estimation of PACF 78

4.​4 Estimation of the Long-Run Variance 79

4.​4.​1 An Example 83

4.​5 Exercises 85

5 Estimation of ARMA Models 87

5.​1 The Yule-Walker Estimator 87

5.​2 OLS Estimation of an AR(p) Model 91

5.​3 Estimation of an ARMA(p,q) Model 94

5.4 Estimation of the Orders p and q 99

5.​5 Modeling a Stochastic Process 102

5.​6 Modeling Real GDP of Switzerland 103

6 Spectral Analysis and Linear Filters 109

6.​1 Spectral Density 110

6.​2 Spectral Decomposition of a Time Series 113

6.​3 The Periodogram and the Estimation of Spectral Densities 117

6.​3.​1 Non-Parametric Estimation 117

6.​3.​2 Parametric Estimation 121

6.​4 Linear Time-Invariant Filters 122

6.​5 Some Important Filters 127

6.​5.​1 Construction of Low- and High-Pass Filters 127

6.​5.​2 The Hodrick-Prescott Filter 128

6.​5.​3 Seasonal Filters 130

6.​5.​4 Using Filtered Data 131

6.​6 Exercises 132

7 Integrated Processes 133

7.​1 Definition, Properties and Interpretation 133

7.​1.​1 Long-Run Forecast 135

7.​1.​2 Variance of Forecast Error 136

7.​1.​3 Impulse Response Function 137

7.​1.​4 The Beveridge-Nelson Decomposition 138

7.​2 Properties of the OLS Estimator in the Case of Integrated Variables 141

7.​3 Unit-Root Tests 145

7.​3.​1 Dickey-Fuller Test 147

7.​3.​2 Phillips-Perron Test 149

7.​3.​3 Unit-Root Test:​ Testing Strategy 150

7.​3.​4 Examples of Unit-Root Tests 152

7.​4 Generalizations of Unit-Root Tests 153

7.​4.​1 Structural Breaks in the Trend Function 153

7.​4.​2 Testing for Stationarity 157

7.​5 Regression with Integrated Variables 158

7.​5.​1 The Spurious Regression Problem 158

7.​5.​2 Bivariate Cointegration 159

7.​5.​3 Rules to Deal with Integrated Times Series 162

8 Models of Volatility 167

8.​1 Specification and Interpretation 168

8.​1.​1 Forecasting Properties of AR(1)-Models 168

8.​1.​2 The ARCH(1) Model 169

8.​1.​3 General Models of Volatility 173

8.​1.​4 The GARCH(1,1) Model 177

8.​2 Tests for Heteroskedastici​ty 183

8.​2.​1 Autocorrelation of Quadratic Residuals 183

8.​2.​2 Engle’s Lagrange-Multiplier Test 184

8.​3 Estimation of GARCH(p,q) Models 184

8.​3.​1 Maximum-Likelihood Estimation 184

8.​3.​2 Method of Moment Estimation 187

8.​4 Example:​ Swiss Market Index (SMI) 188

Part II Multivariate Time Series Analysis

9 Introduction 197

10 Definitions and Stationarity 201

11 Estimation of Covariance Function 207

11.​1 Estimators and Asymptotic Distributions 207

11.​2 Testing Cross-Correlations of Time Series 209

11.​3 Some Examples for Independence Tests 211

12 VARMA Processes 215

12.​1 The VAR(1) Process 216

12.​2 Representation in Companion Form 218

12.​3 Causal Representation 218

12.​4 Computation of Covariance Function 221

13 Estimation of VAR Models 225

13.​1 Introduction 225

13.​2 The Least-Squares Estimator 226

13.​3 Proofs of Asymptotic Normality 231

13.​4 The Yule-Walker Estimator 238

14 Forecasting with VAR Models 241

14.​1 Forecasting with Known Parameters 241

14.​1.​1 Wold Decomposition Theorem 245

14.​2 Forecasting with Estimated Parameters 245

14.​3 Modeling of VAR Models 247

14.​4 Example:​ VAR Model 248

15 Interpretation of VAR Models 255

15.​1 Wiener-Granger Causality 255

15.​1.​1 VAR Approach 256

15.​1.​2 Wiener-Granger Causality and Causal Representation 258

15.​1.​3 Cross-Correlation Approach 259

15.​2 Structural and Reduced Form 260

15.​2.​1 A Prototypical Example 260

15.​2.​2 Identification:​ The General Case 263

15.2.3 Identification: The Case n  = 2 266

15.​3 Identification via Short-Run Restrictions 268

15.​4 Interpretation of VAR Models 270

15.​4.​1 Impulse Response Functions 270

15.​4.​2 Variance Decomposition 270

15.​4.​3 Confidence Intervals 272

15.​4.​4 Example 1:​ Advertisement and Sales 274

15.​4.​5 Example 2:​ IS-LM Model with Phillips Curve 277

15.​5 Identification via Long-Run Restrictions 282

15.​5.​1 A Prototypical Example 282

15.​5.​2 The General Approach 285

15.​6 Sign Restrictions 289

16 Cointegration 295

16.​1 A Theoretical Example 296

16.​2 Definition and Representation 302

16.​2.​1 Definition 302

16.​2.​2 VAR and VEC Models 305

16.​2.​3 Beveridge-Nelson Decomposition 308

16.​2.​4 Common Trend Representation 310

16.​3 Johansen’s Cointegration Test 311

16.​3.​1 Specification of the Deterministic Components 317

16.​3.​2 Testing Cointegration Hypotheses 318

16.​4 Estimation and Testing of Cointegrating Relationships 319

16.​5 An Example 321

17 Kalman Filter 325

17.​1 The State Space Model 326

17.​1.​1 Examples 328

17.​2 Filtering and Smoothing 336

17.​2.​1 The Kalman Filter 339

17.​2.​2 The Kalman Smoother 340

17.​3 Estimation of State Space Models 343

17.​3.​1 The Likelihood Function 344

17.​3.​2 Identification 346

17.​4 Examples 346

17.​4.​1 Estimation of Quarterly GDP 346

17.​4.​2 Structural Time Series Analysis 349

17.​5 Exercises 350

18 Generalizations of Linear Models 353

18.​1 Structural Breaks 353

18.​1.​1 Methodology 354

18.​1.​2 An Example 356

18.​2 Time-Varying Parameters 357

18.​3 Regime Switching Models 364

A Complex Numbers369

B Linear Difference Equations373

C Stochastic Convergence377

D BN-Decomposition383

E The Delta Method387

Bibliography391

Index403

List of Figures

Fig. 1.1 Real gross domestic product (GDP)5

Fig. 1.2 Growth rate of real gross domestic product (GDP)5

Fig. 1.3 Swiss real gross domestic product6

Fig. 1.4 Short- and long-term Swiss interest rates7

Fig. 1.5 Swiss Market Index (SMI). ( a ) Index. ( b ) Daily return 8

Fig. 1.6 Unemployment rate in Switzerland9

Fig. 1.7 Realization of a random walk12

Fig. 1.8 Realization of a branching process12

Fig. 1.9 Processes constructed from a given white noise process. ( a ) White noise. ( b ) Moving-average with θ  = 0. 9. ( c ) Autoregressive with ϕ  = 0. 9. ( d ) Random walk 17

Fig. 1.10 Relation between the autocorrelation coefficient of order one, ρ (1), and the parameter θ of a MA(1) process 23

Fig. 2.1 Realization and estimated ACF of MA(1) process28

Fig. 2.2 Realization and estimated ACF of an AR(1) process31

Fig. 2.3 Autocorrelation function of an ARMA(2,1) process42

Fig. 3.1 Autocorrelation and partial autocorrelation functions. ( a ) Process 1. ( b ) Process 2. ( c ) Process 3. ( d ) Process 4 66

Fig. 4.1 Estimated autocorrelation function of a WN(0,1) process75

Fig. 4.2 Estimated autocorrelation function of MA(1) process76

Fig. 4.3 Estimated autocorrelation function of an AR(1) process77

Fig. 4.4 Estimated PACF of an AR(1) process78

Fig. 4.5 Estimated PACF for a MA(1) process79

Fig. 4.6 Common kernel functions81

Fig. 4.7 Estimated autocorrelation function for the growth rate of GDP84

Fig. 5.1 Parameter space of causal and invertible ARMA(1,1) process100

Fig. 5.2 Real GDP growth rates of Switzerland104

Fig. 5.3 ACF and PACF of GDP growth rate105

Fig. 5.4 Inverted roots of the ARMA(1,3) model106

Fig. 5.5 ACF of the residuals from AR(2) and ARMA(1,3) models107

Fig. 5.6 Impulse responses of the AR(2) and the ARMA(1,3) model107

Fig. 5.7 Forecasts of real GDP growth rates108

Fig. 6.1 Examples of spectral densities with Z t  ∼ WN(0, 1). ( a ) MA(1) process. ( b ) AR(1) process 114

Fig. 6.2 Raw periodogram of a white noise time series ( X t  ∼ WN(0, 1), T  = 200) 120

Fig. 6.3 Raw periodogram of an AR(2) process ( X t  = 0. 9 X t −1 − 0. 7 X t −2 + Z t with Z t  ∼ WN(0, 1), T  = 200) 121

Fig. 6.4 Non-parametric direct estimates of a spectral density121

Fig. 6.5 Nonparametric and parametric estimates of spectral density123

Fig. 6.6 Transfer function of the Kuznets filters127

Fig. 6.7 Transfer function of HP-filter129

Fig. 6.8 HP-filtered US GDP130

Fig. 6.9 Transfer function of growth rate of investment in the construction sector with and without seasonal adjustment131

Fig. 7.1 Distribution of the OLS estimator142

Fig. 7.2 Distribution of t-statistic and standard normal distribution144

Fig. 7.3 ACF of a random walk with 100 observations145

Fig. 7.4 Three types of structural breaks at T B . ( a ) Level shift. ( b ) Change in slope. ( c ) Level shift and change in slope 154

Fig. 7.5 Distribution of OLS-estimate $$\hat{\beta }$$ and t-statistic $$t_{\hat{\beta }}$$ for two independent random walks and two independent AR(1) processes. ( a ) Distribution of $$\hat{\beta }$$ . ( b ) Distribution of $$t_{\hat{\beta }}$$ . ( c ) Distribution of $$\hat{\beta }$$ and t-statistic $$t_{\hat{\beta }}$$ 160

Fig. 7.6 Cointegration of inflation and three-month LIBOR. ( a ) Inflation and three-month LIBOR. ( b ) Residuals from cointegrating regression 163

Fig. 8.1 Simulation of two ARCH(1) processes174

Fig. 8.2 Parameter region for which a strictly stationary solution to the GARCH(1,1) process exists assuming ν t  ∼ IID + N(0, 1) 180

Fig. 8.3 Daily return of the SMI (Swiss Market Index)188

Fig. 8.4 Normal-Quantile Plot of SMI returns189

Fig. 8.5 Histogram of SMI returns190

Fig. 8.6 ACF of the returns and the squared returns of the SMI190

Fig. 11.1 Cross-correlations between two independent AR(1) processes212

Fig. 11.2 Cross-correlations between consumption and advertisement213

Fig. 11.3 Cross-correlations between GDP and consumer sentiment214

Fig. 14.1 Forecast comparison of alternative models. ( a ) log Y t . ( b ) log P t . ( c ) log M t . ( d ) R t 251

Fig. 14.2 Forecast of VAR(8) model and 80% confidence intervals253

Fig. 15.1 Identification in a two-dimensional structural VAR267

Fig. 15.2 Impulse response functions for advertisement and sales276

Fig. 15.3 Impulse response functions of IS-LM model280

Fig. 15.4 Impulse response functions of the Blanchard-Quah model289

Fig. 16.1 Impulse responses of present discounted value model302

Fig. 16.2 Stochastic simulation of present discounted value model303

Fig. 17.1 State space model326

Fig. 17.2 Spectral density of cyclical component334

Fig. 17.3 Estimates of quarterly GDP growth rates349

Fig. 17.4 Components of the basic structural model (BSM) for real GDP of Switzerland. ( a ) Logged Swiss GDP (demeaned). ( b ) Local linear trend (LLT). ( c ) Business cycle component. ( d ) Seasonal component 350

Fig. 18.1 Break date UK357

Fig. A.1 Representation of a complex number370

List of Tables

Table 1.1 Construction of stochastic processes17

Table 3.1 Forecast function for a MA(1) process with θ  = −0. 9 and σ ²  = 1 52

Table 3.2 Properties of the ACF and the PACF65

Table 4.1 Common kernel functions81

Table 5.1 AIC for alternative ARMA(p,q) models105

Table 5.2 BIC for alternative ARMA(p,q) models106

Table 7.1 The four most important cases for the unit-root test147

Table 7.2 Examples of unit root tests153

Table 7.3 Dickey-Fuller regression allowing for structural breaks155

Table 7.4 Critical values of the KPSS test158

Table 7.5 Rules of thumb in regressions with integrated processes165

Table 8.1 AIC criterion for variance equation in GARCH(p,q) model191

Table 8.2 BIC criterion for variance equation in GARCH(p,q) model191

Table 8.3 One percent VaR for the next day of the return on SMI193

Table 8.4 One percent VaR for the next 10 days of the return on SMI193

Table 14.1 Information criteria for the VAR models of different orders249

Table 14.2 Forecast evaluation of alternative VAR models252

Table 15.1 Forecast error variance decomposition (FEVD) in terms of demand, supply, price, wage, and money shocks (percentages)281

Table 16.1 Trend specifications in vector error correction models318

Table 16.2 Evaluation of the results of Johansen’s cointegration test322

List of Definitions

1.3 Model10

1.4 Autocovariance Function13

1.5 Stationarity13

1.6 Strict Stationarity14

1.7 Strict Stationarity14

1.8 Gaussian Process15

1.9 White Noise15

2.1 ARMA Models25

2.2 Causality32

2.3 Invertibility37

3.1 Deterministic Process54

3.2 Partial Autocorrelation Function I62

3.3 Partial Autocorrelation Function II62

6.1 Spectral Density110

6.2 Periodogram118

7.2 Cointegration, Bivariate159

8.1 ARCH(1) Model169

10.2 Stationarity202

10.3 Strict Stationarity203

12.1 VARMA process215

15.2 Sign Restrictions291

16.3 Cointegration305

C.1 Almost Sure Convergence378

C.2 Convergence in Probability378

C.3 Convergence in r-th Mean378

C.4 Convergence in Distribution379

C.5 Characteristic Function380

C.6 Asymptotic Normality381

C.7 m-Dependence381

List of Theorems

3.1 Wold Decomposition55

4.1 Convergence of Arithmetic Average68

4.2 Asymptotic Distribution of Sample Mean69

4.4 Asymptotic Distribution of Autocorrelations74

5.1 Asymptotic Normality of Yule-Walker Estimator89

5.2 Asymptotic Normality of the Least-Squares Estimator92

5.3 Asymptotic Distribution of ML Estimator98

6.1 Properties of a Spectral Density111

6.2 Spectral Representation115

6.3 Spectral Density of ARMA Processes121

6.4 Autocovariance Function of Filtered Process123

7.1 Beveridge-Nelson Decomposition139

13.1 Asymptotic Distribution of OLS Estimator229

16.1 Beveridge-Nelson Decomposition304

18.1 Solution TVC-VAR(1)358

C.1 Cauchy-Bunyakovskii-Schwarz Inequality377

C.2 Minkowski’s Inequality377

C.3 Chebyschev’s Inequality377

C.4 Borel-Cantelli Lemma378

C.5 Kolmogorov’s Strong Law of Large Numbers (SLLN)378

C.6 Riesz-Fisher379

C.9 Continuous Mapping Theorem380

C.10 Slutzky’s Lemma380

C.11 Convergence of Characteristic Functions, Lévy381

C.12 Central Limit Theorem381

C.13 CLT for m-Dependent Processes381

C.14 Basis Approximation Theorem382

Footnotes

1

EVIEWS is a product of IHS Global Inc. MATLAB is a matrix-oriented software developed by MathWorks which is ideally suited for econometric and time series applications.

Part I
Univariate Time Series Analysis

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_1

1. Introduction and Basic Theoretical Concepts

Klaus Neusser1 

(1)

Bern, Switzerland

Time series analysis is an integral part of every empirical investigation which aims at describing and modeling the evolution over time of a variable or a set of variables in a statistically coherent way. The economics of time series analysis is thus very much intermingled with macroeconomics and finance which are concerned with the construction of dynamic models. In principle, one can approach the subject from two complementary perspectives. The first one focuses on descriptive statistics. It characterizes the empirical properties and regularities using basic statistical concepts like mean, variance, and covariance. These properties can be directly measured and estimated from the data using standard statistical tools. Thus, they summarize the external (observable) or outside characteristics of the time series. The second perspective tries to capture the internal data generating mechanism. This mechanism is usually unknown in economics as the models developed in economic theory are mostly of a qualitative nature and are usually not specific enough to single out a particular mechanism.¹ Thus, one has to consider some larger class of models. By far most widely used is the class of autoregressive moving-average (ARMA) models which rely on linear stochastic difference equations with constant coefficients. Of course, one wants to know how the two perspectives are related which leads to the important problem of identifying a model from the data.

The observed regularities summarized in the form of descriptive statistics or as a specific model are, of course, of principal interest to economics. They can be used to test particular theories or to uncover new features. One of the main assumptions underlying time series analysis is that the regularities observed in the sample period are not specific to that period, but can be extrapolated into the future. This leads to the issue of forecasting which is another major application of time series analysis.

Although its roots lie in the natural sciences and in engineering, time series analysis, since the early contributions by Frisch (1933) and Slutzky (1937), has become an indispensable tool in empirical economics. Early applications mostly consisted in making the knowledge and methods acquired there available to economics. However, with the progression of econometrics as a separate scientific field, more and more techniques that are specific to the characteristics of economic data have been developed. I just want to mention the analysis of univariate and multivariate integrated, respectively cointegrated time series (see Chaps. 7 and 16), the identification of vector autoregressive (VAR) models (see Chap. 15), and the analysis of volatility of financial market data in Chap. 8 Each of these topics alone would justify the treatment of time series analysis in economics as a separate subfield.

1.1 Some Examples

Before going into more formal analysis, it is useful to examine some prototypical economic time series by plotting them against time. This simple graphical inspection already reveals some of the issues encountered in this book. One of the most popular time series is the real gross domestic product. Figure 1.1 plots the data for the U.S. from 1947 first quarter to 2011 last quarter on logarithmic scale. Several observations are in order. First, the data at hand cover just a part of the time series. There are data available before 1947 and there will be data available after 2011. As there is no natural starting nor end point, we think of a time series as extending back into the infinite past and into the infinite future. Second, the observations are treated as the realizations of a random mechanism. This implies that we observe only one realization. If we could turn back time and let run history again, we would obtain a second realization. This is, of course, impossible, at least in the macroeconomics context. Thus, typically, we are faced with just one realization on which to base our analysis. However, sound statistical analysis needs many realizations. This implies that we have to make some assumption on the constancy of the random mechanism over time. This leads to the concept of stationarity which will be introduced more rigorously in the next section. Third, even a cursory look at the plot reveals that the mean of real GDP is not constant, but is upward trending. As we will see, this feature is typical of many economic time series.² The investigation into the nature of the trend and the statistical consequences thereof have been the subject of intense research over the last couple of decades. Fourth, a simple way to overcome this problem is to take first differences. As the data have been logged, this amounts to taking growth rates.³ The corresponding plot is given in Fig. 1.2 which shows no trend anymore.

Fig. 1.1

Real gross domestic product (GDP) of the U.S. (chained 2005 dollars; seasonally adjusted annual rate)

Fig. 1.2

Quarterly growth rate of U.S. real gross domestic product (GDP) (chained 2005 dollars)

Another feature often encountered in economic time series is seasonality. This issue arises, for example in the case of real GDP, because of a particular regularity within a year: the first quarter being the quarter with the lowest values, the second and fourth quarter those with the highest values, and the third quarter being in between. These movements are due to climatical and holiday seasonal variations within the year and are viewed to be of minor economic importance. Moreover, these seasonal variations, because of there size, hide the more important business cycle movements. It is therefore customary to work with time series which have been adjusted for seasonality before hand. Figure 1.3 shows the unadjusted and the adjusted real gross domestic product for Switzerland. The adjustment has been achieved by taking a moving-average. This makes the time series much smoother and evens out the seasonal movements.

Fig. 1.3

Comparison of unadjusted and seasonally adjusted Swiss real gross domestic product (GDP)

Other typical economic time series are interest rates plotted in Fig. 1.4. Over the period considered these two variables also seem to trend. However, the nature of this trend must be different because of the theoretically binding zero lower bound. Although the relative level of the two series changes over time—at the beginning of the sample, short-term rates are higher than long-terms ones—they move more or less together. This comovement is true in particular true with respect to the medium- and long-term.

Fig. 1.4

Short- and long-term Swiss interest rates (three-month LIBOR and 10 year government bond)

Other prominent time series are stock market indices. In Fig. 1.5 the Swiss Market Index (SMI) in plotted as an example. The first panel displays the raw data on a logarithmic scale. One can clearly discern the different crises: the internet bubble in 2001 and the most recent financial market crisis in 2008. More interesting than the index itself is the return on the index plotted in the second panel. Whereas the mean seems to stay relatively constant over time, the volatility is not: in the periods of crisis volatility is much higher. This clustering of volatility is a typical feature of financial market data and will be analyzed in detail in Chap. 8

Fig. 1.5

Swiss Market Index (SMI). (a ) Index. (b ) Daily return

Finally, Fig. 1.6 plots the unemployment rate for Switzerland. This is another widely discussed time series. However, the Swiss data have a particular feature in that the behavior of the series changes over time. Whereas unemployment was practically nonexistent in Switzerland up to the end of 1990’s, several policy changes (introduction of unemployment insurance, liberalization of immigration laws) led to drastic shifts. Although such dramatic structural breaks are rare, one has to be always aware of such a possibility. Reasons for breaks are policy changes and simply structural changes in the economy at large.⁴

Fig. 1.6

Unemployment rate in Switzerland

1.2 Formal Definitions

The previous section attempted to give an intuitive approach of the subject. The analysis to follow necessitates, however, more precise definitions and concepts. At the heart of the exposition stands the concept of a stochastic process. For this purpose we view the observation at some time t as the realization of random variable X t . In time series analysis we are, however, in general not interested in a particular point in time, but rather in a whole sequence. This leads to the following definition.

Definition 1.1.

A stochastic process {X t } is a family of random variables indexed by $$t \in \mathcal{T}$$ and defined on some given probability space.

Thereby $$\mathcal{T}$$ denotes an ordered index set which is typically identified with time . In the literature one can encounter the following index sets:

Remark 1.1.

Given that $$\mathcal{T}$$ is identified with time and thus has a direction, a characteristic of time series analysis is the distinction between past, present, and future.

For technical reasons which will become clear later, we will work with $$\mathcal{T} = \mathbb{Z}$$, the set of integers. This choice is consistent with the use of time indices in economics as there is, usually, no natural starting point nor a foreseeable endpoint. Although models in continuous time are well established in the theoretical finance literature, we will disregard them because observations are always of a discrete nature and because models in continuous time would need substantially higher mathematical requirements.

Remark 1.2.

The random variables {X t } take values in a so-called state space. In the first part of this treatise, we take as the state space the space of real numbers $$\mathbb{R}$$ and thus consider only univariate time series. In part II we extend the state space to $$\mathbb{R}^{n}$$ and study multivariate times series. Theoretically, it is possible to consider other state spaces (for example, {0, 1}, the integers, or the complex numbers), but this will not be pursued here.

Definition 1.2.

The function $$t \rightarrow x_{t}$$ which assigns to each point in time t the realization of the random variable X t , x t , is called a realization or a trajectory of the stochastic process. We denote such a realization by {x t }.

We denominate by a time series the realization or trajectory (observations or data), or the underlying stochastic process. Usually, there is no room for misunderstandings. A trajectory therefore represents one observation of the stochastic process. Whereas in standard statistics a sample consists of several, typically, independent draws from the same distribution, a sample in time series analysis is just one trajectory. Thus, we are confronted with a situation where there is in principle just one observation. We cannot turn back the clock and get additional trajectories. The situation is even worse as we typically observe only the realizations in a particular time window. For example, we might have data on US GDP from the first quarter 1960 up to the last quarter in 2011. But it is clear, the United States existed before 1960 and will continue to exist after 2011, so that there are in principle observations before 1960 and after 2011. In order to make a meaningful statistical analysis, it is therefore necessary to assume that the observed part of the trajectory is typical for the time series as a whole. This idea is related to the concept of stationarity which we will introduce more formally below. In addition, we want to require that the observations cover in principle all possible events. This leads to the concept of ergodicity . We avoid a formal definition of ergodicity as this would require a sizeable amount of theoretical probabilistic background material which goes beyond the scope this treatise.⁵

An important goal of time series analysis is to build a model given the realization (data) at hand. This amounts to specify the joint distribution of some set of X t ’s with corresponding realization {x t }.

Definition 1.3 (Model).

A time series model or a model for the observations (data) {x t } is a specification of the joint distribution of {X t } for which {x t } is a realization.

The Kolmogorov existence theorem ensures that the specification of all finite dimensional distributions is sufficient to characterize the whole stochastic process (see Billingsley (1986), Brockwell and Davis (1991), or Kallenberg (2002)).

Most of the time it is too involved to specify the complete distribution so that one relies on only the first two moments. These moments are then given by the means $$\mathbb{E}X_{t}$$, the variances $$\mathbb{V}X_{t}$$, $$t \in \mathbb{Z}$$, and the covariances $$\mathrm{cov}(X_{t},X_{s}) = \mathbb{E}(X_{t} - \mathbb{E}X_{t})(X_{s} - \mathbb{E}X_{s}) = \mathbb{E}(X_{t}\,X_{s}) - \mathbb{E}X_{t}\,\mathbb{E}X_{s}$$, respectively the correlations $$\mathrm{corr}(X_{t},X_{s}) =\mathrm{ cov}(X_{t},X_{s})/(\sqrt{\mathbb{V}X_{t}}\sqrt{\mathbb{V}X_{s}})$$, $$t,s \in \mathbb{Z}$$. If the random variables are jointly normally distributed then the specification of the first two moments is sufficient to characterize the whole distribution.

Examples of Stochastic Processes

{X t } is a sequence of independently distributed random variables with values in { − 1, 1} such that $$\mathbf{P}[X_{t} = 1] = \mathbf{P}[X_{t} = -1] = 1/2$$. X t represents, for example, the payoff after tossing a coin: if head occurs one gets a Euro whereas if tail occurs one has to pay a Euro.

The simple random walk {S t } is defined by

where {X t } is the process from the example just above. In this case S t is the proceeds after t rounds of coin tossing. More generally, {X t } could be any sequence of identically and independently distributed random variables. Figure 1.7 shows a realization of {X t } for t = 1, 2, …, 100 and the corresponding random walk {S t }. For more on random walks see Sect. 1.4.4 and, in particular, Chap. 7.

Fig. 1.7

Realization of a random walk

The simple branching process is defined through the recursion

In this example X t represents the size of a population where each member lives just one period and reproduces itself with some probability. Z t, j thereby denotes the number of offsprings of the j-th member of the population in period t. In the simplest case {Z t, j } is nonnegative integer valued and identically and independently distributed. A realization with X ₀ = 100 and with probabilities of one third each that the member has no, one, or two offsprings is shown as an example in Fig. 1.8.

Fig. 1.8

Realization of a branching process

1.3 Stationarity

An important insight in time series analysis is that the realizations in different periods are related with each other. The value of GDP in some year obviously depends on the values from previous years. This temporal dependence can be represented either by an explicit model or, in a descriptive way, by covariances, respectively correlations. Because the realization of X t in some year t may depend, in principle, on all past realizations $$X_{t-1},X_{t-2},\ldots$$, we do not have to specify just a finite number of covariances, but infinitely many covariances. This leads to the concept of the covariance function. The covariance function is not only a tool for summarizing the statistical properties of a time series, but is also instrumental in the derivation of forecasts (Chap. 3), in the estimation of ARMA models, the most important class of models (Chap. 5), and in the Wold representation (Sect. 3.​2 in Chap. 3). It is therefore of utmost importance to get a thorough understanding of the meaning and properties of the covariance function.

Definition 1.4 (Autocovariance Function).

Let {X t } be a stochastic process with $$\mathbb{V}X_{t} < \infty $$ for all $$t \in \mathbb{Z}$$ then the function which assigns to any two time periods t and s, $$t,s \in \mathbb{Z}$$ , the covariance between X t and X s is called the autocovariance function of {X t }. The autocovariance function is denoted by γ X (t,s). Formally this function is given by

Remark 1.3.

The acronym auto emphasizes that the covariance is computed with respect to the same variable taken at different points in time. Alternatively, one may use the term covariance function for short.

Definition 1.5 (Stationarity).

A stochastic process {X t } is called stationary if and only if for all integers r, s and t the following properties hold:

(i)

$$\mathbb{E}X_{t} =\mu$$ constant;

(ii)

$$\mathbb{V}X_{t} < \infty $$ ;

(iii)

$$\gamma _{X}(t,s) =\gamma _{X}(t + r,s + r)$$.

Remark 1.4.

Processes with these properties are often called weakly stationary, wide-sense stationary, covariance stationary, or second order stationary. As we will not deal with other forms of stationarity, we just speak of stationary processes, for short.

Remark 1.5.

For t = s, we have $$\gamma _{X}(t,s) =\gamma _{X}(t,t) = \mathbb{V}X_{t}$$ which is nothing but the unconditional variance of X t . Thus, if {X t } is stationary $$\gamma _{X}(t,t) = \mathbb{V}X_{t} =\mathrm{ constant}$$.

Remark 1.6.

If {X t } is stationary, by setting $$r = -s$$ the autocovariance function becomes:

Thus the covariance γ X (t, s) does not depend on the points in time t and s, but only on the number of periods t and s are apart from each other, i.e. from t − s. For stationary processes it is therefore possible to view the autocovariance function as a function of just one argument. We denote the autocovariance function in this case by γ X (h), $$h \in \mathbb{Z}$$. Because the covariance is symmetric in t and s, i.e. γ X (t, s) = γ X (s, t), we have

It is thus sufficient to look at the autocovariance function for positive integers only, i.e. for h = 0, 1, 2, …. In this case we refer to h as the order of the autocovariance. For h = 0, we get the unconditional variance of X t , i.e. $$\gamma _{X}(0) = \mathbb{V}X_{t}$$.

In practice it is more convenient to look at the autocorrelation coefficients instead of the autocovariances. The autocorrelation function (ACF) for stationary processes is defined as :

where h is referred to as the order. Note that this definition is equivalent to the ordinary correlation coefficients $$\rho (h) = \tfrac{\mathrm{cov}(X_{t},X_{t-h})} {\sqrt{\mathbb{V}X_{t}}\sqrt{\mathbb{V}X_{t-h}}}$$ because stationarity implies that $$\mathbb{V}X_{t} = \mathbb{V}X_{t-h}$$ so that $$\sqrt{\mathbb{V}X_{t}}\sqrt{\mathbb{V}X_{t-h}} = \mathbb{V}X_{t} =\gamma _{X}(0)$$.

Most of the time it is sufficient to concentrate on the first two moments. However, there are situations where it is necessary to look at the whole distribution. This leads to the concept of strict stationarity.

Definition 1.6 (Strict Stationarity).

A stochastic process is called strictly stationary if the joint distributions of $$\left (X_{t_{1}},\ldots,X_{t_{n}}\right )$$ and $$\left (X_{t_{1}+h},\ldots,X_{t_{n}+h}\right )$$ are the same for all $$h \in \mathbb{Z}$$ and all $$\left (t_{1},\ldots,t_{n}\right ) \in \mathcal{T}^{n}$$ , n = 1,2,…

Definition 1.7 (Strict Stationarity).

A stochastic process is called strictly stationary if for all integers h and n ≥ 1 (X ₁ ,…,X n ) and $$(X_{1+h},\ldots,X_{n+h})$$ have the same distribution.

Remark 1.7.

Both definitions are equivalent.

Remark 1.8.

If {X t } is strictly stationary then X t has the same distribution for all t (n=1). For n = 2 we have that X t+h and X t have a joint distribution which is independent of t. This implies that the covariance, if it exists, depends only on h. Thus, every strictly stationary process with $$\mathbb{V}X_{t} < \infty $$ is also stationary.⁶

The converse is, however, not true as shown by the following example:

whereby the X t ’s are independently distributed. In this example we have:

γ X (0) = 1 and γ X (h) = 0 for h ≠ 0

Thus {X t } is stationary, but not strictly stationary, because the distribution changes depending on whether t is even or uneven.

Definition 1.8 (Gaussian Process).

A stochastic process {X t } is called a Gaussian process if all finite dimensional distributions $$(X_{t_{1}},\ldots,X_{t_{n}})$$ with $$(t_{1},\ldots,t_{n}) \in \mathcal{T}^{n}$$ , n = 1,2,…, are multivariate normally distributed.

Remark 1.9.

A Gaussian process is obviously strictly stationary. For all n, h, t ₁, …, t n , $$\left (X_{t_{1}},\ldots,X_{t_{n}}\right )$$ and $$\left (X_{t_{1}+h},\ldots,X_{t_{n}+h}\right )$$ have the same mean and the same covariance matrix.

At this point we will not delve into the relation between stationarity, strict stationarity and Gaussian processes, rather some of these issues will be further discussed in Chap. 8

1.4 Construction of Stochastic Processes

One important notion in time series analysis is to build up more complicated process from simple ones. The simplest building block is a process with zero autocorrelation called a white noise process which is introduced below. Taking moving-averages from this process or using it in a recursion gives rise to more sophisticated process with more elaborated autocovariance functions. Slutzky (1937) first introduced the idea that moving-averages of simple processes can generate time series whose motion resembles business cycle fluctuations.

1.4.1 White Noise

The simplest building block is a process with zero autocorrelation called a white noise process.

Definition 1.9 (White Noise).

A stationary process {Z t } is called a white noise process if {Z t } satisfies:

We denote this by $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$.

The white noise process is therefore stationary and temporally uncorrelated, i.e. the ACF is always equal to zero, except for h = 0 where it is equal to one. As the ACF possesses no structure, it is impossible to draw inferences from past observations to its future development, at least in a least square setting with linear forecasting functions (see Chap. 3). Therefore one can say that a white noise process has no memory .

If {Z t } is not only temporally uncorrelated, but also independently and identically distributed, we write $$Z_{t} \sim \mathrm{ IID}(0,\sigma ^{2})$$. If in addition Z t is normally distributed, we write $$Z_{t} \sim \mathrm{ IIN}(0,\sigma ^{2})$$. An $$\mathrm{IID}(0,\sigma ^{2})$$ process is always a white noise process. The converse is, however, not true as will be shown in Chap. 8

1.4.2 Construction of Stochastic Processes: Some Examples

We will now illustrate how complex stationary processes can be constructed by manipulating of a white noise process. In Table 1.1 we report in column 2 the first 6 realizations of a white noise process {Z t }. Figure 1.9a plots the first 100 observations. We can now construct a new process {X t (MA)} by taking moving-averages over adjacent periods. More specifically, we take $$X_{t} = Z_{t} + 0.9Z_{t-1}$$, t = 2, 3, …. Thus, the realization of {X t (MA)} in period 2 is $$\{x_{2}^{(\mathrm{MA})}\} = -0.8718 + 0.9 \times 0.2590 = -0.6387$$.⁷ The realization in period 3 is $$\{x_{3}^{(\mathrm{MA})}\} = -0.7879 + 0.9 \times -0.8718 = -1.5726$$, and so on. The resulting realizations of {X t (MA)} for t = 2, …, 6 are reported in the third column of Table 1.1 and the plot is shown in Fig. 1.9b. On can see that the averaging makes the series more smooth. In Sect. 1.4.3 we will provide a more detailed analysis of this moving-average process.

Table 1.1

Construction of stochastic processes assuming $$Z_{0} = X_{0} = 0$$

Time

White

Moving-

Auto-

Random

noise

average

regressive

walk

1

0.2590

0.2590

0.2590

0.2590

2

-0.8718

-0.6387

-0.6387

-0.6128

3

-0.7879

-1.5726

-1.3627

-1.4007

4

-0.3443

-1.0535

-1.5708

-1.7451

5

0.6476

0.3377

-0.7661

-1.0974

6

2.0541

2.6370

1.3646

0.9567

Fig. 1.9

Processes constructed from a given white noise process. (a ) White noise. (b ) Moving-average with $$\theta = 0.9$$. (c ) Autoregressive with ϕ = 0. 9. (d ) Random walk

Another construction device is a recursion: $$X_{t}^{(AR)} =\phi X_{t-1}^{(AR)} + Z_{t}$$, t = 2, 3, …, with starting value X ₁ (AR) = Z ₁. Such a process is called autoregressive because it refers to its own past. Taking ϕ = 0. 9, the realization of {X t (AR)} in period 2 is $$\{x_{2}^{(AR)}\} = -0.6387 = 0.9 \times 0.2590 - 0.8718$$, in period 3 $$\{x_{3}^{(AR)}\} = -1.3627 = 0.9 \times -0.6387 - 0.7879$$, and so on. Again the resulting realizations of {X t (AR)} for t = 2, …, 6 are reported in the fourth column of Table 1.1 and the plot is shown in Fig. 1.9c. On can see how the series becomes more persistent. In Sect. 2.​2.​2 we will provide a more detailed analysis of this autoregressive process.

Finally, we construct a new process by taking cumulative sums: $$X_{t}^{(RW)} =\sum _{ \tau =1}^{t}Z_{\tau }$$. This process can also be obtained from the recursion above by taking ϕ = 1 so that $$X_{t}^{(RW)} = X_{t-1}^{(RW)} + Z_{t}$$. It is called a random walk. Thus, the realization of {X t (RW)} for period 2 is $$\{x_{2}^{(RW)}\} = -0.6128 = 0.2590 - 0.8718$$, for period 3 $$\{x_{3}^{(RW)}\} = -1.4007 = -0.6128 - 0.7879$$, and so on. Again the resulting realizations of {X t (RW)} for t = 2, …, 6 are reported in the last column of Table 1.1 and the plot is shown in Fig. 1.9d. On can see how the series moves away from its mean of zero more persistently than all the other three process considered. In Sect. 1.4.4 we will provide a more detailed analysis of this so-called random walk process and show that it is not stationary.

1.4.3 Moving-Average Process of Order One

The white noise process can be used as a building block to construct more complex processes with a more involved autocorrelation structure. The simplest procedure is to take moving averages over consecutive periods.⁸ This leads to the moving-average processes. The moving-average process of order one, MA(1) process, is defined as

Clearly, $$\mathbb{E}X_{t} = \mathbb{E}Z_{t} +\theta \mathbb{E}Z_{t-1} = 0$$. The mean is therefore constant and equal to zero.

The autocovariance function can be computed as follows:

Recalling that {Z t } is white noise so that $$\mathbb{E}Z_{t}^{2} =\sigma ^{2}$$ and $$\mathbb{E}Z_{t}Z_{t+h} = 0$$ for h ≠ 0, we therefore get the following autocovariance function of {X t }:

(1.1)

Thus {X t } is stationary irrespective of the value of $$\theta$$. The autocorrelation function is:

Note that the newly created process now exhibits a dependence from its past as X t is correlated with X t−1. This correlation is restricted to the interval $$[0, \tfrac{1} {2}]$$, i.e. $$0 \leq \vert \rho _{X}(1)\vert \leq \tfrac{1} {2}$$. As the correlation between X t and X s is zero when t and s are more than one period apart, we call a moving-average process a process with finite memory or a process with finite-range dependence.

Remark 1.10.

To motivate the name moving-average, we can define the MA(1) process more generally as

Thus, X t is a weighted average of Z t and Z t−1. If $$\theta _{0} =\theta _{1} = 1/2$$, X t is just the arithmetic mean of Z t and Z t−1. This process is, however, (observationally) equivalent to the process

where $$\tilde{\theta }=\theta _{1}/\theta _{0}$$ and $$\tilde{\sigma }^{2} =\theta _{ 0}^{2}\sigma ^{2}$$. Both processes would generate the same first two moments and are therefore observationally indistinguishable from each other. Thus, we can set $$\theta _{0} = 1$$ without loss of generality.

1.4.4 Random Walk

Let $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ be a white noise process then the new process {X t } defined as

(1.2)

is called a random walk. Note that, in contrast to {Z t }, {X t } is only defined for t > 0. The random walk may alternatively be defined through the recursion

If in each time period a constant δ is added such that

the process {X t } is called a random walk with drift.

Although the random walk has a constant mean of zero, it is a nonstationary process.

Proposition 1.1.

The random walk {X t } as defined in Eq. (1.2)is nonstationary.

Proof.

The variance of $$X_{t+1} - X_{1}$$ equals $$\mathbb{V}(X_{t+1} - X_{1}) = \mathbb{V}\left (\sum _{j=2}^{t+1}Z_{j}\right ) =\sum _{ j=2}^{t+1}\mathbb{V}Z_{j} = t\sigma ^{2}$$.

Assume for the moment that {X t } is stationary then the triangular inequality implies for t > 0:

where “std” denotes the standard deviation. As the left hand side of the inequality converges to infinity for t going to infinity, also the right hand side must go to infinity. This means that the variance of X ₁ must be infinite. This, however, contradicts the assumption of stationarity. Thus {X t } cannot be stationary. □ 

The random walk represents by far the most widely used nonstationary process in economics. It has proven to be an important ingredient in many economic time series. Typical nonstationary time series which are or are driven by random walks are stock market prices, exchange rates, or the gross domestic product (GDP). Usually it is necessary to apply some transformation (filter) first to achieve stationarity. In the example above, one has to replace {X t } by its first difference $$\{\Delta X_{t}\} =\{ X_{t} - X_{t-1}\} =\{ Z_{t}\}$$ which is stationary by construction. Time series which become stationary after differencing are called integrated processes and are the subject of a more in depth analysis in Chap. 7. Besides ordinary differencing, other transformations are often encountered: seasonal differencing, inclusion of a time trend, seasonal dummies, moving averages, etc. Some of them will be discussed as we go along.

1.4.5 Changing Mean

Finally, here is another simple example of a nonstationary process.

where t c is some specific point in time. {X t } is clearly not stationary because the mean is not constant. In econometrics we refer to such a situation as a structural change which can be accommodated by introducing a so-called dummy variable. Models with more sophisticated forms of structural changes will be discussed in Chap. 18

1.5 Properties of the Autocovariance Function

The autocovariance function represents the directly accessible external properties of the time series. It is therefore important to understand its properties and how it is related to its inner structure. We will deepen the connection between the autocovariance function and a particular class of models in Chap. 2 The estimation of the autocovariance function will be treated in Chap. 4. For the moment we will just give its properties and analyze the case of the MA(1) model as a prototypical example.

Theorem 1.1.

The autocovariance function of a stationary process {X t } is characterized by the following properties:

(i)

γ X (0) ≥ 0;

(ii)

$$0 \leq \left \vert \gamma _{X}(h)\right \vert \leq \gamma _{X}(0)$$ ;

(iii)

$$\gamma _{X}(h) =\gamma _{X}(-h)$$ ;

(iv)

$$\sum _{i,j=1}^{n}a_{i}\gamma _{X}(t_{i} - t_{j})a_{j} \geq 0$$ for all n and all vectors (a ₁ ,…,a n )′ and (t ₁ ,…,t n ). This property is called non-negative definiteness.

Proof.

The first property is obvious as the variance is always nonnegative. The second property follows from the Cauchy-Bunyakovskii-Schwarz inequality(see Theorem C.1) applied to X t and X t+h which yields $$0 \leq \left \vert \gamma _{X}(h)\right \vert \leq \gamma _{X}(0)$$. The third property follows immediately from the definition of the covariance. Define a = (a ₁, …, a n )′ and $$X = \left (X_{t_{1}},\ldots,X_{t_{n}}\right )'$$ then the last property follows from the fact that the variance is always nonnegative: $$0 \leq \mathbb{V}\left (a'X\right ) = a'\mathbb{V}(X)a =\sum _{ i,j=1}^{n}a_{i}\gamma _{X}(t_{i} - t_{j})a_{j}$$. □ 

Similar properties hold for the correlation function ρ X , except that we have ρ X (0) = 1.

Theorem 1.2.

The autocorrelation function of a stationary stochastic process {X t } is characterized by the following properties:

(i)

ρ X (0) = 1;

(ii)

$$0 \leq \left \vert \rho _{X}(h)\right \vert \leq 1$$ ;

(iii)

$$\rho _{X}(h) =\rho _{X}(-h)$$ ;

(iv)

$$\sum _{i,j=1}^{n}a_{i}\rho _{X}(t_{i} - t_{j})a_{j} \geq 0$$ for all n and all vectors (a ₁ ,…,a n )′ and (t ₁ ,…,t n ).

Proof.

The proof follows immediately from the properties of the autocovariance function. □ 

It can be shown that for any given function with the above properties there exists a stationary process (Gaussian process) which has this function as its autocovariance function, respectively autocorrelation function.

1.5.1 Autocovariance Function of MA(1) Processes

The autocovariance function describes the external observable characteristics of a time series which can be estimated from the data. Usually, we want to understand the internal mechanism which generates the data at hand. For this we need a model. Hence it is important to understand the relation between the autocovariance function and a certain class of models. In this section, by analyzing the MA(1) model, we will show that this relationship is not one-to-one. Thus we are confronted with a fundamental identification problem.

In order to make the point, consider the following given autocovariance function:

The problem consists of determining the parameters of the MA(1) model, $$\theta$$ and $$\sigma ^{2}$$, from the values of the autocovariance function. For this purpose we equate $$\gamma _{0} = (1 +\theta ^{2})\sigma ^{2}$$ and $$\gamma _{1} =\theta \sigma ^{2}$$ (see Eq. (1.1)). This leads to an equation system in the two unknowns $$\theta$$ und $$\sigma ^{2}$$. This system can be simplified by dividing the second equation by the first one to obtain: $$\gamma _{1}/\gamma _{0} =\theta /(1 +\theta ^{2})$$. Because $$\gamma _{1}/\gamma _{0} =\rho (1) =\rho _{1}$$ one gets a quadratic equation in $$\theta$$:

The two solutions of this equation are

The solutions are real if and only if the discriminant 1 − 4ρ ₁ ² is positive. This is the case if and only if ρ ₁ ² ≤ 1∕4, respectively | ρ ₁ | ≤ 1∕2. Note that one root is the inverse of the other. The identification problem thus takes the following form:

  | ρ ₁ |  < 1∕2:

there exists two observationally equivalent MA(1) processes corresponding to the two solutions $$\theta _{1}$$ and $$\theta _{2}$$.

$$\rho _{1} = \pm 1/2$$:

there exists exactly one MA(1) process with $$\theta = \pm 1$$.

  | ρ ₁ |  > 1∕2:

there exists no MA(1) process with this autocovariance function.

The relation between the first order autocorrelation coefficient, ρ ₁ = ρ(1), and the parameter $$\theta$$ of the MA(1) process is represented in Fig. 1.10. As can be seen, there exists for each ρ(1) with $$\vert \rho (1)\vert < \frac{1} {2}$$ two solutions. The two solutions are inverses of each other. Hence one solution is absolutely smaller than one whereas the other is bigger than one. In Sect. 2.​3 we will argue in favor of the solution smaller than one. For $$\rho (1) = \pm 1/2$$ there exists exactly one solution, namely $$\theta = \pm 1$$. For | ρ(1) |  > 1∕2 there is no solution. For | ρ ₁ |  > 1∕2, ρ(h) actually does not represent a genuine autocorrelation function as the fourth condition in Theorem 1.1, respectively Theorem 1.2 is violated. For $$\rho _{1} > \frac{1} {2}$$, set $$a = (1,-1,1,-1,\ldots,1,-1)'$$ to get:

For $$\rho _{1} = -\frac{1} {2}$$ one sets a = (1, 1, …, 1)′. Hence the fourth property is violated.

Fig. 1.10

Relation between the autocorrelation coefficient of order one, ρ(1), and the parameter $$\theta$$ of a MA(1) process

1.6 Exercises

Exercise 1.6.1.

Let the process {X t } be generated by a two-sided moving-average process

Determine the autocovariance and the autocorrelation function of {X t }.

Exercise 1.6.2.

Let {X t } be the MA(1) process

(i)

Determine the autocovariance and the autocorrelation function of {X t } for $$\theta = 0.9$$.

(ii)

Determine the variance of the mean $$(X_{1} + X_{2} + X_{3} + X_{4})/4$$.

(iii)

How do the previous results change if $$\theta = -0.9$$ ?

Exercise 1.6.3.

Consider the autocovariance function

Determine the parameters $$\theta$$ and $$\sigma ^{2}$$ , if they exist, of the first order moving-average process $$X_{t} = Z_{t} +\theta Z_{t-1}$$ with $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ such that autocovariance function above is the autocovariance function corresponding to {X t }.

Exercise 1.6.4.

Let the stochastic process {X t } be defined as

where {Z t } is identically and independently distributed as $$Z_{t} \sim \mathrm{ N}(0,1)$$ . Show that $$\{X_{t}\} \sim \mathrm{ WN}(0,1)$$ , but not IID (0,1).

Exercise 1.6.5.

Which of the following processes is stationary?

(i)

(ii)

(iii)

(iv)

In all cases we assume that {Z t } is identically and independently distributed with $$Z_{t} \sim \mathrm{ N}(0,\sigma ^{2})$$. $$\theta$$ and a are arbitrary parameters.

References

Billingsley P (1986) Probability and measure, 2nd edn. Wiley, New York

Brockwell PJ, Davis RA (1991) Time series: theory and methods, 2nd edn. Springer, New YorkCrossRef

Burren D, Neusser K (2013) The role of sectoral shifts in the decline of real GDP volatility. Macroecon Dyn 17:477–500CrossRef

Campbell JY, Lo AW, MacKinlay AC (1997) The econometrics of financial markets. Princeton University Press, Princeton

Frisch R (1933) Propagation problems and impulse problems in dynamic economics. In: Economic essays in honour of Gustav cassel. Frank Cass, London, pp 171–205

Hall RE (1978) Stochastic implications of the life cycle-permanent income hypothesis: Theory and evidence. J Polit Econ 86:971–987CrossRef

Kallenberg O (2002) Foundations of modern probability, 2nd edn. Probability and its applications, Springer, New YorkCrossRef

Samuelson PA (1965) Proof that properly anticipated prices fluctuate randomly. Ind Manage Rev 6:41–49

Slutzky E (1937) The summation of random causes as the source of cyclic processes. Econometrica 5:105–146CrossRef

Footnotes

1

One prominent exception is the random-walk hypothesis of real private consumption first derived and analyzed by Hall (1978). This hypothesis states that the current level of private consumption should just depend on private consumption one period ago and on no other variable, in particular not on disposable income. The random-walk property of asset prices is another very much discussed hypothesis. See Campbell et al. (1997) for a general exposition and Samuelson (1965) for a first rigorous derivation from market efficiency.

2

See footnote 2 for some theories predicting non-stationary behavior.

3

This is obtained by using the approximation $$\ln (1+\varepsilon ) \approx \varepsilon$$ for small $$\varepsilon$$ where $$\varepsilon$$ equals the growth rate of GDP.

4

Burren and Neusser (2013) investigate, for example, how systematic sectoral shifts affect volatility of real GDP growth.

5

In the theoretical probability theory ergodicity is an important concept which asks the question under which conditions the time average of a property is equal to the corresponding ensemble average, i.e. the average over the entire state space. In particular, ergodicity ensures that the arithmetic averages over time converge to their theoretical counterparts. In Chap. 4 we allude to this principle in the estimation of the mean and the autocovariance function of a time series.

6

An example of a process which is strictly stationary, but not stationary, is given by the IGARCH process (see Sect. 8.​1.​4). This process is strictly stationary with infinite variance.

7

The following calculations are subject to rounding to four digits.

8

This procedure is an example of a filter. Section 6.​4 provides a general introduction to filters.

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_2

2. Autoregressive Moving-Average Models

Klaus Neusser1 

(1)

Bern, Switzerland

A basic idea in time series analysis is to construct more complex processes from simple ones. In the previous chapter we showed how the averaging of a white noise process leads to a process with first order autocorrelation. In this chapter we generalize this idea and consider processes which are solutions of linear stochastic difference equations. These so-called ARMA processes constitute the most widely used class of models for stationary processes.

Definition 2.1 (ARMA Models).

A stochastic process {X t } with $$t \in \mathbb{Z}$$ is called an autoregressive moving-average process (ARMA process) of order (p,q), denoted by ARMA(p,q) process, if the process is stationary and satisfies a linear stochastic difference equation of the form

(2.1)

with $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ and $$\phi _{p}\theta _{q}\neq 0$$ . {X t } is called an ARMA(p,q) process with mean μ if {X t −μ} is an ARMA(p,q) process.

The importance of ARMA processes is due to the fact that every stationary process can be approximated arbitrarily well by an ARMA process. In particular, it can be shown that for any given autocovariance function γ with the property $$\lim _{h\rightarrow \infty }\gamma (h) = 0$$ and any positive integer k there exists an autoregressive moving-average process (ARMA process) {X t } such that γ X (h) = γ(h), h = 0, 1, …, k.

For an ARMA process with mean μ one often adds a constant c to the right hand side of the difference equation:

The mean of X t is then: $$\mu = \frac{c} {1-\phi _{1}-\ldots -\phi _{p}}$$. The mean is therefore only well-defined if $$\phi _{1} +\ldots +\phi _{p}\neq 1$$. The case $$\phi _{1} +\ldots +\phi _{p} = 1$$ can, however, be excluded because there exists no stationary solution in this case (see Remark 2.2) and thus no ARMA process.

2.1 The Lag Operator

In times series analysis it is customary to rewrite the above difference equation more compactly in terms of the lag operator L. This is, however, not only a compact notation, but will open the way to analyze the inner structure of ARMA processes. The lag or back-shift operator L moves the time index one period back:

For ease of notation we write: $$\mathrm{L}X_{t} = X_{t-1}$$. The lag operator is a linear operator with the following calculation rules:

(i)

L applied to the process {X t  = c} where c is an arbitrary constant gives:

(ii)

Applying L n times:

(iii)

The inverse of the lag operator is the lead or forward operator. This operator shifts the time index one period into the future.¹ We can write L⁻¹:

(iv)

For any integers m and n we have:

(v)

As $$\mathrm{L}^{-1}\mathrm{L}X_{t} = X_{t}$$ we have that

(vi)

For any real numbers a and b, any integers m and n, and arbitrary stochastic processes {X t } and {Y t } we have:

In this way it is possible to define lag polynomials : $$A(\mathrm{L}) = a_{0} + a_{1}\mathrm{L} + a_{2}\mathrm{L}^{2} +\ldots +a_{p}\mathrm{L}^{p}$$ where a ₀, a ₁, …, a p are any real numbers. For these polynomials the usual calculation rules apply. Let, for example, $$A(\mathrm{L}) = 1 - 0.5\mathrm{L}$$ and $$B(\mathrm{L}) = 1 + 4\mathrm{L}^{2}$$ then $$C(\mathrm{L}) = A(\mathrm{L})B(\mathrm{L}) = 1 - 0.5\mathrm{L} + 4\mathrm{L}^{2} - 2\mathrm{L}^{3}$$.

Applied to the stochastic difference equation, we define the autoregressive and the moving-average polynomial as follows:

The stochastic difference equation defining the ARMA process can then be written compactly as

Thus, the use of lag polynomials provides a compact notation for ARMA processes. Moreover and most importantly, $$\Phi (z)$$ and $$\Theta (z)$$, viewed as polynomials of the complex number z, also reveal much of their inherent structural properties as will become clear in Sect. 2.3.

2.2 Some Important Special Cases

Before we deal with the general theory of ARMA processes, we will analyze some important special cases first:

q = 0:

autoregressive process of order p, AR(p) process

p = 0:

moving-average process of order q, MA(q) process

2.2.1 The Moving-Average Process of Order q (MA(q) Process)

The MA(q) process is defined by the following stochastic difference equation:

and $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$. Obviously,

because $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$. As can be easily verified using the properties of {Z t }, the autocovariance function of the MA(q) processes are:

This implies the following autocorrelation function :

Every MA(q) process is therefore stationary irrespective of its parameters $$\theta _{0},\theta _{1},\ldots,\theta _{q}$$. Because the correlation between X t and X s is equal to zero if the two time points t and s are more than q periods apart, such processes are sometimes called processes with short memory or processes with short range dependence .

Figure 2.1 displays an MA(1) process and its autocorrelation function.

Fig. 2.1

Realization and estimated ACF of a MA(1) process: $$X_{t} = Z_{t} - 0.8Z_{t-1}$$ with $$Z_{t} \sim \mathrm{ IID\,N}(0,1)$$

2.2.2 The First Order Autoregressive Process (AR(1) Process)

The AR(p) process requires a more thorough analysis as will already become clear from the AR(1) process . This process is defined by the following stochastic difference equation:

(2.2)

The above stochastic difference equation has in general several solutions. Given a sequence {Z t } and an arbitrary distribution for X ₀, it determines all random variables X t , $$t \in \mathbb{Z}\setminus \{0\}$$, by applying the above recursion. The solutions are, however, not necessarily stationary. But, according to the Definition 2.1, only stationary processes qualify for ARMA processes. As we will demonstrate, depending on the value of ϕ, there may exist no or just one solution.

Consider first the case of | ϕ |  < 1. Inserting into the difference equation several times leads to:

If {X t } is a stationary solution, $$\mathbb{V}X_{t-k-1}$$ remains constant independently of k. Thus

This shows that $$\sum _{j=0}^{k}\phi ^{j}Z_{t-j}$$ converges in the mean square sense, and thus also in probability, to X t for $$k \rightarrow \infty $$ (see Theorem C.8 in Appendix C). This suggests to take

(2.3)

as the solution to the stochastic difference equation. As $$\sum _{j=0}^{\infty }\vert \phi ^{j}\vert = \frac{1} {1-\phi } < \infty $$ this solution is well-defined according to Theorem 6.​4 and has the following properties:

Thus the solution $$X_{t} =\sum _{ j=0}^{\infty }\phi ^{j}Z_{t-j}$$ is stationary and fulfills the difference equation as can be easily verified. It is also the only stationary solution which is compatible with the difference equation. Assume that there is second solution $$\{\tilde{X}_{t}\}$$ with these properties. Inserting into the difference equation yields again

This variance converges to zero for k going to infinity because | ϕ |  < 1 and because $$\{\tilde{X}_{t}\}$$ is stationary. The two processes $$\{\tilde{X}_{t}\}$$ and {X t } with $$X_{t} =\sum _{ j=0}^{\infty }\phi ^{j}Z_{t-j}$$ are therefore identical in the mean square sense and thus with probability one.

Finally, note that the recursion (2.2) will only generate a stationary process if it is initialized with X ₀ having the stationary distribution, i.e. if $$\mathbb{E}X_{0} = 0$$ and $$\mathbb{V}X_{0} =\sigma ^{2}/(1 -\phi ^{2})$$. If the recursion is initiated with an arbitrary variance of X ₀, $$0 <\sigma _{ 0}^{2} < \infty $$, Eq. (2.2) implies the following difference equation for the variance of X t , $$\sigma _{t}^{2}$$:

The solution of this difference equation is

where $$\sigma _{{\ast}}^{2} =\sigma ^{2}/(1 -\phi ^{2})$$ denotes the variance of the stationary distribution. If $$\sigma _{0}^{2}\neq \sigma _{{\ast}}^{2}$$, $$\sigma _{t}^{2}$$ is not constant implying that the process {X t } is not stationary. However, as | ϕ |  < 1, the variance of X t , $$\sigma _{t}^{2}$$, will converge to the variance of the stationary distribution.²

Figure 2.2 shows a realization of such a process and its estimated autocorrelation function.

Fig. 2.2

Realization and estimated ACF of an AR(1) process: $$X_{t} = 0.8X_{t-1} + Z_{t}$$ with $$Z_{t} \sim \mathrm{ IIN}(0,1)$$

In the case | ϕ |  > 1 the solution (2.3) does not converge. It is, however, possible to iterate the difference equation forward in time to obtain:

This suggests to take

as the solution. Going through similar arguments as before it is possible to show that this is indeed the only stationary solution. This solution is, however, viewed to be inadequate because X t depends on future shocks $$Z_{t+j},j = 1,2,\ldots$$ Note, however, that there exists an AR(1) process with | ϕ |  < 1 which is observationally equivalent, in the sense that it generates the same autocorrelation function, but with a new shock or forcing variable $$\{\widetilde{Z}_{t}\}$$ (see next section).

In the case | ϕ |  = 1 there exists no stationary solution (see Sect. 1.​4.​4) and therefore, according to our definition, no ARMA process. Processes with this property are called random walks, unit root processes or integrated processes. They play an important role in economics and are treated separately in Chap. 7

2.3 Causality and Invertibility

If we interpret {X t } as the state variable and {Z t } as an impulse or shock, we can ask whether it is possible to represent today’s state X t as the outcome of current and past shocks $$Z_{t},Z_{t-1},Z_{t-2},\ldots$$ In this case we can view X t as being caused by past shocks and call this a causal representation. Thus, shocks to current Z t will not only influence current X t , but will propagate to affect also future X t ’s. This notion of causality rest on the assumption that the past can cause the future but that the future cannot cause the past. See Sect. 15.​1 for an elaboration of the concept of causality and its generalization to the multivariate context.

In the case that {X t } is a moving-average process of order q, X t is given as a weighted sum of current and past shocks $$Z_{t},Z_{t-1},\ldots,Z_{t-q}$$. Thus, the moving-average representation is already the causal representation. In the case of an AR(1) process, we have seen that this is not always feasible. For | ϕ |  < 1, the solution (2.3) represents X t as a weighted sum of current and past shocks and is thus the corresponding causal representation. For | ϕ |  > 1, no such representation is possible. The following Definition 2.2 makes the notion of a causal representation precise and Theorem 2.1 gives a general condition for its existence.

Definition 2.2 (Causality).

An ARMA(p,q) process {X t } with $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$ is called causal with respect to {Z t } if there exists a sequence {ψ j } with the property $$\sum _{j=0}^{\infty }\vert \psi _{j}\vert < \infty $$ such that

where $$\Psi (\mathrm{L}) = 1 +\psi _{1}\mathrm{L} +\psi _{2}\mathrm{L}^{2}+\ldots =\sum _{ j=0}^{\infty }\psi _{j}\mathrm{L}^{j}$$ . The above equation is referred to as the causal representation of {X t } with respect to {Z t }.

The coefficients {ψ j } are of great importance because they determine how an impulse or a shock in period t propagates to affect current and future X t+j , j = 0, 1, 2… In particular, consider an impulse $$e_{t_{0}}$$ at time t ₀, i.e. a time series which is equal to zero except for the time t ₀ where it takes on the values $$e_{t_{0}}$$. Then, $$\{\psi _{t-t_{0}}e_{t_{0}}\}$$ traces out the time history of this impulse. For this reason, the coefficients ψ j with $$j = t - t_{0}$$, $$t = t_{0},t_{0} + 1,t_{0} + 2,\ldots$$, are called the impulse response function. If $$e_{t_{0}} = 1$$, it is called a unit impulse. Alternatively, $$e_{t_{0}}$$ is sometimes taken to be equal to $$\sigma$$, the standard deviation of Z t . It is customary to plot ψ j as a function of j, j = 0, 1, 2…

Note that the notion of causality is not an attribute of {X t }, but is defined relative to another process {Z t }. It is therefore possible that a stationary process is causal with respect to one process, but not with respect to another process. In order to make this point more concrete, consider again the AR(1) process defined by the equation $$X_{t} =\phi X_{t-1} + Z_{t}$$ with | ϕ |  > 1. As we have seen, the only stationary solution is given by $$X_{t} = -\sum _{j=1}^{\infty }\phi ^{-j}Z_{t+j}$$ which is clearly not causal with respect {Z t }. Consider as an alternative the process

(2.4)

This new process is white noise with variance $$\tilde{\sigma }^{2} =\phi ^{-2}\sigma ^{2}$$.³ Because {X t } fulfills the difference equation

{X t } is causal with respect to $$\{\tilde{Z}_{t}\}$$. This remark shows that there is no loss of generality involved if we concentrate on causal ARMA processes.

Theorem 2.1.

Let {X t } be an ARMA(p,q) process with $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$ and assume that the polynomials $$\Phi (z)$$ and $$\Theta (z)$$ have no common root. {X t } is causal with respect to {Z t } if and only if $$\Phi (z)\neq 0$$ for |z|≤ 1, i.e. all roots of the equation $$\Phi (z) = 0$$ are outside the unit circle. The coefficients {ψ j } are then uniquely defined by identity:

Proof.

Given that $$\Phi (z)$$ is a finite order polynomial with $$\Phi (z)\neq 0$$ for | z | ≤ 1, there exits ε > 0 such that $$\Phi (z)\neq 0$$ for | z | ≤ 1 +ε. This implies that $$1/\Phi (z)$$ is an analytic function on the circle with radius 1 +ε and therefore possesses a power series expansion:

This implies that $$\xi _{j}(1 +\epsilon /2)^{j}$$ goes to zero for j to infinity. Thus there exists a positive and finite constant C such that

This in turn implies that $$\sum _{j=0}^{\infty }\vert \xi _{j}\vert < \infty $$ and that $$\Xi (z)\Phi (z) = 1$$ for | z | ≤ 1. Applying $$\Xi (\mathrm{L})$$ on both sides of $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$, gives:

Theorem 6.​4 implies that the right hand side is well-defined. Thus $$\Psi (\mathrm{L}) = \Xi (\mathrm{L})\Theta (\mathrm{L})$$ is the sought polynomial. Its coefficients are determined by the relation $$\Psi (z) = \Theta (z)/\Phi (z)$$.

Assume now that there exists a causal representation $$X_{t} =\sum _{ j=0}^{\infty }\psi _{j}Z_{t-j}$$ with $$\sum _{j=0}^{\infty }\vert \psi _{j}\vert < \infty $$. Therefore

Take $$\eta (z) = \Phi (z)\Psi (z) =\sum _{ j=0}^{\infty }\eta _{j}z^{j}$$, | z | ≤ 1. Multiplying the above equation by Z t−k and taking expectations shows that $$\eta _{k} =\theta _{k}$$, k = 0, 1, 2, …, q, and that η k  = 0 for k > q. Thus we get $$\Theta (z) =\eta (z) = \Phi (z)\Psi (z)$$ for | z | ≤ 1. As $$\Theta (z)$$ and $$\Phi (z)$$ have no common roots and because $$\vert \Psi (z)\vert < \infty $$ for | z | ≤ 1, $$\Phi (z)$$ cannot be equal to zero for | z | ≤ 1. □ 

Remark 2.1.

If the AR and the MA polynomial have common roots, there are two possibilities:

No common roots lies on the unit circle. In this situation there exists a unique stationary solution which can be obtained by canceling the common factors of the polynomials.

If at least one common root lies on the unit circle then more than one stationary solution may exist (see the last example below).

Some Examples

We concretize the above Theorem and Remark by investigating some examples starting from the ARMA model $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$ with $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$.

$$\Phi (\mathrm{L}) = 1 - 0.05\mathrm{L} - 0.6\mathrm{L}^{2}$$ and $$\Theta (\mathrm{L}) = 1$$:

The roots of the polynomial $$\Phi (z)$$ are $$z_{1} = -4/3$$ and $$z_{2} = 5/4$$. Because both roots are absolutely greater than one, there exists a causal representation with respect to {Z t }.

$$\Phi (\mathrm{L}) = 1 + 2\mathrm{L} + 5/4\mathrm{L}^{2}$$ and $$\Theta (\mathrm{L}) = 1$$:

In this case the roots are conjugate complex and equal to $$z_{1} = -4/5 + 2/5\imath $$ and $$z_{2} = -4/5 - 2/5\imath $$. The modulus or absolute value of z ₁ and z ₂ equals $$\vert z_{1}\vert = \vert z_{2}\vert = \sqrt{20/25}$$. This number is smaller than one. Therefore there exists a stationary solution, but this solution is not causal with respect to {Z t }.

$$\Phi (\mathrm{L}) = 1 - 0.05\mathrm{L} - 0.6\mathrm{L}^{2}$$ and $$\Theta (\mathrm{L}) = 1 + 0.75\mathrm{L}$$:

$$\Phi (z)$$ and $$\Theta (z)$$ have the common root $$z = -4/3\neq 1$$. Thus one can cancel both $$\Phi (\mathrm{L})$$ and $$\Theta (\mathrm{L})$$ by $$1 + \tfrac{3} {4}\mathrm{L}$$ to obtain the polynomials $$\tilde{\Phi }(\mathrm{L}) = 1 - 0.8\mathrm{L}$$ and $$\tilde{\Theta }(\mathrm{L}) = 1$$. Because the root of $$\tilde{\Phi }(z)$$ equals 5/4 which is greater than one, there exists a unique stationary and causal representation with respect to {Z t }.

$$\Phi (\mathrm{L}) = 1 + 1.2\mathrm{L} - 1.6\mathrm{L}^{2}$$ and $$\Theta (\mathrm{L}) = 1 + 2\mathrm{L}$$:

The roots of $$\Phi (z)$$ are $$z_{1} = 5/4$$ and $$z_{2} = -0.5$$. Thus one root is outside the unit circle whereas one is inside. This would suggest that there is no causal solution. However, the root − 0. 5 ≠ 1 is shared by $$\Phi (z)$$ and $$\Theta (z)$$ and can therefore be canceled to obtain $$\tilde{\Phi }(\mathrm{L}) = 1 - 0.8\mathrm{L}$$ and $$\tilde{\Theta }(\mathrm{L}) = 1$$. Because the root of $$\tilde{\Phi }(z)$$ equals 5∕4 > 1, there exists a unique stationary and causal solution with respect to {Z t }.

$$\Phi (\mathrm{L}) = 1 +\mathrm{ L}$$ and $$\Theta (\mathrm{L}) = 1 +\mathrm{ L}$$:

$$\Phi (z)$$ and $$\Theta (z)$$ have the common root − 1 which lies on the unit circle. As before one might cancel both polynomials by 1 + L to obtain the trivial stationary and causal solution {X t } = { Z t }. This is, however, not the only solution. Additional solutions are given by $$\{Y _{t}\} =\{ Z_{t} + A(-1)^{t}\}$$ where A is an arbitrary random variable with mean zero and finite variance $$\sigma _{A}^{2}$$ which is independent from both {X t } and {Z t }. The process {Y t } has a mean of zero and an autocovariance function γ Y (h) which is equal to

Thus this new process is therefore stationary and fulfills the difference equation.

Remark 2.2.

If the AR and the MA polynomial in the stochastic difference equation $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$ have no common root, but $$\Phi (z) = 0$$ for some z on the unit circle, there exists no stationary solution. In this sense the stochastic difference equation does no longer define an ARMA model. Models with this property are said to have a unit root and are treated in Chap. 7 If $$\Phi (z)$$ has no root on the unit circle, there exists a unique stationary solution.

As explained in the previous Theorem, the coefficients {ψ j } of the causal representation are uniquely determined by the relation $$\Psi (z)\Phi (z) = \Theta (z)$$. If {X t } is a MA process, $$\Phi (z) = 1$$ and the coefficients {ψ j } just correspond to the coefficients of the MA polynomial, i.e. $$\psi _{j} =\theta _{j}$$ for 0 ≤ j ≤ q and ψ j  = 0 for j > q. Thus in this case no additional computations are necessary. In general this is not the case. In principle there are two ways to find the coefficients {ψ j }. The first one uses polynomial division or partial fractions, the second one uses the method of undetermined coefficients. This book relies on the second method because it is more intuitive and presents some additional insides. For this purpose let us write out the defining relation $$\Psi (z)\Phi (z) = \Theta (z)$$:

Multiplying out the left hand side one gets:

Equating the coefficients of the powers of z, z j , $$j = 0,1,2,\ldots$$, one obtains the following equations:

As can be seen, it is possible to solve recursively for the unknown coefficients {ψ j }. This is convenient when it comes to numerical computations, but in some cases one wants an analytical solution. Such a solution can be obtained by observing that, for $$j \geq \max \{ p,q + 1\}$$, the recursion leads to the following difference equation of order p:

This is a linear homogeneous difference equation with constant coefficients. The solution of such an equation is of the form (see Eq. (B.1) in Appendix B):

(2.5)

where z ₁, …, z p denote the roots of $$\Phi (z) = 1 -\phi _{1}z -\ldots -\phi _{p}z^{p} = 0$$.⁴ Note that the roots are exactly those which have been computed to assess the existence of a causal representation. The coefficients c ₁, …, c p can be obtained using the p boundary conditions obtained from $$\psi _{j} =\sum _{0<k\leq j}\phi _{k}\psi _{j-k} =\theta _{j}$$, $$\max \{p,q + 1\} - p \leq j <\max \{ p,q + 1\}$$. Finally, the values for ψ j , $$0 \leq j <\max \{ p,q + 1\} - p$$, must be computed from the first $$\max \{p,q + 1\} - p$$ iterations (see the example in Sect. 2.4).

As mentioned previously, the coefficients {ψ j } are of great importance as they quantify the effect of a shock to Z t−j on X t , respectively of Z t on X t+j . In macroeconomics they are sometimes called dynamic multipliers of a transitory or temporary shock . Because the underlying ARMA process is stationary and causal, the infinite sum $$\sum _{j=0}^{\infty }\vert \psi _{j}\vert $$ converges. This implies that the effect ψ j converges to zero as $$j \rightarrow \infty $$. Thus the effect of a shock dies out eventually⁵:

As can be seen from Eq. (2.5), the coefficients {ψ j } even converge to zero exponentially fast to zero because each term c i z i −j , i = 1, …, p, goes to zero exponentially fast as the roots z i are greater than one in absolute value. Viewing {ψ j } as a function of j one gets the so-called impulse response function which is usually displayed graphically.

The effect of a permanent shock in period t on X t+j is defined as the cumulative effect of a transitory shock. Thus, the effect of a permanent shock to X t+j is given by $$\sum _{i=0}^{j}\psi _{i}$$. Because $$\sum _{i=0}^{j}\psi _{i} \leq \sum _{i=0}^{j}\vert \psi _{i}\vert \leq \sum _{i=0}^{\infty }\vert \psi _{i}\vert < \infty $$, the cumulative effect remains finite.

In time series analysis we view the observations as realizations of {X t } and treat the realizations of {Z t } as unobserved. It is therefore of interest to know whether it is possible to recover the unobserved shocks from the observations on {X t }. This idea leads to the concept of invertibility.

Definition 2.3 (Invertibility).

An ARMA(p,q) process for {X t } satisfying  $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$ is called invertible with respect to {Z t } if and only if there exists a sequence {π j } with the property $$\sum _{j=0}^{\infty }\vert \pi _{j}\vert < \infty $$ such that

Note that like causality, invertibility is not an attribute of {X t }, but is defined only relative to another process {Z t }. In the literature, one often refers to invertibility as the strict miniphase property.⁶

Theorem 2.2.

Let {X t } be an ARMA(p,q) process with $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$ such that polynomials $$\Phi (z)$$ and $$\Theta (z)$$ have no common roots. Then {X t } is invertible with respect to {Z t } if and only if $$\Theta (z)\neq 0$$ for |z|≤ 1. The coefficients {π j } are then uniquely determined through the relation:

Proof.

The proof follows from Theorem 2.1 with X t and Z t interchanged. □ 

The discussion in Sect. 1.​3 showed that there are in general two MA(1) processes compatible with the same autocorrelation function ρ(h) given by ρ(0) = 1, ρ(1) = ρ with $$\vert \rho \vert \leq \tfrac{1} {2}$$, and ρ(h) = 0 for h ≥ 2. However, only one of these solutions is invertible because the two solutions for $$\theta$$ are inverses of each other. As it is important to be able to recover Z t from current and past X t , one prefers the invertible solution. Section 3.​2 further elucidates this issue.

Remark 2.3.

If {X t } is a stationary solution to the stochastic difference equation $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$ with $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ and if $$\Phi (z)\Theta (z)\neq 0$$ for | z | ≤ 1 then

where the coefficients {ψ j } and {π j } are determined for | z | ≤ 1 by $$\Psi (z) = \dfrac{\Theta (z)} {\Phi (z)}$$ and $$\Pi (z) = \dfrac{\Phi (z)} {\Theta (z)}$$, respectively. In this case {X t } is causal and invertible with respect to {Z t }.

Remark 2.4.

If {X t } is an ARMA process with $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$ such that $$\Phi (z)\neq 0$$ for | z |  = 1 then there exists polynomials $$\tilde{\Phi }(z)$$ and $$\tilde{\Theta }(z)$$ and a white noise process $$\{\tilde{Z}_{t}\}$$ such that {X t } fulfills the stochastic difference equation $$\tilde{\Phi }(\mathrm{L})X_{t} =\tilde{ \Theta }(\mathrm{L})\tilde{Z}_{t}$$ and is causal with respect to $$\{\tilde{Z}_{t}\}$$. If in addition $$\Theta (z)\neq 0$$ for | z |  = 1 then $$\tilde{\Theta }(\mathrm{L})$$ can be chosen such that {X t } is also invertible with respect to $$\{\tilde{Z}_{t}\}$$ (see the discussion of the AR(1) process after the definition of causality and Brockwell and Davis (1991, p. 88)). Thus, without loss of generality, we can restrict the analysis to causal and invertible ARMA processes.

2.4 Computation of the Autocovariance Function of an ARMA Process

Whereas the autocovariance function summarizes the external and directly observable properties of a time series, the coefficients of the ARMA process give information of its internal structure. Although there exists for each ARMA model a corresponding autocovariance function, the converse is not true as we have seen in Sect. 1.​3 where we showed that two MA(1) processes are compatible with the same autocovariance function. This brings up a fundamental identification problem. In order to shed some light on the relation between autocovariance function and ARMA models it is necessary to be able to compute the autocovariance function for a given ARMA model. In the following, we will discuss three such procedures. Each procedure relies on the assumption that the ARMA process $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$ with $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ is causal with respect to {Z t }. Thus there exists a representation of X t as a weighted sum of current and past Z t ’s: $$X_{t} =\sum _{ j=0}^{\infty }\psi _{j}Z_{t-j}$$ with $$\sum _{j=0}^{\infty }\vert \psi _{j}\vert < \infty $$.

2.4.1 First Procedure

Starting from the causal representation of {X t }, it is easy to calculate its autocovariance function given that {Z t } is white noise. The exact formula is proved in Theorem (6.​4).

where

The first step consists in determining the coefficients ψ j by the method of undetermined coefficients. This leads to the following system of equations:

This equation system can be solved recursively (see Sect. 2.3):

Alternatively one may view the second part of the equation system as a linear homogeneous difference equation with constant coefficients (see Sect. 2.3). Its solution is given by Eq. (2.5). The first part of the equation system delivers the necessary initial conditions to determine the coefficients $$c_{1},c_{2},\ldots,c_{p}$$. Finally one can insert the ψ’s in the above formula for the autocovariance function.

A Numerical Example

Consider the ARMA(2,1) process with $$\Phi (\mathrm{L}) = 1 - 1.3\mathrm{L} + 0.4\mathrm{L}^{2}$$ and $$\Theta (\mathrm{L}) = 1 + 0.4\mathrm{\mathrm{L}}$$. Writing out the defining equation for $$\Psi (z)$$, $$\Psi (z)\Phi (z) = \Theta (z)$$, gives:

Equating the coefficients of the powers of z leads to the following equation system:

The last equation represents a linear difference equation of order two. Its solution is given by

whereby z ₁ and z ₂ are the two distinct roots of the characteristic polynomial $$\Phi (z) = 1 - 1.3z + 0.4z^{2} = 0$$ (see Eq. (2.5)) and where the coefficients c ₁ and c ₂ are determined from the initial conditions. The two roots are $$\frac{1.3\pm \sqrt{1.69-4\times 0.4}} {2\times 0.4} = 5/4 = 1.25$$ and 2. The general solution to the homogeneous equation therefore is $$\psi _{j} = c_{1}0.8^{j} + c_{2}0.5^{j}$$. The constants c ₁ and c ₂ are determined by the equations:

Solving this equation system in the two unknowns c ₁ and c ₂ gives: c ₁ = 4 and $$c_{2} = -3$$. Thus the solution to the difference equation is given by:

Inserting this solution for ψ j into the above formula for γ(h) one obtains after using the formula for the geometric sum:

Dividing γ(h) by γ(0), one gets the autocorrelation function:

which is represented in Fig. 2.3.

Fig. 2.3

Autocorrelation function of the ARMA(2,1) process: $$(1 - 1.3\mathrm{L} + 0.4\mathrm{L}^{2})X_{t} = (1 + 0.4\mathrm{L})Z_{t}$$

2.4.2 Second Procedure

Instead of determining the ψ j coefficients first, it is possible to compute the autocovariance function directly from the ARMA model. To see this multiply the ARMA equation successively by $$X_{t-h},h = 0,1,\ldots$$ and apply the expectations operator:

This leads to an equation system for the autocovariances γ(h), $$h = 0,1,2,\ldots$$:

The second part of the equation system consists again of a linear homogeneous difference equation in γ(h) whereas the first part can be used to determine the initial conditions. Note that the initial conditions depend ψ ₁, …, ψ q which have to be determined before hand. The general solution of the difference equation is:

(2.6)

where z ₁, …, z p are the distinct roots of the polynomial $$\Phi (z) = 1 -\phi _{1}z -\ldots -\phi _{p}z^{p} = 0$$.⁷ The constants c ₁, …, c p can be computed from the first p initial conditions after the ψ ₁, … ψ q have been calculated like in the first procedure. The form of the solution shows that the autocovariance and hence the autocorrelation function converges to zero exponentially fast.

A Numerical Example

We consider the same example as before. The second part of the above equation system delivers a difference equation for γ(h): $$\gamma (h) =\phi _{1}\gamma (h - 1) +\phi _{2}\gamma (h - 2) = 1.3\gamma (h - 1) - 0.4\gamma (h - 2)$$, h ≥ 2. The general solution of this difference equation is (see Appendix B):

where 0. 8 and 0. 5 are the inverses of the roots computed from the same polynomial $$\Phi (z) = 1 - 1.3z - 0.4z^{2} = 0$$.

The first part of the system delivers the initial conditions which determine the constants c ₁ and c ₂:

where the numbers on the right hand side are taken from the first procedure. Inserting the general solution in this equation system and bearing in mind that $$\gamma (h) =\gamma (-h)$$ leads to:

Solving this equation system in the unknowns c ₁ and c ₂ one gets finally gets: $$c_{1} = (220/9)\sigma ^{2}$$ and $$c_{2} = -8\sigma ^{2}$$.

2.4.3 Third Procedure

Whereas the first two procedures produce an analytical solution which relies on the solution of a linear difference equation, the third procedure is more suited for numerical computation using a computer. It rests on the same equation system as in the second procedure. The first step determines the values γ(0), γ(1), …, γ(p) from the first part of the equation system. The following γ(h),  h > p are then computed recursively using the second part of the equation system.

A Numerical Example

Using again the same example as before, the first of the equation delivers γ(2), γ(1) and γ(0) from the equation system:

Bearing in mind that $$\gamma (h) =\gamma (-h)$$, this system has three equations in three unknowns γ(0), γ(1) and γ(2). The solution is: $$\gamma (0) = (148/9)\sigma ^{2}$$, $$\gamma (1) = (140/9)\sigma ^{2}$$, $$\gamma (2) = (614/45)\sigma ^{2}$$. This corresponds, of course, to the same numerical values as before. The subsequent values for γ(h),  h > 2 are then determined recursively from the difference equation$$\gamma (h) = 1.3\gamma (h - 1) - 0.4\gamma (h - 2)$$.

2.5 Exercises

Exercise 2.5.1.

Consider the AR(1) process $$X_{t} = 0.8X_{t-1} + Z_{t}$$ with $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ . Compute the variance of $$(X_{1} + X_{2} + X_{3} + X_{4})/4$$ .

Exercise 2.5.2.

Check whether the following stochastic difference equations possess a stationary solution. If yes, is the solution causal and/or invertible with respect to $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ ?

(i)

(ii)

(iii)

(iv)

(v)

(vi)

Exercise 2.5.3.

Compute the causal representation with respect to $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ for the following ARMA processes:

(i)

(ii)

(iii)

$$X_{t} =\phi X_{t-1} + Z_{t} +\theta Z_{t-1}$$ with |ϕ| < 1

Exercise 2.5.4.

Compute the autocovariance function of the ARMA processes:

(i)

(ii)

Thereby $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ .

Exercise 2.5.5.

Verify that the process $$\{\widetilde{Z}_{t}\}$$ defined in Eq. (2.4)is white noise with $$\widetilde{Z}_{t} \sim \mathrm{ WN}(0,\theta ^{-2}\sigma ^{2})$$ .

References

Brockwell PJ, Davis RA (1991) Time series: theory and methods, 2nd edn. Springer, New YorkCrossRef

Phillips PCB, Sul D (2007) Some empirics on economic growth under heterogeneous technology. J Macroecon 29:455–469CrossRef

Potter SM (2000) Nonlinear impulse response functions. J Econ Dyn Control 24:1425–1446CrossRef

Footnotes

1

One technical advantage of using the double-infinite index set $$\mathbb{Z}$$ is that the lag operators form a group.

2

Phillips and Sul (2007) provide an application and an in depth discussion of the hypothesis of economic growth convergence.

3

The reader is invited to verify this.

4

In the case of multiple roots one has to modify the formula according to Eq. (B.2).

5

The use of the partial derivative sign actually represents an abuse of notation. It is inspired by an alternative definition of the impulse responses: $$\psi _{j} = \frac{\partial \tilde{\mathbb{P}}_{t}X_{t+j}} {\partial x_{t}}$$ where $$\tilde{\mathbb{P}}_{t}$$ denotes the optimal (in the mean squared error sense) linear predictor of X t+j given a realization back to infinite remote past $$\{x_{t},x_{t-1},x_{t-2},\ldots \}$$ (see Sect. 3.​1.​3). Thus, ψ j represents the sensitivity of the forecast of X t+j with respect to the observation x t . The equivalence of alternative definitions in the linear and especially nonlinear context is discussed in Potter (2000).

6

Without the qualification strict, the miniphase property allows for roots of $$\Theta (z)$$ on the unit circle. The terminology is, however, not uniform in the literature.

7

In case of multiple roots the formula has to be adapted accordingly. See Eq. (B.2) in the Appendix.

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_3

3. Forecasting Stationary Processes

Klaus Neusser1 

(1)

Bern, Switzerland

An important goal of time series analysis is forecasting. In the following we will consider the problem of forecasting X T+h , h > 0, given {X T , …, X ₁} where {X t } is a stationary stochastic process with known mean μ and known autocovariance function γ(h). In practical applications μ and γ are unknown so that we must replace these entities by their estimates. These estimates can be obtained directly from the data as explained in Sect. 4.​2 or indirectly by first estimating an appropriate ARMA model (see Chap. 5) and then inferring the corresponding autocovariance function using one of the methods explained in Sect. 2.​4 Thus the forecasting problem is inherently linked to the problem of identifying an appropriate ARMA model from the data (see Deistler and Neusser 2012).

3.1 The Theory of Linear Least-Squares Forecasts

We restrict our discussion to linear forecast functions, also called linear predictors , $$\mathbb{P}_{T}X_{T+h}$$. Given observation from period 1 up to period T, these predictors take the form:

with unknown coefficients a ₀, a ₁, a ₂, …, a T . In principle, we should index these coefficients by T because they may change with every new observations. See the example of the MA(1) process in Sect. 3.1.2. In order not to overload the notation, we will omit this additional index.

In the Hilbert space of random variables with finite second moments the optimal forecast in the mean squared error sense is given by the conditional expectation $$\mathbb{E}\left (X_{T+h}\vert c,X_{T},X_{T-1},\ldots,X_{1}\right )$$. However, having practical applications in mind, we restrict ourself to linear predictors for the following reasons¹:

(i)

The determination of the conditional expectation is usually very difficult because all possible functions must in principle be considered whereas linear predictors are easy to compute.

(ii)

The coefficients of the optimal (in the sense of means squared errors) linear forecasting function depend only on the first two moments of the time series, i.e. on $$\mathbb{E}X_{t}$$ and γ(j), $$j = 0,1,\ldots,h + T - 1$$.

(iii)

In the case of Gaussian processes the conditional expectation coincides with the linear predictor.

(iv)

The optimal predictor is linear when the process is a causal and invertible ARMA process even when Z t follows an arbitrary distribution with finite variance (see Rosenblatt 2000, chapter 5).

(v)

Practical experience has shown that even non-linear processes can be predicted accurately by linear predictors.

The coefficients a ₀, …, a T of the forecasting function are determined such that the mean squared errors are minimized. The use of mean squared errors as a criterion leads to a compact representation of the solution to the forecasting problem. It implies that over- and underestimation are treated equally. Thus, we have to solve the following minimization problem:

As S is a quadratic function, the coefficients, a j ,  j = 0, 1, …, T, are uniquely determined by the so-called normal equations . These are obtained from the first order conditions of the minimization problem, i.e. from $$\frac{\partial S} {\partial a_{j}} = 0,\ j = 0,1,\ldots,T$$:

(3.1)

(3.2)

The first equation can be rewritten as $$a_{0} =\mu -\sum _{i=1}^{T}a_{i}\mu$$ so that the forecasting function becomes:

The unconditional mean of the forecast error , $$\mathbb{E}\left (X_{T+h} - \mathbb{P}_{T}X_{T+h}\right )$$, is therefore equal to zero. This means that there is no bias, neither upward nor downward, in the forecasts. The forecasts correspond on average to the “true” value.

Inserting in the second normal equation the expression for $$\mathbb{P}_{T}X_{T+h}$$ from above, we get:

The forecast error is therefore uncorrelated with the available information represented by past observations. Thus, the forecast errors $$X_{T+h} - \mathbb{P}_{T}X_{T+h}$$ are orthogonal to X T , X T−1, …, X ₁. Geometrically speaking, the best linear forecast is obtained by finding the point in the linear subspace spanned by {X T , X T−1, …, X ₁} which is closest to X T+h . This point is found by projecting X T+h on this linear subspace.²

The normal equations (3.1) and (3.2) can be rewritten in matrix notation as follows:

(3.3)

(3.4)

Denoting by ι, α T and γ T (h) the vectors (1, 1, …, 1)′, $$\left (a_{1},\ldots,a_{T}\right )'$$ and $$\left (\gamma (h),\ldots,\gamma (h + T - 1)\right )'$$ and by $$\Gamma _{T} = [\gamma (i - j)]_{i,j=1,\ldots,T}$$ the symmetric T × T covariance matrix of $$\left (X_{1},\ldots,X_{T}\right )'$$ the normal equations can be written compactly as:

(3.5)

(3.6)

Dividing the second equation by γ(0), one obtains an equation in terms autocorrelations instead of autocovariances:

(3.7)

where $$R_{T} = \Gamma _{T}/\gamma (0)$$ and $$\rho _{T}(h) = \left (\rho (h),\ldots,\rho (h + T - 1)\right )'$$. The coefficients of the forecasting function α T are then obtained by inverting $$\Gamma _{T}$$, respectively R T :

A sufficient condition which ensures the invertibility of $$\Gamma _{T}$$, respectively R T , is given by assuming γ(0) > 0 and $$\lim _{h\rightarrow \infty }\gamma (h) = 0$$.³ The last condition is automatically satisfied for ARMA processes because γ(h) converges even exponentially fast to zero (see Sect. 2.​4).

The mean squared error or variance of the forecast error for the forecasting horizon h, v T (h), is given by:

because $$\Gamma _{T}\alpha _{T} =\gamma _{T}(h)$$. Bracketing out γ(0), one can write the mean squared forecast error as:

(3.8)

Because the coefficients of the forecast function have to be recomputed with the arrival of every new observation, it is necessary to have a fast and reliable algorithm at hand. These numerical problems have been solved by the development of appropriate computer algorithms, like the Durbin-Levinson algorithm or the innovation algorithm (see Brockwell and Davis 1991, Chapter 5).

3.1.1 Forecasting with an AR(p) Process

Consider first the case of an AR(1) process:

The equation system (3.7) becomes:

The guess-and-verify method immediately leads to the solution:

We therefore get the following predictor:

The forecast therefore just depends on the last observation with the corresponding coefficient a ₁ = ϕ h being independent of T. All previous observations can be disregarded, they cannot improve the forecast further. To put it otherwise, all the useful information about X T+h in the entire realization previous to X T , i.e. in {X T , X T−1, …, X ₁}, is contained in X T .

The variance of the prediction error is given by

For h = 1, the formula simplifies to $$\sigma ^{2}$$ and for $$h \rightarrow \infty $$, $$v_{T}(h) \rightarrow \frac{1} {1-\phi ^{2}} \sigma ^{2}$$ the unconditional variance of X t . Note also that the variance of the forecast error v T (h) increases with h.

The general case of an AR(p) process, p > 1, can be treated in the same way. The autocovariances follow a p-th order difference equation (see Sect. 2.​4):

Applying again the guess-and-verify method for the case h = 1 and assuming that T > p, the solution is given by $$\alpha _{T} = \left (\phi _{1},\phi _{2},\ldots,\phi _{p},0,\ldots,0\right )'$$. Thus the one-step ahead predictor is

(3.9)

The one-step ahead forecast of an AR(p) process therefore depends only on the last p observations.

The above predictor can also be obtained in a different way. View for this purpose $$\mathbb{P}_{T}$$ as an operator with the following meaning: Take the linear least-squares forecast with respect to the information {X T , …, X ₁}. Apply this operator to the defining stochastic difference equation of the AR(p) process forwarded one period:

In period T observations of X T , X T−1, …, X ₁ are known so that $$\mathbb{P}_{T}X_{T-j} = X_{T-j}$$, $$j = 0,1,\ldots,T - 1$$. Because {Z t } is a white noise process and because {X t } is a causal function with respect to {Z t }, Z T+1 is uncorrelated with X T , …, X ₁. This reasoning leads to the same predictor as in Eq. (3.9).

The forecasting functions for h > 1 can be obtained recursively by successively applying the forecast operator. Take, for example, the case h = 2:

In this way forecasting functions for h > 2 can be obtained recursively.

Note that in the case of AR(p) processes the coefficient of the forecast function remain constant as long as T > p. Thus with each new observation it is not necessary to recompute the equation system and solve it again. This will be different in the case of MA processes. In practice, the parameters of the AR model are usually unknown and have therefore be replaced by some estimate. Section 14.​2 investigates in a more general context how this substitution affects the results.

3.1.2 Forecasting with MA(q) Processes

The forecasting problem becomes more complicated in the case of MA(q) processes. In order to get a better understanding we analyze the case of a MA(1) process:

Taking a forecast horizon of one period, i.e. h = 1, the equation system (3.7) in the case of a MA(1) process becomes:

(3.10)

Despite the fact that the equation system has a simple structure, the forecasting function will depend in general on all past observations of X T−j , 0 ≤ j ≤ T. We illustrate this point by a numerical example which will allow us to get a deeper understanding.

Suppose that we know the parameters of the MA(1) process to be $$\theta = -0.9$$ and $$\sigma ^{2} = 1$$. We start the forecasting exercise in period T = 0 and assume that, at this point in time, we have no observation at hand. The best forecast is therefore just the unconditional mean which in this example is zero. Thus, $$\mathbb{P}_{0}X_{1} = 0$$. The variance of the forecast error then is $$\mathbb{V}(X_{1} - \mathbb{P}_{0}X_{1}) = v_{0}(1) =\sigma ^{2} +\theta ^{2}\sigma ^{2} = 1.81$$. This result is summarized in the first row of Table 3.1. In period 1, the realization of X ₁ is observed. This information can be used and the forecasting function becomes $$\mathbb{P}_{1}X_{2} = a_{1}X_{1}$$. The coefficient a ₁ is found by solving the equation system (3.10) for T = 1. This gives $$a_{1} =\theta /(1 +\theta ^{2}) = -0.4972$$. The corresponding variance of the forecast error according to Eq. (3.8) is $$\mathbb{V}(X_{2} - \mathbb{P}_{1}X_{2}) = v_{1}(1) =\gamma (0)(1 -\alpha _{1}'\rho _{1}(1)) = 1.81(1 - 0.4972 \times 0.4972) = 1.3625$$. This value is lower compared to the previous forecast because additional information, the observation of the realization of X ₁, is taken into account. Row 2 in Table 3.1 summarizes these results.

Table 3.1

Forecast function for a MA(1) process with $$\theta = -0.9$$ and $$\sigma ^{2} = 1$$

Time

Forecasting function α T  = (a ₁, a ₂, …, a T )′

Forecast error variance

T = 0: 

v ₀(1) = 1. 8100

T = 1: 

v ₁(1) = 1. 3625

T = 2: 

v ₂(1) = 1. 2155

T = 3: 

v ₃(1) = 1. 1436

T = 4: 

v ₄(1) = 1. 1017

In period 2, not only X ₁, but also X ₂ is observed which allows us to base our forecast on both observations: $$\mathbb{P}_{2}X_{3} = a_{1}X_{2} + a_{2}X_{1}$$. The coefficients can be found by solving the equation system (3.10) for T = 2. This amounts to solving the simultaneous equation system

Inserting $$\theta = -0.9$$, the solution is $$\alpha _{2} = (a_{1},a_{2})' = (-0.6606,-0.3285)'$$. The corresponding variance of the forecast error becomes

These results are summarized in row 3 of Table 3.1.

In period 3, the realizations of X ₁, X ₂ and X ₃ are known so that the forecast function becomes $$\mathbb{P}_{3}X_{4} = a_{1}X_{3} + a_{2}X_{2} + a_{3}X_{1}$$. The coefficients can again be found by solving the equation system (3.10) for T = 3:

For $$\theta = -0.9$$, the coefficients of the linear predictor are $$\alpha _{2} = (a_{1},a_{2},a_{3})' = (-0.7404,-0.4891,-0.2432)'$$. The corresponding variance of the forecast error becomes

These results are summarized in row 4 of Table 3.1. We can, of course, continue in this way and derive successively the forecast functions for T = 4, 5, ….

From this exercise we can make several observations.

In contrast to the AR process, every new information is used. The forecast $$\mathbb{P}_{T}X_{T+1}$$ depends on all available information, in particular on X T , X T−1, …, X ₁.

The coefficients of the forecast function are not constant. They change as more and more information comes in.

The importance of the new information can be “measured” by the last coefficient of α T . These coefficients are termed partial autocorrelations (see Definition 3.2) and are of particular relevance as will be explained in Sect. 3.5. In our example they are $$-0.4972,-0.3285,-0.2432$$, and − 0. 1914.

As more information becomes available, the variance of the forecast error (mean squared error) declines monotonically. It will converge to $$\sigma ^{2} = 1$$. The reason for this result can be explained as follows. Applying the forecasting operator to the defining MA(1) stochastic difference equation forwarded by one period gives: $$\mathbb{P}_{T}X_{T+1} = \mathbb{P}_{T}Z_{T+1} +\theta \mathbb{P}_{T}Z_{T} =\theta \mathbb{P}_{T}Z_{T}$$ with forecast error $$X_{T+1} - \mathbb{P}_{T}X_{T+1} = Z_{T+1}$$. As more and more observation become available, it becomes better and better possible to recover the “true” value of the unobserved Z T from the observations X T , X T−1, …, X ₁. As the process is invertible, in the limit it is possible to recover the value of Z T exactly (almost surely). The only uncertainty remaining is with respect to Z T+1 which has a mean of zero and a variance of $$\sigma ^{2} = 1$$.

3.1.3 Forecasting from the Infinite Past

The forecasting function based on the infinitely remote past is of particular theoretical interest. Thereby we look at the problem of finding the optimal linear forecast of X T+1 given $$X_{T},X_{T-1},\ldots,X_{1},X_{0},X_{-1},\ldots$$ taking the mean squared error again as the criterion function. The corresponding forecasting function (predictor) will be denoted by $$\widetilde{\mathbb{P}}_{T}X_{T+h}$$, h > 0.

Noting that the MA(1) process with $$\vert \theta \vert < 1$$ is invertible, we have

We can therefore write X t+1 as

The predictor of X T+1 from the infinite past, $$\widetilde{\mathbb{P}}_{T}$$, is then given by:

where the mean squared forecasting error is

Applying this result to our example gives:

with $$v_{\infty }(1) = 1$$. See last row in Table 3.1.

Example of an ARMA(1,1) Process

Consider now the case of a causal and invertible ARMA(1,1) process {X t }:

where | ϕ |  < 1, $$\vert \theta \vert < 1$$ and $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$. Because {X t } is causal and invertible with respect to {Z t },

Applying the forecast operator $$\widetilde{\mathbb{P}}_{T}$$ to the second equation and noting that $$\widetilde{\mathbb{P}}_{T}Z_{T+1} = 0$$, one obtains the following one-step ahead predictor

Applying the forecast operator to the first equation, we obtain

This implies that the one-step ahead prediction error is equal to $$X_{T+1} -\widetilde{ \mathbb{P}}_{T}X_{T+1} = Z_{T+1}$$ and that the mean squared forecasting error of the one-step ahead predictor given the infinite past is equal to $$\mathbb{E}Z_{T+1}^{2} =\sigma ^{2}$$.

3.2 The Wold Decomposition Theorem

The Wold Decomposition theorem is essential for the theoretical understanding of stationary stochastic processes. It shows that any stationary process can essentially be represented as a linear combination of current and past forecast errors. Before we can state the theorem precisely, we have to introduce the following definition.

Definition 3.1 (Deterministic Process).

A stationary stochastic process {X t } is called (purely) deterministic or (purely) singular if and only if it can be forecasted exactly from the infinite past. More precisely, if and only if

where $$\widetilde{\mathbb{P}}_{t}X_{t+1}$$ denotes the best linear forecast of X t+1 given its infinite past, i.e. given {X t ,X t−1 ,…}.

The most important class of deterministic processes are the harmonic processes . These processes are characterized by finite or infinite sums of sine and cosine functions with stochastic amplitude.⁴ A simple example of a harmonic process is given by

Thereby, A and B denote two uncorrelated random variables with mean zero and finite variance. One can check that X t satisfies the deterministic difference equation

Thus, X t can be forecasted exactly from its past. In this example the last two observations are sufficient. We are now in a position to state the Wold Decomposition Theorem.

Theorem 3.1 (Wold Decomposition)

Every stationary stochastic process {X t } with mean zero and finite positive variance can be represented as

(3.11)

where

(i)

$$Z_{t} = X_{t} -\widetilde{ \mathbb{P}}_{t-1}X_{t} =\widetilde{ \mathbb{P}}_{t}Z_{t}$$ ;

(ii)

$$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ with $$\sigma ^{2} = \mathbb{E}\left (X_{t+1} -\widetilde{ \mathbb{P}}_{t}X_{t+1}\right )^{2} > 0$$ ;

(iii)

ψ ₀ = 1 and $$\sum _{j=0}^{\infty }\psi _{j}^{2} < \infty $$ ;

(iv)

{V t } is deterministic;

(v)

$$\mathbb{E}(Z_{t}V _{s}) = 0$$ for all $$t,s \in \mathbb{Z}$$ .

The sequences {ψ j }, {Z t }, and {V t } are uniquely determined by (3.11).

Proof.

The proof, although insightful, requires some knowledge about Hilbert spaces which is beyond the scope of this book. A rigorous proof can be found in Brockwell and Davis (1991, Section 5.7).

It is nevertheless instructive to give an intuition of the proof. Following the MA(1) example from the previous section, we start in period 0 and assume that no information is available. Thus, the best forecast $$\mathbb{P}_{0}X_{1}$$ is zero so that trivially

Starting with X ₁ = Z ₁, X ₂, X ₃, … can then be constructed recursively:

where a j (t−1), $$j = 1,\ldots,t - 1$$, denote the coefficients of the forecast function for X t based on X t−1, …, X ₁. This shows how X t unfolds into the sum of forecast errors. The stationarity of {X t } ensures that the coefficients of Z j converge, as t goes to infinity, to ψ j which are independent of t. □ 

Every stationary stochastic process is thus representable as the sum of a moving-average of infinite order and a (purely) deterministic process.⁵ The weights of the infinite moving average are thereby normalized such that ψ ₀ = 1. In addition, the coefficients ψ j are square summable. This property is less strong than absolute summability which is required for a causal representation (see Definition 2.​2).⁶ The process {Z t } is a white noise process with positive variance $$\sigma ^{2} > 0$$. The Z t ’s are called innovations as they represent the one-period ahead forecast errors based on the infinite past, i.e $$Z_{t} = X_{t} -\widetilde{ \mathbb{P}}_{t-1}X_{t}$$. Z t is the additional information revealed from the t-th observation. Thus, the Wold Decomposition Theorem serves as a justification for the use of causal ARMA models. In this instance, the deterministic component {V t } vanishes.

The second part of Property (i) further means that the innovation process {Z t } is fundamental with respect to {X t }, i.e. that Z t lies in the linear space spanned by $$\{X_{t},X_{t-1},X_{t-2},\ldots \}$$ or that $$Z_{t} =\widetilde{ \mathbb{P}}_{t}Z_{t}$$. This implies that $$\Psi (\mathrm{L})$$ must be invertible and that Z t can be perfectly (almost surely) recovered from the observations of X t , X t−1, … Finally, property (v) says that the two components {Z t } and {V t } are uncorrelated with each other at all leads and lags. Thus, in essence, the Wold Decomposition Theorem states that every stationary stochastic process can be uniquely decomposed into a weighted sum of current and past forecast errors plus a deterministic process.

Although the Wold Decomposition is very appealing from a theoretical perspective, it is not directly implementable in practice because it requires the estimation of infinitely many parameters (ψ ₁, ψ ₂, …). This is impossible with only a finite amount of observations. It is therefore necessary to place some assumptions on (ψ ₁, ψ ₂, …). One possibility is to assume that {X t } is a causal ARMA process and to recover the ψ j ’s from the causal representation. This amounts to say that $$\Psi (\mathrm{L})$$ is a rational polynomial which means that

Thus, the process is characterized by only a finite number, p + q, of parameters. Another possibility is to place restrictions on the smoothness of the spectrum (see Chap. 6).

The Wold Decomposition Theorem has several implications which are presented in the following remarks.

Remark 3.1.

In the case of ARMA processes, the purely deterministic part {V t } can be disregarded so that the process is represented only by a weighted sum of current and past innovations. Processes with this property are called purely non-deterministic, linearly regular, or regular for short. Moreover, it can be shown that every regular process {X t } can be approximated arbitrarily well by an ARMA process {X t (ARMA)} meaning that

can be made arbitrarily small. The proof of these results can be found in Hannan and Deistler (1988, Chapter 1).

Remark 3.2.

The process {Z t } is white noise, but not necessarily Gaussian. In particular, {Z t } need not be independently and identically distributed (IID). Thus, $$\mathbb{E}(Z_{t+1}\vert X_{t},X_{t-1},\ldots )$$ need not be equal to zero although $$\widetilde{\mathbb{P}}_{t}Z_{t+1} = 0$$. The reason is that $$\widetilde{\mathbb{P}}_{t}Z_{t+1}$$ is only the best linear forecast function, whereas $$\mathbb{E}(Z_{t+1}\vert X_{t},X_{t-1},\ldots )$$ is the best forecast function among all linear and non-linear functions. Examples of processes which are white noise, but not IID, are GARCH processes discussed in Chap. 8

Remark 3.3.

The innovations {Z t } may not correspond to the “true” shocks of the underlying economic system. In this case, the shocks to the economic system cannot be recovered from the Wold Decomposition. Thus, they are not fundamental with respect to {X t }. Suppose, as a simple example, that {X t } is generated by a noninvertible MA(1) process:

This generates an impulse response function with respect to the true shocks of the system equal to $$(1,\theta,0,\ldots )$$. The above mechanism can, however, not be the Wold Decomposition because the noninvertibility implies that U t cannot be recovered from the observation of {X t }. As shown in the introduction, there is an observationally equivalent MA(1) process, i.e. a process which generates the same ACF. Based on the computation in Sect. 1.​5, this MA(1) process is

with $$\tilde{\theta }=\theta ^{-1}$$ and $$\tilde{\sigma }^{2} = \tfrac{1+\theta ^{2}} {1+\theta ^{-2}} \sigma ^{2}$$. This is already the Wold Decomposition. The impulse response function for this process is $$(1,\theta ^{-1},0,\ldots )$$ which is different from the original system. As $$\vert \tilde{\theta }\vert = \vert \theta ^{-1}\vert < 1$$, the innovations {Z t } can be recovered from the observations as $$Z_{t} =\sum _{ j=0}^{\infty }(-\tilde{\theta })^{j}X_{t-j}$$, but they do not correspond to the shocks of the system {U t }. Hansen and Sargent (1991), Quah (1990), and Lippi and Reichlin (1993) among others provide a deeper discussion and present additional more interesting economic examples.

3.3 Exponential Smoothing

Besides the method of least-squares forecasting exponential smoothing can often be seen as a valid alternative. This method views X t as a function of time:

whereby f(t; β) typically represents a polynomial in t with coefficients β. The above equation is similar to a regression model with error term $$\varepsilon _{t}$$. This error term is usually specified as a white noise process $$\{\varepsilon _{t}\} \sim \mathrm{ WN}(0,\sigma ^{2})$$.

Consider first the simplest case where X t just moves randomly around a fixed mean β. This corresponds to the case where f(t; β) is a polynomial of degree zero:

If β is known then $$\mathbb{P}_{T}X_{T+h}$$, the forecast of X T+h given the observations X T , …, X ₁, clearly is β. If, however, β is unknown, we can substitute β by $$\overline{X}_{T}$$, the average of the observations:

where “$$\hat{\phantom{a}}$$” means that the model parameter β has been replaced by its estimate. The one-period ahead forecast function can then be rewritten as follows:

The first equation represents the forecast for T + 1 as a linear combination of the forecast for T and of the last additional information, i.e. the last observation. The weight given to the last observation is equal to 1∕T because we assumed that the mean remains constant and because the contribution of one observation to the mean is 1∕T. The second equation represents the forecast for T + 1 as the forecast for T plus a correction term which is proportional to the last forecast error. One advantage of this second representation is that the computation of the new forecast, i.e. the forecast for T + 1, only depends on the forecast for T and the additional observation. In this way the storage requirements are minimized.

In many applications, the mean does not remain constant, but is a slowly moving function of time. In this case it is no longer meaningful to give each observation the same weight. Instead, it seems plausible to weigh the more recent observation higher than the older ones. A simple idea is to let the weights decline exponentially which leads to the following forecast function:

ω thereby acts like a discount factor which controls the rate at which agents forget information. 1 −ω is often called the smoothing parameter. The value of ω should depend on the speed at which the mean changes. In case when the mean changes only slowly, ω should be large so that all observations are almost equally weighted; in case when the mean changes rapidly, ω should be small so that only the most recent observations are taken into account. The normalizing constant $$\frac{1-\omega } {1-\omega ^{T}}$$ ensures that the weights sum up to one. For large T the term ω T can be disregarded so that one obtains the following forecasting function based on simple exponential smoothing:

In the economics literature this forecasting method is called adaptive expectation . Similar to the model with constant mean, the new forecast is a weighted average between the old forecast and the last (newest) observation, respectively between the previous forecast and a term proportional to the last forecast error.

One important advantage of adaptive forecasting methods is that they can be computed recursively. Starting with value S ₀, the following values can be computed as follows:

Thereby S ₀ has to be determined. Because

the effect of the starting value declines exponentially with time. In practice, we can take S ₀ = X ₁ or $$S_{0} = \overline{X}_{T}$$. The discount factor ω is usually set a priori to be a number between 0.7 and 0.95. It is, however, possible to determine ω optimally by choosing a value which minimizes the mean squared one-period forecast error:

From a theoretical perspective one can ask the question for which class of models exponential smoothing represents the optimal procedure. Muth (1960) showed that this class of models is given by

Note that the process generated by the above equation is no longer stationary. This has to be expected as the exponential smoothing assumes a non-constant mean. Despite the fact that this class seems rather restrictive at first, practice has shown that it delivers reasonable forecasts, especially in situations when it becomes costly to specify a particular model.⁷ Additional results and more general exponential smoothing methods can be found in Abraham and Ledolter (1983) and Mertens and Rässler (2005).

3.4 Exercises

Exercise 3.4.1.

Compute the linear least-squares predictor $$\mathbb{P}_{T}X_{T+h}$$ , T > 2, and the mean squared error v T (h), h = 1,2,3, if {X t } is given by the AR(2) process

To which values do $$\mathbb{P}_{T}X_{T+h}$$ and v T (h) converge for h going to infinity?

Exercise 3.4.2.

Compute the linear least-squares predictor $$\mathbb{P}_{T}(X_{T+1)}$$ and the mean squared error v T (1), T = 0,1,2,3, if {X t } is given by the MA(1) process

To which values do $$\mathbb{P}_{T}X_{T+h}$$ and v T (h) converge for h going to infinity?

Exercise 3.4.3.

Suppose that you observe {X t } for the two periods t = 1 and t = 3, but not for t = 2.

(i)

Compute the linear least-squares forecast for X ₂ if

Compute the mean squared error for this forecast.

(ii)

Assume now that {X t } is the MA(1) process

Compute the mean squared error for the forecast of X ₂ .

Exercise 3.4.4.

Let

with A and B being two uncorrelated random variables with mean zero and finite variance. Show that {X t } satisfies the deterministic difference equation:

3.5 The Partial Autocorrelation Function

Consider again the problem of forecasting X T+1 from observations X T , X T−1, …, X ₂, X ₁. Denoting, as before, the best linear predictor by $$\mathbb{P}_{T}X_{T+1} = a_{1}X_{T} + a_{2}X_{T-1} + a_{T-1}X_{2} + a_{T}X_{1}$$, we can express X T+1 as

where Z T+1 denotes the forecast error which is uncorrelated with X T , …, X ₁. We can now ask the question whether X ₁ contributes to the forecast of X T+1 controlling for X T , X T−2, …, X ₂ or, equivalently, whether a T is equal to zero. Thus, a T can be viewed as a measure of the importance of the additional information provided by X ₁. It is referred to as the partial autocorrelation. In the case of an AR(p) process, the whole information useful for forecasting X T+1, T > p, is incorporated in the last p observations so that a T  = 0. In the case of the MA process, the observations on X T , …, X ₁ can be used to retrieve the unobserved $$Z_{T},Z_{T-1}\ldots,Z_{t-q+1}$$. As Z t is an infinite weighted sum of past X t ’s, every new observation contributes to the recovering of the Z t ’s. Thus, the partial autocorrelation a T is not zero. Taking T successively equal to 0, 1, 2, etc. we get the partial autocorrelation function (PACF).

We can, however, interpret the above equation as a regression equation. From the Frisch-Lovell-Waugh Theorem (See Davidson and MacKinnon 1993), we can obtain a T by a two-stage procedure. Project (regress) in a first stage X T+1 on X T , …, X ₂ and take the residual. Similarly, project (regress) X ₁ on X T , …, X ₂ and take the residual. The coefficient a T is then obtained by projecting (regressing) the first residual on the second. Stationarity implies that this is nothing but the correlation coefficient between the two residuals.

3.5.1 Definition

The above intuition suggests two equivalent definitions of the partial autocorrelation function (PACF).

Definition 3.2 (Partial Autocorrelation Function I).

The partial autocorrelation function (PACF) α(h),h = 0,1,2,…, of a stationary process is defined as follows:

where a h denotes the last element of the vector $$\alpha _{h} = \Gamma _{h}^{-1}\gamma _{h}(1) = R_{h}^{-1}\rho _{h}(1)$$ (see Sect.  3.1 and Eq. (3.7)).

Definition 3.3 (Partial Autocorrelation Function II).

The partial autocorrelation function (PACF) α(h),h = 0,1,2,…, of a stationary process is defined as follows:

where $$\mathbb{P}\left (X_{h+1}\vert 1,X_{2},\ldots,X_{h}\right )$$ and $$\mathbb{P}\left (X_{1}\vert 1,X_{2},\ldots,X_{h}\right )$$ denote the best, in the sense mean squared forecast errors, linear forecasts of X h+1 , respectively X ₁ given {1,X ₂ ,…,X h }.

Remark 3.4.

If {X t } has a mean of zero, then the constant in the projection operator can be omitted.

The first definition implies that the partial autocorrelations are determined from the coefficients of the forecasting function which are themselves functions of the autocorrelation coefficients. It is therefore possible to express the partial autocorrelations as a function of the autocorrelations. More specifically, the partial autocorrelation functions can be computed recursively from the autocorrelation function according to the Durbin-Levinson algorithm (Durbin 1960):

where $$a_{h,j} = a_{h-1,j} - a_{hh}a_{h-1,h-j}$$ for $$j = 1,2,\ldots,h - 1$$.

Autoregressive Processes

The idea of the PACF can be well illustrated in the case of an AR(1) process

As shown in Chap. 2, X t and X t−2 are correlated with each other despite the fact that there is no direct relationship between the two. The correlation is obtained “indirectly” because X t is correlated with X t−1 which is itself correlated with X t−2. Because both correlation are equal to ϕ, the correlation between X t and X t−2 is equal to ρ(2) = ϕ ². The ACF therefore accounts for all correlation, including the indirect ones. The partial autocorrelation on the other hand only accounts for the direct relationships. In the case of the AR(1) process, there is only an indirect relation between X t and X t−h for h ≥ 2, thus the PACF is zero.

Based on the results in Sect. 3.1 for the AR(1) process, the definition 3.2 of the PACF implies:

The partial autocorrelation function of an AR(1) process is therefore equal to zero for h ≥ 2.

This logic can be easily generalized. The PACF of a causal AR(p) process is equal to zero for h > p, i.e. α(h) = 0 for h > p. This property characterizes an AR(p) process as shown in the next section.

Moving-Average Processes

Consider now the case of an invertible MA process . For this process we have:

X t is therefore “directly” correlated with each X t−h , h = 1, 2, …. Consequently, the PACF is never exactly equal to zero, but converges exponentially to zero. This convergence can be monotonic or oscillating.

Take the MA(1) process as an illustration:

The computations in Sect. 3.1.2 showed that

Thus we get for the MA(1) process:

3.5.2 Interpretation of ACF and PACF

The ACF and the PACF are two important tools to determining the nature of the underlying mechanism of a stochastic process. In particular, they can be used to determine the orders of the underlying AR, respectively MA processes. The analysis of ACF and PACF to identify appropriate models is know as the Box-Jenkins methodology (Box and Jenkins 1976). Table 3.2 summarizes the properties of both tools for the case of a causal AR and an invertible MA process.

Table 3.2

Properties of the ACF and the PACF

Processes

ACF

PACF

AR(p)

Declines exponentially (monotonically or oscillating) to zero

α(h) = 0 for h > p

MA(q)

ρ(h) = 0 for h > q

Declines exponentially (monotonically or oscillating) to zero

If {X t } is a causal and invertible ARMA(p,q) process, we have the following properties. As shown in Sect. 2.​4, the ACF is characterized for $$h >\max \{ p,q + 1\}$$ by the homogeneous difference equation $$\rho (h) =\phi _{1}\rho (h - 1) +\ldots +\phi _{p}\rho (h - p)$$. Causality implies that the roots of the characteristic equation are all inside the unit circle. The autocorrelation coefficients therefore decline exponentially to zero. Whether this convergence is monotonic or oscillating depends on the signs of the roots. The PACF starts to decline to zero for h > p. Thereby the coefficients of the PACF exhibit the same behavior as the autocorrelation coefficients of $$\theta ^{-1}(\mathrm{L})X_{t}$$.

3.6 Exercises

Exercise 3.6.1.

Assign the ACF and the PACF from Fig.  3.1 to the following processes:

with $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ .

Fig. 3.1

Autocorrelation and partial autocorrelation functions. (a ) Process 1. (b ) Process 2. (c ) Process 3. (d ) Process 4

References

Abraham B, Ledolter J (1983) Statistical methods for forecasting. Wiley, New YorkCrossRef

Box GEP, Jenkins GM (1976) Time series analysis: forecasting and control, revised edn. Holden-Day, San Francisco

Brockwell PJ, Davis RA (1991) Time series: theory and methods, 2nd edn. Springer, New YorkCrossRef

Davidson R, MacKinnon JG (1993) Estimation and inference in econometrics. Oxford University Press, Oxford

Deistler M, Neusser K (2012) Prognosen uni- und multivariater Zeitreihen. In: Mertens P (ed) Prognoserechnung. Physica-Verlag, Heidelberg, pp 225–256CrossRef

Durbin J (1960) The fitting of time series models. Revue de l’Institut International de Statistique 28:233–244CrossRef

Elliott G, Timmermann A (2008) Economic forecasting. J Econ Lit 46:3–56CrossRef

Hannan EJ, Deistler M (1988) The statistical theory of linear systems. Wiley, New York

Hansen LP, Sargent TJ (1991) Two difficulties in interpreting vector autoregressions. In: Hansen LP, Sargent TJ (eds) Rational expectations econometrics, underground classics in economics. Westview, Boulder, Colorado, pp 77–119

Lippi M, Reichlin L (1993) The dynamic effects of aggregate demand and supply disturbances: Comment. Am Econ Rev 83:644–652

Mertens P, Rässler S (2005) Prognoserechnung - einführung und Überblick. In: Mertens P, Rässler S (eds) Prognoserechnung. Physica-Verlag, Heidelberg, pp 1–37CrossRef

Muth J (1960) Optimal properties of exponentially weighted forecasts. J Am Stat Assoc 55:299–306CrossRef

Quah D (1990) Permanent and transitory movements in labor income: An explanation for ‘excess smoothnes’ in consumption. J Polit Econ 98:449–475CrossRef

Rosenblatt M (2000) Gaussian and Non-Gaussian linear time series and random fields. Springer, New YorkCrossRef

Footnotes

1

Elliott and Timmermann (2008) provide a general overview of forecasting procedures and their evaluations.

2

Note the similarity of the forecast errors with the least-square residuals of a linear regression.

3

See Brockwell and Davis (1991, p. 167).

4

More about harmonic processes can be found in Sect. 6.​2

5

The Wold Decomposition corresponds to the decomposition of the spectral distribution function of F into the sum of F Z and F V (see Sect. 6.​2). Thereby the spectral distribution function F Z has spectral density $$f_{Z}(\lambda ) = \tfrac{\sigma ^{2}} {2\pi } \vert \Psi (e^{-\imath \lambda })\vert ^{2}$$.

6

The series $$\psi _{j} = 1/j$$, for example, is square summable, but not absolutely summable.

7

This happens, for example, when many, perhaps thousands of time series have to be forecasted in a real time situation.

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_4

4. Estimation of the Mean and the Autocorrelation Function

Klaus Neusser1 

(1)

Bern, Switzerland

In the previous chapters we have seen in which way the mean μ, and, more importantly, the autocovariance function, γ(h),  h = 0, ±1, ±2, …, of a stationary stochastic process {X t } characterize its dynamic properties, at least if we restrict ourself to the first two moments. In particular, we have investigated how the autocovariance function is related to the coefficients of the corresponding ARMA process. Thus the estimation of the ACF is not only interesting for its own sake, but also for the specification and identification of appropriate ARMA models. It is therefore of outmost importance to have reliable (consistent) estimators for these entities. Moreover, we want to test specific features for a given time series. This means that we have to develop corresponding testing theory. As the small sample distributions are hard to get, we rely for this purpose on asymptotic theory.¹

In this section we will assume that the process is stationary and observed for the time periods t = 1, 2, …, T. We will refer to T as the sample size. As mentioned previously, the standard sampling theory is not appropriate in the times series context because the X t ’s are not independent draws from some underlying distribution, but are systematically related to each other.

4.1 Estimation of the Mean

The arithmetic average constitutes a “natural” estimator of the mean μ of the stochastic process. The arithmetic mean $$\overline{X}_{T}$$ is defined as usual by

It is immediately clear that the arithmetic average is an unbiased estimator of the mean:

Of greater interest are the asymptotic properties of the variance of the arithmetic mean $$\mathbb{V}\overline{X}_{T}$$ which are summarized in the following theorem:

Theorem 4.1 (Convergence of Arithmetic Average).

If {X t } is a stationary stochastic process with mean μ and ACF γ(h) then the variance of the arithmetic mean $$\mathbb{V}\overline{X}_{T}$$ has the following asymptotic properties:

for T going to infinity.

Proof.

Immediate algebra establishes:

The assumption $$\gamma (h) \rightarrow 0$$ for $$h \rightarrow \infty$$ implies that for any given $$\varepsilon> 0$$, we can find T ₀ such that $$\vert \gamma (h)\vert <\varepsilon /2$$ for h ≥ T ₀. If T > T ₀ and $$T> 2T_{0}\,\gamma (0)/\varepsilon$$ then

Therefore $$\mathbb{V}\overline{X}_{T}$$ converges to zero for $$T \rightarrow \infty$$ which establishes the first property. Moreover, we have

The infinite sum $$\sum _{h=-\infty }^{\infty }\gamma (h)$$ converges because it converges absolutely by assumption. □ 

This Theorem establishes that the arithmetic average is not only an unbiased estimator of the mean, but also a consistent one. In particular, the arithmetic average converges in the mean-square sense, and therefore also in probability, to the true mean (see appendix C). This result can be interpreted as a reflection of the concept of ergodicity (see Sect. 1.​2). The assumptions are relatively mild and are fulfilled for the ARMA processes because for these processes γ(h) converges exponentially fast to zero (see Sect. 2.​4.​2, in particular Eq. (2.​6)). Under little more restrictive assumptions it is even possible to show that the arithmetic mean is asymptotically normally distributed.

Theorem 4.2 (Asymptotic Distribution of Sample Mean).

For any stationary process {X t } given by

such that $$\sum _{j=-\infty }^{\infty }\vert \psi _{j}\vert <\infty$$ and $$\sum _{j=-\infty }^{\infty }\psi _{j}\neq 0$$ , the arithmetic average $$\overline{X}_{T}$$ is asymptotically normal:

where γ is the autocovariance function of {X t }.

Proof.

The standard proof invokes the Basic Approximation Theorem C.14 and the Central Limit Theorem for m-dependent processes C.13. To this end we define the 2m-dependent approximate process

For {X t (m)}, we have $$V _{m} =\sum _{ h=-m}^{m}\gamma (h) =\sigma ^{2}(\sum _{j=-m}^{m}\psi _{j})^{2}$$. This last assertion can be verified by noting that

Note that the assumption $$\sum _{j=-\infty }^{\infty }\vert \psi _{j}\vert <\infty$$ guarantees the convergence of the infinite sums. Applying this result to the special case ψ j  = 0 for | j | > m, we obtain V m .

The arithmetic average of the approximating process is

The CLT for m-dependent processes C.13 then implies that for $$T \rightarrow \infty$$

As $$m \rightarrow \infty$$, $$\sigma ^{2}(\sum _{j=-m}^{m}\psi _{j})^{2}$$ converges to $$\sigma ^{2}(\sum _{j=-\infty }^{\infty }\psi _{j})^{2}$$ and thus

This assertion can be established by noting that the characteristic functions of X (m) approaches the characteristic function of X so that by Theorem C.11 $$X^{(m)}\mathop{\longrightarrow}\limits_{}^{\mathrm{\quad d\quad }}X$$.

Finally, we show that the approximation error becomes negligible as T goes to infinity:

where the error e t (m) is

Clearly, {e t (m)} is a stationary process with autocovariance function γ e such that $$\sum _{h=-\infty }^{\infty }\gamma _{e}(h) =\sigma ^{2}\left (\sum _{\vert j\vert>m}\psi _{j}\right )^{2} <\infty$$. We can therefore invoke Theorem 4.1 to show that

converges to $$\sigma ^{2}\left (\sum _{\vert j\vert>m}\psi _{j}\right )^{2}$$ as $$T \rightarrow \infty$$. This term converges to zero as $$m \rightarrow \infty$$. The approximation error $$\sqrt{T}\left (\overline{X}_{T}-\mu \right ) -\sqrt{T}\left (\overline{X}_{T}^{(m)}-\mu \right )$$ therefore converges in mean square to zero and thus, using Chebyschev’s inequality (see Theorem C.3 or C.7), also in probability. We have therefore established the third condition of Theorem C.14 as well. Thus, we can conclude that $$\sqrt{T}\left (\overline{X}_{T}-\mu \right )\mathop{\longrightarrow}\limits_{}^{\mathrm{\quad d\quad }}X$$. □ 

Under a more restrictive summability condition which holds, however, within the context of causal ARMA processes, we can provide a less technical proof. This proof follows an idea of Phillips and Solo (1992) and is based on the Beveridge-Nelson decomposition (see Appendix D).²

Theorem 4.3.

For any stationary process

with the properties $$Z_{t} \sim \mathrm{ IID}(0,\sigma ^{2})$$ and $$\sum _{j=0}^{\infty }j^{2}\vert \psi _{j}\vert ^{2} <\infty$$ , the arithmetic average $$\overline{X}_{T}$$ is asymptotically normal:

Proof.

The application of the Beveridge-Nelson decomposition (see Theorem D.1 in Appendix D) leads to

The assumption $$Z_{t} \sim \mathrm{ IID}(0,\sigma ^{2})$$ allows to invoke the Central Limit Theorem C.12 of Appendix C to the first term. Thus, $$\sqrt{T}\;\tfrac{\sum _{t=1}^{T}Z_{ t}} {T}$$ is asymptotical normal with mean zero and variance $$\sigma ^{2}$$. Theorem D.1 also implies $$\vert \Psi (1)\vert <\infty$$. Therefore, the term $$\Psi (1)\sqrt{T}\;\tfrac{\sum _{t=1}^{T}Z_{ t}} {T}$$ is asymptotically normal with mean zero and variance $$\sigma ^{2}\Psi (1)^{2}$$.

The variances of the second and third term are equal to $$\tfrac{\sigma ^{2}} {T}\sum _{j=0}^{T}\tilde{\psi }_{ j}^{2}$$. The summability condition then implies according to Theorem D.1 that $$\sum _{j=0}^{T}\tilde{\psi }_{j}^{2}$$ converges for $$T \rightarrow \infty$$. Thus, the variances of the last two terms converge to zero implying that these terms converge also to zero in probability (see Theorem C.7) and thus also in distribution. We can then invoke Theorem C.10 to establish the Theorem. Finally, the equality of $$\sum _{h=-\infty }^{\infty }\gamma (h)$$ and $$\sigma ^{2}\Psi (1)^{2}$$ can be obtained from direct computations or by the application of Theorem 6.​4. □ 

Remark 4.1.

Theorem 4.2 holds with respect to any causal ARMA process because the ψ j ’s converge exponentially fast to zero (see the discussion following Eq. (2.​5)).

Remark 4.2.

If {X t } is a Gaussian process, then for any given fixed T, $$\overline{X}_{T}$$ is distributed as

According to Theorem 4.2, the asymptotic variance of the average depends on the sum of all covariances γ(h). This entity, denoted by J, is called the long-run variance of {X t }:

(4.1)

Note that the long-run variance equals 2π times the spectral density $$f(\lambda )$$ evaluated at $$\lambda = 0$$ (see the Definition 6.​1 of the spectral density in Sect. 6.​1).

As the long-run variance takes into account the serial properties of the time series, it is also called heteroskedastic and autocorrelation consistent variance (HAC variance). If {X t } has some nontrivial autocorrelation (i.e. ρ(h) ≠ 0 for h ≠ 0), the long-run variance J is different from γ(0). This implies among other things that the construction of the t-statistic for testing the simple hypothesis H₀: μ = μ ₀ should be based on J rather than on γ(0).

In case that {X t } is a causal ARMA process with $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$, $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$, the long-run variance is given by

If {X t } is a AR(1) process with $$X_{t} =\phi X_{t-1} + Z_{t}$$, $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ and | ϕ | < 1, $$\gamma (0) = \frac{\sigma ^{2}} {1-\phi ^{2}}$$ and ρ(h) = ϕ  | h | . Thus the long-run variance is given by $$J = \frac{\sigma ^{2}} {(1-\phi )^{2}} =\gamma (0) \times \frac{1+\phi } {1-\phi }$$. From this example it is clear that the long-run variance can be smaller or larger than γ(0), depending on the sign of ϕ: for negative values of ϕ, γ(0) overestimates the long-run variance; for positive values, it underestimates J. The estimation of the long-run variance is dealt with in Sect. 4.4.

4.2 Estimation of the Autocovariance and the Autocorrelation Function

With some slight, asymptotically unimportant modifications, we can use the standard estimators for the autocovariances , γ(h), and the autocorrelations , ρ(h), of a stationary stochastic process:

(4.2)

(4.3)

These estimators are biased because the sums are normalized (divided) by T rather than T − h. The normalization with T − h delivers an unbiased estimate only if $$\overline{X}_{T}$$ is replaced by μ which, however is typically unknown in practice. The second modification concerns the use of the complete sample for the estimation of μ.³ The main advantage of using the above estimators is that the implied estimator for the covariance matrix, $$\widehat{\Gamma }_{T}$$, respectively the autocorrelation matrix, $$\widehat{R}_{T}$$, of $$\left (X_{1},\ldots,X_{T}\right )'$$,

always delivers, independently of the realized observations, non-negative definite and for $$\hat{\gamma }(0)> 0$$ non-singular matrices. The resulting estimated autocovariance function will then satisfy the characterization given in Theorem 1.​1, in particular property (iv).

According to Box and Jenkins (1976, p. 33), one can expect reasonable estimates for γ(h) and ρ(h) if the sample size is larger than 50 and if the order of the autocorrelation coefficient is smaller than T∕4.

The theorem below establishes that these estimators lead under rather mild conditions to consistent and asymptotically normally distributed estimators.

Theorem 4.4 (Asymptotic Distribution of Autocorrelations).

Let {X t } be the stationary process

with $$Z_{t} \sim \mathrm{ IID}(0,\sigma ^{2})$$ , $$\sum _{j=-\infty }^{\infty }\vert \psi _{j}\vert <\infty$$ and $$\sum _{j=-\infty }^{\infty }j\vert \psi _{j}\vert ^{2} <\infty$$ . Then we have for h = 1,2,…

where the elements of $$W = \left (w_{ij}\right )_{i,j\in \{1,\ldots,h\}}$$ are given by Bartlett’s formula

Proof.

Brockwell and Davis (1991, section 7.​3) □ 

Brockwell and Davis (1991) offer a second version of the above theorem where $$\sum _{j=-\infty }^{\infty }j\vert \psi _{j}\vert ^{2} <\infty$$ is replaced by the assumption of finite fourth moments, i.e. by assuming $$\mathbb{E}Z_{t}^{4} <\infty$$. As we rely mainly on ARMA processes, we do not pursue this distinction further because this class of process automatically fulfills the above assumptions as soon as {Z t } is identically and independently distributed (IID). A proof which relies on the Beveridge-Nelson polynomial decomposition (see Theorem D.1 in Appendix D) can be gathered from Phillips and Solo (1992).

Example:$$\{X_{t}\} \sim \mathrm{ IID}(0,\sigma ^{2})$$

The most important application of Theorem 4.4 is related to the case of a white noise process. For this process ρ(h) is equal to zero for | h | > 0. Theorem 4.4 then implies that

The estimated autocorrelation coefficients converge to the true autocorrelation coefficient, in this case zero. The asymptotic distribution of $$\sqrt{ T}\hat{\rho }(h)$$ converges to the standard normal distribution. This implies that for large T we can approximate the distribution of $$\hat{\rho }(h)$$ by a normal distribution with mean zero and variance 1∕T. This allows the construction of a 95 % confidence interval assuming that the true process is white noise. This confidence interval is therefore given by $$\pm 1.96T^{-\frac{1} {2} }$$. It can be used to verify if the observed process is indeed white noise.

Figure 4.1 plots the empirical autocorrelation function of a WN(0,1) process with a sample size of T = 100. The implied 95 % confidence interval is therefore equal to ± 0. 196. As each estimated autocorrelation coefficient falls within the confidence interval, we can conclude that the observed times series may indeed represent a white noise process.

Fig. 4.1

Estimated autocorrelation function of a WN(0,1) process with 95 % confidence interval for sample size T = 100

Instead of examining each correlation coefficient separately, we can test the joint hypothesis that all correlation coefficients up to order N are simultaneously equal to zero, i.e. $$\rho (1) =\rho (2) =\ldots =\rho (N) = 0$$, N = 1, 2, … As each $$\sqrt{T}\hat{\rho }(h)$$ has an asymptotic standard normal distribution and is as¡mptotically uncorrelated with $$\sqrt{T}\hat{\rho }(k)$$, h ≠ k, the sum of the squared estimated autocorrelation coefficients is χ ² distributed with N degrees of freedom. This test statistic is called Box-Pierce statistic:

A refinement of this test statistic is given by the Ljung-Box statistic :

(4.4)

This test statistic is also asymptotically χ ² distributed with the same degree of freedom N. This statistic accounts for the fact that the estimates for high orders h are based on a smaller number of observations and are thus less precise and more noisy. The two test statistics are used in the usual way. The null hypothesis that all correlation coefficients are jointly equal to zero is rejected if Q, respectively Q′ is larger than the critical value corresponding the χ N ² distribution. The number of summands N is usually taken to be rather large, for a sample size of 150 in the range between 15 and 20. The two test are also referred to as Portmanteau tests .

Example: MA(q) Process:$$X_{t} = Z_{t} +\theta _{1}Z_{t-1} +\ldots +\theta _{q}Z_{t-q}$$ with $$Z_{t} \sim \mathrm{ IID}(0,\sigma ^{2})$$

In this case the covariance matrix is determined as

For i, j > q, w ij is equal to zero. The 95 % confidence interval for the MA(1) process $$X_{t} = Z_{t} - 0.8Z_{t-1}$$ is therefore given for a sample size of T = 200 by $$\pm 1.96T^{-\frac{1} {2} }[1 + 2\rho (1)^{2}]^{\frac{1} {2} } = \pm 0.1684$$.

Figure 4.2 shows the estimated autocorrelation function of the above MA(1) process together with 95 % confidence interval based on a white noise process and a MA(1) process with $$\theta = -0.8$$. As the first order autocorrelation coefficient is clearly outside the confidence interval whereas all other autocorrelation coefficients are inside it, the figure demonstrate that the observations are evidently the realization of MA(1) process.

Fig. 4.2

Estimated autocorrelation function of a MA(1) process with $$\theta = -0.8$$ with corresponding 95 % confidence interval for T = 200

Example: AR(1) Process$$X_{t} -\phi X_{t-1} = Z_{t}$$ with$$Z_{t} \sim \mathrm{ IID}(0,\sigma ^{2})$$

In this case the covariance matrix is determined as

The formula for w ij with i ≠ j are not shown. In any case, this formula is of relatively little importance because the partial autocorrelations are better suited for the identification of AR processes (see Sect. 3.​5 and 4.3).

Figure 4.3 shows an estimated autocorrelation function of an AR(1) process. The autocorrelation coefficients decline exponentially which is a characteristic for an AR(1) process.⁴ Furthermore the coefficients are outside the confidence interval up to order 8 for white noise processes.

Fig. 4.3

Estimated autocorrelation function of an AR(1) process with ϕ = 0. 8 and corresponding 95 % confidence interval for T = 100

4.3 Estimation of the Partial Autocorrelation Function

According to its definition (see Definition 3.​2), the partial autocorrelation of order h, α(h), is equal to a h , the last element of the vector $$\alpha _{h} = \Gamma _{h}^{-1}\gamma _{h}(1) = R_{h}^{-1}\rho _{h}(1)$$. Thus, α h and consequently a h can be estimated by $$\hat{\alpha }_{h} =\widehat{ \Gamma }_{h}^{-1}\hat{\gamma }_{h}(1) =\hat{ R}_{h}^{-1}\hat{\rho }_{h}(1)$$. As ρ(h) can be consistently estimated and is asymptotically normally distributed (see Sect. 4.2), the continuous mapping theorem (see Appendix C) ensures that the above estimator for α(h) is also consistent and asymptotically normal. In particular we have for an AR(p) process (Brockwell and Davis 1991)

This result allows to construct, as in the case of the autocorrelation coefficients, confidence intervals for the partial autocorrelations coefficients. The 95 % confidence interval is given by $$\pm \frac{1.96} {\sqrt{T}}$$. The AR(p) process is characterized by the fact that the partial autocorrelation coefficients are zero for h > p. $$\hat{\alpha }(h)$$ should therefore be inside the confidence interval for h > p and outside for h ≤ p. Figure 4.4 confirms this for an AR(1) process with ϕ = 0. 8.

Fig. 4.4

Estimated PACF for an AR(1) process with ϕ = 0. 8 and corresponding 95 % confidence interval for T = 200

Figure 4.5 shows the estimated PACF for an MA(1) process with $$\theta = 0.8$$. In conformity with the theory, the partial autocorrelation coefficients converge to zero. They do so in an oscillating manner because $$\theta$$ is positive (see formula in Sect. 3.​5).

Fig. 4.5

Estimated PACF for a MA(1) process with $$\theta = 0.8$$ and corresponding 95 % confidence interval for T = 200

4.4 Estimation of the Long-Run Variance

For many applications⁵ it is necessary to estimate the long-run variance J which is defined according to Eq. (4.1) as follows⁶

(4.5)

This can, in principle, be done in two different ways. The first one consists in the estimation of an ARMA model which is then used to derive the implied covariances as explained in Sect. 2.​4 These covariances are then inserted into Eq. (4.5). The second method is a nonparametric one and is the subject for the rest of this Section. It has the advantage that it is not necessary to identify and estimate an appropriate ARMA model, a step which can be cumbersome in practice. Additional and more advanced material on this topic can be found in Andrews (1991), Andrews and Monahan (1992), or among others in Haan and Levin (1997).⁷

If the sample size is T > 1, only the covariances γ(0), …, γ(T − 1) can, in principle, be estimated. Thus, a first naive estimator of J is given by $$\hat{J}_{T}$$ defined as

where $$\hat{\gamma }(h)$$ and $$\hat{\rho }(h)$$ are the estimators for γ(h) and ρ(h), respectively, given in Sect. 4.2. As the estimators of the higher order autocovariances are based on smaller samples, their estimates become more erratic. At the same time, their weight in the above sum is the same as the lower order and more precisely estimated autocovariances. Thus, the higher order autocovariances have a disproportionate hazardous influence on the above estimator.

A remedy for this problem is to use only a certain number ℓ T of autocovariances and/or to use a weighted sum instead of an unweighted one. This idea leads to the following class estimators:

where k is a weighting or kernel function .⁸ The kernel functions are required to have the following properties:

(i)

$$k: \mathbb{R} \rightarrow [-1,1]$$ is, with the exception of a finite number of points a continuous function. In particular, k is continuous at x = 0.

(ii)

k is quadratically integrable, i.e. $$\int _{\mathbb{R}}k(x)^{2}dx <\infty$$;

(iii)

k(0) = 1;

(iv)

k is symmetric, i.e. $$k(x) = k(-x)$$ for all $$x \in \mathbb{R}$$.

The basic idea of the kernel function is to give relatively little weight to the higher order autocovariances and relatively more weight to the smaller order ones. As k(0) equals one, the variance $$\hat{\gamma }(0)$$ receives weight one by construction. The continuity assumption implies that also the covariances of smaller order, i.e. for h small, receive a weight close to one. Table 4.1 lists some of the most popular kernel functions used in practice.

Table 4.1

Common kernel functions

Name

k(x) = 

Boxcar (“truncated”)

1

Bartlett

1 − | x | 

Daniell

Tukey-Hanning

Quadratic Spectral

The function are, with the exception of the quadratic spectral function, only defined for | x | ≤ 1. Outside this interval they are set to zero

Figure 4.6 shows a plot of these functions. The first three functions are nonzero only for | x | < 1. This implies that only the orders h for which | h | ≤ ℓ T are taken into account. ℓ T is called the lag truncation parameter or the bandwidth . The quadratic spectral kernel function is an example of a kernel function which takes all covariances into account. Note that some weights are negative in this case as shown in Fig. 4.6.⁹

Fig. 4.6

Common kernel functions

The estimator for the long-run variance is subject to the correction term $$\frac{T} {T-r}$$. This factor depends on the number of parameters estimated in a first step and is only relevant when the sample size is relatively small. In the case of the estimation of the mean r would be equal to one and the correction term is negligible. If on the other hand X t , t = 1, …, T, are the residuals from multivariate regression, r designates the number of regressors. In many applications the correction term is omitted.

The lag truncation parameter or bandwidth, ℓ T , depends on the number of observations. It is intuitive that the number of autocovariances accounted for in the computation of the long-run variance should increase with the sample size, i.e. we should have $$\ell_{T} \rightarrow \infty$$ for $$T \rightarrow \infty$$.¹⁰ The relevant issue is, at which rate the lag truncation parameter should go to infinity. The literature made several suggestions.¹¹ In the following we concentrate on the Bartlett and the quadratic spectral kernel because these function always deliver a positive long-run variance in small samples. Andrews (1991) proposes the following formula to determine the optimal bandwidth:

where [. ] rounds to the nearest integer. The two coefficients α Bartlett and α QuadraticSpectral are data dependent constants which have to be determined in a first step from the data (see (Andrews 1991, 832–839), (Andrews and Monahan 1992, 958) and (Haan and Levin 1997)). If the underlying process is approximated by an AR(1) model, we get:

where $$\hat{\rho }$$ is the first order empirical autocorrelation coefficient.

In order to avoid the cumbersome determination of the α’s Newey and West (1994) suggest the following rules of thumb:

It has been shown that values of 4 for β Bartlett as well as for β QuadraticSpectral lead to acceptable results. A comparison of these formulas with the ones provided by Andrews shows that the latter imply larger values for ℓ T when the sample sizes gets larger. Both approaches lead to consistent estimates, i.e. $$\hat{J}_{T}(\ell_{T}) - J_{T}\mathop{\longrightarrow}\limits_{}^{\quad p\quad }0$$ for $$T \rightarrow \infty$$.

In practice, a combination of both parametric and nonparametric methods proved to deliver the best results. This combined method consists of five steps:

(i)

The first step is called prewhitening and consists in the estimation of a simple ARMA model for the process {X t } to remove the most obvious serial correlations. The idea, which goes back to Press and Tukey (1956) (see also Priestley (1981)), is to get a process for the residuals $$\hat{Z}_{t}$$ which is close to a white noise process. Usually, an AR(1) model is sufficient.¹²

(ii)

Choose a kernel function and, if the method of Andrews has been chosen, the corresponding data dependent constants, i.e. α Bartlett or α QuadraticSpectral for the Bartlett, respectively the quadratic spectral kernel function.

(iii)

Compute the lag truncation parameter for the residuals using the above formulas.

(iv)

Estimate the long-run variance for the residuals $$\hat{Z_{t}}$$.

(v)

Compute the long-run variance for the original time series {X t }.

If in the first step an AR(1) model, $$X_{t} =\phi X_{t-1} + Z_{t}$$, was used, the last step is given by:

where $$\hat{J}_{T}^{Z}(\ell_{T})$$ and $$\hat{J}_{T}^{X}(\ell_{T})$$ denote the estimated long-run variances of {X t } and $$\{\hat{Z}_{t}\}$$. In the general case, of an arbitrary ARMA model, $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$, we get:

4.4.1 An Example

Suppose we want to test whether the yearly growth rate of Switzerland’s real GDP in the last 25 years was higher than 1 %. For this purpose we compute the percentage change against the corresponding quarter of the last year over the period 1982:1 to 2006:1 (97 observations in total), i.e. we compute $$X_{t} = (1 -\mathrm{ L}^{4})\log (\mathrm{GDP}_{t})$$. The arithmetic average of these growth rates is 1.4960 with a variance of 3.0608 . We test the null hypothesis that the growth rate is smaller than one against the alternative that it is greater than one. The corresponding value of the t-statistic is $$(1.4960 - 1)/\sqrt{3.0608/97} = 2.7922$$. Taking a 5 % significance level, the critical value for this one-sided test is 1.661. Thus the null hypothesis is clearly rejected.

The above computation is, however, not valid because the serial correlation of the time series was not taken into account. Indeed the estimated autocorrelation function shown in Fig. 4.7 clearly shows that the growth rate is subject to high and statistically significant autocorrelations.

Fig. 4.7

Estimated autocorrelation function for Switzerland’s real GDP growth (percentage change against corresponding last year’s quarter)

Taking the Bartlett function as the kernel function, the rule of thumb formula for the lag truncation parameter suggest ℓ T  = 4. The weights in the computation of the long-run variance are therefore

The corresponding estimate for the long-run variance is therefore given by:

Using the long-run variance instead of the simple variance leads to a quite different value of the t-statistic: $$(1.4960 - 1)/\sqrt{9.2783/97} = 1.6037$$. The null hypothesis is thus not rejected at the 5 % significance level when the serial correlation of the process is taken into account.

4.5 Exercises

Exercise 4.5.1.

You regress 100 realizations of a stationary stochastic process {X t } against a constant c. The least-squares estimate of c equals $$\hat{c} = 004$$ with an estimated standard deviation of $$\hat{\sigma }_{c} = 0.15$$ . In addition, you have estimated the autocorrelation function up to order h = 5 and obtained the following values:

(i)

How do you interpret the estimated parameter value of 0.4?

(ii)

Examine the autocorrelation function. Do you think that {X t } is white noise?

(iii)

Why is the estimated standard deviation $$\hat{\sigma }_{c} = 0.15$$ incorrect?

(iv)

Estimate the long-run variance using the Bartlett kernel.

(v)

Test the null hypothesis that {X t } is a mean-zero process.

References

Andrews DWK (1991) Heteroskedasticity and autocorrelation consistent covariance matrix estimation. Econometrica 59:817–858CrossRef

Andrews DWK, Monahan JC (1992) An improved heteroskedasticity and autocorrelation consistent covariance matrix estimator. Econometrica 60:953–966CrossRef

Box GEP, Jenkins GM (1976) Time series analysis: forecasting and control, revised edn. Holden-Day, San Francisco

Brockwell PJ, Davis RA (1991) Time series: theory and methods, 2nd edn. Springer, New YorkCrossRef

Haan WJ, Levin AT (1997) A practitioner’s guide to robust covariance matrix estimation. In: Maddala GS, Rao CR (eds) Handbook of statistics: robust inference, vol 15. Elsevier, New York, pp 299–342CrossRef

Newey WK, West KD (1994) Automatic lag selection in covariance matrix estimation. Rev Econ Stud 61:631–653CrossRef

Ogaki M (1992) An introduction to the generalized method of moments. Working Paper No. 314, University of Rochester

Phillips PCB (2004) HAC estimation by automated regression. Cowles Foundation Discussion Paper 1470, Yale University

Phillips PCB, Solo V (1992) Asymptotics for linear processes. Ann Stat 20:971–1001CrossRef

Press H, Tukey JW (1956) Power spetral methods of analysis and their application to problems in airplane dynamics. In: Flight Test Manual, NATO Advisory Group for Aeronautical Research and Development, pp 1–41

Priestley MB (1981) Spectral analysis and time series, vol 1&2. Academic Press, London

Sul D, Phillips PCB, Choi CY (2005) Prewhitening bias in HAC estimation. Oxford Bull Econ Stat 67:517–546CrossRef

Footnotes

1

Recently, bootstrap methods have also been introduced in the time series context.

2

The Beveridge-Nelson decomposition is an indispensable tool for the understanding of integrated and cointegrated processes analyzed in Chaps. 7 and 16.

3

The standard statistical formulas would suggest to estimate the mean appearing in first multiplicand from X ₁, …, X t−h , and the mean appearing in the second multiplicand from X h+1, …, X T .

4

As a reminder: the theoretical autocorrelation coefficients are ρ(h) = ϕ  | h | .

5

For example, when testing the null hypothesis H₀: μ = μ ₀ in the case of serially correlated observations (see Sect. 4.1); for the Phillips-Perron unit-root test explained in Sect. 7.​3.​2.

6

See Theorem 4.2 and the comments following it.

7

Note the connection between the long-run variance and the spectral density at frequency zero: J = 2π f(0) where f is the spectral density function (see Sect. 6.​3).

8

Kernel functions are also relevant for spectral estimators. See in particular Sect. 6.​3.

9

Phillips (2004) has proposed a nonparametric regression-based method which does not require a kernel function.

10

This is true even when the underlying process is known to be a MA(q) process. Even in this case it is advantageous to include also the autocovariances for h > q. The reason is twofold. First, only when $$\ell_{T} \rightarrow \infty$$ for $$T \rightarrow \infty$$, do we get a consistent estimator, i.e. $$\hat{J}_{T} \rightarrow J_{T}$$, respectively J. Second, the restriction to $$\hat{\gamma }(h),\vert h\vert \leq q$$, does not necessarily lead to positive value for the estimated long-run variance $$\hat{J}_{T}$$, even when the Bartlett kernel is used. See Ogaki (1992) for details.

11

See Haan and Levin (1997) for an overview.

12

If in this step an AR(1) model is used and if a first order correlation $$\hat{\phi }$$ larger in absolute terms than 0. 97 is obtained, Andrews and Monahan (1992, 457) suggest to replace $$\hat{\phi }$$ by − 0. 97, respectively 0.97. Instead of using a arbitrary fixed value, it turns out that a data driven value is superior. Sul et al. (2005) suggest to replace 0.97 by $$1 - 1/\sqrt{T}$$ and − 0. 97 by $$-1 + 1/\sqrt{T}$$.

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_5

5. Estimation of ARMA Models

Klaus Neusser1 

(1)

Bern, Switzerland

The specification and estimation of an ARMA(p,q) model for a given realization involves several intermingled steps. First one must determine the orders p and q. Given the orders one can then estimate the parameters ϕ j , $$\theta _{j}$$ and $$\sigma ^{2}$$. Finally, the model has to pass several robustness checks in order to be accepted as a valid model. These checks may involve tests of parameter constancy, forecasting performance or tests for the inclusion of additional exogenous variables. This is usually an iterative process in which several models are examined. It is rarely the case that one model imposes itself. All too often, one is confronted in the modeling process with several trade-offs, like simple versus complex models or data fit versus forecasting performance. Finding the right balance among the different dimensions therefore requires some judgement based on experience.

We start the discussion by assuming that the orders of the ARMA process is known and the problem just consists in the estimation of the corresponding parameters from a realization of length T. For simplicity, we assume that the data are mean adjusted. We will introduce three estimation methods. The first one is a method of moments procedure where the theoretical moments are equated to the empirical ones. This procedure is known under the name of Yule-Walker estimator. The second procedure interprets the stochastic difference as a regression model and estimates the parameters by ordinary least-squares (OLS). These two methods work well if the underlying model is just an AR model and thus involves no MA terms. If the model comprises MA terms, a maximum likelihood (ML) approach must be pursued.

5.1 The Yule-Walker Estimator

We assume that the stochastic process has mean zero and is governed by a causal purely autoregressive model of order p:

where $$\Phi (\mathrm{L}) = 1 -\phi _{1}\mathrm{L} -\phi _{2}\mathrm{L}^{2} -\ldots -\phi _{p}\mathrm{L}^{p}$$. Causality with respect to {Z t } implies that there exists a sequence {ψ j } with $$\sum _{j=0}^{\infty }\vert \psi _{j}\vert <\infty$$ such that $$X_{t} =\sum _{ j=0}^{\infty }\psi _{j}Z_{t-j} = \Psi (\mathrm{L})Z_{t}$$. Multiplying the above difference equation by $$X_{t-j},j = 0,1,\ldots,p$$ and taking expectations leads to the following equation system for the parameters $$\Phi = (\phi _{1},\ldots,\phi _{p})'$$ and $$\sigma ^{2}$$:

This equation system is known as the Yule-Walker equations. It can be written compactly in matrix algebra as:

respectively

The Yule-Walker estimator is obtained by replacing the theoretical moments by the empirical ones and solving the resulting equation system for the unknown parameters:

Note the recursiveness of the equation system: the estimate $$\widehat{\Phi }$$ is obtained without knowledge of $$\hat{\sigma }^{2}$$ as the estimator $$\widehat{R}_{p}^{-1}\,\hat{\rho }_{p}(1)$$ involves only autocorrelations. The estimates $$\widehat{\Gamma }_{p}$$, $$\widehat{R}_{p}$$, $$\hat{\gamma }_{p}(1)$$, $$\hat{\rho }_{p}(1)$$, and $$\hat{\gamma }(0)$$ are obtained in the usual way as explained in Chap. 4 ¹

The construction of the Yule-Walker estimator implies that the first p values of the autocovariance, respectively the autocorrelation function, implied by the estimated model exactly correspond to their estimated counterparts. It can be shown that this moment estimator always delivers coefficients $$\widehat{\Phi }$$ which imply that {X t } is causal with respect to {Z t }. In addition, the following Theorem establishes that the estimated coefficients are asymptotically normal.

Theorem 5.1 (Asymptotic Normality of Yule-Walker Estimator).

Let {X t } be an AR(p) process which is causal with respect to {Z t } whereby $$\{Z_{t}\} \sim \mathrm{ IID}(0,\sigma ^{2})$$ . Then the Yule-Walker estimator is consistent and $$\widehat{\Phi }$$ is asymptotically normal with distribution given by:

In addition we have that

Proof.

See Brockwell and Davis (1991, 233–234). □ 

Noteworthy, the asymptotic covariance matrix of the Yule-Walker estimate is independent of $$\sigma ^{2}$$. In practice, the unknown parameters $$\sigma ^{2}\Gamma _{p}^{-1}$$ are replaced by their empirical counterparts.

Example: AR(1) Process

In the case of an AR(1) process, the Yule-Walker equation is $$\widehat{\Gamma }_{1}\Phi =\hat{\gamma } _{1}(0)$$ which simplifies to $$\hat{\gamma }(0)\phi =\hat{\gamma } (1)$$. The Yule-Walker estimator thus becomes:

The asymptotic distribution then is

This shows that the assumption of causality, i.e.  | ϕ | < 1, is crucial. Otherwise no strictly positive value for the variance would exist. For the case ϕ = 1 which corresponds to the random walk, the asymptotic distribution of $$\sqrt{ T}(\hat{\phi }-1)$$ becomes degenerate as the variance is equal to zero. This case is, however, of prime importance in economics and is treated detail in Chap. 7

In practice the order of the model is usually unknown. However, one can expect when estimating an AR(m) model whereby the true order p is strictly smaller than m that the estimated coefficients $$\hat{\phi }_{p+1},\ldots,\hat{\phi }_{m}$$ should be close to zero. This is indeed the case as shown in Brockwell and Davis (1991, 241). In particular, under the assumptions of Theorem 5.1 it holds that

(5.1)

This result justifies the following strategy to identify the order of an AR-model. Estimate in a first step a highly parameterized model (overfitted model), i.e. a model with a large value of m, and test via a t-test whether ϕ m is zero. If the hypothesis cannot be rejected, reduce the order of the model from m to m − 1 and repeat the same procedure now with respect to ϕ m−1. This is done until the hypothesis can no longer be rejected.

If the order of the initial model is too low (underfitted model) so that the true order is higher than m, one incurs an “omitted variable bias”. The corresponding estimates are no longer consistent. In Sect. 5.4, we take closer look at the problem of determining the order of a model.

Example: MA(q) Process

The Yule-Walker estimator can, in principle, also be applied to MA(q) or ARMA(p,q) processes with q > 0. However, the analysis of the simple MA(1) process in Sect. 1.​5.​1 showed that the relation between the autocorrelations and the model parameters is nonlinear and may have two, one, or no solution. Consider again the MA(1) process as an example. It is given by the stochastic difference equation $$X_{t} = Z_{t} +\theta Z_{t-1}$$ with $$Z_{t} \sim \mathrm{ IID}(0,\sigma ^{2})$$. The Yule-Walker equations are then as follows:

As shown in Sect. 1.​5.​1, this system of equations has for the case $$\vert \hat{\rho }(1)\vert = \left \vert \hat{\gamma }(1)/\hat{\gamma }(0)\right \vert <1/2$$ two solutions; for $$\vert \hat{\rho }(1)\vert = \left \vert \hat{\gamma }(1)/\hat{\gamma }(0)\right \vert = 1/2$$ one solution; and for $$\vert \hat{\rho }(1)\vert = \left \vert \hat{\gamma }(1)/\hat{\gamma }(0)\right \vert> 1/2$$ no real solution. In the case of several solutions, we usually take the invertible one which leads to $$\vert \theta \vert <1$$. Invertibility is, however, a restriction which is hard to implement in the case of higher order MA processes. Moreover, it can be shown that Yule-Walker estimator is no longer consistent in general (see Brockwell and Davis (1991, 246) for details). For these reasons, it is not advisable to use the Yule-Walker estimator in the case of MA processes, especially when there exist consistent and efficient alternatives.

5.2 Ordinary Least-Squares (OLS) Estimation of an AR(p) Model

An alternative approach is to view the AR model as a regression model for X t with regressors $$X_{t-1},\ldots,X_{t-p}$$ and error term Z t :

Given observation for X ₁, …, X T , the regression model can be compactly written in matrix algebra as follows:

(5.2)

Note that the first p observations are lost and that the effective sample size is thus reduced to T − p. The least-squares estimator (OLS estimator) is obtained as the minimizer of the sum of squares $$S(\Phi )$$:

(5.3)

Note that the optimization problem involves no constraints, in particular causality is not imposed as a restriction. The solution of this minimization problem is given by usual formula:

Though Eq. (5.2) resembles very much an ordinary regression model, there are some important differences. First, the standard orthogonality assumption between regressors and error is violated. The regressors X t−j , j = 1, …, p, are correlated with the error terms $$Z_{t-j},j = 1,2,\ldots$$. Second, there is a dependency on the starting values X p , …, X ₁. The assumption of causality, however, insures that these features do not play a role asymptotically. It can be shown that (X′X)∕T converges in probability to $$\widehat{\Gamma }_{p}$$ and (X′Y )∕T to $$\hat{\gamma }_{p}$$. In addition, under quite general conditions, $$T^{-1/2}\mathbf{X}'Z$$ is asymptotically normally distributed with mean 0 and variance $$\sigma ^{2}\Gamma _{p}$$. Then by Slutzky’s Lemma C.10, $$\sqrt{T}(\widehat{\Phi } - \Phi ) = \left (\tfrac{\mathbf{X}'\mathbf{X}} {T} \right )^{-1}\left (\tfrac{\mathbf{X}'Z} {\sqrt{T}}\right )$$ converges in distribution to $$\mathrm{N}(0,\sigma ^{2}\Gamma _{p}^{-1})$$ Thus, the OLS estimator is asymptotically equivalent to the Yule-Walker estimator.

Theorem 5.2 (Asymptotic Normality of the Least-Squares Estimator).

Under the same assumptions as in Theorem  5.1,the ordinary least-squares estimator (OLS estimator) $$\widehat{\Phi } = \left (\mathbf{X}'\mathbf{X}\right )^{-1}\left (\mathbf{X}'Y \right )$$ is asymptotically distributed as

where $$s_{T}^{2} =\widehat{ Z}'\widehat{Z}/T$$ and $$\widehat{Z}_{t}$$ are the OLS residuals.

Proof.

See Chap. 13 and in particular Sect. 13.​3 for a proof in the multivariate case. Additional details may be gathered from Brockwell and Davis (1991, chapter 8). □ 

Remark 5.1.

In practice $$\sigma ^{2}\Gamma _{p}^{-1}$$ is approximated by $$s_{T}^{2}(\mathbf{X}'\mathbf{X}/T)^{-1}$$. Thus, for large T, $$\widehat{\Phi }$$ can be viewed as being normally distributed as $$\mathrm{N}(\Phi,s_{T}^{2}(\mathbf{X}'\mathbf{X})^{-1})$$. This result allows the application of the usual t- and F-tests.

 Because the regressors X t−j , $$j = 1,\ldots,p$$ are correlated with the errors terms Z t−j , j = 1, 2, …, the Gauss-Markov theorem cannot be applied. This implies that the least-squares estimator is no longer unbiased in finite samples. It can be shown that the estimates of an AR(1) model are downward biased when the true value of ϕ is between zero and one. MacKinnon and Smith (1998, figure 1) plots the bias as a function of the sample size and the true parameter (see also Fig. 7.​1). As the bias function is almost linear in the range − 0. 85 < ϕ < 0. 85, an approximately unbiased estimator for the AR(1) model has been proposed by Marriott and Pope (1954), Kendall (1954), and Orcutt and Winokur (1969) (for further details see MacKinnon and Smith 1998):

Remark 5.2.

The OLS estimator does in general not deliver coefficients $$\hat{\Phi }$$ for which {X t } is causal with respect {Z t }. In particular, in the case of an AR(1) model, it can happen that, in contrast to the Yule-Walker estimator, $$\vert \hat{\phi }\vert$$ is larger than one despite the fact that the true parameter is absolutely smaller than one. Nevertheless, the least-squares estimator is to be preferred in practice because it delivers small-sample biases of the coefficients which are smaller than those of Yule-Walker estimator, especially for roots of $$\Phi (z)$$ close to the unit circle (Tjøstheim and Paulsen 1983; Shaman and Stine 1988; Reinsel 1993).

Appendix: Proof of the Asymptotic Normality of the OLS Estimator

The proofs of Theorems 5.1 and 5.2 are rather involved and will therefore not be pursued here. A proof for the more general multivariate case will be given in Chap. 13. It is, however, instructive to look at a simple case, namely the AR(1) model with | ϕ | < 1, $$Z_{t} \sim \mathrm{ IIN}(0,\sigma ^{2})$$ and X ₀ = 0. Denoting by $$\hat{\phi }_{T}$$ the OLS estimator of ϕ, we have:

(5.4)

Moreover, X t can be written as follows:

By assumption each Z j , j = 1, …, t, is normally distributed so that X t as a sum normally distributed random variables is also normally distributed. Because the Z j ’s are independent we have: $$X_{t} \sim \text{N}\left (0,\sigma ^{2}\frac{1-\phi ^{2t}} {1-\phi ^{2}} \right )$$.

The expected value of $$\frac{1} {\sqrt{T}}\sum _{t=1}^{T}X_{t-1}Z_{t}$$ is zero because $$Z_{t} \sim \text{IIN}(0,\sigma ^{2})$$. The variance of this expression is given by

Moreover, $$\sum _{t=1}^{T}X_{t}^{2} =\sum _{ t=1}^{T}X_{t-1}^{2} - (X_{0}^{2} -X_{T}^{2}) =\phi ^{2}\sum _{t=1}^{T}X_{t-1}^{2} +\sum _{ t=1}^{T}Z_{t}^{2} + 2\phi \sum _{t=1}^{T}X_{t-1}Z_{t}$$ so that

The expected value multiplied by $$\sigma ^{2}/T$$ thus is equal to

For T going to infinity, we finally get:

The numerator in Eq. (5.4) therefore converges to a normal random variable with mean zero and variance $$\frac{\sigma ^{4}} {1-\phi ^{2}}$$.

The denominator in Eq. (5.4) can be rewritten as

The expected value and the variance of X T ²∕T converge to zero. Chebyschev’s inequality (see Theorem C.3 in Appendix C) then implies that the first term converges also in probability to zero. X ₀ is equal to zero by assumption. The second term has a constant mean equal to $$\sigma ^{2}/(1 -\phi ^{2})$$ and a variance which converges to zero. Theorem C.8 in Appendix C then implies that the second term converges in probability to $$\sigma ^{2}/(1 -\phi ^{2})$$. The third term has a mean of zero and a variance which converges to zero. Thus the third term converges to zero in probability. This implies:

Putting the results for the numerator and the denominator together and applying Theorem C.10 and the continuous mapping theorem for the convergence in distribution one finally obtains:

(5.5)

Thereby the value for the variance is derived from

5.3 Estimation of an ARMA(p,q) Model

While the estimation of AR models by OLS is rather straightforward and leads to consistent and asymptotically efficient estimates, the estimation of ARMA models is more complex. The reason is that, in contrast to past X t ’s $$Z_{t},Z_{t-1},\ldots,Z_{t-q}$$ are not directly observable from the data. They must be inferred from the observations of X t . The standard method for the estimation of ARMA models is the method of maximum likelihood which will be explained in this section.

We assume that the process {X t } is a causal and invertible ARMA(p,q) process following the difference equation

with $$Z_{t} \sim \mathrm{ IID}(0,\sigma ^{2})$$. We also assume that $$\Phi (z)$$ and $$\Theta (z)$$ have no roots in common. We then stack the parameters of the model into a vector β and a scalar $$\sigma ^{2}$$:

Given the assumption above the admissible parameter space for β, $$\mathcal{C}$$, is described by the following set:

The estimation by the method of maximum likelihood (ML method) is based on some assumption about the joint distribution of X T  = (X ₁, …, X T )′ given the parameters β and $$\sigma ^{2}$$. This joint distribution function is called the likelihood function. The method of maximum likelihood then determines the parameters such that the probability of observing a given sample x T  = (x ₁, …, x T ) is maximized. This is achieved by maximizing the likelihood function with respect to the parameters.

By far the most important case is given by assuming that {X t } is a Gaussian process with mean zero and autocovariance function γ. This implies that X T  = (X ₁, …, X T )′ is distributed as a multivariate normal with mean zero and variance $$\Gamma _{T}$$.² The Gaussian likelihood function given the observations x T , $$\mathrm{L}_{T}(\beta,\sigma ^{2}\vert \mathbf{x}_{T})$$, is then given by

where $$G_{T} =\sigma ^{-2}\Gamma _{T}$$. Note that, in contrast to $$\Gamma _{T}$$, G T does only depend on β and not on $$\sigma ^{2}$$.³ If one wants to point out the dependence of G T from β we write G T (β). The method of maximum likelihood then consists in the maximization of the likelihood function with respect to β and $$\sigma ^{2}$$ taking the data x T as given.

The first order condition of this maximization problem with respect to $$\sigma ^{2}$$ is obtained by taking the logarithm of the likelihood function $$\mathrm{L}_{T}(\beta,\sigma ^{2}\vert \mathbf{x}_{T})$$ and differentiating with respect $$\sigma ^{2}$$ and setting the resulting equation equal to zero:

Solving this equation with respect to $$\sigma ^{2}$$ we get as the solution: $$\sigma ^{2} = T^{-1}\mathbf{x}_{T}'G_{T}^{-1}\mathbf{x}_{T}$$. Inserting this value into the original likelihood function and taking the logarithm, one gets the concentrated log-likelihood function:

This function is then maximized with respect to $$\beta \in \mathcal{C}$$. This is, however, equivalent to minimizing the function

The value of β which minimizes the above function is called maximum-likelihood estimator of β. It will be denoted by $$\hat{\beta }_{\mathrm{ML}}$$. The maximum-likelihood estimator for $$\sigma ^{2}$$, $$\hat{\sigma }_{\mathrm{ML}}^{2}$$, is then given by

The actual computation of $$\det G_{T}(\beta )$$ and G T (β)⁻¹ is numerically involved, especially when T is large, and should therefore be avoided. It is therefore convenient to rewrite the likelihood function in a different, but equivalent form:

Thereby $$\mathbb{P}_{t-1}X_{t}$$ denotes least-squares predictor of X t given X t−1, …, X ₁ and $$r_{t} = v_{t}/\sigma ^{2}$$ where v t is the mean squared forecast error as defined in Sect. 3.​1 Several numerical algorithms have been developed to compute these forecast in a numerically efficient and stable way.⁴

$$\mathbb{P}_{t-1}X_{t}$$ and r t do not depend on $$\sigma ^{2}$$ so that the partial differentiation of the log-likelihood function $$\ln \mathrm{L}(\beta,\sigma ^{2}\vert \mathbf{x}_{T})$$ with respect to the parameters leads to the maximum likelihood estimator. This estimator fulfills the following equations:

where

and where $$\hat{\beta }_{\mathrm{ML}}$$ denote the value of β which minimizes the function

subject to $$\beta \in \mathcal{C}$$. This optimization problem must be solved numerically. In practice, one chooses as a starting value β ₀ for the iteration an initial estimate such that $$\beta _{0} \in \mathcal{C}$$. In the following iterations this restriction is no longer imposed to enhance speed and reduce the complexity of the optimization problem. This implies that one must check whether the so obtained final estimates are indeed in $$\mathcal{C}$$.

If instead of ℓ T (β | x T ), the function

is minimized subject to constraint $$\beta \in \mathcal{C}$$, we obtain the least-squares estimator of β denoted by $$\hat{\beta }_{\mathrm{LS}}$$. The least-squares estimator of $$\sigma ^{2}$$, $$\hat{\sigma }_{\mathrm{LS}}^{2}$$, is then

The term $$\frac{1} {T}\sum _{t=1}^{T}\ln r_{ t-1}$$ disappears asymptotically because, given the restriction $$\beta \in \mathcal{C}$$, the mean-squared forecast error v T converges to $$\sigma ^{2}$$ and thus r T goes to one as T goes to infinity. This implies that for T going to infinity the maximization of the likelihood function becomes equivalent to the minimization of the least-squares criterion. Thus the maximum-likelihood estimator and the least-squares estimator share the same asymptotic normal distribution.

Note also that in the case of autoregressive models r t is constant and equal to one. In this case, the least-squares criterion S(β) reduces to the criterion (5.3) discussed in the previous Sect. 5.2.

Theorem 5.3 (Asymptotic Distribution of ML Estimator).

If {X t } is an ARMA process with true parameters $$\beta \in \mathcal{C}$$ and $$Z_{t} \sim \mathrm{ IID}(0,\sigma ^{2})$$ with $$\sigma ^{2}> 0$$ then the maximum-likelihood estimator and the least-squares estimator have asymptotically the same normal distribution:

The asymptotic covariance matrix V (β) is thereby given by

where {u t } and {v t } denote autoregressive processes defined as $$\Phi (\mathrm{L})u_{t} = w_{t}$$ and $$\Theta (\mathrm{L})v_{t} = w_{t}$$ with $$w_{t} \sim \mathrm{ WN}(0,1)$$ .

Proof.

See Brockwell and Davis (1991, Section 8.8). □ 

It can be shown that both estimators are asymptotically efficient.⁵ Note that the asymptotic covariance matrix V (β) is independent of $$\sigma ^{2}$$.

The use of the Gaussian likelihood function makes sense even when the process is not Gaussian. First, the Gaussian likelihood can still be interpreted as a measure of fit of the ARMA model to the data. Second, the asymptotic distribution is still Gaussian even when the process is not Gaussian as long as $$Z_{t} \sim \mathrm{ IID}(0,\sigma ^{2})$$. The Gaussian likelihood is then called the quasi Gaussian likelihood. The use of the Gaussian likelihood under this circumstance is, however, in general no longer efficient.

Example: AR(p) Process

In this case β = (ϕ ₁, …, ϕ p ) and $$V (\beta ) = \left (\mathbb{E}U_{t}U_{t}'\right )^{-1} =\sigma ^{2}\Gamma _{p}^{-1}$$. This is, however, the same asymptotic distribution as the Yule-Walker estimator. The Yule-Walker, the least-squares, and the maximum likelihood estimator are therefore asymptotically equivalent in the case of an AR(p) process. The main difference lies in the treatment of the first p observations.

In particular, we have:

Example: MA(q) Process

Similarly, one can compute the asymptotic distribution for an MA(q) process . In particular, we have:

Example: ARMA(1,1) Process

For an ARMA(1,1) process the asymptotic covariance matrix is given by

Therefore we have:

5.4 Estimation of the Orders p and q

Up to now we have always assumed that the true orders of the ARMA model p and q are known. This is, however, seldom the case in practice. As economic theory does usually not provide an indication, it is all too often the case that the orders of the ARMA model must be identified from the data. In such a situation one can make two type of errors: p and q are too large in which case we speak of overfitting; p and q are too low in which case we speak of underfitting.

In the case of overfitting, the maximum likelihood estimator is no longer consistent for the true parameter, but still consistent for the coefficients of the causal representation ψ j , j = 0, 1, 2, …, where $$\psi (z) = \frac{\theta (z)} {\phi (z)}$$. This can be illustrated by the following example. Suppose that {X t } is a white noise process, i.e. $$X_{t} = Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$, but we fit an ARMA(1,1) model given by $$X_{t} -\phi X_{t-1} = Z_{t} +\theta Z_{t-1}$$. Then, the maximum likelihood estimator does not converge to $$\phi =\theta = 0$$, but only to the line-segment $$\phi = -\theta$$ with | ϕ | < 1 and $$\vert \theta \vert <1$$. For values of ϕ and $$\theta$$ on this line-segment we have $$\psi (z) =\theta (z)/\phi (z) = 1$$. The maximum likelihood estimator converges to the true values of ψ j , i.e. to the values ψ ₀ = 1 and ψ j  = 0 for j > 0. The situation is depicted in Fig. 5.1. There it is shown that the estimator has a tendency to converge to the points (−1, 1) and (1, −1), depending on the starting values. This indeterminacy of the estimator manifest itself as a numerical problem in the optimization of the likelihood function. Thus models with similar roots for the AR and MA polynomials which are close in absolute value to the unit circle are probably overparametrized. The problem can be overcome by reducing the orders of the AR and MA polynomial by one.

Fig. 5.1

Parameter space of a causal and invertible ARMA(1,1) process

This problem does not appear in a purely autoregressive models. As explained in section “Example: AR(1) Process”, the estimator for the redundant coefficients converges to zero with asymptotic distribution N(0, 1∕T) (see the result in Eq. (5.1)). This is one reason why purely autoregressive models are often preferred. In addition the estimator is easily implemented and every stationary stochastic process can be arbitrarily well approximated by an AR process. This approximation may, however, necessitate high order models when the true process encompasses a MA component.

In the case of underfitting the maximum likelihood estimator converges to those values which are closest to the true parameters given the restricted parameter space. The estimates are, however, inconsistent due to the “omitted variable bias”.

For these reasons the identification of the orders is an important step. One method which goes back to Box and Jenkins (1976) consists in the analysis of the autocorrelation function (ACF) and the partial autocorrelation function (PACF) (see Sect. 3.​5). Although this method requires some experience, especially when the process is not a purely AR or MA process, the analysis of the ACF und PACF remains an important first step in every practical investigation of a time series.

An alternative procedure relies on the automatic order selection. The objective is to minimize a so-called information criterion over different values of p and q. These criteria are based on the following consideration. Given a fixed number of observations, the successive increase of the orders p and q increases the fit of the model so that variance of the residuals $$\hat{\sigma }_{p,q}^{2}$$ steadily decreases. In order to compensate for this tendency to overfitting a penalty is introduced. This penalty term depends on the number of free parameters and on the number of observations at hand.⁶ The most important information criteria have the following additive form:

where $$\ln \hat{\sigma }_{p,q}^{2}$$ measures the goodness of fit of the ARMA(p,q) model and $$(p + q)\frac{C(T)} {T}$$ denotes the penalty term. Thereby C(T) represents a nondecreasing function of T which governs the trade-off between goodness of fit and complexity (dimension) of the model. Thus, the information criteria chooses higher order models for larger sample sizes T. If the model includes a constant term or other exogenous variables, the criterion must be adjusted accordingly. However, this will introduce, for a given sample size, just a constant term in the objective function and will therefore not influence the choice of p and q.

The most common criteria are the Akaike information criterion (AIC), the Schwarz or Bayesian information criterion (BIC), and the Hannan-Quinn information criterion (HQ criterion):

Because AIC < HQC < BIC for a given sample size T ≥ 16, Akaike’s criterion delivers the largest models, i.e. the highest order p + q; the Bayesian criterion is more restrictive and delivers therefore the smallest models, i.e. the lowest p + q. Although Akaike’s criterion is not consistent with respect to p and q and has a tendency to deliver overfitted models, it is still widely used in practice. This feature is sometimes desired as overfitting is seen as less damaging than underfitting.⁷ Only the BIC and HQC lead to consistent estimates of the orders p and q.

5.5 Modeling a Stochastic Process

The identification of a satisfactory ARMA model typically involves in practice several steps.

Step 1: Transformations to Achieve Stationary Time Series

Economic time series are often of a non-stationary nature. It is therefore necessary to transform the time series in a first step to achieve stationarity. Time series which exhibit a pronounced trend (GDP, stock market indices, etc.) should not be modeled in levels, but in differences. If the variable under consideration is already in logs, as is often case, then taking first differences effectively amounts to working with growth rates. Sometimes first differences are not enough and further differences have to be taken. Price indices or monetary aggregates are typical examples where first differences may not be sufficient to achieve stationarity. Thus instead of X t one works with the series $$Y _{t} = (1 -\mathrm{ L})^{d}X_{t}$$ with d = 1, 2, …. A non-stationary process {X t } which needs to be differentiated d-times to arrive at a stationary time series is called integrated of order d, $$X_{t} \sim I(d)$$.⁸ If $$Y _{t} = (1 -\mathrm{ L})^{d}X_{t}$$ is generated by an ARMA(p,q) process, {X t } is said to be an ARIMA(p,d,q) process.

An alternative method to eliminate the trend is to regress the time series against a polynomial in t of degree s, i.e. against (1, t, …, t s ), and to proceed with the residuals. These residuals can then be modeled as an ARMA(p,q) process. Chapter 7 discusses in detail which of the two detrending methods is to be preferred under which circumstances.

Often the data are subject to seasonal fluctuations. As with the trend there are several alternative available. The first possibility is to pass the time series through a seasonal filter and work with the seasonally adjusted data. The construction of seasonal filters is discussed in Chap. 6 A second alternative is to include seasonal dummies in the ARMA model. A third alternative is to take seasonal differences. In the case of quarterly observations, this amounts to work with $$Y _{t} = (1 -\mathrm{ L}^{4})X_{t}$$. As $$1 -\mathrm{ L}^{4} = (1 -\mathrm{ L})(1 +\mathrm{ L} +\mathrm{ L}^{2} +\mathrm{ L}^{3})$$, this transformation involves a first difference and will therefore also account for the trend.

Step 2: Finding the Orders p and q

Having achieved stationarity, one has to find the appropriate orders p and q of the ARMA model. Thereby one can rely either on the analysis of the ACF and the PACF, or on the information criteria outlined in the previous Sect. 5.4.

Step 3: Checking the Plausibility

After having identified a particular model or a set of models, one has to inspect its adequacy. There are several dimensions along which the model(s) can be checked.

(i)

Are the residuals white noise? This can be checked by investigating at the ACF of the residuals and by applying the Ljung-Box test (4.​4). If they are not this means that the model failed to capture all the dynamics inherent in the data.

(ii)

Are the parameters plausible?

(iii)

Are the parameters constant over time? Are there structural breaks? This can be done by looking at the residuals or by comparing parameter estimates across subsamples. More systematic approaches are discussed in Perron (2006). These involve the revolving estimation of parameters by allowing the break point to vary over the sample. Thereby different type of structural breaks can be distinguished. A more in depth analysis of structural breaks is presented in Sect. 18.​1

(iv)

Does the model deliver sensible forecasts? It is particularly useful to investigate the out-of-sample forecasting performance. If one has several candidate models, one can perform a horse-race among them.

In case the model turns out to be unsatisfactory, one has to go back to steps 1 and 2.

5.6 An example: Modeling Real GDP in the Case of Switzerland

This section illustrates the concepts and ideas just presented by working out a specific example. We take the seasonally unadjusted Swiss real GDP as an example. The data are plotted in Fig. 1.​3 To take the seasonality into account we transform the logged time series by taking first seasonal differences, i.e. $$X_{t} = (1 -\mathrm{ L}^{4})\ln \mathrm{GDP}_{t}$$. Thus, the variable corresponds to the growth rate with respect to quarter of the previous year. The data are plotted in Fig. 5.2. A cursory inspection of the plot reveals that this transformation eliminated the trend as well as the seasonality.

Fig. 5.2

Real GDP growth rates of Switzerland

First we analyze the ACF and the PACF. They are plotted together with corresponding confidence intervals in Fig. 5.3. The slowly monotonically declining ACF suggests an AR process. As only the first two orders of the PACF are significantly different from zero, it seems that an AR(2) model is appropriate. The least-squares estimate of this model are:

Fig. 5.3

Autocorrelation (ACF) and partial autocorrelation (PACF) function (PACF) of real GDP growth rates of Switzerland with 95 % confidence interval

The numbers in parenthesis are the estimated standard errors of the corresponding parameter above. The roots of the AR-polynomial are 1.484 and 2.174. They are clearly outside the unit circle so that there exists a stationary and causal representation.

Next, we investigate the information criteria AIC and BIC to identify the orders of the ARMA(p,q) model. We examine all models with 0 ≤ p, q ≤ 4. The AIC and the BIC values, are reported in Tables 5.1 and 5.2. Both criteria reach a minimum at (p, q) = (1, 3) (bold numbers) so that both criteria prefer an ARMA(1,3) model. The parameters of this models are as follows:

Table 5.1

Values of Akaike’s information criterium (AIC) for alternative ARMA(p,q) models

q

p

0

1

2

3

4

0

0.3021

0.0188

−0.2788

−0.3067

1

−0.2174

−0.2425

−0.2433

−0.3446

−0.2991

2

−0.2721

−0.2639

−0.2613

−0.3144

−0.2832

3

−0.2616

−0.2276

−0.2780

−0.2663

−0.2469

4

−0.2186

−0.1990

−0.2291

−0.2574

−0.2099

Minimum in bold

Table 5.2

Values of Bayes’ information criterium (BIC) for alternative ARMA(p,q) models

q

p

0

1

2

3

4

0

0.3297

0.0740

−0.1961

−0.1963

1

−0.1896

−0.1869

−0.1600

−0.2335

−0.1603

2

−0.2162

−0.1801

−0.1495

−0.1746

−0.1154

3

−0.1772

−0.1150

−0.1373

−0.0974

−0.0499

4

−0.1052

−0.0573

−0.0591

−0.0590

0.0169

Minimum in bold

The estimated standard errors of the estimated parameters are again reported in parenthesis below. The AR(2) model is not considerably worse than the ARMA(1,3) model, according to the BIC criterion it is even the second best model.

The inverted roots of the AR- and the MA-polynomial are plotted together with their corresponding 95 % confidence regions in Fig. 5.4.⁹ As the confidence regions are all inside the unit circle, also the ARMA(1,3) has a stationary and causal representation. Moreover, the estimated process is also invertible. In addition, the roots of the AR- and the MA-polynomial are distinct.

Fig. 5.4

Inverted roots of the AR- and the MA-polynomial of the ARMA(1,3) model together with the corresponding 95 % confidence regions

The autocorrelation functions of the AR(2) and the ARMA(1,3) model are plotted in Fig. 5.5. They show no sign of significant autocorrelations so that both residual series are practically white noise. We can examine this hypothesis formally by the Ljung-Box test (see Sect. 4.​2 Eq. (4.​4)). Taking N = 20 the values of the test statistics are Q′AR(2) = 33. 80 and Q′ARMA(1, 3) = 21. 70, respectively. The 5 % critical value according to the χ ₂₀ ² distribution is 31. 41. Thus the null hypothesis $$\rho (1) =\ldots =\rho (20) = 0$$ is rejected for the AR(2) model, but not for the ARMA(1,3) model. This implies that the AR(2) model does not capture the full dynamics of the data.

Fig. 5.5

Autocorrelation function (ACF) of the residuals from the AR(2) and the ARMA(1,3) model

Although the AR(2) and the ARMA(1,3) model seem to be quite different at first glance, they deliver similar impulse response functions as can be gathered from Fig. 5.6. In both models, the impact of the initial shock is first built up to values higher than 1. 1 in quarters one and two, respectively. Then the effect monotonically declines to zero. After 10 to 12 quarters the effect of the shock has practically dissipated.

Fig. 5.6

Impulse responses of the AR(2) and the ARMA(1,3) model

As a final exercise, we use both models to forecast real GDP growth over the next nine quarters, i.e. for the period fourth quarter 2003 to fourth quarter 2005. As can be seen from Fig. 5.7, both models predict that the Swiss economy should move out of recession in the coming quarters. However, the ARMA(1,3) model indicates that the recovery is taking place more quickly and the growth overshooting its long-run mean of 1.3 % in about a year. The forecast of the AR(2) predicts a more steady approach to the long-run mean.

Fig. 5.7

Forecasts of real GDP growth rates for Switzerland

References

Box GEP, Jenkins GM (1976) Time series analysis: forecasting and control, revised edn. Holden-Day, San Francisco

Brockwell PJ, Davis RA (1991) Time series: theory and methods, 2nd edn. Springer, New YorkCrossRef

Fan J, Yao Q (2003) Nonlinear time series. Springer, New York

Kendall MG (1954) Note on the bias in the estimation of autocorelation. Biometrika 41:403–404CrossRef

MacKinnon JG, Smith AA Jr (1998) Approximate bias correction in econometrics. J Econ 85: 205–230CrossRef

Marriott FHC, Pope JA (1954) Bias in the estimation of autocorrelation. Biometrika 41:393–402CrossRef

Orcutt GH, Winokur HS Jr (1969) First order autoregression inference, estimation, and prediction. Econometrica 37:1–14CrossRef

Perron P (2006) Dealing with structural breaks. In: Hassani H, Mills TC, Patterson K (eds) Palgrave handbook in econometrics, vol 1. Econometric theory. Palgrave Macmillan, Hampshire, pp 278–352

Reinsel GC (1993) Elements of multivariate time series analysis. Springer Series in statistics. Springer, New YorkCrossRef

Shaman P, Stine RA (1988) The bias of autoregressive coefficient estimators. J Am Stat Assoc 83:842–848CrossRef

Tjøstheim D, Paulsen J (1983) Bias of some commonly-used time series estimates. Biometrika 70:389–399; corrigendum (1984), 71:656

Footnotes

1

Note that the application of the estimator introduced in Sect. 4.​2 guarantees that $$\widehat{\Gamma }_{p}$$ is always invertible.

2

If the process does not have a mean of zero, we can demean the data in a preliminary step.

3

In Sect. 2.​4 we showed how the autocovariance function γ and as a consequence $$\Gamma _{T}$$, respectively G T can be inferred from a given ARMA model, i.e from a given β.

4

One such algorithm is the innovation algorithm. See Brockwell and Davis (1991, section 5) for details.

5

See Brockwell and Davis (1991) and Fan and Yao (2003) for details.

6

See Brockwell and Davis (1991) for details and a deeper appreciation.

7

See, for example, the section on the unit root tests 7.3.

8

An exact definition will be provided in Chap. 7 In this chapter we will analyze the consequences of non-stationarity and discuss tests for specific forms of non-stationarity.

9

The confidence regions are determined by the delta-method (see Appendix E).

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_6

6. Spectral Analysis and Linear Filters

Klaus Neusser1 

(1)

Bern, Switzerland

Up to now we have viewed a time series as a time indexed sequence of random variables. The class of ARMA process was seen as an adequate class of models for the analysis of stationary time series. This approach is usually termed as time series analysis in the time domain. There is, however, an equivalent perspective which views a time series as overlayed waves of different frequencies. This view point is termed in time series analysis as the analysis in the frequency domain. The decomposition of a time series into sinusoids of different frequencies is called the spectral representation. The estimation of the importance of the waves at particular frequencies is referred to as spectral or spectrum estimation. Priestley (1981) provides an excellent account of these methods. The use of frequency domain methods, in particular spectrum estimation, which originated in the natural sciences was introduced to economics by Granger (1964).¹ Notably, he showed that most of the fluctuations in economic time series can be attributed low frequencies cycles (Granger 1966).

Although both approaches are equivalent, the analysis in the frequency domain is more convenient when it comes to the analysis and construction of linear filters. The application of a filter to a time series amounts to take some moving-average of the time series. These moving-average may extend, at least in theory, into the infinite past, but also into the infinite future. A causal ARMA process {X t } may be regarded as filtered white-noise process with filter weights given by ψ j , $$j = 1,2,\ldots$$ In economics, filters are usually applied to remove cycles of a particular frequency, like seasonal cycles (for example Christmas sales in a store), or to highlight particular cycles, like business cycles.

From a mathematical point of view, the equivalence between time and frequency domain analysis rest on the theory of Fourier series. An adequate representation of this theory is beyond the scope of this book. The interested reader may consult Brockwell and Davis (1991, chapter 4). An introduction to the underlying mathematical theory can be found in standard textbooks like Rudin (1987).

6.1 The Spectral Density

In the following, we assume that {X t } is a mean-zero (centered) stationary stochastic process with autocovariance function γ(h), $$h = 0,\pm 1,\pm 2,\ldots$$ Mathematically, γ(h) represents an double-infinite sequence which can be mapped into a real valued function $$f(\lambda )$$, $$\lambda \in \mathbb{R}$$, by the Fourier transform. This function is called the spectral density function or spectral density. Conversely, we retrieve from the spectral density each covariance. Thus, we have a one-to-one relation between autocovariance functions and spectral densities: both objects summarize the same properties of the time series, but represent them differently.

Definition 6.1 (Spectral Density).

Let {X t } be a mean-zero stationary stochastic process absolutely summable autocovariance function γ then the function

(6.1)

is called the spectral density function or spectral density of {X t }. Thereby $$\imath $$ denotes the imaginary unit (see Appendix A).

The sine is an odd function whereas the cosine and the autocovariance function are even functions.² This implies that the spectral density can be rewritten as:

(6.2)

Because of the periodicity of the cosine function, i.e. because

it is sufficient to consider the spectral density only in the interval (−π, π]. As the cosine is an even function so is f. Thus, we restrict the analysis of the spectral density $$f(\lambda )$$ further to the domain $$\lambda \in [0,\pi ]$$.

In practice, we often use the period or oscillation length instead of the radiant $$\lambda$$. They are related by the formula:

(6.3)

If, for example, the data are quarterly observations, a value of 0.3 for $$\lambda$$ corresponds to a period length of approximately 21 quarters.

Remark 6.1.

We gather some properties of the spectral density function f:

Because $$f(0) = \tfrac{1} {2\pi }\sum _{h=-\infty }^{\infty }\gamma (h)$$, the long-run variance of {X t } J (see Sect. 4.​4) equals 2π f(0), i.e. 2π f(0) = J.

f is an even function so that $$f(\lambda ) = f(-\lambda )$$.

$$f(\lambda ) \geq 0$$ for all $$\lambda \in (-\pi,\pi ]$$. The proof of this proposition can be found in Brockwell and Davis (1996, chapter 4). This property corresponds to the non-negative definiteness of the autocovariance function (see property 4 in Theorem 1.​1 of Sect. 1.​3).

The single autocovariances are the Fourier-coefficients of the spectral density f:

For h = 0, we therefore get $$\gamma (0) =\int _{ -\pi }^{\pi }f(\lambda )\mathrm{d}\lambda$$.

The last property allows us to compute the autocovariances from a given spectral density. It shows how time and frequency domain analysis are related to each other and how a property in one domain is reflected as a property in the other.

These properties of a non-negative definite function can be used to characterize the spectral density of a stationary process {X t } with autocovariance function γ.

Theorem 6.1 (Properties of a Spectral Density).

A function f defined on (−π,π] is the spectral density of a stationary process if and only if the following properties hold:

$$f(\lambda ) = f(-\lambda )$$ ;

$$f(\lambda ) \geq 0$$ ;

$$\int _{-\pi }^{\pi }f(\lambda )\mathrm{d}\lambda < \infty $$ .

Corollary 6.1.

An absolutely summable function γ is the autocovariance function of a stationary process if and only if

In this case f is called the spectral density of γ.

The function $$f(\lambda )/\gamma (0)$$ can be considered as a density function of some probability distribution defined on [−π, π] because $$\frac{f(\lambda )} {\gamma (0)} \geq 0$$ and

The corresponding cumulative distribution function G is then defined as:

It satisfies: $$G(-\pi ) = 0$$, G(π) = 1, $$1 - G(\lambda ) = G(\lambda )$$, und $$G(0) = 1/2$$. The autocorrelation function ρ is then given by

Some Examples

Some relevant examples illustrating the above are:

white noise:

Let {X t } be a white noise process with $$X_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$. For this process all autocovariances, except γ(0), are equal to zero. The spectral density therefore is equal to

Thus, the spectral density is equal to a constant which is proportional to the variance. This means that no particular frequency dominates the spectral density. This is the reason why such a process is called white noise.

MA(1):

Let {X t } be a MA(1) process with autocovariance function

The spectral density therefore is:

Thus, $$f(\lambda ) \geq 0$$ if and only | ρ | ≤ 1∕2. According to Corollary 6.1 above, γ is the autocovariance function of a stationary stochastic process if and only if | ρ | ≤ 1∕2. This condition corresponds exactly to the condition derived in the time domain (see Sect. 1.​3). The spectral density for ρ = 0. 4 or equivalently $$\theta = 0.5$$, respectively for $$\rho = -0.4$$ or equivalently $$\theta = -0.5$$, and $$\sigma ^{2} = 1$$ is plotted in Fig. 6.1a. As the process is rather smooth when the first order autocorrelation is positive, the spectral density is large in the neighborhood of zero and small in the neighborhood of π. For a negative autocorrelation the picture is just reversed.

Fig. 6.1

Examples of spectral densities with $$Z_{t} \sim \mathrm{ WN}(0,1)$$. (a ) MA(1) process. (b ) AR(1) process

AR(1):

The spectral density of an AR(1) process $$X_{t} =\phi X_{t-1} + Z_{t}$$ with $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ is:

The spectral density for ϕ = 0. 6 and $$\phi = -0.6$$ and $$\sigma ^{2} = 1$$ are plotted in Fig. 6.1b. As the process with ϕ = 0. 6 exhibits a relatively large positive autocorrelation so that it is rather smooth, the spectral density takes large values for low frequencies. In contrast, the process with $$\phi = -0.6$$ is rather volatile due to the negative first order autocorrelation. Thus, high frequencies are more important than low frequencies as reflected in the corresponding figure.

Note that, as ϕ approaches one, the spectral density evaluated at zero tends to infinity, i.e. $$\lim _{\lambda \downarrow 0}f(\lambda ) = \infty $$. This can be interpreted in the following way. As the process gets closer to a random walk more and more weight is given to long-run fluctuations (cycles with very low frequency or very high periodicity) (Granger 1966).

6.2 Spectral Decomposition of a Time Series

Consider the simple harmonic process {X t } which just consists of a cosine and a sine wave:

(6.4)

Thereby A and B are two uncorrelated random variables with $$\mathbb{E}A = \mathbb{E}B = 0$$ and $$\mathbb{V}A = \mathbb{V}B = 1$$. The autocovariance function of this process is $$\gamma (h) =\cos (\omega \,h)$$. This autocovariance function cannot be represented as $$\int _{-\pi }^{\pi }\mathrm{e}^{\imath h\lambda }f(\lambda )\mathrm{d}\lambda$$. However, it can be regarded as the Fourier transform of a discrete distribution function F:

where

(6.5)

The integral with respect to the discrete distribution function is a so-called Riemann-Stieltjes integral.³ F is a step function with jumps at −ω and ω and step size of 1∕2 so that the above integral equals $$\tfrac{1} {2}\mathrm{e}^{-\imath \,h\omega } + \tfrac{1} {2}\mathrm{e}^{-\imath \,h\omega } =\cos (h\omega )$$.

These considerations lead to a representation, called the spectral representation, of the autocovariance function as the Fourier transform a distribution function over [−π, π].

Theorem 6.2 (Spectral Representation).

γ is the autocovariance function of a stationary process {X t } if and only if there exists a right-continuous, nondecreasing, bounded function F on (−π,π] with the properties $$F(-\pi ) = 0$$ and

(6.6)

F is called the spectral distribution function of γ.

Remark 6.2.

If the spectral distribution function F has a density f such that $$F(\lambda ) =\int _{ -\pi }^{\lambda }f(\omega )\mathrm{d}\omega$$ then f is called the spectral density and the time series is said to have a continuous spectrum.

Remark 6.3.

According to the Lebesgue-Radon-Nikodym Theorem (see, for example, Rudin (1987)), the spectral distribution function F can be represented uniquely as the sum of a distribution function F Z which is absolutely continuous with respect to the Lebesgue measure and a discrete distribution function F V . The distribution function F Z corresponds to the regular part of the Wold Decomposition (see Theorem 3.​1 in Sect. 3.​2) and has spectral density

The discrete distribution F V corresponds to the deterministic part {V t }.

The process (6.4) considers just a single frequency ω. We may, however, generalize this process by superposing several sinusoids. This leads to the class of harmonic processes:

(6.7)

where $$A_{1},B_{1},\ldots,A_{k},B_{k}$$ are random variables which are uncorrelated with each other and which have means $$\mathbb{E}A_{j} = \mathbb{E}B_{j} = 0$$ and variances $$\mathbb{V}A_{j} = \mathbb{V}B_{j} =\sigma _{ j}^{2}$$, $$j = 1,\ldots,k$$. The autocovariance function of such a process is given by $$\gamma (h) =\sum _{ j=1}^{k}\sigma _{j}^{2}\cos (\omega _{j}\,h)$$. According to the spectral representation theorem the corresponding distribution function F can be represented as a weighted sum of distribution functions like those in Eq. (6.5):

with

This generalization points to the following properties:

Each of the k components $$A_{j}\cos (\omega _{j}\,t) + B_{j}\sin (\omega _{j}\,t)$$, j = 1, …, k, is completely associated to a specific frequency ω j .

The k components are uncorrelated with each other.

The variance of each component is $$\sigma _{j}^{2}$$. The contribution of each component to the variance of X t given by $$\sum _{j=1}^{k}\sigma _{j}^{2}$$ therefore is $$\sigma _{j}^{2}$$.

F is a nondecreasing step-function with jumps at frequencies ω = ±ω j and step sizes $$\tfrac{1} {2}\sigma _{j}^{2}$$.

The corresponding probability distribution is discrete with values $$\tfrac{1} {2}\sigma _{j}^{2}$$ at the frequencies ω = ±ω j and zero otherwise.

The interesting feature of harmonic processes as represented in Eq. (6.7) is that every stationary process can be represented as the superposition of uncorrelated sinusoids. However, in general infinitely many (even uncountably many) of these processes have to be superimposed. The generalization of (6.7) then leads to the spectral representation of a stationary stochastic process:

(6.8)

Thereby $$\{Z(\lambda )\}$$ is a complex-valued stochastic process with uncorrelated increments defined on the interval (−π, π]. The above representation is known as the spectral representation of the process {X t }.⁴ Note the analogy to the spectral representation of the autocovariance function in Eq. (6.6).

For the harmonic processes in Eq. (6.7), we have:

In this case the variance of dZ is given by:

In general, we have:

Thus, a large jump of the spectrum at frequency $$\lambda$$ is associated with a large sinusoidal component with frequency $$\lambda$$.⁵

6.3 The Periodogram and the Estimation of Spectral Densities

Although the spectral distribution function is uniquely determined, its estimation from a finite sample with realizations $$\{x_{1},x_{2},\ldots,x_{T}\}$$ is not easy. This has to do with the problem of estimating a function from a finite number of points. We will present two-approaches: a non-parametric and a parametric one.

6.3.1 Non-Parametric Estimation

A simple estimator of the spectral density, $$\hat{f}_{T}(\lambda )$$, can be obtained by replacing in the defining equation (6.1) the theoretical autocovariances γ by their estimates $$\hat{\gamma }$$. However, instead of a simple sum, we consider a weighted sum:

(6.9)

The weighting function k, also known as the lag window , is assumed to have exactly the same properties as the kernel function introduced in Sect. 4.​4. This correspondence is not accidental, indeed the long-run variance defined in Eq. (4.​1) is just 2π times the spectral density evaluated at $$\lambda = 0$$. Thus, one might choose a weighting, kernel or lag window from Table 4.​1, like the Bartlett-window, and use it to estimate the spectral density. The lag truncation parameter is chosen in such a way that $$\ell_{T} \rightarrow \infty $$ as $$T \rightarrow \infty $$. The rate of divergence should, however, be smaller than T so that $$\tfrac{\ell_{T}} {T}$$ approaches zero as T goes to infinity. As an estimator of the autocovariances one uses the estimator given in Eq. (4.​2) of Sect. 4.​2

The above estimator is called an indirect spectral estimator because it requires the estimation of the autocovariances in the first step. The periodogram provides an alternative direct spectral estimator . For this purpose, we represent the observations as linear combinations of sinusoids of specific frequencies. These so-called Fourier frequencies are defined as $$\omega _{k} = \tfrac{2\pi k} {T}$$, $$k = -\lfloor \tfrac{T-1} {2} \rfloor,\ldots,\lfloor \tfrac{T} {2} \rfloor $$. Thereby ⌊x⌋ denotes the largest integer smaller or equal to x. With this notation, the observations x t , t = 1, …, T, can be represented as a sum of sinusoids:

The coefficients {a k } are the discrete Fourier-transform of the observations $$\{x_{1},x_{2},\ldots,x_{T}\}$$. The periodogram I T is then defined as follows.

Definition 6.2 (Periodogram).

Given observations $$\{x_{1},x_{2},\ldots,x_{T}\}$$ , the periodogram is defined as the function

For each Fourier-frequency ω k , the periodogram I T (ω k ) equals | a k  | ². This implies that

The value of the periodogram evaluated at the Fourier-frequency ω k is therefore nothing but the contribution of the sinusoid with frequency ω k to the variation of {x t } as measured by sum of squares. In particular, for any Fourier-frequency different from zero we have that

Thus the periodogram represents, disregarding the proportionality factor 2π, the sample analogue of the spectral density and therefore carries the same information.

Unfortunately, it turns out that the periodogram is not a consistent estimator of the spectral density. In particular, the covariance between $$I_{T}(\lambda _{1})$$ and $$I_{T}(\lambda _{2})$$, $$\lambda _{1}\neq \lambda _{2}$$, goes to zero for T going to infinity. The periodogram thus has a tendency to get very jagged for large T leading to the detection of spurious sinusoids. A way out of this problem is to average the periodogram over neighboring frequencies, thereby reducing its variance. This makes sense because the variance is relatively constant within a small frequency band. The averaging (smoothing) of the periodogram over neighboring frequencies leads to the class of discrete spectral average estimators which turn out to be consistent:

(6.10)

where $$\tilde{\omega }_{T,\lambda }$$ denotes the multiple of $$\tfrac{2\pi } {T}$$ which is closest to $$\lambda$$. ℓ T is the bandwidth of the estimator, i.e. the number of ordinates over which the average is taken. ℓ T satisfies the same properties as in the case of the indirect spectral estimator (6.9): $$\ell_{T} \rightarrow \infty $$ and ℓ T ∕T → 0 for $$T \rightarrow \infty $$. Thus, as T goes to infinity, on the one hand the average is taken over more and more values, but on the other hand the frequency band over which the average is taken is getting smaller and smaller. The spectral weighting function or spectral window K T is a positive even function satisfying $$\sum _{\vert h\vert \leq \ell_{T}}K_{T}(h) = 1$$ and $$\sum _{\vert h\vert \leq \ell_{T}}K_{T}^{2}(h) \rightarrow 0$$ for $$T \rightarrow \infty $$. It can be shown that under these conditions the discrete spectral average estimator is mean-square consistent. Moreover, the estimator in Eq. (6.9) can be approximated by a corresponding discrete spectral average estimator by defining the spectral window as

or vice versa

Thus, the lag and the spectral window are related via the Fourier transform. For details and the asymptotic distribution the interested reader is referred to Brockwell and Davis (1991, Chapter10). Although the indirect and the direct estimator give approximately the same result when the kernels used are related as in the equation above, the direct estimator (6.10) is usually preferred in practice because it is, especially for long times series, computationally more efficient, in particular in connection with the fast fourier transformation (FFT).⁶

A simple spectral weighting function, known as the Daniell spectral window, is given by $$K_{T}(h) = (2\ell_{T} + 1)^{-1}$$ when | h | ≤ ℓ T and 0 otherwise and where $$\ell_{T} = \sqrt{T}$$. It averages over 2ℓ T + 1 values within a frequency band of approximate width $$\tfrac{4\pi } {\sqrt{T}}$$. This function corresponds to the Daniell kernel function or Daniell lag window $$k(x) =\sin (\pi x)/(\pi x)$$ for | x | ≤ 1 and zero otherwise (see Sect. 4.​4). In practice, the sample size is fixed and the researcher is faced with a trade-off between variance and bias. On the one hand, a weighting function which averages over a wide frequency band produces a smooth spectral density, but has probably a large bias because the estimate of $$f(\lambda )$$ depends on frequencies which are rather far away from $$\lambda$$. On the other hand, a weighting function which averages only over a small frequency band produces a small bias, but probably a large variance. It is thus advisable in practice to work with alternative weighting functions and to choose the one which delivers a satisfying balance between bias and variance.

The following two examples demonstrate the large variance of the periodogram. The first example consists of 200 observations from a simulated white noise time series with variance equal. Whereas the true spectrum is constant equal to one, the raw periodogram, i.e. the periodogram without smoothing, plotted in Fig. 6.2 is quite erratic. However, it is obvious that by taking averages of adjacent frequencies the periodogram becomes smoother and more in line with the theoretical spectrum. The second example consists of 200 observations of a simulated AR(2) process. Figure 6.3 demonstrates again the jaggedness of the raw periodogram. However, these erratic movements are distributed around the true spectrum. Thus, by smoothing one can hope to get closer to the true spectrum and even detect the dominant cycle with radian equal to one. It is also clear that by smoothing over a too large range, in the extreme over all frequencies, no cycle could be detected.

Fig. 6.2

Raw periodogram of a white noise time series ($$X_{t} \sim \mathrm{ WN}(0,1)$$, T = 200)

Fig. 6.3

Raw periodogram of an AR(2) process ($$X_{t} = 0.9X_{t-1} - 0.7X_{t-2} + Z_{t}$$ with $$Z_{t} \sim \mathrm{ WN}(0,1)$$, T = 200)

Figure 6.4 illustrates these considerations with real life data by estimating the spectral density of quarterly growth rates of real investment in constructions for the Swiss economy using alternative weighting functions. To obtain a better graphical resolution we have plotted the estimates on a logarithmic scale. All three estimates show a peak (local maximum) at the frequency $$\lambda = \tfrac{\pi } {2}$$. This corresponds to a wave with a period of one year. The estimator with a comparably wide frequency band (dotted line) smoothes the minimum $$\lambda = 1$$ away. The estimator with a comparable small frequency band (dashed line), on the contrary, reveals additional waves with frequencies $$\lambda = 0.75$$ and 0. 3 which correspond to periods of approximately two, respectively five years. Whether these waves are just artifacts of the weighting function or whether there really exist cycles of that periodicity remains open.

Fig. 6.4

Non-parametric direct estimates of a spectral density with alternative weighting functions

6.3.2 Parametric Estimation

An alternative to the nonparametric approaches just outlined consists in the estimation of an ARMA model and followed by deducing the spectral density from it. This approach was essentially first proposed by Yule (1927).

Theorem 6.3 (Spectral Density of ARMA Processes).

Let {X t } be a causal ARMA(p,q) process given by $$\Phi (L)X_{t} = \Theta (L)Z_{t}$$ and $$Z_{t} \sim WN(0,\sigma ^{2})$$ . Then the spectral density f X is given by

(6.11)

Proof.

{X t } is generated by applying the linear filter $$\Psi (L)$$ with transfer function $$\Psi (\mathrm{e}^{-\imath \lambda }) = \frac{\Theta (\mathrm{e}^{-\imath \lambda })} {\Phi (\mathrm{e}^{-\imath \lambda })}$$ to {Z t } (see Sect. 6.4). Formula (6.11) is then an immediate consequence of Theorem 6.5 because the spectral density of {Z t } is equal to $$\tfrac{\sigma ^{2}} {2\pi }$$. □ 

Remark 6.4.

As the spectral density of an ARMA process {X t } is given by a quotient of trigonometric functions, the process is said to have a rational spectral density.

The spectral density of the AR(2) process $$X_{t} =\phi _{1}X_{t-1} +\phi _{2}X_{t-2} + Z_{t}$$ with $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$, for example, is then given by

The spectral density of an ARMA(1,1) process $$X_{t} =\phi X_{t-1} + Z_{t} +\theta Z_{t-1}$$ with $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ is

An estimate of the spectral density is then obtained by replacing the unknown coefficients of the ARMA model by their corresponding estimates and by applying Formula (6.11) to the estimated model. Figure 6.5 compares the nonparametric to the parametric method based on an AR(4) model using the same data as in Fig. 6.4. Both methods produce similar estimates. They clearly show waves of periodicity of half a year and a year, corresponding to frequencies $$\tfrac{\pi }{2}$$ and π. The nonparametric estimate is, however, more volatile in the frequency band [0. 6, 1] and around 2. 5.

Fig. 6.5

Comparison of nonparametric and parametric estimates of the spectral density of the growth rate of investment in the construction sector

6.4 Linear Time-Invariant Filters

Time-invariant linear filters are an indispensable tool in time series analysis. Their objective is to eliminate or amplify waves of a particular periodicity. For example, they may be used to purge a series from seasonal movements. The seasonally adjusted time series should then reflect more strongly the business cyclical movements which are viewed to have period length between two and eight years. The spectral analysis provides just the right tools to construct and analyze such filters.

Definition 6.3.

{Y t } is the output of the linear time-invariant filter (LTF) $$\Psi =\{\psi _{j},j = 0,\pm 1,\pm 2,\ldots \}$$ applied to the input {X t } if

The filter is called causal or one-sided if ψ j = 0 for j < 0; otherwise it is called two-sided.

Remark 6.5.

Time-invariance in this context means that the lagged process {Y t−s } is obtained for all $$s \in \mathbb{Z}$$ from {X t−s } by applying the same filter $$\Psi $$.

Remark 6.6.

MA processes, causal AR processes and causal ARMA processes can be viewed as filtered white noise processes.

It is important to recognize that the application of a filter systematically changes the autocorrelation properties of the original time series. This may be warranted in some cases, but may lead to the “discovery” of spurious regularities which just reflect the properties of the filter. See the example of the Kuznets filter below.

Theorem 6.4 (Autocovariance Function of Filtered Process).

Let {X t } be a mean-zero stationary process with autocovariance function γ X . Then the filtered process {Y t } defined as

with $$\sum _{j=-\infty }^{\infty }\vert \psi _{j}\vert < \infty $$ is also a mean-zero stationary process with autocovariance function γ Y . Thereby the two autocovariance functions are related as follows:

Proof.

We first show the existence of the output process {Y t }. For this end, consider the sequence of random variables $$\{Y _{t}^{(m)}\}_{m=1,2,\ldots }$$ defined as

To show that the limit for $$m \rightarrow \infty $$ exists in the mean square sense, it is, according to Theorem C.6, enough to verify the Cauchy criterion

Taking without loss of generality m > n, Minkowski’s inequality (see Theorem C.2 or triangular inequality) leads to

Using the Cauchy-Bunyakovskii-Schwarz inequality and the stationarity of {X t }, the first term on the right hand side is bounded by

As $$\sum _{j=-\infty }^{\infty }\vert \psi _{j}\vert < \infty $$ by assumption, the last term converges to zero. Thus, the limit of {Y t (m)}, $$m \rightarrow \infty $$, denoted by S t , exists in the mean square sense with $$\mathbb{E} S_{t}^{2} < \infty $$.

In remains to show that S t and $$\sum _{j=-\infty }^{\infty }\psi _{j}X_{t-j}$$ are actually equal with probability one. This is established by noting that

where use has been of Fatou’s lemma.

The stationarity of {Y t } can be checked as follows:

Thus, $$\mathbb{E}Y _{t}$$ and $$\mathbb{E}Y _{t}Y _{t-h}$$ are finite and independent of t. {Y t } is therefore stationary. □ 

Corollary 6.2.

If $$X_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ and $$Y _{t} =\sum _{ j=0}^{\infty }\psi _{j}X_{t-j}$$ with $$\sum _{j=0}^{\infty }\vert \psi _{j}\vert < \infty $$ then the above expression for γ Y (h) simplifies to

Remark 6.1.

In the proof of the existence of {Y t }, the assumption of the stationarity of {X t } can be weakened by assuming only $$\sup _{t}\mathbb{E}X_{t}^{2} < \infty $$.

Theorem 6.5.

Under the conditions of Theorem  6.4,the spectral densities of {X t } and {Y t } are related as

where $$\Psi (\mathrm{e}^{-\imath \lambda }) =\sum _{ j=-\infty }^{\infty }\psi _{j}\mathrm{e}^{-\imath \,j\lambda }$$ . $$\Psi (\mathrm{e}^{-\imath \lambda })$$ is called the transfer function of the filter.

To understand the effect of the filter $$\Psi $$, consider the simple harmonic process $$X_{t} = 2\cos (\lambda \,t) =\mathrm{ e}^{\imath \lambda \,t} +\mathrm{ e}^{-\imath \lambda \,t}$$. Passing {X t } through the filter $$\Psi $$ leads to a transformed time series {Y t } defined as

where $$\theta (\lambda ) =\arg \Psi (\mathrm{e}^{-\imath \lambda })$$. The filter therefore amplifies some frequencies by the factor $$g(\lambda ) = \left \vert \Psi (\mathrm{e}^{-\imath \lambda })\right \vert $$ and delays X t by $$\tfrac{\theta (\lambda )} {\lambda }$$ periods. Thus, we have a change in amplitude given by the amplitude gain function $$g(\lambda )$$ and a phase shift given by the phase gain function $$\theta (\lambda )$$ . If the gain function is bigger than one the corresponding frequency is amplified. On the other hand, if the value is smaller than one the corresponding frequency is dampened.

Examples of Filters

First differences (changes with respect to previous period):

The transfer function of this filter is $$(1 -\mathrm{ e}^{-\imath \lambda })$$ and the gain function is $$2(1-\cos \lambda )$$. These functions take the value zero for $$\lambda = 0$$. Thus, the filter eliminates the trend which can be considered as a wave with an infinite period length.

Change with respect to same quarter last year, assuming that the data are quarterly observations:

The transfer function and the gain function are $$1 -\mathrm{ e}^{-4\imath \lambda }$$ and $$2(1 -\cos (4\lambda ))$$, respectively. Thus, the filter eliminates all frequencies which are multiples of $$\tfrac{\pi }{2}$$ including the zero frequency. In particular, it eliminates the trend and waves with periodicity of four quarters.

A famous example of a filter which led to wrong conclusions is the Kuznets filter (see Sargent 1987, 273–276). Assuming yearly data, this filter is obtained from two transformations carried out in a row. The first transformation which should eliminate cyclical movements takes centered five year moving averages. The second one take centered non-overlapping first differences. Thus, the filter can be written as:

Figure 6.6 gives a plot of the transfer function of the Kuznets filter. Thereby it can be seen that all frequencies are dampened, except those around $$\lambda = 0.2886$$. The value $$\lambda = 0.2886$$ corresponds to a wave with periodicity of approximately $$2\pi /0.2886 = 21.77 \approx 22$$ years. Thus, as first claimed by Howrey (1968), even a filtered white noise time series would exhibit a 22 year cycle. This demonstrates that cycles of this length, related by Kuznets (1930) to demographic processes and infrastructure investment swings, may just be an artefact produced by the filter and are therefore not endorsed by the data.

Fig. 6.6

Transfer function of the Kuznets filters

6.5 Some Important Filters

6.5.1 Construction of Low- and High-Pass Filters

For some purposes it is desirable to eliminate specific frequencies. Suppose, we want to purge a time series from all movements with frequencies above $$\lambda _{c}$$, but leave those below this value unchanged. The transfer function of such an ideal low-pass filter would be:

By expanding $$\Psi (\mathrm{e}^{-\imath \lambda })$$ into a Fourier-series $$\Psi (\mathrm{e}^{-\imath \lambda }) =\sum _{ j=-\infty }^{\infty }\psi _{j}\mathrm{e}^{-\imath \,j\lambda }$$, it is possible to determine the appropriate filter coefficients {ψ j }. In the case of a low-pass filter they are given by:

The implementation of the filter in practice is not straightforward because only a finite number of coefficients can be used. Depending on the number of observations, the filter must be truncated such that only those ψ j with | j | ≤ q are actually employed. The problem becomes more severe as one gets to the more recent observations because less future observations are available. For the most recent period even no future observation is available. This problem is usually overcome by replacing the missing future values by their corresponding forecast. Despite this remedy, the filter works best in the middle of the sample and is more and more distorted as one approaches the beginning or the end of the sample.

Analogously for low-pass filters, it is possible to construct high-pass filters. Figure 6.7 compares the transfer function of an ideal high-pass filter with two filters truncated at q = 8 and q = 32, respectively. Obviously, the transfer function with the higher q approximates the ideal filter better. In the neighborhood of the critical frequency, in our case π∕16, however, the approximation remains inaccurate. This is known as the Gibbs phenomenon.

Fig. 6.7

Transfer function of HP-filter in comparison to high-pass filters

6.5.2 The Hodrick-Prescott Filter

The Hodrick-Prescott filter (HP-Filter) has gained great popularity in the macroeconomic literature, particularly in the context of the real business cycles theory. This high-pass filter is designed to eliminate the trend and cycles of high periodicity and to emphasize movements at business cycles frequencies (see Hodrick and Prescott 1980; King and Rebelo 1993; Brandner and Neusser 1992).

One way to introduce the HP-filter is to examine the problem of decomposing a time series {X t } additively into a growth component {G t } and a cyclical component {C t }:

This decomposition is, without further information, not unique. Following the suggestion of Whittaker (1923), the growth component should be approximated by a smooth curve. Based on this recommendation Hodrick and Prescott suggest to solve the following restricted least-squares problem given a sample {X t } t = 1, …, T :

The above objective function has two terms. The first one measures the fit of {G t } to the data. The closer {G t } is to {X t } the smaller this term becomes. In the limit when G t  = X t for all t, the term is minimized and equal to zero. The second term measures the smoothness of the growth component by looking at the discrete analogue to the second derivative. This term is minimized if the changes of the growth component from one period to the next are constant. This, however, implies that G t is a linear function. Thus the above objective function represents a trade-off between fitting the data and smoothness of the approximating function. This trade-off is governed by the meta-parameter $$\lambda$$ which must be fixed a priori.

The value of $$\lambda$$ depends on the critical frequency and on the periodicity of the data (see Uhlig and Ravn 2002, for the latter). Following the proposal by Hodrick and Prescott (1980) the following values for $$\lambda$$ are common in the literature:

It can be shown that these choices for $$\lambda$$ practically eliminate waves of periodicity longer than eight years. The cyclical or business cycle component is therefore composed of waves with periodicity less than eight years. Thus, the choice of $$\lambda$$ implicitly defines the business cycle. Figure 6.7 compares the transfer function of the HP-filter to the ideal high-pass filter and two approximate high-pass filters.⁷

As an example, Fig. 6.8 displays the HP-filtered US logged GDP together with the original series in the upper panel and the implied business cycle component in the lower panel.

Fig. 6.8

HP-filtered US logged GDP (upper panel) and cyclical component (lower panel)

6.5.3 Seasonal Filters

Besides the elimination of trends, the removal of seasonal movements represents another important application in practice. Seasonal movements typically arise when a time series is observed several times within a year giving rise to the possibility of waves with periodicity less than twelve month in the case of monthly observations, respectively of four quarters in the case of quarterly observations. These cycles are usually considered to be of minor economic interest because they are due to systematic seasonal variations in weather conditions or holidays (Easter, for example).⁸ Such variations can, for example, reduce construction activity during the winter season or production in general during holiday time. These cycles have usually quite large amplitude so that they obstruct the view to the economically and politically more important business cycles. In practice, one may therefore prefer to work with seasonally adjusted series. This means that one must remove the seasonal components from the time series in a preliminary stage of the analysis. In section, we will confine ourself to only few remarks. Comprehensive treatment of seasonality can be found in Hylleberg (1986) and Ghysels and Osborn (2001).

Two simple filters for the elimination of seasonal cycles in the case of quarterly data are given by the one-sided filter

or the two-sided filter

In practice, the so-called X–11-Filter or its enhanced versions X–12 and X–13 filter developed by the United States Census Bureau are often applied. This filter is a two-sided filter which makes, in contrast to two examples above, use of all sample observations. As this filter not only adjusts for seasonality, but also corrects for outliers, a blind mechanical use is not recommended. Gómez and Maravall (1996) developed an alternative method known under the name TRAMO-SEATS. More details on the implementation of both methods can be found in Eurostat (2009).

Figure 6.9 shows the effect of seasonal adjustment using TRAMO-SEATS by looking at the corresponding transfer functions of the growth rate of construction investment. One can clearly discern how the yearly and the half-yearly waves corresponding to the frequencies π∕2 and π are dampened. On the other hand, the seasonal filter weakly amplifies a cycle of frequency 0. 72 corresponding to a cycle of periodicity of two years.

Fig. 6.9

Transfer function of growth rate of investment in the construction sector with and without seasonal adjustment

6.5.4 Using Filtered Data

Whether or not to use filtered, especially seasonally adjusted, data is still an ongoing debate. Although the use of unadjusted data together with a correctly specified model is clearly the best choice, there is a nonnegligible uncertainty in modeling economic time series. Thus, in practice one faces several trade-offs which must be taken into account and which may depend on the particular context (Sims 1974, 1993; Hansen and Sargent 1993). One the one hand, the use of adjusted data may disregard important information on the dynamics of the time series and introduce some biases. On the other hand, the use of unadjusted data encounters the risk of misspecification, especially because usual measures of fit may put too large emphasis on fitting the seasonal frequencies thereby neglecting other frequencies.

6.6 Exercises

Exercise 6.6.1.

(i)

Show that the process defined in Eq. (6.4)has an autocovariance function equal to $$\gamma (h) =\cos (\omega h)$$ .

(ii)

Show that the process defined in Eq. (6.7)has autocovariance function

Exercise 6.6.2.

Compute the transfer and the gain function for the following filters:

(i)

(ii)

References

Brandner P, Neusser K (1992) Business cycles in open economies: Stylized facts for Austria and Germany. Weltwirtschaftliches Arch 128:67–87CrossRef

Brockwell PJ, Davis RA (1991) Time series: theory and methods, 2nd edn. Springer, New YorkCrossRef

Brockwell PJ, Davis RA (1996) Introduction to time series and forecasting. Springer, New YorkCrossRef

Eurostat (2009) ESS guidelines on seasonal adjustment. Eurostat, Luxembourg

Ghysels E, Osborn DR (2001) The econometric analysis of seasonal time series. Cambridge University Press, CambridgeCrossRef

Gómez V, Maravall A (1996) Programs TRAMO and SEATS. Instructions for the user (with some updates). Working Paper 9628, Servicio de Estudios, Banco de España

Granger CWJ (1964) Spectral analysis of economic time series. Princeton University Press, Princeton

Granger CWJ (1966) The typical spectral shape of an economic variable. Econometrica 34:150–161CrossRef

Hansen LP, Sargent TJ (1993) Saesonality and approximation errors in rational expectations models. J Econ 55:21–55CrossRef

Harvey AC, Jaeger A (1993) Detrending, stylized facts and the business cycle. J Appl Econ 8:231–247CrossRef

Hodrick RJ, Prescott EC (1980) Post-war U.S. business cycles: An empirical investigation. Discussion Paper 451, Carnegie-Mellon University, Pittsburgh

Howrey EP (1968) A spectrum analysis of the long swing hypothesis. Int Econ Rev 9:228–252CrossRef

Hylleberg S (1986) Seasonality in regression. Academic Press, Orlando, FL

King RG, Rebelo ST (1993) Low frequency filtering and real business cycles. J Econ Dyn Control 17:201–237CrossRef

Kuznets S (1930) Secular movements in production and prices. Their nature and their bearing upon cyclical fluctuations. Houghton Mifflin, Boston

Miron JA (1996) The economics of seasonal cycles. MIT Press, Cambridge

Priestley MB (1981) Spectral analysis and time series, vol 1&2. Academic Press, London

Robinson EA (1982) A historical perspective of spectrum estimation. Proc IEEE 70:885–907CrossRef

Rudin W (1987) Real and complex analysis, 3rd edn. McGraw-Hill, Boston

Sargent TJ (1987) Macroeconomic theory, 2nd edn. Academic Press, Orlando, FL

Sims CA (1974) Seasonality in regression. J Am Stat Assoc 69:618–626CrossRef

Sims CA (1993) Rational expectations modeling with seasonally adjsuted data. J Econ 55:9–19CrossRef

Uhlig H, Ravn M (2002) On adjusting the HP-filter for the frequency of observations. Rev Econ Stat 84:371–376CrossRef

Whittaker ET (1923) On a new method of graduation. Proc Edinbrough Math Soc 41:63–75

Yule GU (1927) On a method of investigating periodicities in disturbed series, with special reference to Wolfer’ sunspot numbers. Philos Trans R Soc A 226:267–298CrossRef

Footnotes

1

The use of spectral methods in the natural sciences can be traced many centuries back. The modern statistical approach builds on to the work of N. Wiener, G. U. Yule, J. W. Tukey, and many others. See the interesting survey by Robinson (1982).

2

A function f is called even if $$f(-x) = f(x)$$; the function is called odd if $$f(-x) = -f(x)$$. Thus, we have $$\sin (-\theta ) = -\sin (\theta )$$ and $$\cos (-\theta ) =\cos (\theta )$$.

3

The Riemann-Stieltjes integral is a generalization of the Riemann integral. Let f and g be two bounded functions defined on the interval [a, b] then the Riemann-Stieltjes integral $$\int _{a}^{b}f(x)\mathrm{d}g(x)$$ is defined as $$\lim _{n\rightarrow \infty }\sum _{i=1}^{n}f(\xi _{i})[g(x_{i}) - g(x_{i-1})]$$ where $$a = x_{1} < x_{2} <\ldots < x_{n-1} < x_{n} = b$$. For g(x) = x we obtain the standard Riemann integral. If g is a step function with a countable number of steps x i of height h i then $$\int _{a}^{b}f(x)\mathrm{d}g(x) =\sum _{i}f(x_{i})h_{i}$$.

4

A mathematically precise statement is given in Brockwell and Davis (1991, chapter 4) where also the notion of stochastic integration is explained.

5

Thereby $$F(\lambda ^{-})$$ denotes the left-sided limit, i.e. $$F(\lambda ^{-}) =\lim _{\omega \uparrow \lambda }F(\omega )$$.

6

The FFT is seen as one of the most important numerical algorithms ever as it allows a rapid computation of Fourier transformations and its inverse. The FFT is widely in digital signal processing.

7

As all filters, the HP-filter systematically distorts the properties of the time series. Harvey and Jaeger (1993) show how the blind application of the HP-filter can lead to the detection of spurious cyclical behavior.

8

An exception to this view is provided by Miron (1996).

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_7

7. Integrated Processes

Klaus Neusser1 

(1)

Bern, Switzerland

7.1 Definition, Properties and Interpretation

Up to now the discussion concentrated on stationary processes and in particular ARMA processes. According to the Wold decomposition theorem (see Theorem 3.​1) every purely non-deterministic processes possesses the following representation:

where $$\{Z_{t}\} \sim \mathrm{ WN}(0,\sigma ^{2})$$ and $$\sum _{j=0}^{\infty }\psi _{j}^{2} <\infty$$. Typically, we model X t as an ARMA process so that $$\Psi (\mathrm{L}) = \frac{\Theta (\mathrm{L})} {\Phi (\mathrm{L})}$$. This representation implies:

$$\qquad \mathbb{E}X_{t} =\mu$$,

$$\qquad \lim _{h\rightarrow \infty }\mathbb{P}_{t}X_{t+h} =\mu$$.

The above property is often referred to as mean reverting because the process moves around a constant mean. Deviations from this mean are only temporary or transitory. Thus, the best long-run forecast is just the mean of the process.

This property is often violated by economic time series which typically show a tendency to growth. Classic examples are time series for GDP (see Fig. 1.​3) or some stock market index (see Fig. 1.​5). This trending property is not compatible with stationarity as the mean is no longer constant. In order to cope with this characteristic of economic time series, two very different alternatives have been proposed. The first one consists in letting the mean μ be a function of time μ(t). The most popular specification for μ(t) is a linear function, i.e. $$\mu (t) =\alpha +\delta t$$. In this case we get:

The process {X t } is then referred to as a trend-stationary process . In practice one also encounters quadratic polynomials of t or piecewise linear functions. For example, $$\mu (t) =\alpha _{1} +\delta _{1}t$$ for t ≤ t ₀ and $$\mu (t) =\alpha _{2} +\delta _{2}t$$ for t > t ₀. In the following, we restrict ourself to linear trend functions.

The second alternative assumes that the time series becomes stationary after differentiation. The number of times one has to differentiate the process to achieve stationarity is called the order of integration. If d times differentiation is necessary, the process is called integrated of order d and is denoted by $$X_{t} \sim \mathrm{ I}(d)$$. If the resulting time series, $$\Delta ^{d}X_{t} = (1 -\mathrm{ L})^{d}X_{t}$$, is an ARMA(p,q) process, the original process is called an ARIMA(p,d,q) process. Usually it is sufficient to differentiate the time series only once, i.e. d = 1. For expositional purposes we will stick to this case.

The formal definition of an I(1) process is given as follows.

Definition 7.1.

The stochastic process {X t } is called integrated of order one or difference-stationary , denoted as $$X_{t} \sim \mathrm{ I}(1)$$ , if and only if $$\Delta X_{t} = X_{t} - X_{t-1}$$ can be represented as

with $$\{Z_{t}\} \sim \mathrm{ WN}(0,\sigma ^{2})$$ and $$\sum _{j=0}^{\infty }j\vert \psi _{j}\vert <\infty$$.

The qualification $$\Psi (1)\neq 0$$ is necessary to avoid trivial and uninteresting cases. Suppose for the moment that $$\Psi (1) = 0$$, then it would be possible to write $$\Psi (\mathrm{L})$$ as $$(1 -\mathrm{ L})\widetilde{\Psi }(\mathrm{L})$$ for some lag polynomial $$\widetilde{\Psi }(\mathrm{L})$$. This would, however, imply that the factor 1 − L could be canceled in the above definition so that {X t } is already stationary and that the differentiation would be unnecessary. The assumption $$\Psi (1)\neq 0$$ thus excludes the case where a trend-stationary process could be regarded as an integrated process. For each trend-stationary process $$X_{t} =\alpha +\delta t +\widetilde{ \Psi }(\mathrm{L})Z_{t}$$ we have $$\Delta X_{t} =\delta +\Psi (\mathrm{L})Z_{t}$$ with $$\Psi (\mathrm{L}) = (1 -\mathrm{ L})\widetilde{\Psi }(\mathrm{L})$$. This would violate the condition $$\Psi (1)\neq 0$$. Thus a trend-stationary process cannot be a difference-stationary process.

The condition $$\sum _{j=0}^{\infty }j\vert \psi _{j}\vert <\infty$$ implies $$\sum _{j=0}^{\infty }\psi _{j}^{2} <\infty$$ and is therefore stronger than necessary for the Wold representation to hold. In particular, it implies the Beveridge-Nelson decomposition of integrated processes into a linear trend, a random walk, and a stationary component (see Sect. 7.1.4). The condition is automatically fulfilled for all ARMA processes because {ψ j } decays exponentially to zero.

Integrated processes with d > 0 are also called unit-root processes. This designation results from the fact that ARIMA processes with d > 0 can be viewed as ARMA processes, whereby the AR polynomial has a d-fold root of one.¹ An important prototype of an integrated process is the random walk with drift δ:

Trend-stationary and difference-stationary processes have quite different characteristics. In particular, they imply different behavior with respect to the long-run forecast, the variance of the forecast error, and the impulse response function. In the next section, we will explore these properties in detail.

7.1.1 Long-Run Forecast

The optimal forecast in the least-squares sense given the infinite past of a trend-stationary process is given by

Thus we have

because $$\sum _{j=0}^{\infty }\psi _{j}^{2} <\infty$$. Thus the long-run forecast is given by the linear trend. Even if X t deviates temporarily from the trend line, it is assumed to return to it. A trend-stationary process therefore behaves in the long-run like $$\mu (t) =\alpha +\delta t$$.

The forecast of the differentiated series is

The level of X t+h is by definition

so that

This shows that also for the integrated process the long-run forecast depends on a linear trend with slope δ. However, the intercept is no longer a fixed number, but given by X t which is stochastic. With each new realization of X t the intercept changes so that the trend line moves in parallel up and down. This issue can be well illustrated by the following two examples.

Example 1.

Let {X t } be a random walk with drift δ. Then best forecast of X t+h , $$\mathbb{P}_{t}X_{t+h}$$, is

The forecast thus increases at rate δ starting from the initial value of X t . δ is therefore the slope of a linear trend. The intercept of this trend is stochastic and equal to X t . Thus the trend line moves in parallel up or down depending on the realization of X t .

Example 2.

Let {X t } be an ARIMA(0,1,1) process given by $$\Delta X_{t} =\delta +Z_{t} +\theta Z_{t-1}$$ with $$\vert \theta \vert <1$$. The best forecast of X t+h is then given by

As before the intercept changes in a stochastic way, but in contrary to the previous example it is now given by $$X_{t} +\theta Z_{t}$$. If we consider the forecast given the infinite past, the invertibility of the process implies that Z t can be expressed as a weighted sum of current and past realizations of $$\Delta X_{t}$$ (see Sects. 2.​3 and 3.​1).

7.1.2 Variance of Forecast Error

In the case of a trend-stationary process the forecast error is

As the mean of the forecast error is zero, the variance is

For h going to infinity this expression converges to $$\sigma ^{2}\sum _{j=0}^{\infty }\psi _{j}^{2} <\infty$$. This is nothing but the unconditional variance of X t . Thus the variance of the forecast error increases with the length of the forecasting horizon, but remains bounded.

For the integrated process the forecast error can be written as

The forecast error variance therefore is

This expression increases with the length of the forecast horizon h, but is no longer bounded. It increases linearly in h to infinity.² The precision of the forecast therefore not only decreases with the forecasting horizon h as in the case of the trend-stationary model, but converges to zero. In the example above of the ARIMA(0,1,1) process the forecasting error variance is

This expression clearly increases linearly with h.

7.1.3 Impulse Response Function

The impulse response function (dynamic multiplier) is an important analytical tool as it gives the response of the variable X t to the underlying shocks. In the case of the trend-stationary process the impulse response function is

The effect of a shock thus declines with time and dies out. Shocks have therefore only transitory or temporary effects. In the case of an ARMA process the effect even declines exponentially (see the considerations in Sect. 2.​3).³

In the case of integrated processes the impulse response function for $$\Delta X_{t}$$ implies:

For h going to infinity, this expression converges $$\sum _{j=0}^{\infty }\psi _{j} = \Psi (1)\neq 0$$. This implies that a shock experienced in period t will have a long-run or permanent effect. This long-run effect is called persistence. If $$\{\Delta X_{t}\}$$ is an ARMA process then the persistence is given by the expression

Thus, for an ARIMA(0,1,1) the persistence is $$\Psi (1) = \Theta (1) = 1+\theta$$. In the next section we will discuss some examples.

7.1.4 The Beveridge-Nelson Decomposition

The Beveridge-Nelson decomposition represents an important tool for the understanding of integrated processes.⁴ It shows how an integrated time series of order one can be represented as the sum of a linear trend, a random walk, and a stationary series. It may therefore be used to extract the cyclical component (business cycle component) of a time series and can thus be viewed as an alternative to the HP-filter (see Sect. 6.​5.​2) or to more elaborated so-called structural time series models (see Sects. 17.​1 and 17.​4.​2).

Assuming that {X t } is an integrated process of order one, there exists, according to Definition 7.1, a causal representation for $$\{\Delta X_{t}\}$$:

with the property $$\Psi (1)\neq 0$$ and $$\sum _{j=0}^{\infty }j\vert \psi _{j}\vert <\infty$$. Before proceeding to the main theorem, we notice the following simple, but extremely useful polynomial decomposition of $$\Psi (\mathrm{L})$$:

We state this results in the following Lemma:

Lemma 7.1.

Let $$\Psi (\mathrm{L}) =\sum _{j=0}\psi _{j}\mathrm{L}^{j}$$ , then

where $$\widetilde{\Psi }(\mathrm{L}) =\sum _{ j=0}^{\infty }\widetilde{\psi }_{j}\mathrm{L}^{j}$$ with $$\tilde{\psi }_{j} =\sum _{ i=j+1}^{\infty }\psi _{i}$$.

As {X t } is integrated and because $$\Psi (1)\neq 0$$ we can express X t as follows:

This leads to the following theorem.

Theorem 7.1 (Beveridge-Nelson Decomposition).

Every integrated process {X t } has a decomposition of the following form:

The above representation is referred to as the Beveridge-Nelson decomposition .

Proof.

The only substantial issue is to show that $$\widetilde{\Psi }(\mathrm{L})Z_{0} -\widetilde{ \Psi }(\mathrm{L})Z_{t}$$ defines a stationary process. According to Theorem 6.​4 it is sufficient to show that the coefficients of $$\widetilde{\Psi }(\mathrm{L})$$ are absolutely summable. We have that:

where the first inequality is a consequence of the triangular inequality and the second inequality follows from the Definition 7.1 of an integrated process. □ 

Shocks of a random walk component have a permanent effect. This effect is measured by the persistence $$\Psi (1)$$, the coefficient of the random walk component. In macroeconomics aggregate supply shocks are ascribed to have a long-run effect as they affect productivity. In contrast monetary or demand shocks are viewed to have temporary effects only. Thus the persistence $$\Psi (1)$$ can be interpreted as a measure for the importance of supply shocks (see Campbell and Mankiw (1987), Cochrane (1988) or Christiano and Eichenbaum (1990)). For a critical view from an econometric standpoint see Hauser et al. (1999). A more sophisticated multivariate approach to identify supply and demand shocks and to disentangle their relative importance is provided in Sect. 15.​5

In business cycle analysis it is often useful to decompose {X t } into a sum of a trend component μ t and a cyclical component $$\varepsilon _{t}$$:

In the case of a difference-stationary series, the cyclical component can be identified with the stationary component in the Beveridge-Nelson decomposition and the trend component with the random walk plus the linear trend. Suppose that $$\{\Delta X_{t}\}$$ follows an ARMA process $$\Phi (\mathrm{L})\Delta X_{t} = c + \Theta (\mathrm{L})Z_{t}$$ then $$\Delta \mu _{t} =\delta +\Psi (1)Z_{t}$$ can be identified as the trend component. This means that the trend component can be recursively determined from the observations by applying the formula

The cyclical component is then simply the residual: $$\varepsilon _{t} = X_{t} -\mu _{t}$$.

In the above decomposition both the permanent (trend) component as well as the stationary (cyclical) component are driven by the same shock Z t . A more sophisticated model would, however, allow that the two components are driven by different shocks. This idea is exploited in the so-called structural time series analysis where the different components (trend, cycle, season, and irregular) are modeled as being driven by separated shocks. As only the series {X t } is observed, not its components, this approach leads to serious identification problems. See the discussion in Harvey (1989), Hannan and Deistler (1988), or Mills (2003). In Sects. 17.​1 and 17.​4.​2 we will provide an overall framework to deal with these issues.

Examples

Let $$\{\Delta X_{t}\}$$ be a MA(q) process with $$\Delta X_{t} =\delta +Z_{t} +\ldots +\theta _{q}Z_{t-q}$$ then the persistence is given simply by the sum of the MA-coefficients: $$\Psi (1) = 1 +\theta _{1} +\ldots +\theta _{q}$$. Depending on the value of these coefficients. The persistence can be smaller or greater than one.

If $$\{\Delta X_{t}\}$$ is an AR(1) process with $$\Delta X_{t} =\delta +\phi \Delta X_{t-1} + Z_{t}$$ and assuming | ϕ | < 1 then we get: $$\Delta X_{t} = \frac{\delta } {1-\phi } +\sum _{ j=0}^{\infty }\phi ^{j}Z_{t-j}$$. The persistence is then given as $$\Psi (1) =\sum _{ j=0}^{\infty }\phi ^{j} = \frac{1} {1-\phi }$$. For positive values of ϕ, the persistence is greater than one. Thus, a shock of one is amplified to have an effect larger than one in the long-run.

If $$\{\Delta X_{t}\}$$ is assumed to be an ARMA(1,1) process with $$\Delta X_{t} =\delta +\phi \Delta X_{t-1} + Z_{t} +\theta Z_{t-1}$$ and | ϕ | < 1 then $$\Delta X_{t} = \frac{\delta } {1-\phi } + Z_{t} + (\phi +\theta )\sum _{j=0}^{\infty }\phi ^{j}Z_{t-j-1}$$. The persistence is therefore given by $$\Psi (1) = 1 + (\phi +\theta )\sum _{j=0}^{\infty }\phi ^{j} = \frac{1+\theta } {1-\phi }$$.

The computation of the persistence for the model estimated for Swiss GDP in Sect. 5.​6 is more complicated because a fourth order difference 1 − L⁴ has been used instead of a first order one. As $$1 -\mathrm{ L}^{4} = (1 -\mathrm{ L})(1 +\mathrm{ L} +\mathrm{ L}^{2} +\mathrm{ L}^{3})$$, it is possible to extend the above computations also to this case. For this purpose we compute the persistence for $$(1 +\mathrm{ L} +\mathrm{ L}^{2} +\mathrm{ L}^{3})\ln \mathrm{BIP}_{t}$$ in the usual way. The long-run effect on $$\ln \mathrm{BIP}_{t}$$ is therefore given by $$\Psi (1)/4$$ because $$(1 +\mathrm{ L} +\mathrm{ L}^{2} +\mathrm{ L}^{3})\ln \mathrm{BIP}_{t}$$ is nothing but four times the moving-average of the last four values. For the AR(2) model we get a persistence of 1.42 whereas for the ARMA(1,3) model the persistence is 1.34. Both values are definitely above one so that the permanent effect of a one-percent shock to Swiss GDP is amplified to be larger than one in the long-run. Campbell and Mankiw (1987) and Cochrane (1988) report similar values for the US.

7.2 Properties of the OLS Estimator in the Case of Integrated Variables

The estimation and testing of coefficients of models involving integrated variables is not without complications and traps because the usual asymptotic theory may become invalid. The reason being that the asymptotic distributions are in general no longer normal so that the usual critical values for the test statistics are no longer valid. A general treatment of these issues is beyond this text, but can be found in Banerjee et al. (1993) and Stock (1994). We may, however, illustrate the kind of problems encountered by looking at the Gaussian AR(1) case⁵:

where $$Z_{t} \sim \mathrm{ IID\,N}(0,\sigma ^{2})$$ and X ₀ = 0. For observations on X ₁, X ₂, …, X T the OLS-estimator of ϕ is given by the usual expression:

For | ϕ | < 1, the OLS estimator of ϕ, $$\hat{\phi }_{T}$$, converges in distribution to a normal random variable (see Chap. 5 and in particular Sect. 5.​2):

The estimated density of the OLS estimator of ϕ for different values of ϕ is represented in Fig. 7.1. This figure was constructed using a Monte-Carlo simulation of the above model for a sample size of T = 100 using 10, 000 replications for each value of ϕ.⁶ The figure shows that the distribution of $$\hat{\phi }_{T}$$ becomes more and more concentrated if the true value of ϕ gets closer and closer to one. Moreover the distribution gets also more and more skewed to the left. This implies that the OLS estimator is downward biased and that this bias gets relatively more and more pronounced in small samples as ϕ approaches one.

Fig. 7.1

Distribution of the OLS estimator of ϕ for T = 100 and 10,000 replications

The asymptotic distribution would be degenerated for ϕ = 1 because the variance approaches zero as ϕ goes to one. Thus the asymptotic distribution becomes useless for statistical inferences under this circumstance. In order to obtain a non-degenerate distribution the estimator must be scaled by T instead by $$\sqrt{ T}$$. It can be shown that

This result was first established by Dickey and Fuller (1976) and Dickey and Fuller (1981). However, the asymptotic distribution ν need no longer be normal. It was first tabulated in Fuller (1976). The scaling with T instead of $$\sqrt{T}$$ means that the OLS-estimator converges, if the true value of ϕ equals one, at a higher rate to ϕ = 1. This property is known as superconsistency .

In order to understand this result better, in particular in the light of the derivation in the Appendix of Sect. 5.​2, we take a closer look at the asymptotic distribution of $$T\left (\hat{\phi }_{T}-\phi \right )$$:

Under the assumption ϕ = 1, X t becomes a random walk so that X t can be written as $$X_{t} = Z_{t} +\ldots +Z_{1}$$. Moreover, as a sum of normally distributed random variables X t becomes itself normally distributed as $$X_{t} \sim \mathrm{ N}(0,\sigma ^{2}t)$$. In addition, we get

The numerator therefore converges to a χ ₁ ² distribution. The distribution of the denominator is more involved, but its expected value is given by:

because $$X_{t-1} \sim \mathrm{ N}\left (0,\sigma ^{2}(t - 1)\right )$$. To obtain a nondegenerate random variable one must scale by T ². Thus, intuitively, $$T(\hat{\phi }_{T}-\phi )$$ will no longer converge to a degenerate distribution.

Using similar arguments it can be shown that the t-statistic

with $$s_{T}^{2} = \frac{1} {T-2}\sum _{t=2}^{T}\left (X_{ t} -\hat{\phi }_{T}X_{t-1}\right )^{2}$$ is not asymptotically normal. Its distribution was also first tabulated by Fuller (1976). Figure 7.2 compares its density with the standard normal distribution in a Monte-Carlo experiment using again a sample of T = 100 and 10, 000 replications. It is obvious that the t-distribution is shifted to the left. This implies that the critical values will be absolutely higher than for the standard case. In addition, one may observe a slight skewness.

Fig. 7.2

Distribution of t-statistic for T = 100 and 10,000 replications and standard normal distribution

Finally, we also want to investigate the autocovariance function of a random walk. Using similar arguments as in Sect. 1.​3 we get:

Thus the correlation coefficient between X T and X T−h is:

The autocorrelation coefficient ρ(h) therefore monotonically decreases with h, holding the sample size T constant. The rate at which ρ(h) falls is, however, smaller than for ARMA processes for which ρ(h) declines exponentially fast to zero. Given h, the autocorrelation coefficient converges to one for $$T \rightarrow \infty$$. Figure 7.3 compares the theoretical and the estimated ACF of a simulated random walk with T = 100. Typically, the estimated coefficients lie below the theoretical ones. In addition, we show the estimated ACF for an AR(1) process with ϕ = 0. 9 and using the same realizations of the white noise process as in the construction of the random walk. Despite the large differences between the ACF of an AR(1) process and a random walk, the ACF is only of limited use to discriminate between an (stationary) ARMA process and a random walk.

Fig. 7.3

ACF of a random walk with 100 observations

The above calculation also shows that ρ(1) < 1 so that the expected value of the OLS estimator is downward biased in finite samples: $$\mathbb{E}\hat{\phi }_{T} <1$$.

7.3 Unit-Root Tests

The previous Sects. 7.1 and 7.2 have shown that, depending on the nature of the non-stationarity (trend versus difference stationarity), the stochastic process has quite different algebraic (forecast, forecast error variance, persistence) and statistical (asymptotic distribution of OLS-estimator) properties. It is therefore important to be able to discriminate among these two different types of processes. This also pertains to standard regression models for which the presence of integrated variables can lead to non-normal asymptotic distributions.

The ability to differentiate between trend- and difference-stationary processes is not only important from a statistical point of view, but can be given an economic interpretation. In macroeconomic theory, monetary and demand disturbances are alleged to have only temporary effects whereas supply disturbances, in particular technology shocks, are supposed to have permanent effects. To put it in the language of time series analysis: monetary and demand shocks have a persistence of zero whereas supply shocks have nonzero (positive) persistence. Nelson and Plosser (1982) were the first to investigate the trend properties of economic time series from this angle. In their influential study they reached the conclusion that, with the important exception of the unemployment rate, most economic time series in the US are better characterized as being difference stationary. Although this conclusion came under severe scrutiny (see Cochrane (1988) and Campbell and Perron (1991)), this issue resurfaces in many economic debates. The latest discussion relates to the nature and effect of technology shocks (see Galí (1999) or Christiano et al. (2003)).

The following exposition focuses on the Dickey-Fuller test (DF-test) and the Phillips-Perron test(PP-test) . Although other test procedures and variants thereof have been developed in the meantime, these two remain the most widely applied in practice. These types of tests are also called unit-root tests.

Both the DF- as well as the PP-test rely on a regression of X t on X t−1 which may include further deterministic regressors like a constant or a linear time trend. We call this regression the Dickey-Fuller regression:

(7.1)

Alternatively and numerically equivalent, one may run the Dickey-Fuller regression in difference form:

with $$\beta =\phi -1$$. For both tests, the null hypothesis is that the process is integrated of order one, difference stationary, or has a unit-root. Thus we have

The alternative hypothesis H ₁ is that the process is trend-stationary or stationary with constant mean and is given by:

Thus the unit root test is a one-sided test. The advantage of the second formulation of the Dickey-Fuller regression is that the corresponding t-statistic can be readily read off from standard outputs of many computer packages which makes additional computations unnecessary.

7.3.1 The Dickey-Fuller Test (DF-Test)

The Dickey-Fuller test comes in two forms. The first one, sometimes called the ρ-test, takes $$T(\hat{\phi }-1)$$ as the test statistic. As shown previously, this statistic is no longer asymptotically normally distributed. However, it was first tabulated by Fuller and can be found in textbooks like Fuller (1976) or Hamilton (1994b). The second and much more common one relies on the usual t-statistic for the hypothesis ϕ = 1:

This test-statistic is also not asymptotically normally distributed. It was for the first time tabulated by Fuller (1976) and can be found, for example, in Hamilton (1994b). Later MacKinnon (1991) presented much more detailed tables where the critical values can be approximated for any sample size T by using interpolation formulas (see also Banerjee et al. (1993)).⁷

The application of the Dickey-Fuller test as well as the Phillips-Perron test is obfuscated by the fact that the asymptotic distribution of the test statistic (ρ- or t-test) depends on the specification of the deterministic components and on the true data generating process. This implies that depending on whether the Dickey-Fuller regression includes, for example, a constant and/or a time trend and on the nature of the true data generating process one has to use different tables and thus different critical values. In the following we will focus on the most common cases listed in Table 7.1.

Table 7.1

The four most important cases for the unit-root test

Data generating process

Estimated regression

ρ-test:

(null hypothesis)

(Dickey-Fuller regression)

t-test

Case 1

Case 1

Case 2

Case 2

N(0,1)

α ≠ 0

Case 4

Case 4

In case 1 the Dickey-Fuller regression includes no deterministic component. Thus, a rejection of the null hypothesis implies that {X t } has to be a mean zero stationary process. This specification is, therefore, only warranted if one can make sure that the data have indeed mean zero. As this is rarely the case, except, for example, when the data are the residuals from a previous regression,⁸ case 1 is very uncommon in practice. Thus, if the data do not display a trend, which can be checked by a simple time plot, the Dickey-Fuller regression should include a constant. A rejection of the null hypothesis then implies that {X t } is a stationary process with mean $$\mu = \frac{c} {1-\phi }$$. If the data display a time trend, the Dickey-Fuller regression should also include a linear time trend as in case 4. A rejection of the null hypothesis then implies that the process is trend-stationary. In the case that the Dickey-Fuller regression contains no time trend and there is no time trend under the alternative hypothesis, asymptotic normality holds. This case is only of theoretical interest as it should a priori be clear whether the data are trending or not. In the instance where one is not confident about the trending nature of the time series see the procedure outlined in Sect. 7.3.3.

In the cases 2 and 4 it is of interest to investigate the joint hypothesis H ₀: α = 0 and ϕ = 1, and H ₀: δ = 0 and ϕ = 1 respectively. Again the corresponding F-statistic is no longer F-distributed, but has been tabulated (see Hamilton (1994b, Table B7)). The trade-off between t- and F-test is discussed in Sect. 7.3.3.

Most economic time series display a significant amount of autocorrelation. To take this feature into account it is necessary to include lagged differences $$\Delta X_{t-1},\ldots,\Delta X_{t-p+1}$$ as additional regressors. The so modified Dickey-Fuller regression then becomes:

This modified test is called the augmented Dickey-Fuller test (ADF-test). This autoregressive correction does not change the asymptotic distribution of the test statistics. Thus the same tables can be used as before. For the coefficients of the autoregressive terms asymptotic normality holds. This implies that the standard testing procedures (t-test, F-test) can be applied in the usual way. This is true if instead of autoregressive correction terms moving-average terms are used instead (see Said and Dickey (1984)).

For the ADF-test the order p of the model should be chosen such that the residuals are close to being white noise. This can be checked, for example, by looking at the ACF of the residuals or by carrying out a Ljung-Box test (see Sect. 4.​2). In case of doubt, it is better to choose a higher order. A consistent procedure to find the right order is to use the Akaike’s criterion (AIC). Another alternative strategy advocated by Ng and Perron (1995) is an iterative testing procedure which makes use of the asymptotic normality of the autoregressive correction terms. Starting from a maximal order $$p - 1 = p_{max}$$, the method amounts to the test whether the coefficient corresponding to the highest order is significantly different from zero. If the null hypothesis that the coefficient is zero is not rejected, the order of the model is reduced by one and the test is repeated. This is done as long as the null hypothesis is not rejected. If the null hypothesis is finally rejected, one sticks with the model and performs the ADF-test. The successive test are standard t-tests. It is advisable to use a rather high significance level, for example a 10 % level. The simulation results by Ng and Perron (1995) show that this procedure leads to a smaller bias compared to using the AIC criterion and that the reduction in power remains negligible.

7.3.2 The Phillips-Perron Test (PP-Test)

The Phillips-Perron test represents a valid alternative to the ADF-test. It is based on the simple Dickey-Fuller regression (without autoregressive correction terms) and corrects for autocorrelation by modifying the OLS-estimate or the corresponding value of the t-statistic. The simple Dickey-Fuller regression with either constant and/or trend is:

where {Z t } need no longer be a white noise process, but can be any mean zero stationary process. {Z t } may, for example, be an ARMA process. In principle, the approach also allows for heteroskedasticity.⁹

The first step in the Phillips-Perron unit-root test estimates the above appropriately specified Dickey-Fuller regression. The second step consists in the estimation of the unconditional variance γ Z (0) and the long-run variance J of the residuals $$\hat{Z} _{t}$$. This can be done using one of the methods prescribed in Sect. 4.​4 These two estimates are then used in a third step to correct the ρ- and the t-test statistics. This correction would then take care of the autocorrelation present in the data. Finally, one can use the so modified test statistics to carry out the unit-root test applying the same tables for the critical values as before.

In case 1 where no deterministic components are taken into account (see case 1 in Table 7.1) the modified test statistics according to Phillips (1987) are:

If {Z t } would be white noise so that J = γ(0), respectively $$\widehat{\mathrm{J}}_{T} \approx \hat{\gamma }_{Z}(0)$$ one gets the ordinary Dickey-Fuller test statistic. Similar formulas can be derived for the cases 2 and 4. As already mentioned these modifications will not alter the asymptotic distributions so the same critical values as for the ADF-test can be used.

The main advantage of the Phillips-Perron test is that the non-parametric correction allows for very general {Z t } processes. The PP-test is particularly appropriate if {Z t } has some MA-components which can be only poorly approximated by low order autoregressive terms. Another advantage is that one can avoid the exact modeling of the process. It has been shown by Monte-Carlo studies that the PP-test has more power compared to the DF-test, i.e. the PP-test rejects the null hypothesis more often when it is false, but that, on the other hand, it has also a higher size distortion, i.e. that it rejects the null hypothesis too often.

7.3.3 Unit-Root Test: Testing Strategy

Independently whether the Dickey-Fuller or the Phillips-Perron test is used, the specification of the deterministic component is important and can pose a problem in practice. On the one hand, if the deterministic part is underrepresented, for example when only a constant, but no time trend is used, the test results are biased in favor of the null hypothesis, if the data do indeed have a trend. On the other hand, if too many deterministic components are used, the power of the test is reduced. It is therefore advisable to examine a plot of the series in order to check whether a long run trend is visible or not. In some circumstances economic reasoning may help in this regard.

Sometimes, however, it is difficult to make an appropriate choice a priori. We therefore propose the following testing strategy based on Elder and Kennedy (2001).

X t has a long-run trend:

As X t grows in the long-run, the Dickey-Fuller regression

should contain a linear trend.¹⁰ In this case either ϕ = 1, δ = 0 and α ≠ 0 (unit root case) or ϕ < 1 with δ ≠ 0 (trend stationary case). We can then test the joint null hypothesis

by a corresponding F-test. Note that the F-statistic, like the t-test, is not distributed according to the F-distribution. If the test does not reject the null, we conclude that {X t } is a unit root process with drift or equivalently a difference-stationary (integrated) process. If the F-test rejects the null hypothesis, there are three possible situations:

(i)

The possibility ϕ < 1 and δ = 0 contradicts the primary observation that {X t } has a trend and can therefore be eliminated.

(ii)

The possibility ϕ = 1 and δ ≠ 0 can also be excluded because this would imply that {X t } has a quadratic trend, which is unrealistic.

(iii)

The possibility ϕ < 1 and δ ≠ 0 represents the only valid alternative. It implies that {X t } is stationary around a linear trend, i.e. that {X t } is trend-stationary.

Similar conclusions can be reached if, instead of the F-test, a t-test is used to test the null hypothesis H₀: ϕ = 1 against the alternative H₁: ϕ < 1. Thereby a non-rejection of H₀ is interpreted that δ = 0. If, however, the null hypothesis H₀ is rejected, this implies that δ ≠ 0, because {X t } exhibits a long-run trend. The F-test is more powerful than the t-test. The t-test, however, is a one-sided test, which has the advantage that it actually corresponds to the primary objective of the test. In Monte-Carlo simulations the t-test has proven to be marginally superior to the F-test.

X t has no long-run trend:

In this case δ = 0 and the Dickey-Fuller regression should be run without a trend¹¹:

Thus we have either ϕ = 1 and α = 0 or ϕ < 1 and α ≠ 0. The null hypothesis in this case therefore is

A rejection of the null hypothesis can be interpreted in three alternative ways:

(i)

The case ϕ < 1 and α = 0 can be eliminated because it implies that {X t } would have a mean of zero which is unrealistic for most economic time series.

(ii)

The case ϕ = 1 and α ≠ 0 can equally be eliminated because it implies that {X t } has a long-run trend which contradicts our primary assumption.

(iii)

The case ϕ < 1 and α ≠ 0 is the only realistic alternative. It implies that the time series is stationary around a constant mean given by $$\frac{\alpha }{1-\phi }$$.

As before one can use, instead of a F-test, a t-test of the null hypothesis H₀: ϕ = 1 against the alternative hypothesis H₁: ϕ < 1. If the null hypothesis is not rejected, we interpret this to imply that α = 0. If, however, the null hypothesis H₀ is rejected, we conclude that α ≠ 0. Similarly, Monte-Carlo simulations have proven that the t-test is superior to the F-test.

The trend behavior of X t is uncertain:

This situation poses the following problem. Should the data exhibit a trend, but the Dickey-Fuller regression contains no trend, then the test is biased in favor of the null hypothesis. If the data have no trend, but the Dickey-Fuller regression contains a trend, the power of the test is reduced. In such a situation one can adapt a two-stage strategy. Estimate the Dickey-Fuller regression with a linear trend:

Use the t-test to test the null hypothesis H₀: ϕ = 1 against the alternative hypothesis H₁: ϕ < 1. If H₀ is not rejected, we conclude the process has a unit root with or without drift. The presence of a drift can then be investigated by a simple regression of $$\Delta X_{t}$$ against a constant followed by a simple t-test of the null hypothesis that the constant is zero against the alternative hypothesis that the constant is nonzero. As $$\Delta X_{t}$$ is stationary, the usual critical values can be used. ¹² If the t-test rejects the null hypothesis H₀, we conclude that there is no unit root. The trend behavior can then be investigated by a simple t-test of the hypothesis H₀: δ = 0. In this test the usual critical values can be used as {X t } is already viewed as being stationary.

7.3.4 Examples of Unit-Root Tests

As our first example, we examine the logged real GDP for Switzerland, $$\ln (\mathrm{BIP}_{t})$$, where we have adjusted the series for seasonality by taking a moving-average. The corresponding data are plotted in Fig. 1.​3. As is evident from this plot, this variable exhibits a clear trend so that the Dickey-Fuller regression should include a constant and a linear time trend. Moreover, $$\{\Delta \ln (\mathrm{BIP}_{t})\}$$ is typically highly autocorrelated which makes an autoregressive correction necessary. One way to make this correction is by augmenting the Dickey-Fuller regression by lagged $$\{\Delta \ln (\mathrm{BIP}_{t})\}$$ as additional regressors. Thereby the number of lags is determined by AIC. The corresponding result is reported in the first column of Table 7.2. It shows that AIC chooses only one autoregressive correction term. The value of t-test statistic is − 3. 110 which is just above the 5-% critical value. Thus, the null hypothesis is not rejected. If the autoregressive correction is chosen according to the method proposed by Ng and Perron five autoregressive lags have to be included. With this specification, the value of the t-test statistic is clearly above the critical value, implying that the null hypothesis of the presence of a unit root cannot be rejected (see second column in Table 7.2).¹³ The results of the ADF-tests is confirmed by the PP-test (column 3 in Table 7.2) with quadratic spectral kernel function and band width 20. 3 chosen according to Andrews’ formula (see Sect. 4.​4).

Table 7.2

Examples of unit root tests

R3M t

R3M t

Test

ADF

ADF

PP

PP

PP

Autoregressive

Ng and

Quadratic

correction

AIC

Perron

spectral

Bartlett

Bartlett

Band width

20.3

5

5

α

0.337

0.275

0.121

0.595

−0.014

δ

0.0001

0.0001

0.0002

−0.0021

ϕ

0.970

0.975

0.989

0.963

−0.996

γ ₁

0.885

1.047

γ ₂

−0.060

γ ₃

−0.085

γ ₄

−0.254

γ ₅

0.231

−3.110

−2.243

−1.543

−2.142

−0.568

Critical value

−3.460

−3.463

−3.460

−3.435

−2.878

(5 %)

Critical values from MacKinnon (1996)

The second example, examines the three-month LIBOR, {R3M t }. The series is plotted in Fig. 1.​4 The issue whether this series has a linear trend or not is not easy to decide. On the one hand, the series clearly has a negative trend over the sample period considered. On the other hand, a negative time trend does not make sense from an economic point of view because interest rates are bounded from below by zero. Because of this uncertainty, it is advisable to include in the Dickey-Fuller regression both a constant and a trend to be on the safe side. Column 5 in Table 7.2 reports the corresponding results. The value of the t-statistic of the PP-test with Bartlett kernel function and band width of 5 according to the Newey-West rule of thumb is − 2. 142 and thus higher than the corresponding 5-% critical of − 3. 435. Thus, we cannot reject the null hypothesis of the presence of a unit root. We therefore conclude that the process {R3M t } is integrated of order one, respectively difference-stationary. Based on this conclusion, the issue of the trend can now be decided by running a simple regression of $$\Delta \mathrm{R3M}_{t}$$ against a constant. This leads to the following results:

where e t denotes the least-squares residual. The mean of $$\Delta \mathrm{R3M}_{t}$$ is therefore − 0. 0315. This value is, however, statistically not significantly different from zero as indicated by the estimated standard error in parenthesis. Note that this estimate of the standard error has been corrected for autocorrelation (Newey-West correction). Thus, {R3M t } is not subject to a linear trend. One could have therefore run the Dickey-Fuller regression without the trend term. The result of corresponding to this specification is reported in the last column of Table 7.2. It confirms the presence of a unit root.

7.4 Generalizations of Unit-Root Tests

7.4.1 Structural Breaks in the Trend Function

As we have seen, the unit-root test depends heavily on the correct specification of the deterministic part. Most of the time this amounts to decide whether a linear trend is present in the data or not. In the previous section we presented a rule how to proceed in case of uncertainty about the trend. Sometimes, however, the data exhibit a structural break in their deterministic component. If this structural break is ignored, the unit-root test is biased in favor of the null hypothesis (i. e. in favor of a unit root) as demonstrated by Perron (1989). Unfortunately, the distribution of the test statistic under the null hypothesis, in our case the t-statistic, depends on the exact nature of the structural break and on its date of occurrence in the data. Following Perron (1989) we concentrate on three exemplary cases: a level shift, a change in the slope (change in the growth rate), and a combination of both possibilities. Figure 7.4 shows the three possibilities assuming that a break occurred in period T B . Thereby an AR(1) process with ϕ = 0. 8 was superimposed on the deterministic part.

Fig. 7.4

Three types of structural breaks at T B . (a ) Level shift. (b ) Change in slope. (c ) Level shift and change in slope

The unit-root test with the possibility of a structural break in period T B is carried out using the Dickey-Fuller test. Thereby the date of the structural break is assumed to be known. This assumption, although restrictive, is justifiable in many applications. The first oil price shock in 1973 or the German reunification in 1989 are examples of structural breaks which can be dated exactly. Other examples would include changes in the way the data are constructed. These changes are usually documented by the data collecting agencies. Table 7.3 summarizes the three variants of Dickey-Fuller regression allowing for structural breaks.¹⁴

Table 7.3

Dickey-Fuller regression allowing for structural breaks

Model A: Level Shift

H ₀: 

H ₁: 

$$X_{t} =\alpha +\delta t + \mathbf{1}_{\{t>T_{B}\}}(\alpha _{B}-\alpha ) +\phi X_{t-1} + Z_{t}$$, ϕ < 1

Model B: Change in Slope (Change in Growth Rate)

H ₀: 

H ₁: 

$$X_{t} =\alpha +\delta t + \mathbf{1}_{\{t>T_{B}\}}(\delta _{B}-\delta )(t - T_{B}) +\phi X_{t-1} + Z_{t}$$, ϕ < 1

Model C: Level Shift and Change in Slope

H ₀: 

H ₁: 

$$\mathbf{1}_{\{t=T_{B}+1\}}$$ and $$\mathbf{1}_{\{t>T_{B}\}}$$ denotes the indicator function which takes the value one if the condition is satisfied and the value zero otherwise

Model A allows only for a level shift. Under the null hypothesis the series undergoes a one-time shift at time T B . This level shift is maintained under the null hypothesis which posits a random walk. Under the alternative, the process is viewed as being trend-stationary whereby the trend line shifts parallel by α B −α at time T B . Model B considers a change in the mean growth rate from α to α B at time T B . Under the alternative, the slope of time trend changes from δ to δ B . Model C allows for both types of break to occur at the same time.

The unit-root test with possible structural break for a time series X t , t = 0, 1, …, T, is implemented in two stages as follows. In the first stage, we regress X t on the corresponding deterministic component using OLS. The residuals $$\widetilde{X}_{0},\widetilde{X}_{1},\ldots,\widetilde{X}_{T}$$ from this regression are then used to carry out a Dickey-Fuller test:

The distribution of the corresponding t-statistic under the null hypothesis depends not only on the type of the structural break, but also on the relative date of the break in the sample. Let this relative date be parameterized by $$\lambda = T_{B}/T$$. The asymptotic distribution of the t-statistic has been tabulated by Perron (1989). This table can be used to determine the critical values for the test. These critical values are smaller than those from the normal Dickey-Fuller table. Using a 5 % significance level, the critical values range between − 3. 80 and − 3. 68 for model A, between − 3. 96 and − 3. 65 for model B, and between − 4. 24 and − 3. 75 for model C, depending on the value of $$\lambda$$. These values also show that the dependence on $$\lambda$$ is only weak.

In the practical application of the test one has to control for the autocorrelation in the data. This can be done by using the Augmented Dickey-Fuller (ADF) test. This amounts to the introduction of $$\Delta \widetilde{X}_{t-j}$$, $$t = 1,2,\ldots,p - 1$$, as additional regressors in the above Dickey-Fuller regression. Thereby the order p can be determined by Akaike’s information criterion (AIC) or by the iterative testing procedure of Ng and Perron (1995). Alternatively, one may use, instead of the ADF test, the Phillips-Perron test. In this case one computes the usual t-statistic for the null hypothesis ϕ = 1 and corrects it using the formulas in Phillips and Perron (1988) as explained in Sect. 7.3.2. Which of the two methods is used, is irrelevant for the determination of the critical values which can be extracted from Perron (1989).

Although it may be legitimate in some cases to assume that the time of the structural break is known, we cannot take this for granted. It is therefore important to generalize the test allowing for an unknown date for the occurrence of a structural break. The work of Zivot and Andrews (1992) has shown that the procedure proposed by Perron can be easily expanded in this direction. We keep the three alternative models presented in Table 7.3, but change the null hypothesis to a random walk with drift with no exogenous structural break. Under the null hypothesis, {X t } is therefore assumed to be generated by

The time of the structural T B , respectively $$\lambda = T_{B}/T$$, is estimated in such a way that {X t } comes as close as possible to a trend-stationary process. Under the alternative hypothesis {X t } is viewed as a trend-stationary process with unknown break point. The goal of the estimation strategy is to chose T B , respectively $$\lambda$$, in such a way that the trend-stationary alternative receives the highest weight. Zivot and Andrews (1992) propose to estimate $$\lambda$$ by minimizing the value of the t-statistic $$t_{\hat{\phi }}(\lambda )$$ under the hypothesis ϕ = 1:

(7.2)

where $$\Lambda$$ is a closed subinterval of (0, 1).¹⁵ The distribution of the test statistic under the null hypothesis for the three cases is tabulated in Zivot and Andrews (1992). This table then allows to determine the appropriate critical values for the test. In practice, one has to take the autocorrelation of the time series into account by one of the methods discussed previously.

This testing strategy can be adapted to determine the time of a structural break in the linear trend irrespective of whether the process is trend-stationary or integrated of order one. The distributions of the corresponding test statistics have been tabulated by Vogelsang (1997).¹⁶

7.4.2 Testing for Stationarity (KPSS Test)

The unit-root tests we discussed so far tested the null hypothesis that the process is integrated of order one against the alternative hypothesis that the process is integrated of order zero (i.e. is stationary). However, one may be interested in reversing the null and the alternative hypothesis and test the hypothesis of stationarity against the alternative that the process is integrated of order one. Such a test has been proposed by Kwiatkowski et al. (1992), called the KPSS-Test. This test rests on the idea that according to the Beveridge-Nelson decomposition (see Sect. 7.1.4) each integrated process of order one can be seen as the sum of a linear time trend, a random walk and a stationary process:

where {U t } denotes a stationary process. If d = 0 then the process becomes trend-stationary, otherwise it is integrated of order one.¹⁷ Thus, one can state the null and the alternative hypothesis as follows:

Denote by {S t } the process of partial sums obtained from the residuals {e t } of a regression of X t against a constant and a linear time trend, i.e. $$S_{t} =\sum _{ j=1}^{t}e_{j}$$.¹⁸ Under the null hypothesis d = 0, {S t } is integrated of order one whereas under the alternative {S t } is integrated of order two. Based on this consideration Kwiatkowski et al. propose the following test statistic for a time series consisting of T observations:

(7.3)

where $$\widehat{J}_{T}$$ is an estimate of the long-run variance of {U t } (see Sect. 4.​4). As {S t } is an integrated process under the null hypothesis, the variance of {S t } grows linearly in t (see Sect. 1.​4.​4 or 7.2) so that the sum of squared S t diverges at rate T ². Thus, the test statistic remains bounded and can be shown to converge. Note that the test statistic is independent from further nuisance parameters. Under the alternative hypothesis, however, {S t } is integrated of order two. Thus, the null hypothesis will be rejected for large values of W T . The corresponding asymptotic critical values of the test statistic are reported in Table 7.4.

Table 7.4

Critical values of the KPSS test

Regression without time trend

Significance level

0.1

0.05

0.01

Critical value

0.347

0.463

0.739

Regression with time trend

Significance level

0.1

0.05

0.01

Critical value

0.119

0.146

0.216

See Kwiatkowski et al. (1992)

7.5 Regression with Integrated Variables

7.5.1 The Spurious Regression Problem

The discussion on the Dickey-Fuller and Phillips-Perron tests showed that in a regression of the integrated variables X t on its past X t−1 the standard $$\sqrt{ T}$$-asymptotics no longer apply. A similar conclusion also holds if we regress an integrated variable X t against another integrated variable Y t . Suppose that both processes {X t } and {Y t } are generated as a random walk:

where the processes {U t } and {V t } are uncorrelated with each other at all leads and lags. Thus,

Consider now the regression of Y t on X t and a constant:

As {X t } and {Y t } are two random walks which are uncorrelated with each other by construction, one would expect that the OLS-estimate of the coefficient of X t , $$\hat{\beta }$$, should tend to zero as the sample size T goes to infinity. The same is expected for the coefficient of determination R². This is, however, not true as has already been remarked by Yule (1926) and, more recently, by Granger and Newbold (1974). The above regression will have a tendency to “discover” a relationship between Y t and X t despite the fact that there is none. This phenomenon is called spurious correlation or spurious regression. Similarly, unreliable results would be obtained by using a simple t-test for the null hypothesis β = 0 against the alternative hypothesis β ≠ 0. The reason for these treacherous findings is that the model is incorrect under the null as well as under the alternative hypothesis. Under the null hypothesis $$\{\varepsilon _{t}\}$$ is an integrated process which violates the standard assumption for OLS. The alternative hypothesis is not true by construction. Thus, OLS-estimates should be interpreted with caution when a highly autocorrelated process {Y t } is regressed on another highly correlated process {X t }. A detailed analysis of the spurious regression problem is provided by Phillips (1986).

The spurious regression problem can be illustrated by a simple Monte Carlo study. Specifying $$U_{t} \sim \mathrm{ IIDN}(0,1)$$ and $$V _{t} \sim \mathrm{ IIDN}(0,1)$$, we constructed N = 1000 samples for {Y t } and {X t } of size T = 1000 according to the specification above. The sample size was chosen especially large to demonstrate that this is not a small sample issue. As a contrast, we constructed two independent AR(1) processes with AR-coefficients ϕ X  = 0. 8 and $$\phi _{Y } = -0.5$$.

Figures 7.5a,b show the drastic difference between a regression with stationary variables and integrated variables. Whereas the distribution of the OLS-estimates of β is highly concentrated around the true value β = 0 in the stationary case, the distribution is very flat in the case of integrated variables. A similar conclusion holds for the corresponding t-value. The probability of obtaining a t-value greater than 1. 96 is bigger than 0. 9. This means that in more than 90 % of the time the t-statistic leads to a rejection of the null hypothesis and therefore suggests a relationship between Y t and X t despite their independence. In the stationary case, this probability turns out to be smaller than 0. 05. These results are also reflected in the coefficient of determination R². The median R² is approximately 0. 17 in the case of the random walks, but only 0. 0002 in the case of AR(1) processes.

Fig. 7.5

Distribution of OLS-estimate $$\hat{\beta }$$ and t-statistic $$t_{\hat{\beta }}$$ for two independent random walks and two independent AR(1) processes. (a ) Distribution of $$\hat{\beta }$$. (b ) Distribution of $$t_{\hat{\beta }}$$. (c ) Distribution of $$\hat{\beta }$$ and t-statistic $$t_{\hat{\beta }}$$

The problem remains the same if {X t } and {Y t } are specified as random walks with drift:

where {U t } and {V t } are again independent from each other at all leads and lags. The regression would be same as above:

7.5.2 Bivariate Cointegration

The spurious regression problem cannot be circumvented by first testing for a unit root in Y t and X t and then running the regression in first differences in case of no rejection of the null hypothesis. The reason being that a regression in the levels of Y t and X t may be sensible even when both variables are integrated. This is the case when both variables are cointegrated. The concept of cointegration goes back to Engle and Granger (1987) and initiated a literal research boom. We will give a more general definition in Chap. 16 when we deal with multivariate time series. Here we stick to the case of two variables and present the following definition.

Definition 7.2 (Cointegration, Bivariate).

Two stochastic processes {X t } and {Y t } are called cointegrated if the following two conditions are fulfilled:

(i)

{X t } and {Y t } are both integrated processes of order one, i.e. $$X_{t} \sim \mathrm{ I}(1)$$ and $$Y _{t} \sim \mathrm{ I}(1)$$ ;

(ii)

there exists a constant β ≠ 0 such that {Y t −βX t } is a stationary process, i.e. $$\{Y _{t} -\beta X_{t}\} \sim \mathrm{ I}(0)$$.

The issue whether two integrated processes are cointegrated can be decided on the basis of a unit root test. Two cases can be distinguished. In the first one, β is assumed to be known. Thus, one can immediately apply the augmented Dickey-Fuller (ADF) or the Phillips-Perron (PP) test to the process {Y t −β X t }. Thereby the same issue regarding the specification of the deterministic part arises. The critical values can be retrieved from the usual tables (for example from MacKinnon 1991). In the second case, β is not known and must be estimated from the data. This can be done running, as a first step, a simple (cointegrating) regression of Y t on X t including a constant and/or a time trend.¹⁹ Thereby the specification of the deterministic part follows the same rules as before. The unit root test is then applied, in the second step, to the residuals from this regression. As the residuals have been obtained from a preceding regression, we are faced with the so-called “generated regressor problem”.²⁰ This implies that the usual Dickey-Fuller tables can no longer be used, instead the tables provided by Phillips and Ouliaris (1990) become the relevant ones. As before, the corresponding asymptotic distribution depends on the specification of the deterministic part in the cointegrating regression. If this regression included a constant, the residuals have necessary a mean of zero so that the Dickey-Fuller regression should include no constant (case 1 in Table 7.1):

where e t and $$\xi _{t}$$ denote the residuals from the cointegrating and the residuals of the Dickey-Fuller regression, respectively. In most applications it is necessary to correct for autocorrelation which can be done by including additional lagged differences $$\Delta \hat{\varepsilon }_{t-1},\ldots,\Delta \hat{\varepsilon }_{t-p+1}$$ as in the ADF-test or by adjusting the t-statistic as in the PP-test. The test where β is estimated from a regression is called the regression test for cointegration . Note that if the two series are cointegrated then the OLS estimate of β is (super) consistent.

In principle it is possible the generalize this single equation approach to more than two variables. This encounters, however, some conceptual problems. First, there is the possibility of more than one linearly independent cointegrating relationships which cannot be detected by a single regression. Second, the dependent variable in the regression may not be part of the cointegrating relation which might involves only the other variables. In such a situation the cointegrating regression is again subject to the spurious regression problem. These issues turned the interest of the profession towards multivariate approaches. Chapter 16 presents alternative procedures and discusses the testing, estimation, and interpretation of cointegrating relationships in detail.

An Example for Bivariate Cointegration

As an example, we consider the relation between the short-term interest rate, {R3M t }, and inflation, {INFL t }, in Switzerland over the period January 1989 to February 2012. As the short-term interest rate we take the three month LIBOR. Both time series are plotted in Fig. 7.6a. As they are integrated according to the unit root tests (not shown here), we can look for cointegration. The cointegrating regression delivers:

(7.4)

The residuals from this regression, denoted by e t , are represented in Fig. 7.6b. The ADF unit root test of these residuals leads to a value of − 3. 617 for the t-statistic. Thereby an autoregressive correction of 13 lags was necessary according to the AIC criterion. The corresponding value of the t-statistic resulting from the PP unit root test using a Bartlett window with band width 7 is − 4. 294. Taking a significance level of 5 %, the critical value according to Phillips and Ouliaris (1990, Table IIb) is − 3. 365.²¹ Thus, the ADF as well as the PP test reject the null hypothesis of a unit root in the residuals. This implies that inflation and the short-term interest rate are cointegrated.

Fig. 7.6

Cointegration of inflation and three-month LIBOR. (a ) Inflation and three-month LIBOR. (b ) Residuals from cointegrating regression

7.5.3 Rules to Deal with Integrated Times Series

The previous sections demonstrated that the handling of integrated variables has to be done with care. We will therefore in this section examine some rules of thumb which should serve as a guideline in practical empirical work. These rules are summarized in Table 7.5. In that this section follows very closely the paper by Stock and Watson (1988b) (see also Campbell and Perron 1991).²² Consider the linear regression model:

(7.5)

This model is usually based on two assumptions:

(1)

The disturbance term $$\varepsilon _{t}$$ is white noise and is uncorrelated with any regressor. This is, for example, the case if the regressors are deterministic or exogenous.

(2)

All regressors are either deterministic or stationary processes.

Table 7.5

Rules of thumb in regressions with integrated processes

Remarks

Assumptions

OLS-estimator

Consistency

Standard asymptotics

(1)

(2)

Yes

Yes

Classic results for OLS

(1)

(2.a)

Yes

Yes

(1)

(2.b)

Yes

Yes

(1)

(2.c)

Yes

No

(1.a)

(2.a)

No

No

Omitted variable bias

(1.a)

(2.b)

No

No

Omitted variable bias

(1.a)

(2.c)

Yes

No

Neither (1) nor (1.a)

(2.c)

No

No

Spurious regression

Source: Stock and Watson (1988b); results for the coefficients of interest

If Eq. (7.5) represents the true data generating process, {Y t } must be a stationary process. Under the above assumptions, the OLS-estimator is consistent and the OLS-estimates are asymptotically normally distributed so that the corresponding t- and F-statistics will be approximately distributed as t- and F- distributions.

Consider now the case that assumption 2 is violated and that some or all regressors are integrated, but that instead one of the two following assumptions holds:

(2.a)

The relevant coefficients are coefficients of mean-zero stationary variables.

(2.b)

Although the relevant coefficients are those of integrated variables, the regression can be rearranged in such a way that the relevant coefficients become coefficients of mean-zero stationary variables.

Under assumptions 1 and 2.a or 2.b the OLS-estimator remains consistent. Also the corresponding t- and F-statistics remain valid so that the appropriate critical values can be retrieved from the t-, respectively F-distribution. If neither assumption 2.a nor 2.b holds, but the following assumption:

(2.c)

The relevant coefficients are coefficients of integrated variables and the regression cannot be rewritten in a way that they become coefficients of stationary variables.

If assumption 1 remains valid, but assumption 2.c holds instead of 2.a and 2.b, the OLS-estimator is still consistent. However, the standard asymptotic theory for the t- and the F-statistic fails so that they become useless for normal statistical inferences.

If we simply regress one variable on another in levels, the error term $$\varepsilon _{t}$$ is likely not to follow a white noise process. In addition, it may even be correlated with some regressors. Suppose that we replace assumption 1 by:

(1.a)

The integrated dependent variable is cointegrated with at least one integrated regressor such that the error term is stationary, but may remain autocorrelated or correlated with the regressors.

Under assumptions 1.a and 2.a, respectively 2.b, the regressors are stationary, but correlated with the disturbance term, in this case the OLS-estimator becomes inconsistent. This situation is known as the classic omitted variable bias, simultaneous equation bias or errors-in-variable bias. However, under assumptions 1.a and 2.c, the OLS-estimator is consistent for the coefficients of interest. However, the standard asymptotic theory fails. Finally, if both the dependent variable and the regressors are integrated without being cointegrated, then the disturbance term is integrated and the OLS-estimator becomes inconsistent. This is the spurious regression problem treated in Sect. 7.5.1.

Example: Term Structure of Interest

We illustrate the above rules of thumb by investigating again the relation between inflation ({INFL t }) and the short-term interest rate ({R3M t }). In Sect. 7.5.2.1 we found that the two variables are cointegrated with coefficient $$\hat{\beta }= 0.5535$$ (see Eq. (7.4)). In a further step we want to investigate a dynamic relation between the two variables and estimate the following equation:

where $$\varepsilon _{t} \sim \mathrm{WN}(0,\sigma ^{2})$$. In this regression we want to test the hypotheses ϕ ₃ = 0 against ϕ ₃ ≠ 0 and δ ₁ = 0 against δ ₁ ≠ 0 by examining the corresponding simple t-statistics. Note that we are in the context of integrated variables so that the rules of thumb summarized in Table 7.5 apply. We can rearrange the above equation as

ϕ ₃ is now a coefficient of a stationary variable in a regression with a stationary dependent variables. In addition $$\varepsilon _{t} \sim \mathrm{WN}(0,\sigma ^{2})$$ so that assumptions (1) und (2.b) are satisfied. We can therefore use the ordinary t-statistic to test the hypothesis ϕ ₃ = 0 against ϕ ₃ ≠ 0. Note that it is not necessary to actually carry out the rearrangement of the equation. All relevant item can be retrieved from the original equation.

To test the hypothesis δ ₁ = 0, we rearrange the equation to yield:

As {R3M t } and {INFL t } are cointegrated, $$\mathrm{INFL}_{t-1} -\hat{\beta }\mathrm{ R3M}_{t-1}$$ is stationary. Thus assumptions (1) and (2.b) hold again and we use once more the simple t-test. As before it is not necessary to actually carry out the transformation.

References

Banerjee A, Dolado J, Galbraith JW, Hendry DF (1993) Co-integration, error-correction, and the econometric analysis of non-stationary data. Oxford University Press, OxfordCrossRef

Campbell JY, Mankiw NG (1987) Are output fluctuations transitory? Q J Econ 102:857–880CrossRef

Campbell JY, Perron P (1991) Pitfalls and opportunities: What macroeconomists should know about unit roots. In: Blanchard OJ, Fischer S (eds) Macroeconomics annual 1991, vol 6. MIT Press, Cambridge, pp 141–201

Christiano LJ, Eichenbaum M (1990) Unit roots in real GNP: Do we know and do care? Carn-Roch Conf Ser Public Pol 32:7–62CrossRef

Christiano LJ, Eichenbaum M, Vigfusson RJ (2003) What happens after a technology shock? Working Paper No. 9819, NBER

Cochrane JH (1988) How big is the random walk in GNP? J Polit Econ 96(5):893–920CrossRef

Dickey D, Fuller WA (1976) Distribution of the estimators for autoregressive time series with a unit root. J Am Stat Assoc 74:427–431

Dickey DA, Fuller WA (1981) Likelihood ratio statistics for autoregressive time series with a unit root. Econometrica 49:1057–1072CrossRef

Elder J, Kennedy PE (2001) Testing for unit roots: What should students be taught? J Econ Educ 32:137–146CrossRef

Engle RF, Granger CWJ (1987) Co-integration and error correction: Representation, estimation and testing. Econometrica 55:251–276CrossRef

Fuller WA (1976) Introduction to statistical time series. Wiley, New York

Galí J (1999) Technology, employment, and the business cycle: Do technology shocks explain aggregate fluctuations? Am Econ Rev 89:249–271CrossRef

Granger CWJ, Newbold P (1974) Spurious regression in econometrics. J Econ 2:111–120CrossRef

Hamilton JD (1994b) Time series analysis. Princeton University Press, Princeton

Hannan EJ, Deistler M (1988) The statistical theory of linear systems. Wiley, New York

Harvey AC (1989) Forecasting, structural time series models and the kalman filter. Cambridge University Press, Cambridge

Hauser MA, Pötscher BM, Reschenhofer E (1999) Measuring persistence in aggregate output: ARMA models, fractionally integrated ARMA models and nonparametric procedures. Empir Econ 24:243–269CrossRef

Kwiatkowski D, Phillips PC, Schmidt P, Shin Y (1992) Testing the null hypothesis of stationarity against the alternative of a unit root. How sure are we that economic time series have a unit root? J Econ 54:159–178CrossRef

MacKinnon JG (1991) Critical values for co-integration tests. In: Engle RF, Granger CWJ (eds) Long-run economic relationships. Oxford University Press, Oxford, pp 267–276

MacKinnon JG (1996) Numerical distribution functions for unit root and cointegration tests. J Appl Econ 11:601–618CrossRef

Mills TC (2003) Modelling trends and cycles in economic time series. Palgrave texts in econometrics. Palgrave Macmillan, HampshireCrossRef

Nelson CR, Plosser CI (1982) Trends and random walks in macro-economic time series: Some evidence and implications. J Monet Econ 10:139–162CrossRef

Neusser K (2000) An algebraic interpretation of cointegration. Econ Lett 67:273–281CrossRef

Ng S, Perron P (1995) Unit root tests in ARMA models with data dependent methods for the selection of the truncation lag. J Am Stat Assoc 90:268–281CrossRef

Nicholls DF, Pagan AR (1984) Estimating predictions, prediction errors and their standard deviations using constructed variables. J Econ 24:293–310CrossRef

Pagan AR (1984) Econometric issues in the analysis of regressions with generated regressors. Int Econ Rev 25:183–209CrossRef

Perron P (1989) The great crash, the oil price shock, and the unit root hypothesis. Econometrica 57:1361–1401CrossRef

Perron P (2006) Dealing with structural breaks. In: Hassani H, Mills TC, Patterson K (eds) Palgrave handbook in econometrics, vol 1. Econometric theory. Palgrave Macmillan, Hampshire, pp 278–352

Phillips PC (1987) Time series regression with a unit root. Econometrica 55:277–301CrossRef

Phillips PCB (1986) Understanding spurious regressions in econometrics. J Econ 33:311–340CrossRef

Phillips PCB, Ouliaris S (1990) Asymptotic properties of residual based tests of cointegration. Econometrica 58(1):165–193CrossRef

Phillips PCB, Perron P (1988) Testing for a unit root in time series regression. Biometrika 75:335–346CrossRef

Said SE, Dickey DA (1984) Testing for unit roots in autoregressive-moving average models of unknown order. Biometrika 71:599–607CrossRef

Silverman BW (1986) Density estimation. Chapman and Hall, LondonCrossRef

Sims CA, Stock JH, Watson MW (1990) Inference in linear time series with some unit roots. Econometrica 58:113–144CrossRef

Stock JH (1994) Unit roots, structural breaks and trends. In: Engle RF, McFadden DL (eds) Handbook of econometrics, vol IV. Elsevier Science B.V., Amsterdam, pp 2739–2841

Stock JH, Watson MW (1988b) Variable trends in economic time series. J Econ Perspect 2(3):147–174

Vogelsang TJ (1997) Wald-type tests for detecting breaks in the trend function of a dynamic time series. Econ Theory 13:818–849CrossRef

Yule GU (1926) Why do we sometimes get nonsense correlations between time series? A study in sampling and the nature of time series. J R Stat Soc 89:1–64CrossRef

Zivot E, Andrews DW (1992) Further evidence on the great crash, the oil-price shock, and the unit-root hypothesis. J Bus Econ Stat 10:251–270

Footnotes

1

Strictly speaking this does not conform to the definitions used in this book because our definition of ARMA processes assumes stationarity.

2

Proof: By assumption {ψ j } is absolutely summable so that $$\Psi (1)$$ converges. Moreover, as $$\Psi (1)\neq 0$$, there exists $$\varepsilon> 0$$ and an integer m such that $$\left \vert \sum _{j=0}^{h}\psi _{j}\right \vert>\varepsilon$$ for all h > m. The squares are therefore bounded from below by $$\varepsilon ^{2}> 0$$ so that their infinite sum diverges to infinity.

3

The use of the partial derivative is just for convenience. It does not mean that X t+h is differentiated in the literal sense.

4

Neusser (2000) shows how a Beveridge-Nelson decomposition can also be derived for higher order integrated processes.

5

We will treat more general cases in Sect. 7.5 and Chap. 16

6

The densities were estimated using an adaptive kernel density estimator with Epanechnikov window (see Silverman (1986)).

7

These interpolation formula are now implemented in many software packages, like EVIEWS, to compute the appropriate critical values.

8

This fact may pose a problem by itself.

9

The exact assumptions can be read in Phillips (1987) and Phillips and Perron (1988).

10

In case of the ADF-test additional regressors, $$\Delta X_{t-j},j> 0$$, might be necessary.

11

In case of the ADF-test additional regressors, $$\Delta X_{t-j},j> 0$$, might be necessary.

12

Eventually, one must correct the corresponding standard deviation by taking the autocorrelation in the residual into account. This can be done by using the long-run variance. In the literature this correction is known as the Newey-West correction.

13

The critical value changes slightly because the inclusion of additional autoregressive terms changes the sample size.

14

See Eq. (7.1) and Table 7.1 for comparison.

15

Taking the infimum over $$\Lambda$$ instead over (0, 1) is for theoretical reasons only. In practice, the choice of $$\Lambda$$ plays no important role. For example, one may take $$\Lambda = [0.01,\: 0.99]$$.

16

See also the survey by Perron (2006).

17

If the data exhibit no trend, one can set δ equal to zero.

18

This auxiliary regression may include additional exogenous variables.

19

Thereby, in contrast to ordinary OLS regressions, it is irrelevant which variable is treated as the left hand, respectively right hand variable.

20

This problem was first analyzed by Nicholls and Pagan (1984) and Pagan (1984) in a stationary context.

21

For comparison, the corresponding critical value according to MacKinnon (1991) is − 2. 872.

22

For a thorough analysis the interested reader is referred to Sims et al. (1990).

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_8

8. Models of Volatility

Klaus Neusser1 

(1)

Bern, Switzerland

The prices of financial market securities are often shaken by large and time-varying shocks. The amplitudes of these price movements are not constant. There are periods of high volatility and periods of low volatility. Within these periods volatility seems to be positively autocorrelated: high amplitudes are likely to be followed by high amplitudes and low amplitudes by low amplitudes. This observation which is particularly relevant for high frequency data such as, for example, daily stock market returns implies that the conditional variance of the one-period forecast error is no longer constant (homoskedastic), but time-varying (heteroskedastic). This insight motivated Engle (1982) and Bollerslev (1986) to model the time-varying variance thereby triggering a huge and still growing literature.¹ The importance of volatility models stems from the fact that the price of an option crucially depends on the variance of the underlying security price. Thus with the surge of derivative markets in the last decades the application of such models has seen a tremendous rise. Another use of volatility models is to assess the risk of an investment. In the computation of the so-called value at risk (VaR), these models have become an indispensable tool. In the banking industry, due to the regulations of the Basel accords, such assessments are in particular relevant for the computation of the required equity capital backing-up assets of different risk categories.

The following exposition focuses on the class of autoregressive conditional heteroskedasticity models (ARCH models) and their generalization the generalized autoregressive conditional heteroskedasticity models (GARCH models). These models form the basis for even more generalized models (see Bollerslev et al. (1994) or Gouriéroux (1997)). Campbell et al. (1997) provide a broader economically motivated approach to the econometric analysis of financial market data.

8.1 Specification and Interpretation

8.1.1 Forecasting Properties of AR(1)-Models

Models of volatility play an important role in explaining the behavior of financial market data. They start from the observation that periods of high (low) volatility are clustered in specific time intervals. In these intervals high (low) volatility periods are typically followed by high (low) volatility periods. Thus volatility is usually positively autocorrelated as can be observed in Fig. 8.3. In order to understand this phenomenon we recapitulate the forecasting properties of the AR(1) model.²Starting from the model

the best linear forecast in the mean-squared-error sense of X t+1 conditional on {X t , X t−1, …}, denoted by $$\mathbb{P}_{t}X_{t+1}$$, is given by (see Chap. 3)

In practice the parameters c and ϕ are replaced by an estimate.

Fig. 8.1

Simulation of two ARCH(1) processes (α ₁ = 0. 9 and α ₁ = 0. 5)

Fig. 8.2

Parameter region for which a strictly stationary solution to the GARCH(1,1) process exists assuming $$\nu _{t} \sim \mathrm{ IID\,N}(0,1)$$

Fig. 8.3

Daily return of the SMI (Swiss Market Index) computed as $$\Delta \log (\mathrm{SMI}_{t})$$ between January 3rd 1989 and February 13th 2004

The conditional variance of the forecast error then becomes:

where $$\mathbb{E}_{t}$$ denotes the conditional expectation operator based on information X t , X t−1, …. The conditional variance of the forecast error is therefore constant, irrespective of the current state.

The unconditional forecast is simply the expected value of $$\mathbb{E}X_{t+1} =\mu = \frac{c} {1-\phi }$$ with forecast error variance:

Thus the conditional as well as the unconditional variance of the forecast error are constant. In addition, the conditional variance is smaller and thus more precise because it uses more information. Similar arguments can be made for ARMA models in general.

8.1.2 The ARCH(1) Model

The volatility of financial market prices exhibit a systematic behavior so that the conditional forecast error variance is no longer constant. This observation led Engle (1982) to consider the following simple model for heteroskedasticity (non-constant variance).

Definition 8.1 (ARCH(1) Model).

A stochastic process {Z t }, $$t \in \mathbb{Z}$$ , is called an autoregressive conditional heteroskedastic process of order one, ARCH(1) process, if it is the solution of the following stochastic difference equation:

(8.1)

where $$\nu _{t} \sim \mathrm{ IID\,N}(0,1)$$ and where ν t and Z t−1 are independent from each other for all $$t \in \mathbb{Z}$$.

We will discuss the implications of this simple model below and consider generalizations in the next sections. First we prove the following theorem.

Theorem 8.1.

Under conditions stated in the definition of the ARCH(1) process, the difference equation  (8.1) possesses a unique and strictly stationary solution with $$\mathbb{E}Z_{t}^{2} < \infty $$ . This solution is given by

(8.2)

Proof.

Define the process

(8.3)

Iterating backwards k times we get:

Define the process {Y ′ t } as

The right-hand side of the above expression just contains nonnegative terms. Moreover, making use of the IID N(0, 1) assumption of {ν t },

Thus, $$0 \leq Y _{t}' < \infty $$ a.s. Therefore, {Y t ′} is strictly stationary and satisfies the difference equation (8.3). This implies that $$Z_{t} = \sqrt{Y _{t}'}$$ is also strictly stationary and satisfies the difference equation (8.1).

To prove uniqueness, we follow Giraitis et al. (2000). For any fixed t, it follows from the definitions of Y t and Y t ′ that for any k ≥ 1

The expectation of the right-hand side is bounded by

Define the event A k by $$A_{k} =\{ \vert Y _{t} - Y _{t}'\vert > 1/k\}$$. Then,

where the first inequality follows from Chebyschev’s inequality setting r = 1 (see Theorem C.3). Thus, $$\sum _{k=1}^{\infty }\mathbf{P}(A_{k}) < \infty $$. The Borel-Cantelli lemma (see Theorem C.4) then implies that P{A k  i. o. } = 0. However, as $$A_{k} \subset A_{k+1}$$, P(A k ) = 0 for any k. Thus, Y t  = Y t ′ a.s. □ 

Remark 8.1.

Note that the normality assumption is not necessary for the proof. The assumption $$\nu _{t} \sim \mathrm{ IID}(0,1)$$ would be sufficient. Indeed, in practice it has been proven useful to adopt distributions with fatter tail than the normal, like the t-distribution (see the discussion in Sect. 8.1.3).

Given the assumptions made above {Z t } has the following properties:

(i)

The expected value of Z t is:

This follows from the assumption that ν t and Z t−1 are independent.

(ii)

The covariances between Z t and Z t−h , $$\mathbb{E}Z_{t}Z_{t-h}$$, for h ≠ 0 are given by:

This is also a consequence of the independence assumption between ν t and Z t−1, respectively between ν t−h and $$Z_{t-h-1}$$.

(iii)

The variance of Z t is:

This follows from the independence assumption between ν t and Z t−1 and from the stationarity of {Z t }. Because α ₀ > 0 and 0 < α ₁ < 1, the variance is always strictly positive and finite.

(iv)

As ν t is normally distributed, its skewness, $$\mathbb{E}\nu _{t}^{3}$$, equals zero. The independence assumption between ν t and Z t−1 ² then implies that the skewness of Z t is also zero, i.e.

Z t therefore has a symmetric distribution.

The properties (i), (ii) and (iii) show that {Z t } is a white noise process. According to Theorem 8.1 it is not only stationary but even strictly stationary. Thus {Z t } is uncorrelated with $$Z_{t-1},Z_{t-2},\ldots$$, but not independent from its past! In particular we have:

The conditional variance of Z t therefore depends on Z t−1. Note that this dependence is positive because α ₁ > 0.

In order to guarantee that this conditional variance is always positive, we must postulate that α ₀ > 0 and α ₁ > 0. The stability of the difference equation requires in addition that α ₁ < 1.³ Thus high volatility in the past, a large realization of Z t−1, is followed by high volatility in the future. The precision of the forecast, measured by the conditional variance of the forecast error, thus depends on the history of the process. This feature is not compatible with linear models and thus underlines the non-linear character of the ARCH model and its generalizations.

Despite the fact that ν t was assumed to be normally distributed, Z t is not normally distributed. Its distribution deviates from the normal distribution in that extreme realizations are more probable. This property is called the heavy-tail property . In particular we have⁴:

The strict stationarity of {Z t } implies $$\mathbb{E}Z_{t}^{4} = \mathbb{E}Z_{t-1}^{4}$$ so that

$$\mathbb{E}Z_{t}^{4}$$ is therefore positive and finite if and only if 3α ₁ ² < 1, respectively if $$0 <\alpha _{1} < 1/\sqrt{3} = 0.5774$$. For high correlation of the conditional variance, i.e. high $$\alpha _{1} > 1/\sqrt{3}$$, the fourth moment and therefore also all higher even moments will no longer exist. The kurtosis κ is

if $$\mathbb{E}Z_{t}^{4}$$ exists. The heavy-tail property manifests itself by a kurtosis greater than 3 which is the kurtosis of the normal distribution. The distribution of Z t is therefore leptokurtic and thus more vaulted than the normal distribution.

Finally, we want to examine the autocorrelation function of Z t ². This will lead to a test for ARCH effects, i.e. for time varying volatility (see Sect. 8.2 below).

Theorem 8.2.

Assuming that $$\mathbb{E}Z_{t}^{4}$$ exists, $$Y _{t} = \frac{Z_{t}^{2}} {\alpha _{0}}$$ has the same autocorrelation function as the AR(1) process $$W_{t} =\alpha _{1}W_{t-1} + U_{t}$$ with $$U_{t} \sim \mathrm{ WN}(0,1)$$ . In addition, under the assumption 0 < α ₁ < 1, the process {W t } is also causal with respect to {U t }.

Proof.

From $$Y _{t} =\nu _{ t}^{2}(1 +\alpha _{1}Y _{t-1})$$ we get:

Therefore, $$\gamma _{Y }(h) =\alpha _{ 1}^{h}\gamma _{Y }(0) \Rightarrow \rho (h) =\alpha _{ 1}^{h}$$. □ 

The unconditional variance of X t is:

The unconditional variance of X t involves all parameters of the model. Thus modeling the variance of X t induces a trade-off between ϕ, α ₀ and α ₁.

Figure 8.1 plots the realizations of two AR(1)-ARCH(1) processes.Both processes have been generated with the same realization of {ν t } and the same parameters ϕ = 0. 9 and α ₀ = 1. Whereas the first process (shown on the left panel of the figure) was generated with a value of α ₁ = 0. 9, the second one had a value of α ₁ = 0. 5. In both cases the stability condition, α ₁ < 1, is fulfilled, but for the first process 3α ₁ ² > 1, so that the fourth moment does not exist. One can clearly discern the large fluctuations, in particular for the first process.

8.1.3 General Models of Volatility

The simple ARCH(1) model can be and has been generalized in several directions. A straightforward generalization proposed by Engle (1982) consists by allowing further lags to enter the ARCH equation (8.1). This leads to the ARCH(p) model :

(8.4)

where α ₀ ≥ 0, α j  ≥ 0 and $$\nu _{t} \sim \mathrm{ IID\,N}(0,1)$$ with ν t independent from Z t−j , j ≥ 1. A further popular generalization was proposed by Bollerslev (1986):

(8.5)

where we assume α ₀ ≥ 0, α j  ≥ 0, β j  ≥ 0 and $$\nu _{t} \sim \mathrm{ IID\,N}(0,1)$$ with ν t independent from Z t−j , j ≥ 1, as before. This model is analogous the ordinary ARMA model and allows for a parsimonious specification of the volatility process. All coefficients should be positive to guarantee that the variance is always positive. In addition it can be shown (see for example Fan and Yao (2003, 150) and the literature cited therein) that {Z t } is (strictly) stationary with finite variance if and only if $$\sum _{j=1}^{p}\alpha _{j} +\sum _{ j=1}^{q}\beta _{j} < 1$$.⁵ Under this condition $$\{Z_{t}\} \sim \mathrm{ WN}(0,\sigma _{Z}^{2})$$ with

As ν t is still normally distributed the uneven moments of the distribution of Z t are zero and the distribution is thus symmetric. The fourth moment of Z t , $$\mathbb{E}Z_{t}^{4}$$, exists if

This condition is sufficient, but not necessary.⁶ Furthermore, {Z t } is a white noise process with heavy-tail property if {Z t } is strictly stationary with finite fourth moment.

In addition, {Z t ²} is a causal and invertible ARMA$$(\max \{p,q\},q)$$ process satisfying the following difference equation:

where $$\alpha _{p+j} =\beta _{q+j} = 0$$ for j ≥ 1 and “error term”

Note, however, there is a circularity here because the noise process {e t } is defined in terms of Z t ² and is therefore not an exogenous process driving Z t ². Thus, one has to be precautious in the interpretation of {Z t ²} as an ARMA process.

Further generalizations of the GARCH(p,q) model can be obtained by allowing deviations from the normal distribution for ν t . In particular, distributions such as the t-distribution which put more weight on extreme values have become popular. This seems warranted as prices on financial markets exhibit large and sudden fluctuations.⁷

The Threshold GARCH Model

Assuming a symmetric distribution for ν t and specifying a linear relationship between $$\sigma _{t}^{2}$$ and Z t−j ² bzw. $$\sigma _{t-j}^{2}$$, j > 0, leads to a symmetric distribution for Z t . It has, however, been observed that downward movements seem to be different from upward movements. This asymmetric behavior is accounted for by the asymmetric GARCH(1,1) model or threshold GARCH(1,1) model (TGARCH(1,1) model). This model was proposed by Glosten et al. (1993) and Zakoïan (1994):

$$\mathbf{1}_{\{Z_{t-1}<0\}}$$ denotes the indicator function which takes on the value one if Z t−1 is negative and the value zero otherwise. Assuming, as before, that all parameters α ₀, α ₁, β and γ are greater than zero, this specification postulates a leverage effect because negative realizations have a greater impact than positive ones. In order to obtain a stationary process the condition $$\alpha _{1} +\beta +\gamma /2 < 1$$ must hold. This model can be generalized in an obvious way by allowing additional lags Z t−j ² and $$\sigma _{t-j}^{2}$$, j > 1 to enter the above specification.

The Exponential GARCH Model

Another interesting and popular class of volatility models was introduced by Nelson (1991). The so-called exponential GARCH models or EGARCH models are defined as follows:

Note that, in contrast to the previous specifications, the dependent variable is the logarithm of $$\sigma _{t}^{2}$$ and not $$\sigma _{t}^{2}$$ itself. This has the advantage that the variance is always positive irrespective of the values of the coefficients. Furthermore, the leverage effect is exponential rather than quadratic because $$Z_{t} =\nu _{t}\exp (\sigma _{t}/2)$$. The EGARCH model is also less recursive than the GARCH model as the volatility is specified directly in terms of the noise process {ν t }. Thus, the above EGARCH model can be treated as an AR(1) model of $$\log \sigma _{t}^{2}$$ with noise process $$\gamma \vert \nu _{t-1}\vert +\delta \nu _{t-1}$$. It is obvious that the model can be generalized to allow for additional lags both in $$\sigma _{t}^{2}$$ and ν t . This results in an ARMA process for $$\{\log \sigma _{t}^{2}\}$$ for which the usual conditions for the existence of a causal and invertible solution can be applied (see Sect. 2.​3). A detailed analysis and further properties of this model class can be found in Bollerslev et al. (1994), Gouriéroux (1997) and Fan and Yao (2003, 143–180).

The ARCH-in-Mean Model

The ARCH-in-mean model or ARCH-M model was introduced by Engle et al. (1987) to allow for a feedback of volatility into the mean equation. More specifically, assume for the sake of simplicity that the variance equation is just represented the ARCH(1) model

Then, the ARCH-M model is given

(8.6)

where g is a function of the volatility $$\sigma _{t}^{2}$$ and where M t ′ consists of a vector of regressors, including lagged values of X t . If $$M_{t} = (1,X_{t-1})$$ then we get the AR(1)-ARCH-M model. The most commonly used specification for g is a linear function: $$g(\sigma _{t}^{2}) =\delta _{0} +\delta _{1}\sigma _{t}^{2}$$. In the asset pricing literature, higher volatility would require a higher return to compensate the investor for the additional risk. Thus, if X t denotes the return on some asset, we expect δ ₁ to be positive. Note that any time variation in $$\sigma _{t}^{2}$$ translates into a serial correlation of {X t } (see Hong 1991, for details). Of course, one could easily generalize the model to allow for more sophisticated mean and variance equations.

8.1.4 The GARCH(1,1) Model

The Generalized Autoregressive Conditional Heteroskedasticity model of order (1,1), GARCH(1,1) model for short, is considered as a benchmark for more general specifications and often serves as a starting point for further empirical investigations. We therefore want to explore its properties in more detail. Many of its properties generalize in a straightforward way to the GARCH(p,q) process. According to Eq. (8.5) the GARCH(1,1) model is defined as:

(8.7)

where α ₀, α ₁, and β ≥ 0. We assume α ₁ +β > 0 to avoid the degenerate case $$\alpha _{1} =\beta = 0$$ which implies that {Z t } is just a sequence of IID random variables. Moreover, $$\nu _{t} \sim \mathrm{ IID}(0,1)$$ with ν t being independent of Z t−j , j ≥ 1. Note that we do not make further distributional assumption. In particular, ν t need not required to be normally distributed. For this model, we can formulate a similar theorem as for the ARCH(1) model (see Theorem: 8.1):

Theorem 8.3.

Let {Z t } be a GARCH(1,1) process as defined above. Under the assumption

the difference equation  (8.7) possess strictly stationary solution:

where $$\prod _{i}^{j} = 1$$ whenever i > j. The solution is also unique given the sequence {ν t }. The solution is unique and (weakly) stationary with variance $$\mathbb{E}Z_{t}^{2} = \tfrac{\alpha _{0}} {1-\alpha _{1}-\beta } < \infty $$ if α ₁ + β < 1.

Proof.

The proof proceeds similarly to Theorem 8.1. For this purpose, we define $$Y _{t} =\sigma _{ t}^{2}$$ and rewrite the GARCH(1,1) model as

This defines an AR(1) process with time-varying coefficients $$\xi _{t} =\alpha _{1}\nu _{t}^{2}+\beta \geq 0$$. Iterate this equation backwards k times to obtain:

Taking the limit $$k \rightarrow \infty $$, we define the process {Y t ′}

(8.8)

The right-hand side of this expression converges almost surely as can be seen from the following argument. Given that $$\nu _{t} \sim \mathrm{ IID}$$ and given the assumption $$\mathbb{E}\log (\alpha _{1}\nu _{t}^{2}+\beta ) < 0$$, the strong law of large numbers (Theorem C.5) implies that

or equivalently,

Thus,

The application of the root test then shows that the infinite series (8.8) converges almost surely. Thus, {Y t ′} is well-defined. It is easy to see that {Y t ′} is strictly stationary and satisfies the difference equation. Moreover, if α ₁ +β < 1, we get

Thus, $$\mathbb{E}Z_{t}^{2} = \tfrac{\alpha _{0}} {1-\alpha _{1}-\beta } < \infty $$.

To show uniqueness, we assume that there exists another strictly stationary process {Y t } which also satisfies the difference equation. This implies that

The assumption $$\mathbb{E}\log \xi _{t} = \mathbb{E}\log (\alpha _{1}\nu _{t}^{2}+\beta ) < 0$$ together with the strong law of large numbers (Theorem C.5) imply

As both solutions are strictly stationary so that the distribution of | Y t−k  | and | Y t−k ′ | do not depend on t, this implies that both $$\left (\prod _{i=1}^{k}\xi _{t-i}\right )\,\vert Y _{t-k}\vert $$ and $$\left (\prod _{i=1}^{k}\xi _{t-i}\right )\,\vert Y _{t-k}'\vert $$ converge in probability to zero. Thus, Y t  = Y t ′ a.s. once the sequence $$\xi _{t}$$, respectively ν t , is given. Because $$Z_{t} =\nu _{t}\sqrt{Y _{t}'}$$ this completes the proof. □ 

Remark 8.2.

Using Jensen’s inequality, we see that

Thus, the condition α ₁ +β < 1 is sufficient, but not necessary, to ensure the existence of a strictly stationary solution. Thus even when $$\alpha _{1}+\beta = 1$$, a strictly stationary solution exists, albeit one with infinite variance. This case is known as the IGARCH model and is discussed below. In the case α ₁ +β < 1, the Borel-Cantelli lemma can be used as Theorem 8.1 to establish the uniqueness of the solution. Further details can be found in the references listed in footnote 7.

Assume that α ₁ +β < 1, then a unique strictly stationary process {Z t } with finite variance which satisfies the above difference equation exists. In particular $$Z_{t} \sim \mathrm{ WN}(0,\sigma _{Z}^{2})$$ such that

The assumption $$1 -\alpha _{1}-\beta > 0$$ guarantees that the variance exists. The third moment of Z t is zero due to the assumption of a symmetric distribution for ν t . The condition for the existence of the fourth moment is: $$\sqrt{ 3} \frac{\alpha _{1}} {1-\beta } < 1$$.⁸ The kurtosis is then

if $$\mathbb{E}Z_{t}^{4}$$ exists.⁹ Therefore the GARCH(1,1) model also possesses the heavy-tail property because Z t is more peaked than the normal distribution.

Figure 8.2 shows how the different assumptions and conditions divide up the parameter space. In region I, all conditions are fulfilled. The process has a strictly stationary solution with finite variance and kurtosis. In region II, the kurtosis does no longer exist, but the variance does as α ₁ +β < 1 still holds. In region III, the process has infinite variance, but a strictly stationary solution yet exists. In region IV, no such solution exists.

Viewing the equation for $$\sigma _{t}^{2}$$ as a stochastic difference equation, its solution is given by

(8.9)

This expression is well-defined because 0 < β < 1 so that the infinite sum converges. The conditional variance given the infinite past is therefore equal to

Thus, the conditional variance depends on the entire history of the time series and not just on Z t−1 as in the case of the ARCH(1) model. As all coefficients are assumed to be positive, the clustering of volatility is more persistent than for the ARCH(1) model.

Defining a new time series {e t } by $$e_{t} = Z_{t}^{2} -\sigma _{t}^{2} = (\nu _{t}^{2} - 1)(\alpha _{0} +\alpha _{1}Z_{t-1}^{2} +\beta \sigma _{ t-1}^{2})$$, one can verify that Z t ² obeys the stochastic difference equation

(8.10)

This difference equation defines an ARMA(1,1) process if e t has finite variance which is the case if the fourth moment of Z t exists. In this case, it is easy to verify that {e t } is white noise. The so-defined ARMA(1,1) process is causal and invertible with respect to {e t } because 0 < α ₁ +β < 1 and 0 < β < 1. The autocorrelation function (ACF), $$\rho _{Z^{2}}(h)$$, can be computed using the methods laid out Sect. 2.​4 This gives

(8.11)

with $$\varphi =\alpha _{1}+\beta$$ (see also Bollerslev (1988)).

The IGARCH Model

Practice has shown that the sum α ₁ +β is often close to one. Thus, it seems interesting to examine the limiting case where $$\alpha _{1}+\beta = 1$$. This model was proposed by Engle and Bollerslev (1986) and was termed the integrated GARCH (IGARCH) model in analogy to the notion of integrated processes (see Chap. 7). From Eq. (8.10) we get

with $$e_{t} = Z_{t}^{2} -\sigma _{t}^{2} = (\nu _{t}^{2} - 1)(\alpha _{0} + (1-\beta )Z_{t-1}^{2} +\beta \sigma _{ t-1}^{2})$$. As {e t } is white noise, the squared innovations Z t ² behave like a random walk with a MA(1) error term. Although the variance of Z t becomes infinite, the difference equation still allows for a strictly stationary solution provided that $$\mathbb{E}\log (\alpha _{1}\nu _{t}^{2}+\beta ) < 0$$ (see Theorem 8.3 and the citations in footnote 7 for further details).¹⁰ It has been shown by Lumsdaine (1986) and Lee and Hansen (1994) that standard inferences can still be applied although $$\alpha _{1}+\beta = 1$$. The model may easily generalized to higher lag orders.

Forecasting

On many occasions it is necessary to obtain forecasts of the conditional variance $$\sigma _{t}^{2}$$. An example is given in Sect. 8.4 where the value at risk (VaR) of a portfolio several periods ahead must be evaluated. Denote by $$\mathbb{P}_{t}\sigma _{t+h}^{2}$$ the h period ahead forecast based on information available in period t. We assume that predictions are based on the infinite past. Then the one-period ahead forecast based on Z t and $$\sigma _{t}^{2}$$, respectively ν t and $$\sigma _{t}^{2}$$, is:

(8.12)

As $$\nu _{t} \sim \mathrm{ IID}(0,1)$$ and independent of Z t−j , j ≥ 1,

Thus, forecast for h ≥ 2 can be obtained recursively as follows:

(8.13)

Assuming α ₁ +β < 1, the second term in the above expression vanishes as h goes to infinity. Thus, the contribution of the current conditional variance vanishes when we look further and further into the future. The forecast of the conditional variance then approaches the unconditional one: $$\lim _{h\rightarrow \infty }\mathbb{P}_{t}\sigma _{t+h}^{2} = \frac{\alpha _{0}} {1-\alpha _{1}-\beta }$$. If $$\alpha _{1}+\beta = 1$$ as in the IGARCH model, the contribution of the current conditional variance is constant, but diminishes to zero relative to the first term. Finally, if α ₁ +β > 1, the two terms are of the same order and we have a particularly persistent situation.

In practice, the parameters of the model are unknown and have therefore be replaced by an estimate. The method can be easily adapted for higher order models. Instead of using the recursive approach outlined above, it is possible to use simulation methods by drawing repeatedly from the actual empirical distribution of the $$\hat{\nu }_{t} = \hat{Z} _{t}/\hat{\sigma }_{t}$$. This has the advantage to capture deviations from the underlying distributional assumptions (see Sect. 8.4 for a comparison of both methods). Such methods must be applied if nonlinear models for the conditional variance, like the TARCH model, are employed.

8.2 Tests for Heteroskedasticity

Before modeling the volatility of a time series it is advisable to test whether heteroskedasticity is actually present in the data. For this purpose the literature proposed several tests of which we are going to examine two. For both tests the null hypothesis is that there is no heretoskedasticity i.e. that there are no ARCH effects. These tests can also be useful in a conventional regression setting.

8.2.1 Autocorrelation of Quadratic Residuals

The first test is based on the autocorrelation function of squared residuals from a preliminary regression. This preliminary regression or mean regression produces a series $$\widehat{Z}_{t}$$ which should be approximately white noise if the equation is well specified. Then we can look at the ACF of the squared residuals $$\{\widehat{Z}_{t}^{2}\}$$ and apply the Ljung-Box test (see Eq. (4.​4)). Thus the test can be broken down into three steps.

(i)

Estimate an ARMA model for {X t } and retrieve the residuals $$\widehat{Z}_{t}$$ from this model. Compute $$\widehat{Z}_{t}^{2}$$. These data can be used to estimate $$\sigma ^{2}$$ as

Note that the ARMA model should be specified such that the residuals are approximately white noise.

[(ii)

] Estimate the ACF for the squared residuals in the usual way:

(iii)

It is now possible to use one of the methods laid out in Chap. 4 to test the null hypothesis that {Z t ²} is white noise. It can be shown that under the null hypothesis $$\sqrt{T}\hat{\rho }_{Z^{2}}(h)\mathop{\longrightarrow}\limits_{}^{\quad \mathrm{d}\quad }\mathrm{N}(0,1)$$. One can therefore construct confidence intervals for the ACF in the usual way. Alternatively, one may use the Ljung-Box test statistic (see Eq. (4.​4)) to test the hypothesis that all correlation coefficients up to order N are simultaneously equal to zero.

Under the null hypothesis this statistic is distributed as χ N ². To carry out the test, N should be chosen rather high, for example equal to T∕4.

8.2.2 Engle’s Lagrange-Multiplier Test

Engle (1982) proposed a Lagrange-Multiplier test. This test rests on an ancillary regression of the squared residuals against a constant and lagged values of $$\widehat{Z}_{t-1}^{2},\widehat{Z}_{t-2}^{2},\ldots,\widehat{Z}_{t-p}^{2}$$ where the $$\{\widehat{Z}_{t}\}$$ is again obtained from a preliminary regression. The auxiliary regression thus is

where $$\varepsilon _{t}$$ denotes the error term. Then the null hypothesis $$\mathrm{H}_{0}:\alpha _{1} =\alpha _{2} =\ldots =\alpha _{p} = 0$$ is tested against the alternative hypothesis H₁: α j ≠ 0 for at least one j. As a test statistic one can use the coefficient of determination times T, i.e. TR². This test statistic is distributed as a χ ² with p degrees of freedom. Alternatively, one may use the conventional F-test.

8.3 Estimation of GARCH(p,q) Models

8.3.1 Maximum-Likelihood Estimation

The literature has proposed several approaches to estimate models of volatility (see Fan and Yao (2003, 156–162)). The most popular one, however, rest on the method of maximum-likelihood. We will describe this method using the GARCH(p,q) model . Related and more detailed accounts can be found in Weiss (1986), Bollerslev et al. (1994) and Hall and Yao (2003).

In particular we consider the following model:

where

The mean equation represents a simple AR(r) process for which we assume that it is causal with respect to {Z t }, i.e. that all roots of $$\Phi (z)$$ are outside the unit circle. The method demonstrated here can be easily generalized to ARMA processes or even ARMA process with additional exogenous variables (so-called ARMAX processes) as noted by Weiss (1986). The method also incorporates the ARCH-in-mean model (see equation (8.6)) which allows for an effect of the conditional variance $$\sigma _{t}$$ on X t . In addition, we assume that the coefficients of the variance equation are all positive, that $$\sum _{j=1}^{p}\alpha _{j} +\sum _{ j=1}^{q}\beta _{j} < 1$$ and that $$\mathbb{E}Z_{t}^{4} < \infty $$ exists.¹¹

As ν t is identically and independently standard normally distributed, the distribution of X t conditional on $$\mathcal{X}_{t-1} =\{ X_{t-1},X_{t-2},\ldots \}$$ is normal with mean $$c +\phi _{1}X_{t-1} +\ldots +\phi _{r}X_{t-r}$$ and variance $$\sigma _{t}^{2}$$. The conditional density, $$f(X_{t}\vert \mathcal{X}_{t-1})$$, therefore is:

where Z t equals $$X_{t} - c -\phi _{1}X_{t-1} -\ldots -\phi _{r}X_{t-r}$$ and $$\sigma _{t}^{2}$$ is given by the variance equation.¹² The joint density f(X ₁, X ₂, …, X T ) of a random sample (X ₁, X ₂, …, X T ) can therefore be factorized as

where s is an integer greater than p. The necessity, not to factorize the first s − 1 observations, relates to the fact that $$\sigma _{t}^{2}$$ can only be evaluated for s > p in the ARCH(p) model. For the ARCH(p) model s can be set to p + 1. In the case of a GARCH model $$\sigma _{t}^{2}$$ is given by weighted infinite sum of the $$Z_{t-1}^{2},Z_{t-2}^{2},\ldots$$ (see the expression (8.9) for $$\sigma _{t}^{2}$$ in the GARCH(1,1) model). For finite samples, this infinite sum must be approximated by a finite sum of s summands such that the numbers of summands s is increasing with the sample size.(see Hall and Yao (2003)).

We then merge all parameters of the model as follows: ϕ = (c, ϕ ₁, …, ϕ r )′, α = (α ₀, α ₁, …, α p )′ and β = (β ₁, …, β q )′. For a given realization x = (x ₁, x ₂, …, x T ) the likelihood function conditional on x, L(ϕ, α, β | x), is defined as

where in $$\mathcal{X}_{t-1}$$ the random variables are replaced by their realizations. The likelihood function can be seen as the probability of observing the data at hand given the values for the parameters. The method of maximum likelihood then consist in choosing the parameters (ϕ, α, β) such that the likelihood function is maximized. Thus we chose the parameter so that the probability of observing the data is maximized. In this way we obtain the maximum likelihood estimator. Taking the first s realizations as given deterministic starting values, we then get the conditional likelihood function.

In practice we do not maximize the likelihood function but the logarithm of it where we take f(x ₁, …, x s−1) as a fixed constant which can be neglected in the optimization:

where $$z_{t} = x_{t} - c -\phi _{1}x_{t-1} -\ldots -\phi _{r}x_{t-r}$$ denotes the realization of Z t . The maximum likelihood estimator is obtained by maximizing the likelihood function over the admissible parameter space. Usually, the implementation of the stationarity condition and the condition for the existence of the fourth moment turns out to be difficult and cumbersome so that often these conditions are neglected and only checked in retrospect or some ad hoc solutions are envisaged. It can be shown that the (conditional) maximum likelihood estimator leads to asymptotically normally distributed estimates.¹³ The maximum likelihood estimator remains meaningful even when {ν t } is not normally distributed. In this case the quasi maximum likelihood estimator is obtained (see Hall and Yao (2003) and Fan and Yao (2003)).

For numerical reasons it is often convenient to treat the mean equation and the variance equation separately. As the mean equation is a simple AR(r) model, it can be estimated by ordinary least-squares (OLS) in a first step. This leads to consistent parameter estimates. However, due to the heteroskedasticity, this no longer true for the covariance matrix of the coefficients so that the usual t- and F-tests are nor reliable. This problem can be circumvented by the use of the White correction (see White (1980)). In this way it is possible to find an appropriate specification for the mean equation without having to estimate the complete model. In the second step, one can then work with the residuals to find an appropriate ARMA model for the squared residuals. This leads to consistent estimates of the parameters of the variance equation. These estimates are under additional weakly assumptions asymptotically normally distributed (see Weiss (1986)). It should, however, be noted that this way of proceeding is, in contrast to the maximum likelihood estimator, not efficient because it neglect the nonlinear character of the GARCH model. The parameters found in this way can, however, serve as meaningful starting values for the numerical maximization procedure which underlies the maximum likelihood estimation.

A final remark concerns the choice of the parameter r, p and q. Similarly to the ordinary ARMA models, one can use information criteria such as the Akaike or the Bayes criterion, to determine the order of the model (see Sect. 5.​4).

8.3.2 Method of Moment Estimation

The maximization of the likelihood function requires the use of numerical optimization routines. Depending on the routine actually used and on the starting value, different results may be obtained if the likelihood function is not well-behaved. It is therefore of interest to have alternative estimation methods at hand. The method of moments is such an alternative. It is similar to the Yule-Walker estimator (see Sect. 5.​1) applied to the autocorrelation function of {Z t ²}. This method not only leads to an analytic solution, but can also be easily implemented. Following Kristensen and Linton (2006), we will illustrate the method for the GARCH(1,1) model.

Equation (8.11) applied to $$\rho _{Z^{2}}(1)$$ and $$\rho _{Z^{2}}(2)$$ constitutes a nonlinear equation system in the unknown parameters β and α ₁. This system can be reparameterized to yield an equation system in $$\varphi =\alpha _{1}+\beta$$ and β which can be reduced to a single quadratic equation in β:

The parameter b is well-defined because $$\varphi =\alpha _{1}+\beta \geq \rho _{Z^{2}}(1)$$ with equality only if β = 0. In the following we will assume that β > 0. Under this assumption b > 2 so that the only solution with the property 0 < β < 1 is given by

The moment estimator can therefore be constructed as follows:

(i)

Estimate the correlations $$\rho _{Z^{2}}(1)$$ and $$\rho _{Z^{2}}(2)$$ and $$\sigma ^{2}$$ based on the formulas (8.11) in Sect. 8.2.

(ii)

An estimate for $$\varphi =\alpha _{1}+\beta$$ is then given by

(iii)

use the estimate $$\hat{\varphi }$$ to compute an estimate for b:

The estimate $$\hat{\beta }$$ for β is then

(iv)

The estimate for α ₁ is $$\hat{\alpha }_{1} =\hat{\varphi } -\hat{\beta }$$. Because $$\alpha _{0} =\sigma ^{2}(1 - (\alpha _{1}+\beta ))$$, the estimate for α ₀ is equal to $$\hat{\alpha }_{0} =\hat{\sigma } ^{2}(1-\hat{\varphi })$$.

Kristensen and Linton (2006) show that, given the existence of the fourth moment of Z t , this method of moment leads to consistent and asymptotically normal distributed estimates. These estimates may then serve as starting values for the maximization of the likelihood function to improve efficiency.

8.4 Example: Swiss Market Index (SMI)

In this section, we will illustrate the methods discussed previously to analyze the volatility of the Swiss Market Index (SMI). The SMI is the most important stock market index for Swiss blue chip companies. It is constructed solely from stock market prices, dividends are not accounted for. The data are the daily values of the index between the 3rd of January 1989 and the 13th of February 2004. Figure 1.​5 shows a plot of the data. Instead of analyzing the level of the SMI, we will investigate the daily return computed as the logged difference. This time series is denoted by X t and plotted in Fig. 8.3. One can clearly discern phases of high (observations around t = 2500 and t = 3500) and low (t = 1000 and t = 2000) volatility. This represents a first sign of heteroskedasticity and positively correlated volatility.

Figure 8.4 shows a normal-quantile plot to compare the empirical distribution of the returns with those from a normal distribution. This plot clearly demonstrates that the probability of observing large returns is bigger than warranted from a normal distribution. Thus the distribution of returns exhibits the heavy-tail property. A similar argument can be made by comparing the histogram of the returns and the density of a normal distribution with the same mean and the same variance. shown in Fig. 8.5. Again one can seen that absolutely large returns are more probable than expected from a normal distribution. Moreover, the histogram shows no obvious sign for an asymmetric distribution, but a higher peakedness.

Fig. 8.4

Normal-Quantile Plot of the daily returns of the SMI (Swiss Market Index)

Fig. 8.5

Histogram of the daily returns of the SMI (Swiss Market Index) and the density of a fitted normal distribution (red line)

After the examination of some preliminary graphical devices, we are going to analyze the autocorrelation functions of {X t } and {X t ²}. Figure 8.6 shows the estimated ACFs. The estimated ACF of {X t } shows practically no significant autocorrelation so that we can consider {X t } be approximately white noise. The corresponding Ljung-Box statistic with L = 100, however, has a value of 129.62 which is just above the 5 % critical value of 124.34. Thus there is some sign of weak autocorrelation. This feature is not in line with efficiency of the Swiss stock market (see Campbell et al. (1997)). The estimated ACF of X t ² is clearly outside the 95 % confidence interval for at least up to order 20. Thus we can reject the hypothesis of homoskedasticity in favor of heteroskedasticity. This is confirmed by the Ljung-Box statistic with L = 100 with a value of 2000.93 which is much higher than the critical value of 124.34.

Fig. 8.6

ACF of the daily returns and the squared daily returns of the SMI

After these first investigations, we want to find an appropriate model for the mean equation. We will use OLS with the White-correction. It turns out that a MA(1) model fits the data best although an AR(1) model leads to almost the same results. In the next step we will estimate a GARCH(p,q) model with the method of maximum likelihood where p is varied between p 1 and 3 and q between 0 and 3. The values of the AIC, respectively BIC criterion corresponding to the variance equation are listed in Tables 8.1 and 8.2.

Table 8.1

AIC criterion for the variance equation in the GARCH(p,q) model

q

p

0

1

2

3

1

3.0886

2.9491

2.9491

2.9482

2

3.0349

2.9496

2.9491

2.9486

3

2.9842

2.9477

2.9472

2.9460

Minimum value in bold

Table 8.2

BIC criterion for the variance equation in the GARCH(p,q) model

q

p

0

1

2

3

1

3.0952

2.9573

2.9590

2.9597

2

3.0431

2.9595

2.9606

2.9617

3

2.9941

2.9592

2.9604

2.9607

Minimum value in bold

The results reported in these tables show that the AIC criterion favors a GARCH(3,3) model corresponding to the bold number in Table 8.1 whereas the BIC criterion opts for a GARCH(1,1) model corresponding to the bold number in Table 8.2. It also turns out that high dimensional models, in particular those for which q > 0, the maximization algorithm has problems to find an optimum. Furthermore, the roots of the implicit AR and the MA polynomial corresponding to the variance equation of the GARCH(3,3) model are very similar. These two arguments lead us to prefer the GARCH(1,1) over the GARCH(3,3) model. This model was estimated to have the following mean equation:

with the corresponding variance equation

where the estimated standard deviations are reported below the corresponding coefficient estimate. The small, but significant value of 0.0484 for the MA(1) coefficient shows that there is a small but systematic correlation of the returns from one day to the next. The coefficients of the GARCH model are all positive and their sum $$\alpha _{1}+\beta = 0.1388 + 0.8081 = 0.9469$$ is statistically below one so that all conditions for a stationary process are fulfilled.¹⁴ Because $$\sqrt{3} \frac{\alpha _{1}} {1-\beta } = \sqrt{3} \frac{0.1388} {1-0.8081} = 1.2528 > 1$$, the condition for the existence of the fourth moment of Z t is violated.

As a comparison we also estimate the GARCH(1,1) model using the methods of moments. First we estimate a MA(1) model for $$\Delta \log SMI$$. This results in an estimate $$\hat{\theta }= 0.034$$ (compare this with the ML estimate). The squared residuals have correlation coefficients

The estimate of b therefore is $$\hat{b} = 2.241$$ which leads to an estimate of β equal to $$\hat{\beta }= 0.615$$. This finally results in the estimates of α ₁ and α ₀ equal to $$\hat{\alpha }_{1} = 0.179$$ and $$\hat{\alpha }_{0} = 0.287$$ with an estimate for $$\sigma ^{2}$$ equal to $$\hat{\sigma }^{2} = 1.391$$. Thus these estimates are quite different from those obtained by the ML method.

Value at Risk

We are now in a position to use our ML estimates to compute the Value-at-risk (VaR). The VaR is a very popular measure to estimate the risk of an investment. In our case we consider the market portfolio represented by the stocks in the SMI. The VaR is defined as the maximal loss (in absolute value) of an investment which occurs with probability α over a time horizon h. Thus a 1 % VaR for the return on the SMI for the next day is the threshold value of the return such that one can be confident with 99 % that the loss will not exceed this value. Thus the α VaR at time t for h periods, VaR t, t+h α , is nothing but the α-quantile of the distribution of the forecast of the return in h periods given information X t−k , k = 0, 1, 2, …. Formally, we have:

where $$\widetilde{X}_{t+h}$$ is the return of the portfolio over an investment horizon of h periods. This return is approximately equal to the sum of the daily returns: $$\widetilde{X}_{t+h} =\sum _{ j=1}^{h}X_{t+j}$$.

The one period forecast error is given by $$X_{t+1} -\widetilde{ \mathbb{P}}_{t}X_{t+1}$$ which is equal to $$Z_{t+1} =\sigma _{t+1}\nu _{t+1}$$. Thus the VaR for the next day is

This entity can be computed by replacing the forecast given the infinite past, $$\widetilde{\mathbb{P}}_{t}X_{t+1}$$, by a forecast given the finite sample information X t−k , $$k = 0,1,2,\ldots,t - 1$$, $$\mathbb{P}_{t}X_{t+1}$$, and by substituting $$\sigma _{t+1}$$ by the corresponding forecast from variance equation, $$\hat{\sigma }_{t,t+1}$$. Thus we get:

The computation of $$\widehat{\mathrm{VaR}_{t,t+1}^{\alpha }}$$ requires to determine the α-quantile of the distribution of ν t . This can be done in two ways. The first one uses the assumption about the distribution of ν t explicitly. In the simplest case, ν t is distributed as a standard normal so that the appropriate quantile can be easily retrieved. The 1 % quantile for the standard normal distribution is −2.33. The second approach is a non-parametric one and uses the empirical distribution function of $$\hat{\nu }_{t} =\hat{ Z}_{t}/\hat{\sigma }_{t}$$ to determine the required quantile. This approach has the advantage that deviations from the standard normal distribution are accounted for. In our case, the 1 % quantile is − 2. 56 and thus considerably lower than the − 2. 33 obtained from the normal distribution. Thus the VaR is under estimated by using the assumption of the normal distribution.

The corresponding computations for the SMI based on the estimated ARMA(0,1)-GARCH(1,1) model are reported in Table 8.3. A value of 5.71 for 31st of December 2001 means that one can be 99 % sure that the return of an investment in the stocks of the SMI will not be lower than −5.71 %. The values for the non-parametric approach are typically higher. The comparison of the VaR for different dates clearly shows how the risk evolves over time.

Table 8.3

One percent VaR for the next day of the return to the SMI according to the ARMA(0,1)-GARCH(1,1) model

VaR ($$\widehat{\mathrm{VaR}_{t,t+1}^{0,01}}$$)

Date

Parametric

Non-parametric

31.12.2001

0.28

6.61

5.71

6.30

5.2.2002

−0.109

6.80

6.19

6.79

24.7.2003

0.0754

0.625

1.77

1.95

Due to the nonlinear character of the model, the VaR for more than one day can only be gathered from simulating the one period returns over the corresponding horizon. Starting from a given date 10’000 realizations of the returns over the next 10 days have been simulated whereby the corresponding values for ν t are either drawn from a standard normal distribution (parametric case) or from the empirical distribution function of $$\hat{\nu }_{t}$$ (non-parametric case). The results from this exercise are reported in Table 8.4. Obviously, the risk is much higher for a 10 day than for a one day investment. Alternatively, one may use the forecasting equation (8.12) and the corresponding recursion formula (8.13).

Table 8.4

One percent VaR for the next 10 days of the return to the SMI according to the ARMA(0,1) -GARCH(1,1) model

VaR ($$\widehat{\mathrm{VaR}_{t,t+10}^{0,01}}$$)

Date

Parametric

Non-parametric

31.12.2001

0.84

18.39

22.28

5.2.2002

0.65

19.41

21.53

24.7.2003

0.78

6.53

7.70

References

Bollerslev T (1986) Generalized autoregressive conditional heteroskedasticity. J Econ 31:307–327CrossRef

Bollerslev T (1988) On the correlation structure for the generalized autoregressive conditional heteroskedastic process. J Time Ser Anal 9:121–131CrossRef

Bollerslev T, Engle RF, Nelson DB (1994) ARCH models. In: Engle RF, McFadden DL (eds) Handbook of econometrics, vol IV. Elsevier Science B.V., Amsterdam, pp 2959–3038

Bougerol P, Picard N (1992a) Stationarity of GARCH processes and some nonnegative time series. J Econ 52:115–127

Campbell JY, Lo AW, MacKinlay AC (1997) The econometrics of financial markets. Princeton University Press, Princeton

Engle RF (1982) Autoregressive conditional heteroskedasticity with estimates of the variance of the United Kingdom inflation. Econometrica 50:987–1007CrossRef

Engle RF (2004) Risk and volatility: Econometric models and financial practice. Am Econ Rev 94:405–420CrossRef

Engle RF, Bollerslev T (1986) Modeling the persistence of conditional variances. Econ Rev 5:1–50CrossRef

Engle RF, Lilien D, Robins R (1987) Estimating time varying risk premia in the term structure: The ARCH–M model. Econometrica 55:391–407CrossRef

Fan J, Yao Q (2003) Nonlinear time series. Springer, New York

Giraitis L, Kokoszka P, Leipus R (2000) Stationary ARCH models: Dependence structure and central limit theorem. Econ Theory 16:3–22CrossRef

Glosten LR, Jagannathan R, Runkle DE (1993) On the relation between expected value and the volatility of the nominal excess returns on stocks. J Finance 48:1779–1801CrossRef

Gouriéroux C (1997) ARCH models and financial applications. Springer, New YorkCrossRef

Hall P, Yao Q (2003) Inference in ARCH and GARCH models with heavy-tailed errors. Econometrica 71(1):285–317CrossRef

Hong EP (1991) The autocorrelation structure for the GARCH-M process. Econ Lett 37:129–132CrossRef

Jensen ST, Rahbek A (2004) Asymptotic normality for non-stationary, explosive GARCH. Econ Theory 20(6):1203–1226CrossRef

Klüppelberg C, Lindner A, Maller R (2004) A continuous time GARCH process driven by a Lévy process: Stationarity and second order behaviour. J Appl Probab 41:601–622CrossRef

Kristensen D, Linton O (2006) A closed-form estimator for the GARCH(1,1)-model. Econ Theory 22:323–337CrossRef

Lee SW, Hansen BE (1994) Asymptotic theory for the GARCH(1,1) quasi-maximum likelihood estimator. Econ Theory 10:29–52CrossRef

Lindner AM (2009) Stationarity, Mixing, Distributional Properties and Moments of GARCH(p,q)-Processes. In Anderson TG, Davis RA, Kreiß JP, Mikosch T (eds.) Handbook of Financial Time Series, Springer-Verlag, Berlin Heidelberg

Lumsdaine RL (1986) Consistency and asymptotic normality of the quasi-maximum likelihood estimator in IGARCH(1,1) and covariance stationary GARCH(1,1) models. Econometrica 64:575–596CrossRef

Nelson DB (1990) Stationarity and persistence in the GARCH(1,1) model. Econ Theory 6:318–334CrossRef

Nelson DB (1991) Conditional heteroskedasticity in asset returns: A new approach. Econometrica 59:347–370CrossRef

Weiss AA (1986) Asymptotic theory for ARCH models: Estimation and testing. Econ Theory 2:107–131CrossRef

White H (1980) A heteroskedasticity consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica 48:817–838CrossRef

Zadrozny PA (2005) Necessary and sufficient restrictions for existence of a unique fourth moment of a univariate GARCH(p,q) process, cESifo Working Paper No.1505

Zakoïan JM (1994) Threshold heteroskedastic models. J Econ Dyn Control 18:931–955CrossRef

Footnotes

1

Robert F. Engle III was awarded the Nobel prize in 2003 for his work on time-varying volatility. His Nobel lecture (Engle 2004) is a nice and readable introduction to this literature.

2

Instead of assuming $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$, we make for convenience the stronger assumption that $$Z_{t} \sim \mathrm{ IID}(0,\sigma ^{2})$$.

3

The case α ₁ = 1 is treated in Sect. 8.1.4.

4

As $$\nu _{t} \sim \mathrm{ N}(0, 1)$$ its even moments, $$m_{2k} = \mathbb{E}\nu _{t}^{2k}$$, k = 1, 2, …, are given by $$m_{2k} =\prod _{ j=1}^{k}(2j - 1)$$. Thus we get m ₄ = 3, m ₆ = 15, etc. As the normal distribution is symmetric, all odd moments are equal to zero.

5

A detailed exposition of the GARCH(1,1) model is given in Sect. 8.1.4.

6

Zadrozny (2005) derives a necessary and sufficient condition for the existence of the fourth moment.

7

A thorough treatment of the probabilistic properties of GARCH processes can be found in Nelson (1990), Bougerol and Picard (1992a), Giraitis et al. (2000), Klüppelberg et al. (2004, theorem 2.1), and Lindner (2009).

8

A necessary and sufficient condition is $$(\alpha _{1}+\beta )^{2} + 2\alpha _{1}^{2} < 1$$ (see Zadrozny (2005)).

9

The condition for the existence of the fourth moment implies 3α ₁ ² < (1 −β)² so that the denominator $$1 -\beta ^{2} - 2\alpha _{1}\beta - 3\alpha _{1}^{2} > 1 -\beta ^{2} - 2\alpha _{1}\beta - 1 -\beta ^{2} + 2\beta = 2\beta (1 -\alpha _{1}-\beta ) > 0$$.

10

As the variance becomes infinite, the IGARCH process is an example of a stochastic process which is strictly stationary, but not stationary.

11

The existence of the fourth moment is necessary for the asymptotic normality of the maximum-likelihood estimator, but not for the consistence. It is possible to relax this assumption somewhat (see Hall and Yao (2003)).

12

If ν t is assumed to follow another distribution than the normal, one may use this distribution instead.

13

Jensen and Rahbek (2004) showed that, at least for the GARCH(1,1) case, the stationarity condition is not necessary.

14

The corresponding Wald test clearly rejects the null hypothesis $$\alpha _{1}+\beta = 1$$ at a significance level of 1 %.

Part II
Multivariate Time Series Analysis

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_9

9. Introduction

Klaus Neusser1 

(1)

Bern, Switzerland

The Keynesian macroeconomic theory developed in the 1930s and 1940s, in particular its representation in terms of IS- and LM-diagram, opened a new area in the application of statistical methods to economics. Based on the path breaking work by Tinbergen (1939) and Klein (1950) this research gave rise to simultaneous equation systems which should capture all relevant aspect of an economy. The goal was to establish an empirically grounded tool which would enable the politicians to analyze the consequences of their policies and thereby fine tune the economy to overcome or at least mitigate major business cycle fluctuations. This development was enhanced by the systematic compilation of national accounting data and by the advances in computer sciences.¹ These systems were typically built by putting together single equations such as consumption, investment, money demand, export- and import, Phillips-curve equations to an overall model. The Klein–Goldberger model for the United States was a first successful attempt in this direction (Klein and Goldberger 1955). Shocked by disturbances, Adelman and Adelman (1959) showed that these type of models exhibited cycles with properties similar to those found for the United States Economy.

As the model became more and detailed over time, they could, in the end, well account for several hundreds or even thousands of equations. The climax of this development was the project LINK which linked the different national models to a world model by accounting for their interrelation through trade flows (Klein 1985). Although this research program brought many insights and spurred the development of econometrics as a separate field, by the mid 1970s one had to admit that the idea to use large and very detailed models for forecasting and policy analysis was overly optimistic. In particular, the inability to forecast and cope with the oil crisis of the beginning 1970s raised doubts about the viability of this research strategy. In addition, more and more economist had concerns about the theoretical foundations of these models.

The critique had several facets. First, it was argued that the bottom-up strategy of building a system from single equations is not compatible with general equilibrium theory which stresses the interdependence of economic activities. This insight was even reinforced by the advent of the theory of rational expectations. This theory postulated that expectations should be formed on the basis of all available information and not just by mechanically extrapolating from the past. This implies that developments in every part of the economy, in particular in the realm of economic policy making, should in principle be taken into account and shape the expectation formation. As expectations are omnipresent in almost every economic decision, all aspects of economic activities (consumption capital accumulation, investment, etc.) are inherently linked. Thus the strategy of using zero restrictions—which meant that certain variables were omitted from a particular equation—to identify the parameters in a simultaneous equation system was considered to be flawed. Second, the theory of rational expectations implied that the typical behavioral equations underlying these models are not invariant to changes in policies because economic agents would take into account systematic changes in the economic environment in their decision making. This so-called Lucas-critique (Lucas 1976) undermined the basis for the existence of large simultaneous equation models. Third, simple univariate ARMA models proved to be as good in forecasting as the sophisticated large simultaneous models. Thus it was argued that the effort or at least part of the effort devoted to these models was wasted.

In 1980 Sims (1980b) and proposed an alternative modeling strategy. This strategy concentrates the modeling activity to only a few core variables, but places no restrictions what so ever on the dynamic interrelation among them. Thus every variable is considered to be endogenous and, in principle, dependent on all other variables of the model. In the linear context, the class of vector autoregressive (VAR) models has proven to be most convenient to capture this modeling strategy. They are easy to implement and to analyze. In contrast to the simultaneous equation approach, however, it is no longer possible to perform comparative static exercises and to analyze the effect of one variable on another one because every variable is endogenous a priori. Instead, one tries to identify and quantify the effect of shocks over time. These shocks are usually given some economic content, like demand or supply disturbances. However, these shocks are not directly observed, but are disguised behind the residuals from the VAR. Thus, the VAR approach also faces a fundamental identification problem. Since the seminal contribution by Sims, the literature has proposed several alternative identification schemes which will be discussed in Chap. 15 under the header of structural vector autoregressive (SVAR) models. The effects of these shocks are then further analyzed by computing impulse responses and forecast error variance decompositions.²

The reliance on shocks can be seen as a substitute for the lack of experiments in macroeconomics. The approach can be interpreted as a statistical analogue to the identification of specific episodes where some unforseen event (shock) impinges on and propagates throughout the economy. Singling-out these episodes of quasi “natural experiments” as in Friedman and Schwartz (1963) convinced many economist of the role and effects of monetary policy.

Many concepts which have been introduced in the univariate context carry over in a straightforward manner to the multivariate context. However there are some new aspects. First, we will analyze the interaction among several variables. This can be done in a nonparametric way by examining the cross-correlations between time series or by building an explicit model. We will restrict ourself to the class of VAR models as they are easy to handle and are overwhelmingly used in practice. Second, we will discuss several alternative approaches to identify the structural shocks from VAR models. After analyzing the identification problem in general, we describe short-run, long-run, and sign restrictions as possible remedies. Third, we will discuss the modeling of integrated variables in a more systematic way. In particular, we will extend the concept of cointegration to more than two variables. Finally, we will provide an introduction to the state space models as a general modeling approach. State space models are becoming increasingly popular in economics as they can be more directly linked to theoretical economic models.

References

Adelman I, Adelman FL (1959) The dynamic properties of the Klein-Goldberger model. Econometrica 27:596–625CrossRef

Epstein RJ (1987) A history of econometrics. North-Holland, Amsterdam

Friedman M, Schwartz AJ (1963) A monetary history of the United States, 1867–1960. Priceton University Press, Princeton

Kilian L (2013) Structural vector autoregressions. In: Hashimzade N, Thornton M (eds) Handbook of research methods and applications in empirical macroeconomics. Edward Elgar, Cheltenham, pp 515–554CrossRef

Klein LR (1950) Economic fluctuations in United States 1921–1941. Wiley, New York

Klein LR (1985) New developments in project LINK. Am Econ Rev 75:223–227

Klein LR, Goldberger AS (1955) An econometric model of the United States, 1929–1952. North Holland, Amsterdam

Lucas RE (1976) Econometric policy evaluation: A critique. In: Brunner K, Meltzer AH (eds) The phillips curve and labor markets. Carnegie-Rochester Conference Series on Public Policy, vol 1. North-Holland, Amsterdam, pp 19–46

Sims CA (1980b) Macroeconomics and reality. Econometrica 48:1–45

Tinbergen J (1939) Statistical testing of business cycle theories. League of Nations, Genf

Watson MW (1994) Vector autoregressions and cointegration. Handbook of econometrics, vol 4, chap 47. North-Holland, Amsterdam, pp 2843–2915

Footnotes

1

See Epstein (1987) for an historical overview.

2

Watson (1994) and Kilian (2013) provide a general introduction to this topic.

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_10

10. Definitions and Stationarity

Klaus Neusser1 

(1)

Bern, Switzerland

Similarly to the univariate case, we start our exposition with the concept of stationarity which is also crucial in the multivariate setting. Before doing so let us define the multivariate stochastic process.

Definition 10.1.

A multivariate stochastic Process , {X t }, is a family of random variables indexed by t, $$t \in \mathbb{Z}$$ , which take values in $$\mathbb{R}^{n}$$ , n ≥ 1. n is called the dimension of the process.

Setting n = 1, the above definition includes as a special case univariate stochastic processes. This implies that the statements for multivariate processes carry over analogously to the univariate case. We view X t as a column vector:

Each element {X it } thereby represents a particular variable which may be treated as a univariate process. As in the example of Sect. 15.​4.​5, {X t } represents the multivariate process consisting of the growth rate of GDP Y t , the unemployment rate U t , the inflation rate P t , the wage inflation rate W t , and the growth rate of money M t . Thus, X t  = (Y t , U t , P t , W t , M t )′.

As in the univariate case, we characterize the joint distribution of the elements X it and X jt by the first two moments (if they exist), i.e. by the mean and the variance, respectively covariance:

(10.1)

It is convenient to write these entities compactly as vectors and matrices:

Thus, we apply the expectations operator element-wise to vectors and matrices. The matrix-valued function $$\Gamma (t,s)$$ is called the covariance function of {X t }.

In analogy to the univariate case, we define stationarity as the invariance of the first two moments to time shifts:

Definition 10.2 (Stationarity).

A multivariate stochastic process {X t } is stationary if and only if for all integers r, s and t we have:

(i)

$$\mu =\mu _{t} = \mathbb{E}X_{t}$$ is constant (independent of t);

(ii)

$$\mathbb{E}X_{t}'X_{t} < \infty $$;

(iii)

$$\Gamma (t,s) = \Gamma (t + r,s + r)$$.

In the literature these properties are often called weak stationarity, covariance stationarity, or stationarity of second order. If {X t } is stationary, the covariance function only depends on the number of periods between t and s (i.e. on t − s) and not on t or s themselves. This implies that by setting $$r = -s$$ and $$h = t - s$$ the covariance function simplifies to

For h = 0, $$\Gamma (0)$$ is the unconditional covariance matrix of X t . Using the definition of the covariances in Eq. (10.1) we get:

Note that $$\Gamma (h)$$ is in general not symmetric for h ≠ 0 because γ ij (h) ≠ γ ji (h) for h ≠ 0.

Based on the covariance function of a stationary process, we can define the correlation function R(h) where R(h) = (ρ ij (h)) i, j with

In the case i ≠ j we refer to the cross-correlations between two variables {X it } and {X jt }. The correlation function can be written in matrix notation as

where V represents a diagonal matrix with diagonal elements equal to γ ii (0). Clearly, ρ ii (0) = 1. As for the covariance matrix we have that in general ρ ij (h) ≠ ρ ji (h) for h ≠ 0. It is possible that ρ ij (h) > ρ ij (0). We can summarize the properties of the covariance function by the following theorem.¹

Theorem 10.1.

The covariance function of a stationary process {X t } has the following properties:

(i)

For all $$h \in \mathbb{Z}$$ , $$\Gamma (h) = \Gamma (-h)'$$;

(ii)

for all $$h \in \mathbb{Z}$$ , $$\vert \gamma _{ij}(h)\vert \leq \sqrt{\gamma _{ii } (0) \times \gamma _{jj } (0)}$$;

(iii)

for each i = 1,…,n, γ ii (h) is a univariate autocovariance function;

(iv)

$$\sum _{r,k=1}^{m}a_{r}'\Gamma (r - k)a_{k} \geq 0$$ for all $$m \in \mathbb{N}$$ and all $$a_{1},\ldots,a_{m} \in \mathbb{R}^{n}$$ . This property is called non-negative definiteness (see Property 4 in Theorem  1.1 of Sect. 1.3 in the univariate case).

Proof.

Property (i) follows immediately from the definition. Property (ii) follows from the fact that the correlation coefficient is always smaller or equal to one. γ ii (h) is the autocovariance function of {X it } which delivers property (iii). Property (iv) follows from $$\mathbb{E}\left (\sum _{k=1}^{m}a_{k}'(X_{t-k}-\mu )\right )^{2} \geq 0$$. □ 

If not only the first two moments are invariant to time shifts, but the distribution as a whole we arrive at the concept of strict stationarity.

Definition 10.3 (Strict Stationarity).

A process multivariate {X t } is called strictly stationary if and only if, for all $$n \in \mathbb{N}$$ , t ₁ ,…,t n , $$h \in \mathbb{Z}$$ , the joint distributions of $$(X_{t_{1}},\ldots,X_{t_{n}})$$ and $$(X_{t_{1}+h},\ldots,X_{t_{n}+h})$$ are the same.

An Example

Consider the following example for n = 2:

where $$Z_{t} \sim \mathrm{ WN}(0,1)$$. We then have $$\mu =\mathrm{ E}X_{t} = 0$$. The covariance function is given by

The covariance function is zero for h > 2. The values for h < 0 are obtained from property (i) in Theorem 10.1. The correlation function is:

The correlation function is zero for h > 2. The values for h < 0 are obtained from property (i) in Theorem 10.1.

One idea in time series analysis is to construct more complicated process from simple ones, for example by taking moving-averages. The simplest process is the white noise process which is uncorrelated with its own past. In the multivariate context the white noise process is defined as follows.

Definition 10.4.

A stochastic process {Z t } is called (multivariate) white noise process with mean zero and covariance matrix $$\Sigma > 0$$ , denoted by $$Z_{t} \sim \mathrm{ WN}(0,\Sigma )$$ , if it is stationary {Z t } and

If {Z t } is not only white noise, but independently and identically distributed we write $$Z_{t} \sim \mathrm{ IID}(0,\Sigma )$$.

Remark 10.1.

Even if each component of {Z it } is univariate white noise, this does not imply that {Z t } = { (Z 1t , …, Z nt )′} is multivariate white noise. Take, for example the process $$Z_{t} = (u_{t},u_{t-1})'$$ where $$u_{t} \sim \mathrm{ WN}(0,\sigma _{u}^{2})$$. Then $$\Gamma (1) = \left (\begin{array}{*{10}c} 0 &0\\ \sigma _{ u}^{2} & 0\\ \end{array} \right )\neq 0$$.

Taking moving averages of a white noise process it is possible to generate new stationary processes. This leads to the definition of a linear process.

Definition 10.5.

A stochastic process {X t } is called linear if there exists a representation

where $$Z_{t} \sim \mathrm{ IID}(0,\Sigma )$$ and where the sequence $$\{\Psi _{j}\}$$ of the n × n matrices is absolutely summable, i.e. $$\sum _{j=-\infty }^{\infty }\|\Psi _{j}\| < \infty $$ . If for all j < 0 $$\Psi _{j} = 0$$ , the linear process is also called an $$\mathrm{MA}(\infty )$$ process.

Theorem 10.2.

A linear process is stationary with a mean of zero and with covariance function

Proof.

The required result is obtained by applying the properties of {Z t } to $$\Gamma (h) = \mathbb{E}X_{t+h}X_{t}' =\lim _{m\rightarrow \infty }\mathbb{E}\left (\sum _{j=-m}^{m}\Psi _{j}Z_{t+h-j}\right )\left (\sum _{k=-m}^{m}\Psi _{k}Z_{t-k}\right )'$$. □ 

Remark 10.1.

The same conclusion is reached if {Z t } is a white noise process and not an IID process.

Appendix: Norm and Summability of Matrices

As in the definition of a linear process it is often necessary to analyze the convergence of a sequence of matrices $$\{\Psi _{j}\}$$, j = 0, 1, 2, …. For this we need to define a norm for matrices. The literature considers different alternative approaches. For our purposes, the choice is not relevant as all norms are equivalent in the finite dimensional vector space. We therefore choose the Frobenius, Hilbert-Schmidt or Schur norm which is easy to compute.² This norm treats the elements of a m × n matrix A = (a ij ) as an element of the $$\mathbb{R}^{m\times n}$$ Euclidean space and defines the length of A, denoted by $$\|A\|$$, as $$\|A\| = \sqrt{\sum _{i,j } a_{ij }^{2}}$$. This leads to the formal definition below.

Definition 10.6.

The Frobenius, Hilbert-Schmidt or Schur norm of a m × n matrix A, denoted by $$\|A\|$$ , is defined as:

where tr (A′A) denotes the trace of A′A, i.e. the sum of the diagonal elements of A′A, and where $$\lambda _{i}$$ are the n eigenvalues of A′A.

The matrix norm has the following properties:

The last property is called submultiplicativity .

A sequence of matrices $$\{\Psi _{j}\}$$, $$j = 0,1,\ldots$$, is called absolutely summable if and only if $$\sum _{j=0}^{\infty }\|\Psi _{j}\| < \infty $$; the sequence is said to be quadratic summable if and only if $$\sum _{j=0}^{\infty }\|\Psi _{j}\|^{2} < \infty $$. The absolute summability implies the quadratic summability, but not vice versa.

Corollary 10.1.

Absolute summability of $$\{\Psi _{j}\}$$ is equivalent to the absolute summability of each sequence $$\{[\Psi _{j}]_{kl}\}$$ , $$k,l = 1,\ldots,n$$ , i.e. to $$\lim _{j\rightarrow \infty }\vert [\Psi _{j}]_{kl}\vert $$ exists and is finite.

Proof.

In particular, he have:

Summation over j gives

The absolute convergence of each sequence $$\{[\Psi _{j}]_{kl}\}$$, $$k,l = 1,\ldots,n$$, follows from the absolute convergence of $$\{\Psi _{j}\}$$ by the first inequality. Conversely, the absolute convergence of $$\{\Psi _{j}\}$$ is implied by the absolute convergence of each sequence $$\{[\Psi _{j}]_{kl}\}$$, $$k,l = 1,\ldots,n$$, from the second inequality. □ 

References

Meyer CD (2000) Matrix analysis and applied linear algebra. Society for Industrial and Applied Mathematics, Philadelphia

Footnotes

1

We leave it to the reader to derive an analogous theorem for the correlation function.

2

For details see Meyer (2000, 279ff).

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_11

11. Estimation of Mean and Covariance Function

Klaus Neusser1 

(1)

Bern, Switzerland

11.1 Estimators and Their Asymptotic Distributions

We characterize the stationary process {X t } by its mean and its (matrix) covariance function. In the Gaussian case, this already characterizes the whole distribution. The estimation of these entities becomes crucial in the empirical analysis. As it turns out, the results from the univariate process carry over analogously to the multivariate case. If the process is observed over the periods t = 1, 2, …, T, then a natural estimator for the mean μ is the arithmetic mean or sample average:

We get a theorem analogously to Theorem 4.​1 in Sect. 4.​1

Theorem 11.1.

Let {X t } be stationary process with mean μ and covariance function $$\Gamma (h)$$ then asymptotically, for $$T \rightarrow \infty $$ , we get

Proof.

The Theorem can be established by applying Theorem 4.​1 individually to each time series {X it }, $$i = 1,2,\ldots,n$$. □ 

Thus, the sample average converges in mean square and therefore also in probability to the true mean. Thereby the second condition is more restrictive than the first one. They are, in particular, fulfilled for all VARMA processes (see Chap. 12). As in the univariate case analyzed in Sect. 4.​1, it can be shown with some mild additional assumptions that $$\overline{X}_{T}$$ is also asymptotically normally distributed.

Theorem 11.2.

For any stationary process {X t }

with $$Z_{t} \sim \mathrm{ IID}(0,\Sigma )$$ and $$\sum _{j=-\infty }^{\infty }\|\Psi _{j}\| < \infty $$ , the arithmetic average $$\overline{X}_{T}$$ is asymptotically normal:

Proof.

The proof is a straightforward extension to the multivariate case of the one given for Theorem 4.​2 of Sect. 4.​1 □ 

The summability condition is quite general. It is, in particular, fulfilled by causal VARMA processes (see Chap. 12) as their coefficients matrices $$\Psi _{j}$$ go exponentially fast to zero. Remarks similar to those following Theorem 4.​2 apply also in the multivariate case.

The above formula can be used to construct confidence regions for μ. This turns out, however, to be relatively complicated in practice so that often univariate approximations are used instead (Brockwell and Davis 1996, 228–229).

As in the univariate case, a natural estimator for the covariance matrix function $$\Gamma (h)$$ is given by the corresponding empirical moments $$\widehat{\Gamma }(h)$$:

The estimator of the covariance function can then be applied to derive an estimator for the correlation function:

where $$\hat{V }^{1/2} =\mathrm{ diag}\left (\sqrt{\hat{\gamma }_{11 } (0)},\ldots,\sqrt{\hat{\gamma }_{nn } (0)}\right )$$. Under the conditions given in Theorem 11.2 the estimator of the covariance matrix of order h, $$\widehat{\Gamma }(h)$$, converges to the true covariance matrix $$\Gamma (h)$$. Moreover, $$\sqrt{T}\left (\widehat{\Gamma }(h) - \Gamma (h)\right )$$ is asymptotically normally distributed. In particular, we can state the following Theorem:

Theorem 11.3.

Let {X t } be a stationary process with

where $$Z_{t} \sim \mathrm{ IID}(0,\Sigma )$$, $$\sum _{j=-\infty }^{\infty }\|\Psi _{j}\| < \infty $$ , and $$\sum _{j=-\infty }^{\infty }\Psi _{j}\neq 0$$ . Then, for each fixed h, $$\widehat{\Gamma }(h)$$ converges in probability as $$T \rightarrow \infty $$ to $$\Gamma (h)$$:

Proof.

A proof can be given along the lines given in Proposition 13.​1. □ 

As for the univariate case, we can define the long-run covariance matrix J as

(11.1)

As a non-parametric estimator we can again consider the following class of estimators:

where k(x) is a kernel function and where $$\widehat{\Gamma }(h)$$ is the corresponding estimate of the covariance matrix at lag h. For the choice of the kernel function and the lag truncation parameter the same principles apply as in the univariate case (see Sect. 4.​4 and Haan and Levin (1997)).

11.2 Testing Cross-Correlations of Bivariate Time Series

The determination of the asymptotic distribution of $$\widehat{\Gamma }(h)$$ is complicated. We therefore restrict ourselves to the case of two time series.

Theorem 11.4.

Let {X t } be a bivariate stochastic process whose components can be described by

where {Z 1t } and {Z 2t } are independent from each other at all leads and lags and where $$\sum _{j}\vert \alpha _{j}\vert < \infty $$ and $$\sum _{j}\vert \beta _{j}\vert < \infty $$ . Under these conditions the asymptotic distribution of the estimator of the cross-correlation function ρ ₁₂ (h) between {X 1t } and {X 2t } is

(11.2)

For all h and k with h ≠ k, $$(\sqrt{T}\hat{\rho }_{12}(h),\sqrt{T}\hat{\rho }_{12}(k))'$$ converges in distribution to a bivariate normal distribution with mean zero and variances and covariances given by $$\sum _{j=-\infty }^{\infty }\rho _{11}(j)\rho _{22}(j)$$ and $$\sum _{j=-\infty }^{\infty }\rho _{11}(j)\rho _{22}(j + k - h)$$ , respectively.

This result can be used to construct a test of independence, respectively uncorrelatedness, between two time series. The above theorem, however, shows that the asymptotic distribution of $$\sqrt{ T}\hat{\rho }_{12}(h)$$ depends on ρ ₁₁(h) and ρ ₂₂(h) and is therefore unknown. Thus, the test cannot be based on the cross-correlation alone.¹

This problem can, however, be overcome by implementing the following two-step procedure suggested by Haugh (1976).

First step:

Estimate for each times series separately a univariate invertible ARMA model and compute the resulting residuals $$\hat{Z}_{it}$$ as $$\hat{Z}_{it} =\sum _{ j=0}^{\infty }\hat{\pi }_{j}^{(i)}X_{i,t-j}$$, i = 1, 2. If the ARMA models correspond to the true ones, these residuals should approximately be white noise. This first step is called pre-whitening.

Second step:

Under the null hypothesis the two time series {X 1t } and {X 2t } are uncorrelated with each other. This implies that the residuals {Z 1t } and {Z 2t } should also be uncorrelated with each other. The variance of the cross-correlations between {Z 1t } and {Z 2t } are therefore asymptotically equal to 1∕T under the null hypothesis. Thus, one can apply the result of Theorem 11.4 to construct confidence intervals based on formula (11.2). A 95-% confidence interval is therefore given by $$\pm 1.96T^{-1/2}$$. The Theorem may also be used to construct a test whether the two series are uncorrelated.

If one is not interested in modeling the two time series explicitly, the simplest way is to estimate a high order AR model in the first step. Thereby, the order should be chosen high enough to obtain white noise residuals in the first step. Instead of looking at each cross-correlation separately, one may also test the joint null hypothesis that all cross-correlations are simultaneously equal to zero. Such a test can be based on T times the sum of the squared cross-correlation coefficients. This statistic is distributed as a χ ² with L degrees of freedom where L is the number of summands (see the Haugh-Pierce statistic (15.​1) in Sect. 15.​1).

11.3 Some Examples for Independence Tests

Two Independent AR Processes

Consider two AR(1) process {X 1t } and {X 2t } governed by the following stochastic difference equation $$X_{it} = 0.8X_{i,t-1} + Z_{it}$$, i = 1, 2. The two white noise processes {Z 1t } and {Z 2t } are such that they are independent from each other. {X 1t } and {X 2t } are therefore independent from each other too. We simulate realizations of these two processes over 400 periods. The estimated cross-correlation function of these so-generated processes are plotted in the upper panel of Fig. 11.1. From there one can see that many values are outside the 95 % confidence interval given by $$\pm 1.96T^{-1/2} = 0.098$$, despite the fact that by construction both series are independent of each other. The reason is that the so computed confidence interval is not correct because it does not take the autocorrelation of each series into account. The application of Theorem 11.4 leads to the much larger 95-% confidence interval of

which is more than twice as large. This confidence interval then encompasses most the cross-correlations computed with respect to the original series.

If one follows the testing procedure outline above instead and fits an AR(10) model for each process and then estimates the cross-correlation function of the corresponding residual series (filtered or pre-whitened time series), the plot in the lower panel of Fig. 11.1 is obtained.² This figure shows no significant cross-correlation anymore so that one cannot reject the null hypothesis that both time series are independent from each other.

Fig. 11.1

Cross-correlations between two independent AR(1) processes with ϕ = 0. 8

Consumption Expenditure and Advertisement Expenses

This application focuses on the interaction between nominal aggregate private consumption expenditure and nominal aggregate advertisement expenditures. Such an investigation was first conducted by Ashley et al. (1980) for the United States.³ The upper panel of Fig. 11.2 shows the raw cross-correlations between the two time series where the order h runs from − 20 to + 20. Although almost all cross-correlations are positive and outside the conventional confidence interval, it would be misleading to infer a statistically significant positive cross-correlation. In order to test for independence, we filter both time series by an AR(10) model and estimate the cross-correlations for the residuals.⁴ These are displayed in the lower panel of Fig. 11.2. In this figure, only the correlations of order 0 and 16 fall outside the confidence interval and can thus be considered as statistically significant. Thus, we can reject the null hypothesis of independence between the two series. However, most of the interdependence seems to come from the correlation within the same quarter. This is confirmed by a more detailed investigation in Berndt (1991) where no significant lead and/or lag relations are found.

Fig. 11.2

Cross-correlations between aggregate nominal private consumption expenditures and aggregate nominal advertisement expenditures

Real Gross Domestic Product and Consumer Sentiment

The procedure outlined above can be used to examine whether one of the two time series is systematically leading the other one. This is, for example, important in the judgment of the current state of the economy because first provisional national accounting data are usually published with a lag of at least one quarter. However, in the conduct of monetary policy more up-to-date knowledge is necessary. Such a knowledge can be retrieved from leading indicator variables. These variables should be available more quickly and should be highly correlated with the variable of interest at a lead.

We investigate whether the Consumer Sentiment Index is a leading indicator for the percentage changes in real Gross Domestic Product (GDP).⁵ The raw cross-correlations are plotted in the upper panel of Fig. 11.3. It shows several correlations outside the conventional confidence interval. The use of this confidence interval is, however, misleading as the distribution of the raw cross-correlations depends on the autocorrelations of each series. Thus, instead we filter both time series by an AR(8) model and investigate the cross-correlations of the residuals.⁶ The order of the AR model was chosen deliberately high to account for all autocorrelations. The cross-correlations of the filtered data are displayed in the lower panel of Fig. 11.3. As it turns out, only the cross-correlation which is significantly different from zero is for h = 1. Thus the Consumer Sentiment Index is leading the growth rate in GDP. In other words, an unexpected higher consumer sentiment is reflected in a positive change in the GDP growth rate of next quarter.⁷

Fig. 11.3

Cross-correlations between real growth of GDP and the consumer sentiment index

References

Ashley R, Granger CWJ, Schmalensee R (1980) Advertising and aggregate consumption: An analysis of causality. Econometrica 48(5):1149–1168CrossRef

Berndt ER (1991) The practice of econometrics. Addison Wesley, Reading,

Brockwell PJ, Davis RA (1996) Introduction to time series and forecasting. Springer, New YorkCrossRef

Haan WJ, Levin AT (1997) A practitioner’s guide to robust covariance matrix estimation. In: Maddala GS, Rao CR (eds) Handbook of statistics: robust inference, vol 15. Elsevier, New York, pp 299–342CrossRef

Haugh LD (1976) Checking the independence of two covariance stationary time series: A univariate residual cross-correlation approach. J Am Stat Assoc 71:378–385CrossRef

Footnotes

1

The theorem may also be used to conduct a causality test between two times series (see Sect. 15.​1).

2

The order of the AR processes are set arbitrarily equal to 10 which is more than enough to obtain white noise residuals.

3

The quarterly data are taken from Berndt (1991). They cover the period from the first quarter 1956 to the fourth quarter 1975. In order to achieve stationarity, we work with first differences.

4

The order of the AR processes are set arbitrarily equal to 10 which is more than enough to obtain white noise residuals.

5

We use data for Switzerland as published by the State Secretariat for Economic Affairs SECO.

6

With quarterly data it is wise to set to order as a multiple of four to account for possible seasonal movements. As it turns out p = 8 is more than enough to obtain white noise residuals.

7

During the interpretation of the cross-correlations be aware of the ordering of the variables because $$\rho _{12}(1) =\rho _{21}(-1)\neq \rho _{21}(1)$$.

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_12

12. Stationary Time Series Models: Vector Autoregressive Moving-Average Processes (VARMA Processes)

Klaus Neusser1 

(1)

Bern, Switzerland

The most important class of models is obtained by requiring {X t } to be the solution of a linear stochastic difference equation with constant coefficients. In analogy to the univariate case, this leads to the theory of vector autoregressive moving-average processes (VARMA processes or just ARMA processes).

Definition 12.1 (VARMA process).

A multivariate stochastic process {Xt} is a vector autoregressive moving-average process of order (p,q), denoted as VARMA(p,q) process, if it is stationary and fulfills the stochastic difference equation

(12.1)

where $$\Phi _{p}\neq 0$$ , $$\Theta _{q}\neq 0$$ and $$Z_{t} \sim \mathrm{ WN}(0,\Sigma )$$ . {X t } is called a VARMA(p,q) process with mean μ if {X t −μ} is a VARMA(p,q) process.

With the aid of the lag operator we can write the difference equation more compactly as

where $$\Phi (\mathrm{L}) =\mathrm{ I}_{n} - \Phi _{1}\mathrm{L} -\ldots -\Phi _{p}\mathrm{L}^{p}$$ and $$\Theta (\mathrm{L}) =\mathrm{ I}_{n} + \Theta _{1}\mathrm{L} +\ldots +\Theta _{q}\mathrm{L}^{q}$$. $$\Phi (\mathrm{L})$$ and $$\Theta (\mathrm{L})$$ are n × n matrices whose elements are lag polynomials of order smaller or equal to p, respectively q. If q = 0, $$\Theta (\mathrm{L}) =\mathrm{ I}_{n}$$ so that there is no moving-average part. The process is then a purely autoregressive one which is simply called a VAR(p) process. Similarly if p = 0, $$\Phi (\mathrm{L}) =\mathrm{ I}_{n}$$ and there is no autoregressive part. The process is then a purely moving-average one and simply called a VMA(q) process. The importance of VARMA processes stems from the fact that every stationary process can be arbitrarily well approximated by a VARMA process, VAR process, or VMA process.

12.1 The VAR(1) Process

We start our discussion by analyzing the properties of the VAR(1) process which is defined as the solution the following stochastic difference equation:

We assume that all eigenvalues of $$\Phi $$ are absolutely strictly smaller than one. As the eigenvalues correspond to the inverses of the roots of the matrix polynomial $$\det (\Phi (z)) =\det (I_{n} - \Phi z)$$, this assumption implies that all roots must lie outside the unit circle:

For the sake of exposition, we will further assume that $$\Phi $$ is diagonalizable, i.e. there exists an invertible matrix P such that $$J = P^{-1}\Phi P$$ is a diagonal matrix with the eigenvalues of $$\Phi $$ on the diagonal.¹

Consider now the stochastic process

We will show that this process is stationary and fulfills the first order difference equation above. For {X t } to be well-defined, we must show that $$\sum _{j=0}^{\infty }\|\Phi ^{j}\| < \infty $$. Using the properties of the matrix norm we get:

where $$\lambda _{\max }$$ denotes the maximal eigenvalue of $$\Phi $$ in absolute terms. As all eigenvalues are required to be strictly smaller than one, this clearly also holds for $$\lambda _{\max }$$ so that infinite matrix sum converges. This implies that the process {X t } is stationary. In addition, we have that

Thus, the process {X t } also fulfills the difference equation.

Next we demonstrate that this process is also the unique stationary solution to the difference equation. Suppose that there exists another stationary process {Y t } which also fulfills the difference equation. By successively iterating the difference equation one obtains:

Because {Y t } is assumed to be stationary, $$\mathbb{V}Y _{t} = \mathbb{V}Y _{t-k-1} = \Gamma (0)$$ so that

The submultiplicativity of the norm then implies:

As all eigenvalues of $$\Phi $$ are absolutely strictly smaller than one, the right hand side of the above expression converges to zero for k going to infinity. This implies that Y t and $$X_{t} =\sum _{ j=0}^{\infty }\Phi ^{j}Z_{t-j}$$ are equal in the mean square sense and thus also in probability.

Based on Theorem 10.​2, the mean and the covariance function of the VAR(1) process is:

Analogously to the univariate case, it can be shown that there still exists a unique stationary solution if all eigenvalues are absolutely strictly greater than one. This solution is, however, no longer causal with respect to {Z t }. If some of the eigenvalues of $$\Phi $$ are on the unit circle, there exists no stationary solution.

12.2 Representation in Companion Form

A VAR(p) process of dimension n can be represented as a VAR(1) process of dimension p × n. For this purpose we define the pn vector $$Y _{t} = (X_{t}',X_{t-1}',\ldots,X_{t-p+1}')'$$. This new process {Y t } is characterized by the following first order stochastic difference equation:

where U t  = (Z t , 0, 0, …, 0)′ with $$U_{t} \sim \mathrm{ WN}\left (0,\left (\begin{array}{*{10}c} \Sigma &0\\ 0 &0\\ \end{array} \right )\right )$$. This representation is also known as the companion form or state space representation (see also Chap. 17). In this representation the last p(n − 1) equations are simply identities so that there is no error term attached. The latter name stems from the fact that Y t encompasses all the information necessary to describe the state of the system. The matrix $$\Phi $$ is called the companion matrix of the VAR(p) process.²

The main advantage of the companion form is that by studying the properties of the VAR(1) model, one implicitly encompasses VAR models of higher order and also univariate AR(p) models which can be considered as special cases. The relation between the eigenvalues of the companion matrix and the roots of the polynomial matrix $$\Phi (z)$$ is given by the formula (Gohberg et al. 1982):

(12.2)

In the case of the AR(p) process the eigenvalues of $$\Phi $$ are just the inverses of the roots of the polynomial $$\Phi (z)$$. Further elaboration of state space models is given in Chap. 17.

12.3 Causal Representation

As will become clear in Chap. 15 and particularly in Sect. 15.​2, the issue of the existence of a causal representation is even more important than in the univariate case. Before stating the main theorem let us generalize the definition of a causal representation from the univariate case (see Definition 2.​2 in Sect. 2.​3) to the multivariate one.

Definition 12.2.

A VARMA((p,q) process {Xt} with $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$ is called causal with respect to {Zt} if and only if there exists a sequence of absolutely summable matrices $$\{\Psi _{j}\}$$, j = 0,1,2,…, i.e. $$\sum _{j=0}^{\infty }\|\Psi _{j}\| < \infty $$, such that

Theorem 12.1.

Let {X t } be a VARMA(p,q) process with $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$ and assume that

then the stochastic difference equation $$\Phi (\mathrm{L})X_{t} = \Theta (\mathrm{L})Z_{t}$$ has exactly one stationary solution with causal representation

whereby the sequence of matrices $$\{\Psi _{j}\}$$ is absolutely summable and where the matrices are uniquely determined by the identity

Proof.

The proof is a straightforward extension of the univariate case. □ 

As in the univariate case, the coefficient matrices which make up the causal representation can be found by the method of undetermined coefficients, i.e. by equating $$\Phi (z)\Psi (z) = \Theta (z)$$. In the case of the VAR(1) process, the $$\{\Psi _{j}\}$$ have to obey the following recursion:

The recursion in the VAR(2) case is:

Remark 12.1.

Consider a VAR(1) process with $$\Phi = \left (\begin{array}{*{10}c} 0& \phi \\ 0 &0\\ \end{array} \right )$$ with ϕ ≠ 0 then the matrices in the causal representation are $$\Psi _{j} = \Phi ^{j} = 0$$ for j > 1. This means that {X t } has an alternative representation as a VMA(1) process because $$X_{t} = Z_{t} + \Phi Z_{t-1}$$. This simple example demonstrates that the representation of {X t } as a VARMA process is not unique. It is therefore impossible to always distinguish between VAR and VMA process of higher orders without imposing additional assumptions. These additional assumptions are much more complex in the multivariate case and are known as identifying assumptions. Thus, a general treatment of this identification problem is outside the scope of this book. See Hannan and Deistler (1988) for a general treatment of this issue. For this reason we will concentrate exclusively on VAR processes where these identification issues do not arise.

Example

We illustrate the above concept by the following VAR(2) model:

In a first step, we check whether the VAR model admits a causal representation with respect to {Z t }. For this purpose we have to compute the roots of the equation $$\det (I_{2} - \Phi _{1}z - \Phi _{2}z^{2}) = 0$$:

The four roots are: − 1. 1973, 0. 8828 ± 1. 6669ι, 1. 5650. As they are all outside the unit circle, there exists a causal representation which can be found from the equation $$\Phi (z)\Psi (z) = I_{2}$$ by the method of undetermined coefficients. Multiplying the equation system out, we get:

Equating the coefficients corresponding to z j , j = 1, 2, …:

The last equation shows how to compute the sequence $$\{\Psi _{j}\}$$ recursively:

12.4 Computation of the Covariance Function of a Causal VAR Process

As in the univariate case, it is important to be able to compute the covariance and the correlation function of VARMA process (see Sect. 2.​4). As explained in Remark 12.1 we will concentrate on VAR processes. Consider first the case of a causal VAR(1) process:

Multiplying the above equation first by X t ′ and then successively by X t−h ′ from the left, h = 1, 2, …, and taking expectations, we obtain the Yule-Walker equations:

Knowing $$\Gamma (0)$$ and $$\Phi $$, $$\Gamma (h)$$, h > 0, can be computed recursively from the second equation as

(12.3)

Given $$\Phi $$ and $$\Sigma $$, we can compute $$\Gamma (0)$$. For h = 1, the second equation above implies $$\Gamma (1) = \Phi \Gamma (0)$$. Inserting this expression in the first equation and using the fact that $$\Gamma (-1) = \Gamma (1)'$$, we get an equation in $$\Gamma (0)$$:

This equation can be solved for $$\Gamma (0)$$:

where ⊗ and “vec” denote the Kronecker-product and the vec-operator, respectively.³ Thus,

(12.4)

The assumption that {X t } is causal with respect to {Z t } guarantees that the eigenvalues of $$\Phi \otimes \Phi $$ are strictly smaller than one in absolute value, implying that $$I_{n^{2}} - \Phi \otimes \Phi $$ is invertible.⁴

If the process is a causal VAR(p) process the covariance function can be found in two ways. The first one rewrites the process in companion form as a VAR(1) process and applies the procedure just outlined. The second way relies on the Yule-Walker equation. This equation is obtained by multiplying the stochastic difference equation from the left by X t ′ and then successively by X t−h ′, h > 0, and taking expectations:

(12.5)

The second equation can be used to compute $$\Gamma (h)$$, h ≥ p, recursively taking $$\Phi _{1},\ldots,\Phi _{p}$$ and the starting values $$\Gamma (p - 1),\ldots,\Gamma (0)$$ as given. The starting value can be retrieved by transforming the VAR(p) model into the companion form and proceeding as explained above.

Example

We illustrate the computation of the covariance function using the same example as in Sect. 12.3. First, we transform the model into the companion form:

Equation (12.4) implies that $$\Gamma _{Y }(0)$$ is given by:

so that $$\Gamma _{X}(0)$$ and $$\Gamma _{X}(1)$$ become:

The other covariance matrices can then be computed recursively according to Eq. (12.5):

Appendix: Autoregressive Final Form

Definition 12.1 defined the VARMA process {X t } as a solution to the corresponding multivariate stochastic difference equation (12.1). However, as pointed out by Zellner and Palm (1974) there is an equivalent representation in the form of n univariate ARMA processes, one for each X it . Formally, these representations, also called autoregressive final form or transfer function form (Box and Jenkins 1976), can be written as

where the index i• indicates the i-th row of $$\Phi ^{{\ast}}(\mathrm{L})\Theta (\mathrm{L})$$. Thereby $$\Phi ^{{\ast}}(\mathrm{L})$$ denotes the adjugate matrix of $$\Phi (\mathrm{L})$$.⁵ Thus each variable in X t may be investigated separately as an univariate ARMA process. Thereby the autoregressive part will be the same for each variable. Note, however, that the moving-average processes will be correlated across variables.

The disadvantage of this approach is that it involves rather long AR and MA lags as will become clear from the following example.⁶ Take a simple two-dimensional VAR of order one, i.e. $$X_{t} = \Phi X_{t-1} + Z_{t}$$, $$Z_{t} \sim \mathrm{ WN}(0,\Sigma )$$. Then the implied univariate processes will be ARMA(2,1) processes. After some straightforward manipulations we obtain:

It can be shown by the means given in Sects. 1.​4.​3 and 1.​5.​1 that the right hand sides are observationally equivalent to MA(1) processes.

References

Box GEP, Jenkins GM (1976) Time series analysis: forecasting and control, revised edn. Holden-Day, San Francisco

Gohberg I, Lancaster P, Rodman L (1982) Matrix polynomials. Academic Press, New York

Hannan EJ, Deistler M (1988) The statistical theory of linear systems. Wiley, New York

Magnus JR, Neudecker H (1988) Matrix differential calculus with applications in statistics and econometrics. Wiley, Chichester

Meyer CD (2000) Matrix analysis and applied linear algebra. Society for Industrial and Applied Mathematics, PhiladelphiaCrossRef

Zellner A, Palm F (1974) Time series analysis and simultaneous equation econometric models. J Econ 2:17–54CrossRef

Footnotes

1

The following exposition remains valid even if $$\Phi $$ is not diagonalizable. In this case one has to rely on the Jordan form which complicates the computations (Meyer 2000).

2

The representation of a VAR(p) process in companion form is not uniquely defined. Permutations of the elements in Y t will lead to changes in the companion matrix.

3

The vec-operator stacks the column of a n × m matrix to a column vector of dimension nm. The properties of ⊗ and vec can be found, e.g. in Magnus and Neudecker (1988).

4

If the eigenvalues of $$\Phi $$ are $$\lambda _{i}$$, i = 1, …, n, then the eigenvalues of $$\Phi \otimes \Phi $$ are $$\lambda _{i}\lambda _{j}$$, i, j = 1, …, n (see Magnus and Neudecker (1988)).

5

The elements of the adjugate matrix A ∗ of some matrix A are given by $$[A^{{\ast}}]_{ij} = (-1)^{i+j}M_{ij}$$ where M ij is the minor (minor determinant) obtained by deleting the i-th column and the j-th row of A (Meyer 2000, p. 477).

6

The degrees of the AR and the MA polynomial can be as large as np and $$(n - 1)p + q$$, respectively.

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_13

13. Estimation of Vector Autoregressive Models

Klaus Neusser1 

(1)

Bern, Switzerland

13.1 Introduction

In this chapter we derive the Least-Squares (LS) estimator for vectorautoregressive (VAR) models and its asymptotic distribution. For this end, we have to make several assumption which we maintain throughout this chapter.

Assumption 13.1.

The VAR process {X t } is generated by

$$\Sigma $$ nonsingular, and admits a stationary and causal representation with respect to {Z t }:

with $$\sum _{j=0}^{\infty }\|\Psi _{j}\| < \infty $$ .

Assumption 13.2.

The residual process {Z t } is not only white noise, but also independently and identically distributed:

Assumption 13.3.

All fourth moments of Z t exist. In particular, there exists a finite constant c > 0 such that

Note that the moment condition is automatically fulfilled by Gaussian processes. For the ease of exposition, we omit a constant in the VAR. Thus, we consider the demeaned process.

13.2 The Least-Squares Estimator

Let us denote by ϕ ij (k) the (i, j)-th element of the matrix $$\Phi _{k}$$, k = 1, 2, …, p, then the i-th equation, i = 1, …, n, can be written as

We can view this equation as a regression equation of X it on all lagged variables $$X_{1,t-1},\ldots,X_{n,t-1},\ldots,X_{1,t-p},\ldots,X_{n,t-p}$$ with error term Z it . Note that the regressors are the same for each equation. The np regressors have coefficient vector $$\left (\phi _{i1}^{(1)},\ldots,\phi _{in}^{(1)},\ldots,\phi _{i1}^{(p)},\ldots,\phi _{in}^{(p)}\right )'$$. Thus, the complete VAR(p) model has n ² p coefficients in total to be estimated. In addition, there are $$n(n + 1)/2$$ independent elements of the covariance matrix $$\Sigma $$ that have to be estimated too.

It is clear that the n different equations are linked through the regressors and the errors terms which in general have non-zero covariances $$\sigma _{ij} = \mathbb{E}Z_{it}Z_{jt}$$. Hence, it seems warranted to take a systems approach and to estimate all equations of the VAR jointly. Below, we will see that an equation-by-equation approach is, however, still appropriate.

Suppose that we have T + p observations with $$t = -p + 1,\ldots,0,1,\ldots,T$$, then we can write the regressor matrix for each equation compactly as a T × np matrix X:

Using this notation, we can write the VAR for observations t = 1, 2, …, T as

or more compactly

There are two ways to bring this equation system in the usual multivariate regression framework. One can either arrange the data according to observations or according to equations. Ordered in terms of observations yields:

(13.1)

with $$\mathop{\mathrm{vec}}Y = (X_{11},X_{21},\ldots,X_{n1},X_{12},X_{22},\ldots,X_{n2},\ldots,X_{1T},X_{2T},\ldots,X_{nT})'$$. If the data are arranged equation by equation, the dependent variable is $$\mathop{\mathrm{vec}}Y ' = (X_{11},X_{12},\ldots,X_{1T},X_{21},X_{22},\ldots,X_{2T},\ldots,X_{n1},X_{n2},\ldots,X_{nT})'$$. As both representations, obviously, contain the same information, there exists a nT × nT permutation or commutation matrix K nT such that $$\mathop{\mathrm{vec}}Y ' = K_{nt}\mathop{ \mathrm{vec}}Y$$. Using the computation rules for the Kronecker product, the vec operator, and the permutation matrix (see Magnus and Neudecker 1988), we get for the ordering in terms of equations

(13.2)

where $$K_{n^{2}p}$$ is the corresponding n ² × p permutation matrix relating $$\mathop{\mathrm{vec}}\Phi $$ and $$\mathop{\mathrm{vec}}\Phi '$$.

The error terms of the different equations are correlated because, in general, the covariances $$\sigma _{ij} = \mathbb{E}Z_{it}Z_{jt}$$ are nonzero. In the case of an arrangement by observation the covariance matrix of the error term $$\mathop{\mathrm{vec}}Z$$ is

In the second case, the arrangement by equation, the covariance matrix of the error term $$\mathop{\mathrm{vec}}Z'$$ is

Given that the covariance matrix is not a multiple of the identity matrix, efficient estimation requires the use of generalized least squares (GLS). The GLS estimator minimizes the weighted sum of squared errors

The solution of this minimization problem can be found in standard econometric textbooks like (Dhrymes 1978; Greene 2008; Hamilton 1994b) and is given by

As the covariance matrix $$\Sigma $$ cancels, the GLS and the OLS-estimator deliver numerically exactly the same solution. The reason for this result is that the regressors are the same in each equation. If this does not hold, for example when some coefficients are set a priori to zero, efficient estimation would require the use of GLS.

Further insights can be gained by rewriting the estimation problem in terms of the arrangement by equation (see Eq. (13.2)). For this purpose, multiply the above estimator from the left by the commutation matrix $$K_{n^{2}p}$$ ¹:

This can be written in a more explicit form as

where Y i , i = 1, …, n, stacks the observations of the i-th variable such that Y i  = (X i1, X i2, …, X iT )′. Thus, the estimation of VAR as a system can be broken down into the estimation of n regression equations with dependent variable X it . Each of these equations can then be estimated by OLS.

Thus, we have proven that

(13.3)

(13.4)

The least squares estimator can also be rewritten without the use of the ve c-operator:

Under the assumptions stated in the Introduction Sect. 13.1, these estimators are consistent and asymptotically normal.

Theorem 13.1 (Asymptotic Distribution of OLS Estimator).

Under the assumption stated in the Introduction Sect.  13.1,it holds that

and that

respectively,

where $$\boldsymbol{\Gamma }_{p} =\mathop{ \mathrm{plim}} \frac{1} {T}(\mathbf{X}'\mathbf{X})$$ .

Proof.

See Sect. 13.3. □ 

In order to make use of this result in practice, we have to replace the matrices $$\Sigma $$ and $$\boldsymbol{\Gamma }_{p}$$ by some estimate. A natural consistent estimate of $$\boldsymbol{\Gamma }_{p}$$ is given according to Proposition 13.1 by

In analogy to the multivariate regression model, a natural estimator for $$\Sigma $$ can be obtained from the Least-Squares residuals $$\widehat{Z}$$:

The property of this estimator is summarized in the proposition below.

Theorem 13.2.

Under the condition of Theorem  13.1

Proof.

See Sect. 13.3. □ 

An alternative, but asymptotically equivalent estimator $$\widetilde{\Sigma }$$ is obtained by adjusting $$\widehat{\Sigma }$$ for the degrees of freedom:

(13.5)

If the VAR contains a constant, as is normally the case in practice, the degrees of freedom correction should be $$T - np - 1$$.

Small sample inference with respect to the parameters $$\Phi $$ can therefore be carried out using the approximate distribution

(13.6)

This implies that hypothesis testing can be carried out using the conventional t- and F-statistics. From a system perspective, the appropriate degree of freedom for the t-ratio would be $$nT - n^{2}p - n$$, taking a constant in each equation into account. However, as that the system can be estimated on an equation by equation basis, it seems reasonable to use $$T - np - 1$$ instead. This corresponds to a multivariate regression setting with T observation and np + 1 regressors, including a constant.

However, as in the univariate case the Gauss Markov theorem does not apply because the lagged regressors are correlated with past error terms. This results in biased estimates in small samples. The amount of the bias can be assessed and corrected either by analytical or bootstrap methods. For an overview, a comparison of the different corrections proposed in the literature, and further references see Engsteg and Pedersen (2014).

13.3 Proofs of the Asymptotic Properties of the Least-Squares Estimator

Lemma 13.1.

Given the assumptions made in Sect.  13.1,the process $$\{\mathop{\mathrm{vec}}Z_{t-j}Z_{t-i}'\}$$ , $$i,j \in \mathbb{Z}$$ and i ≠ j, is white noise.

Proof.

Using the independence assumption of {Z t }, we immediately get

 □ 

Under the assumption put forward in the Introduction, $$\tfrac{1} {T}(\mathbf{X}'\mathbf{X})$$ converges in probability for $$T \rightarrow \infty $$ to a np × np matrix $$\boldsymbol{\Gamma }_{p}$$. This matrix consists of p ² blocks where each (i, j)-th block corresponds to the covariance matrix $$\Gamma (i - j)$$. Thus we have the following proposition:

Proposition 13.1.

Under the assumption stated in the Introduction Sect.  13.1

with $$\boldsymbol{\Gamma }_{p}$$ being nonsingular.

Proof.

Write $$\tfrac{1} {T}(\mathbf{X}'\mathbf{X})$$ as

where

We will show that each component $$\widehat{\Gamma }(h)$$ of $$\tfrac{1} {T}\mathbf{X}'\mathbf{X}$$ converges in probability to $$\Gamma (h)$$. Taking the causal representation of {X t } into account

According to Lemma 13.1 above $$\{Z_{t-j}Z_{t-i}'\}$$, i ≠ j, is white noise. Thus,

according to Theorem 11.​1. Hence, for m fixed,

Taking absolute values and expectations element-wise,

As the bound is independent of T and converges to 0 as $$m \rightarrow \infty $$, we have

The Basic Approximation Theorem C.14 then establishes that

Henceforth

where the remainder only depends on initial conditions² and is therefore negligible as $$T \rightarrow \infty $$. As

we finally get

The last equality follows from Theorem 10.​2. □ 

Proposition 13.2.

Under the assumption stated in the Introduction Sect.  13.1

Proof.

The idea of the proof is to approximate {X t } by some simpler process {X t (m)} which allows the application of the CLT for dependent processes (Theorem C.13). This leads to an asymptotic distribution which by the virtue of the Basic Approximation Theorem C.14 converges to the asymptotic distribution of the original process. Define X t (m) as the truncated process from the causal presentation of X t :

Using this approximation, we can then define the process {Y t (m)} as

Due to the independence of {Z t } this process is a mean zero white noise process, but is clearly not independent. It is easy to see that the process is actually (m + p)-dependent with variance V m given by

where $$\Gamma _{p}^{(m)}$$ is composed of

Thus, we can invoke the CLT for (m + p)-dependent process (see Theorem C.13) to establish that

For $$m \rightarrow \infty $$, $$\Gamma ^{(m)}(h)$$ converges to $$\Gamma (h)$$ and thus $$\boldsymbol{\Gamma }_{p}^{(m)}$$ to $$\boldsymbol{\Gamma }_{p}$$. Therefore, $$\mathbf{V}_{m}\longrightarrow \boldsymbol{\Gamma }_{p} \otimes \Sigma $$.

The variance of the approximation error is equal to

The absolute summability of $$\Psi _{j}$$ then implies that the infinite sums converge to zero as $$m \rightarrow \infty $$. As $$X_{t}^{(m)}\mathop{\longrightarrow}\limits_{}^{\quad \text{m.s.}\quad }X_{t}$$ for $$m \rightarrow \infty $$, we can apply the Basic Approximation Theorem C.14 to reach the required conclusion

 □ 

Proof of Theorem 13.1

Proof. We prove the Theorem for the arrangement by observation. The prove for the arrangement by equation can be proven in a completely analogous way. Inserting the regression formula (13.1) into the least-squares formula (13.3) leads to:

(13.7)

Bringing $$\mathop{\mathrm{vec}}\Phi $$ to the left hand side and taking the probability limit, we get using Slutzky’s Lemma C.10 for the product of probability limits

The last equality follows from the observation that Proposition 13.1 implies $$\mathop{\mathrm{plim}} \frac{\mathbf{X}'\mathbf{X}} {T} =\boldsymbol{ \Gamma }_{p}$$ nonsingular and that Proposition 13.2 implies $$\mathop{\mathrm{plim}} \frac{Z\mathbf{X}} {T} = 0$$. Thus, we have established that the Least-Squares estimator is consistent.

Equation (13.7) further implies:

As $$\mathop{\mathrm{plim}} \frac{\mathbf{X}'\mathbf{X}} {T} =\boldsymbol{ \Gamma }_{p}$$ nonsingular, the above expression converges in distribution according to Theorem C.10 and Proposition 13.2 to a normally distributed random variable with mean zero and covariance matrix

 □ 

Proof of Theorem 13.2

Proof.

Applying Theorem C.7 and the results of Propositions 13.1 and 13.2 shows that

and

Hence,

 □ 

13.4 The Yule-Walker Estimator

An alternative estimation method can be derived from the Yule-Walker equations . Consider first a VAR(1) model. The Yule-Walker equation in this case simply is:

or

The solution of this system of equations is:

Replacing the theoretical moments by their empirical counterparts, we get the Yule-Walker estimator for $$\Phi $$ and $$\Sigma $$:

In the general case of a VAR(p) model the Yule-Walker estimator is given as the solution of the equation system

As the least-squares and the Yule-Walker estimator differ only in the treatment of the starting values, they are asymptotically equivalent. In fact, they yield very similar estimates even for finite samples (see e.g. Reinsel (1993)). However, as in the univariate case, the Yule-Walker estimator always delivers, in contrast to the least-square estimator, coefficient estimates with the property $$\det (I_{n} -\hat{ \Phi }_{1}z -\ldots -\hat{\Phi }_{p}z^{p})\neq 0$$ for all $$z \in \mathbb{C}$$ with | z | ≤ 1. Thus, the Yule-Walker estimator guarantees that the estimated VAR possesses a causal representation. This, however, comes at the price that the Yule-Walker estimator has a larger small-sample bias than the least-squares estimator, especially when the roots of $$\Phi (z)$$ get close to the unit circle (Tjøstheim and Paulsen 1983; Shaman and Stine 1988; Reinsel 1993). Thus, it is generally preferable to use the least-squares estimator in practice.

References

Brockwell PJ, Davis RA (1991) Time series: theory and methods, 2nd edn. Springer, New YorkCrossRef

Dhrymes PJ (1978) Introductory econometrics. Springer, New YorkCrossRef

Engsteg T, Pedersen TQ (2014) Bias–correction in vectorautoregressive models: A simulation study. Econometrics 2:45–71CrossRef

Greene WH (2008) Econometric anlysis, 7th edn. Prentice Hall, New Jersey

Hamilton JD (1994b) Time series analysis. Princeton University Press, Princeton

Magnus JR, Neudecker H (1988) Matrix differential calculus with applications in statistics and econometrics. Wiley, Chichester

Reinsel GC (1993) Elements of multivariate time series analysis. Springer Series in statistics. Springer, New YorkCrossRef

Shaman P, Stine RA (1988) The bias of autoregressive coefficient estimators. J Am Stat Assoc 83:842–848CrossRef

Tjøstheim D, Paulsen J (1983) Bias of some commonly-used time series estimates. Biometrika 70:389–399; corrigendum (1984), 71:656

Footnotes

1

Alternatively, one could start from scratch and investigate the minimization problem $$S(\mathop{\mathrm{vec}} \Phi ') = (\mathop{\mathrm{vec}} Z')'(\Sigma ^{-1} \otimes I_{T})(\mathop{\mathrm{vec}} Z') \rightarrow \min _{\Phi }$$.

2

See the proof of Theorem 11.2.2 in Brockwell and Davis (1991) for details.

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_14

14. Forecasting with VAR Models

Klaus Neusser1 

(1)

Bern, Switzerland

14.1 Forecasting with Known Parameters

The discussion of forecasting with VAR models proceeds in two steps. First, we assume that the parameters of the model are known. Although this assumption is unrealistic, it will nevertheless allow us to introduce and analyze important concepts and ideas. In a second step, we then investigate how the results established in the first step have to be amended if the parameters are estimated. The analysis will focus on stationary and causal VAR(1) processes. Processes of higher order can be accommodated by rewriting them in companion form. Thus we have:

where $$\Psi _{j} = \Phi ^{j}$$. Consider then the following forecasting problem: Given observations {X T , X T−1, …, X ₁}, find a linear function, called predictor or forecast function, $$\mathbb{P}_{T}X_{T+h}$$, h ≥ 1, which minimizes the expected quadratic forecast error

Thereby “tr” denotes the trace operator which takes the sum of the diagonal elements of a matrix. As we rely on linear forecasting functions, $$\mathbb{P}_{T}X_{T+h}$$ can be expressed as

(14.1)

with matrices A ₁, A ₂, …, A T still to be determined. In order to simplify the exposition, we already accounted for the fact that the mean of {X t } is zero.¹ A justification for focusing on linear least-squares forecasts is given in Chap. 3 The first order conditions for the least-squares minimization problem are given by the normal equations:

These equations state that the forecast error $$(X_{T+h} - \mathbb{P}_{T}X_{T+h})$$ must be uncorrelated with the available information X s , s = 1, 2, …, T. The normal equations can be written as

Denoting by $$\boldsymbol{\Gamma }_{T}$$ the matrix

the normal equations can be written more compactly as

Using the assumption that {X t } is a VAR(1), $$\Gamma (h)$$ can be expressed as $$\Gamma (h) = \Phi ^{h}\Gamma (0)$$ (see Eq. (12.​3)) so that the normal equations become

The easily guessed solution is given by $$A_{1} = \Phi ^{h}$$ and $$A_{2} =\ldots = A_{T} = 0$$. Thus, the sought-after forecasting function for the VAR(1) process is

(14.2)

The forecast error $$X_{T+h} - \mathbb{P}_{T}X_{T+h}$$ has expectation zero. Thus, the linear least-squares predictor delivers unbiased forecasts. As

the expected squared forecast error (mean squared error) MSE(h) is

(14.3)

In order to analyze the case of a causal VAR(p) process with T > p, we transform the model into the companion form. For h = 1, we can apply the result above to get:

This implies that

(14.4)

The forecast error is $$X_{T+1} - \mathbb{P}_{T}X_{T+1} = Z_{t}$$ which has mean zero and covariance variance matrix $$\Sigma$$. In general we have that $$\mathbb{P}_{T}Y _{T+h} = \Phi ^{h}Y _{T}$$ so that $$\mathbb{P}_{T}X_{T+h}$$ is equal to

where $$\Phi _{i}^{(h)}$$, i = 1, …, p, denote the blocks in the first row of $$\Phi ^{h}$$. Alternatively, the forecast for h > 1 can be computed recursively. For h = 2 this leads to:

For h > 2 we proceed analogously. This way of producing forecasts is sometimes called iterated forecasts.

In general, the forecast error of a causal VAR(p) process can be expressed as

The MSE(h) then is:

(14.5)

Example

Consider again the VAR(2) model of Sect. 12.​3 The forecast function in this case is then:

Based on the results computed in Sect. 12.​3, we can calculate the corresponding mean squared errors (MSE):

A practical forecasting exercise with additional material is presented in Sect. 14.4.

14.1.1 Wold Decomposition Theorem

At this stage we note that Wold’s theorem or Wold’s Decomposition carries over to the multivariate case (see Sect. 3.​2 for the univariate case). This Theorem asserts that there exists for each purely non-deterministic stationary process² a decomposition, respectively representation, of the form:

where $$\Psi _{0} = I_{n}$$, $$Z_{t} \sim \mathrm{ WN}(0,\Sigma )$$ with $$\Sigma> 0$$ and $$\sum _{j=0}^{\infty }\|\Psi _{j}\|^{2} <\infty$$. The innovations {Z t } have the property $$Z_{t} = X_{t} -\widetilde{ \mathbb{P}}_{t-1}X_{t}$$ and consequently $$Z_{t} =\widetilde{ \mathbb{P}}_{t}Z_{t}$$. Thereby $$\widetilde{\mathbb{P}}_{t}$$ denotes the linear least-squares predictor based on the infinite past {X t , X t−1, …}. The interpretation of the multivariate case is analogous to the univariate one.

14.2 Forecasting with Estimated Parameters

In practice the parameters of the VAR model are usually unknown and have therefore to be estimated. In the previous Section we have demonstrated that

where $$\mathbb{P}_{T}X_{T+h-j} = Y _{T+h-j}$$ if j ≥ h. Replacing the true parameters by their estimates, we get the forecast function

where a hat indicates the use of estimates. The forecast error can then be decomposed into two components:

(14.6)

Dufour (1985) has shown that, under the assumption of symmetrically distributed Z t ’s (i.e. if Z t and − Z t have the same distribution) the expectation of the forecast error is zero even when the parameters are replaced by their least-squares estimates. This result holds despite the fact that these estimates are biased in small samples. Moreover, the results do not assume that the model is correctly specified in terms of the order p. Thus, under quite general conditions the forecast with estimated coefficients remains unbiased so that $$\mathbb{E}\left (X_{T+h} -\widehat{ \mathbb{P}}_{T}X_{T+h}\right ) = 0$$.

If the estimation is based on a different sample than the one used for forecasting, the two terms in the above expression are uncorrelated so that its mean squared error is by the sum of the two mean squared errors:

(14.7)

The last term can be evaluated by using the asymptotic distribution of the coefficients as an approximation. The corresponding formula turns out to be cumbersome. The technical details can be found in Lütkepohl (2006) and Reinsel (1993). The formula can, however, be simplified considerably if we consider a forecast horizon of only one period. We deduce the formula for a VAR of order one, i.e. taking $$X_{t} = \Phi X_{t-1} + Z_{t}$$, $$Z_{t} \sim \mathrm{ WN}(0,\Sigma )$$.

This implies that

Thereby, we have used the asymptotic normality of the least-squares estimator (see Theorem 13.​1) and the assumption that forecasting and estimation uses different realizations of the stochastic process. Thus, for h = 1 and p = 1, we get

Higher order models can be treated similarly using the companion form of VAR(p). In this case:

(14.8)

This is only an approximation as we applied asymptotic results to small sample entities. The expression shows that the effect of the substitution of the coefficients by their least-squares estimates vanishes as the sample becomes large. However, in small sample the factor $$\tfrac{T+np} {T}$$ can be sizeable. In the example treated in Sect. 14.4, the covariance matrix $$\Sigma$$, taking the use of a constant into account and assuming 8 lags, has to be inflated by $$\tfrac{T+np+1} {T} = \tfrac{196+4\times 8+1} {196} = 1.168$$. Note also that the precision of the forecast, given $$\Sigma$$, diminishes with the number of parameters.

14.3 Modeling of VAR Models

The previous section treated the estimation of VAR models under the assumption that the order of the VAR, p, is known. In most cases, this assumption is unrealistic as the order p is unknown and must be retrieved from the data. We can proceed analogously as in the univariate case (see Sect. 5.​1) and iteratively test the hypothesis that coefficients corresponding to the highest lag, i.e. $$\Phi _{p} = 0$$, are simultaneously equal to zero. Starting from a maximal order p max , we test the null hypothesis that $$\Phi _{p_{max}} = 0$$ in the corresponding VAR(p max ) model. If the hypothesis is not rejected, we reduce the order by one to p max − 1 and test anew the null hypothesis $$\Phi _{p_{max}-1} = 0$$ using the smaller VAR(p max − 1) model. One continues in this way until the null hypothesis is rejected. This gives, then, the appropriate order of the VAR. The different tests can be carried out either as Wald-tests (F-tests) or as likelihood-ratio tests (χ ²-tests) with n ² degrees of freedom.

An alternative procedure to determine the order of the VAR relies on some information criteria. As in the univariate case, the most popular ones are the Akaike (AIC) , the Schwarz or Bayesian (BIC) and the Hannan-Quinn criterion (HQC) . The corresponding formula are:

where $$\widetilde{\Sigma }_{p}$$ denotes the degree of freedom adjusted estimate of the covariance matrix $$\Sigma$$ for a model of order p (see equation(13.​5)). n ² p is the number of estimated coefficients. The estimated order is then given as the minimizer of one of these criteria. In practice the Akaike’s criterion is the most popular one although it has a tendency to deliver orders which are too high. The BIC and the HQ-criterion on the other hand deliver the correct order on average, but can lead to models which suffer from the omitted variable bias when the estimated order is too low. Examples are discussed in Sects. 14.4 and 15.4.5.

Following Lütkepohl (2006), Akaike’s information criterion can be rationalized as follows. Take as a measure of fit the determinant of the one period approximate mean-squared errors $$\widehat{\mathrm{MSE}}(1)$$ from Eq. (14.8) and take as an estimate of $$\Sigma$$ the degrees of freedom corrected version in Eq. (13.​5). The resulting criterion is called according to Akaike (1969) the final prediction error (FPE):

(14.9)

Taking logs and using the approximations $$\tfrac{T+np} {T-np} \approx 1 + \tfrac{2np} {T}$$ and $$\log (1 + \tfrac{2np} {T} ) \approx \tfrac{2np} {T}$$, we arrive at

14.4 Example: A VAR Model for the U.S. Economy

In this section, we illustrate how to build and use VAR models for forecasting key macroeconomic variables. For this purpose, we consider the following four variables: GDP per capita ({Y t }), price level in terms of the consumer price index (CPI) ({P t }), real money stock M1 ({M t }), and the three month treasury bill rate ({R t }). All variables are for the U.S. and are, with the exception of the interest rate, in logged differences.³ The components of X t are with the exception of the interest rate stationary.⁴ Thus, we aim at modeling $$X_{t} = (\Delta \log Y _{t},\Delta \log P_{t},\Delta \log M_{t},R_{t})'$$. The sample runs from the first quarter 1959 to the first quarter 2012. We estimate our models, however, only up to the fourth quarter 2008 and reserve the last thirteen quarters, i.e. the period from the first quarter 2009 to first quarter of 2012, for an out-of-sample evaluation of the forecast performance. This forecast assessment has the advantage to account explicitly of the sampling variability in estimated parameter models.

The first step in the modeling process is the determination of the lag-length. Allowing for a maximum of twelve lags, the different information criteria produce the values reported in Table 14.1. Unfortunately, the three criteria deliver different orders: AIC suggests 8 lags, HQ 5 lags, and BIC 2 lags. In such a situation it is wise to keep all three models and to perform additional diagnostic tests.⁵ One such test is to run a horse-race between the three models in terms of their forecasting performance.

Table 14.1

Information criteria for the VAR models of different orders

Order

AIC

BIC

HQ

0

−14.498

−14.429

−14.470

1

−17.956

−17.611

−17.817

2

−18.638

−18.016

−18.386

3

−18.741

−17.843

−18.377

4

−18.943

−17.768

−18.467

5

−19.081

−17.630

−18.493

6

−19.077

−17.349

−18.377

7

−19.076

−17.072

−18.264

8

−19.120

−16.839

−18.195

9

−18.988

−16.431

−17.952

10

−18.995

−16.162

−17.847

11

−18.900

−15.789

−17.639

12

−18.884

−15.497

−17.512

Minimum in bold

We evaluate the forecasts according to the two criteria: the root-mean-squared-error (RMSE) and the mean-absolute-error (MAE)⁶:

(14.10)

(14.11)

where $$\widehat{X}_{it}$$ and X it denote the forecast and the actual value of variable i in period t. Forecasts are computed for a horizon h starting in period T. We can gain further insights by decomposing the mean-squared-error additively into three components:

The first component measures how far the mean of the forecasts $$\frac{1} {h}\sum _{T+1}^{T+h}\widehat{X}_{ it}$$ is away from the actual mean of the data $$\overline{X}_{i}$$. It therefore measures the bias of the forecasts. The second one compares the standard deviation of the forecast $$\sigma _{\widehat{X}_{i}}$$ to those of the data $$\sigma _{X_{i}}$$. Finally, the last component is a measure of the unsystematic forecast errors where ρ denotes the correlation between the forecast and the data. Ideally, each of the three components should be close to zero: there should be no bias, the variation of the forecasts should correspond to those of the data, and the forecasts and the data should be highly positively correlated. In order to avoid scaling problems, all three components are usually expressed as a proportion of $$\frac{1} {h}\sum _{T+1}^{T+h}(\widehat{X}_{ it} - X_{it})^{2}$$:

(14.12)

(14.13)

(14.14)

We use these models to produce dynamic or iterated forecasts $$\mathbb{P}_{T}X_{T+1},\mathbb{P}_{T}X_{T+2},\ldots, \mathbb{P}_{T}X_{T+h}$$. Forecasts for h ≥ 2 are computed iteratively by inserting for the lagged variables the forecasts obtained in the previous steps. For details see Chap. 14 Alternatively, one may consider a recursive or rolling out-of-sample strategy where the model is reestimated each time a new observation becomes available. Thus, we would evaluate the one-period-ahead forecasts $$\mathbb{P}_{T}X_{T+1}, \mathbb{P}_{T+1}X_{T+2},\ldots, \mathbb{P}_{T+h-1}X_{T+h}$$, the two-period-ahead forecasts $$\mathbb{P}_{T}X_{T+2}, \mathbb{P}_{T+1}X_{T+3},\ldots, \mathbb{P}_{T+h-2}X_{T+h}$$, and so on. The difference between the recursive and the rolling strategy is that in the first case all observations are used for estimation whereas in the second case the sample is rolled over so that its size is kept fixed at T.

Figure 14.1 displays dynamic or iterated forecasts for the four variables expressed in log-levels, respectively in levels for the interest rate. Forecast are evaluated according to the performance measures explained above. The corresponding values are reported in Table 14.2. All models see a quick recovery after the recession in 2008 and are thus much too optimistic. The lowest RMSE for $$\log Y _{t}$$ is 5.678 for the VAR(8) model. Thus, GDP per capita is predicted to be on average almost 6 % too high over the forecast period. This overly optimistic forecast is reflected in a large bias proportion which amounts to more than 95 %. The situation looks much better for the price level. All models see an increase in inflation starting in 2009. Especially, the two higher order models fare much better. Their RMSE is just over 1 %. The bias proportion is practically zero for the VAR(8) model. The forecast results of the real money stock are mixed. All models predict a quick recovery. This took indeed place, but first at a more moderate pace. Starting in mid-2010 the unconventional monetary policy of quantitative easing, however, led to an unforeseen acceleration so that the forecasts turned out to be systematically too low for the later period. Interestingly, the smallest model fared significantly better than the other two. Finally, the results for the interest rates are very diverse. Whereas the VAR(2) model predicts a rise in the interest rate, the other models foresee a decline. The VAR(8) model even predicts a very drastic fall. However, all models miss the continuation of the low interest rate regime and forecasts an increase starting already in 2009. This error can again be attributed to the unforeseen low interest rate monetary policy which was implemented in conjunction with the quantitative easing. This misjudgement resulted in a relatively large bias proportion.

Fig. 14.1

Forecast comparison of alternative models. (a ) $$\log Y _{t}$$. (b ) $$\log P_{t}$$. (c ) $$\log M_{t}$$. (d ) R t

Table 14.2

Forecast evaluation of alternative VAR models

VAR(2)

VAR(5)

VAR(8)

RMSE

8.387

6.406

5.678

Bias proportion

0.960

0.961

0.951

Variance proportion

0.020

0.010

0.001

Covariance proportion

0.020

0.029

0.048

MAE

8.217

6.279

5.536

RMSE

3.126

1.064

1.234

Bias proportion

0.826

0.746

0.001

Variance proportion

0.121

0.001

0.722

Covariance proportion

0.053

0.253

0.278

MAE

2.853

0.934

0.928

RMSE

5.616

6.780

9.299

Bias proportion

0.036

0.011

0.002

Variance proportion

0.499

0.622

0.352

Covariance proportion

0.466

0.367

0.646

MAE

4.895

5.315

7.762

R t

RMSE

2.195

2.204

2.845

Bias proportion

0.367

0.606

0.404

Variance proportion

0.042

0.337

0.539

Covariance proportion

0.022

0.057

0.057

MAE

2.125

1.772

2.299

RMSE and MAE for $$\log Y _{t}$$, $$\log P_{t}$$, and $$\log M_{t}$$ are multiplied by 100

Up to now, we have just been concerned with point forecasts. Point forecasts, however, describe only one possible outcome and do not reflect the inherent uncertainty surrounding the prediction problem. It is, thus, a question of scientific integrity to present in addition to the point forecasts also confidence intervals. One straightforward way to construct such intervals is by computing the matrix of mean-squared-errors MSE using Eq. (14.5). The diagonal elements of this matrix can be interpreted as a measure of the forecast error variances for each variable. Under the assumption that the innovations {Z t } are Gaussian, such confidence intervals can be easily computed. However, in practice this assumption is likely to be violated. This problem can be circumvented by using the empirical distribution function of the residuals to implement a bootstrap method similar to the computation of the Value-at-Risk in Sect. 8.​4 Figure 14.2 plots the forecasts of the VAR(8) model together with a 80 % confidence interval computed from the bootstrap approach. It shows that, with the exception of the logged price level, the actual realizations fall out of the confidence interval despite the fact that the intervals are already relatively large. This documents the uniqueness of the financial crisis and gives a hard time for any forecasting model.

Fig. 14.2

Forecast of VAR(8) model and 80 % confidence intervals (red dotted lines). (a ) $$\log Y _{t}$$. (b ) $$\log P_{t}$$. (c ) $$\log M_{t}$$. (d ) R t

Instead of computing a confidence interval, one may estimate the probability distribution of possible future outcomes. This provides a complete description of the uncertainty related to the prediction problem (Christoffersen 1998; Diebold et al. 1998; Tay and Wallis 2000; Corradi and Swanson 2006). Finally, one should be aware that the innovation uncertainty is not the only source of uncertainty. As the parameters of the model are themselves estimated, there is also a coefficient uncertainty. In addition, we have to face the possibility that the model is misspecified.

The forecasting performance of the VAR models may seem disappointing at first. However, this was only be a first attempt and further investigations are usually necessary. These may include the search for structural breaks (See Bai et al. 1998; Perron 2006). This topic is treated in Sect. 18.​1 Another reason for the poor forecasting may be due to the over-parametrization of VAR models. The VAR(8) model, for example, 32 lagged dependent variables plus a constant in each of the four equations which leads to a total 132 parameters. This problem can be dealt with by applying Bayesian shrinkage techniques. This approach, also known as Bayesian VAR (BVAR), was particularly successful when using the so-called Minnesota prior (See Doan et al. 1984; Litterman 1986; Kunst and Neusser 1986; Banbura et al. 2010). This prior is presented in Sect. 18.​2

Besides these more fundamental issues, one may rely on more technical remedies. One such remedy is the use of direct rather iterated forecasts. This difference is best explained in the context of the VAR(1) model $$X_{t} = \Phi X_{t-1} + Z_{t}$$, $$Z_{t} \sim \mathrm{ WN}(0,\Sigma )$$. The iterated forecast for X T+h uses the OLS-estimate $$\widehat{\Phi }$$ to compute the forecast $$\widehat{\Phi }^{h}X_{T}$$ (see Chap. 14). Alternatively, one may estimate instead of the VAR(1), the model $$X_{t} = \Upsilon X_{t-h} + Z_{t}$$ and compute the direct forecast for X T+h as $$\widehat{\Upsilon }X_{T}$$. Although $$\widehat{\Upsilon }$$ has larger variance than $$\widehat{\Phi }$$ if the VAR(1) is correctly specified, it is robust to misspecification (see Bhansali 1999; Schorfheide 2005; Marcellino et al. 2006).

Another interesting and common device is intercept correction or residual adjustment. Thereby the constant terms are adjusted in such a way that the residuals of the most recent observation become zero. The model is thereby set back on track. In this way the forecaster can guard himself against possible structural breaks. Residual adjustment can also serve as a device to incorporate anticipated events, like announced policies, which are not yet incorporated into the model. See Clements and Hendry (1996, 2006) for further details and additional forecasting devices.

References

Akaike H (1969) Fitting autoregressive models for prediction. Ann Inst Stat Math 21:243–247CrossRef

Bai J, Lumsdaine RL, Stock JH (1998) Testing for and dating common breaks in multivariate time series. Rev Econ Stud 65:395–432CrossRef

Banbura M, Giannone D, Reichlin L (2010) Large Bayesian vector autoregressions. J Appl Econ 25:71–92CrossRef

Bhansali RJ (1999) Parameter estimation and model selection for multistep prediction of a time series: A review. In: Gosh S (ed) Asymptotics, Nonparametrics, and Time Series. Marcel Dekker, New York, pp 201–225

Christoffersen PF (1998) Evaluating interval forecasts. Int Econ Rev 39:841–862CrossRef

Clements MP, Hendry DF (1996) Intercept corrections and structural change. J Appl Econ 11:475–494CrossRef

Clements MP, Hendry DF (2006) Forecasting with breaks. In: Handbook of economic forecasting, vol 1, Elsevier, Amsterdam, pp 605–657

Corradi V, Swanson NR (2006) Predictive density evaluation. In: Handbook of economic forecasting, vol 1, Elsevier, Amsterdam, pp 197–284

Diebold FX, Gunther TA, Tay AS (1998) Evaluating density forecasts, with applications to financial risk management. Int Econ Rev 39:863–883CrossRef

Doan T, Litterman RB, Sims CA (1984) Forecasting and conditional projection using realistic prior distributions. Econ Rev 3:1–100CrossRef

Dufour JM (1985) Unbiasedness of predictions from estimated autoregressions. Econ Theory 1:387–402CrossRef

Kunst R, Neusser K (1986) A forecasting comparison of some var techniques. Int J Forecast 2:447–456CrossRef

Litterman RB (1986) Forecasting with Bayesian vector autoregressions: Five years of experience. J Bus Econ Stat 4:25–38

Lütkepohl H (2006) New introduction to multiple time series analysis. Springer, Berlin

Marcellino M, Stock JH, Watson MW (2006) A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series. J Econ 135:499–526CrossRef

Perron P (2006) Dealing with structural breaks. In: Hassani H, Mills TC, Patterson K (eds) Palgrave handbook in econometrics, vol 1. Econometric theory. Palgrave Macmillan, Hampshire, pp 278–352

Reinsel GC (1993) Elements of multivariate time series analysis. Springer Series in statistics. Springer, New YorkCrossRef

Schorfheide F (2005) VAR forecasting under misspecification. J Econ 128:99–136CrossRef

Sims CA, Stock JH, Watson MW (1990) Inference in linear time series with some unit roots. Econometrica 58:113–144CrossRef

Tay AS, Wallis KF (2000) Density forecasting: A Survey. J Forecast 19:235–254CrossRef

Footnotes

1

If the mean is non-zero, a constant A ₀ must be added to the forecast function.

2

A stationary stochastic process is called deterministic if it can be perfectly forecasted from its infinite past. It is called purely non-deterministic if there is no deterministic component (see Sect. 3.​2).

3

Thus, $$\Delta \log P_{t}$$ equals the inflation rate.

4

Although the unit root test indicate that R t is integrated of order one, we do not difference this variable. This specification will not affect the consistency of the estimates nor the choice of the lag-length (Sims et al. 1990), but has the advantage that each component of X t is expressed in percentage points which facilitates the interpretation.

5

Such tests would include an analysis of the autocorrelation properties of the residuals and tests of structural breaks.

6

Alternatively one could use the mean-absolute-percentage-error (MAPE). However, as all variables are already in percentages, the MAE is to be preferred.

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_15

15. Interpretation and Identification of VAR Models

Klaus Neusser1 

(1)

Bern, Switzerland

Although the estimation of VAR models poses no difficulties as outlined in the previous chapter, the individual coefficients are almost impossible to interpret. On the one hand, there are usually many coefficients, a VAR(4) model with three variables, for example, already has twelve coefficients per equation and thus 36 coefficients in total to interpret; on the other hand, there is in general no unambiguous relation of the VAR coefficients to the coefficients of a particular model. The last problem is known as the identification problem. To overcome this identification problem, many techniques have been developed which should allow to give the estimated VAR model an explicit economic interpretation.

15.1 Wiener-Granger Causality

As a first technique for the understanding of VAR processes, we analyze the concept of causality which was introduced by Granger (1969). The concept is also known as Wiener-Granger causality because Granger’s idea goes back to the work of Wiener (1956). Take a multivariate time series {X t } and consider the forecast of X 1, T+h , h ≥ 1, given X T , X T−1, … where {X t } has not only X 1t as a component, but also another variable or group of variables X 2t . X t may contain even further variables than X 1, t and X 2, t . The mean-squared forecast error is denoted by MSE₁(h). Consider now an alternative forecast of X 1, T+h given $$\widetilde{X}_{T},\widetilde{X}_{T-1},\ldots$$ where $$\{\widetilde{X}_{t}\}$$ is obtained from {X t } by eliminating the component X 2, t . The mean-squared error of this forecast is denoted by $$\widetilde{\mathrm{MSE}}_{1}(h)$$. According to Granger, we can say that the second variable X 2, t causes or is causal for X 1, t if and only if

This means that the information contained in {X 2t } and its past improves the forecast of {X 1t } in the sense of the mean-squared forecast error. Thus the concept of Wiener-Granger causality makes only sense for purely non-deterministic processes and rest on two principles¹:

The future cannot cause the past. Only the past can have a causal influence on the future.²

A specific cause contains information not available otherwise.

The concept of Wiener-Granger causality played an important role in the debate between monetarists and Keynesians over the issue whether the money stock has an independent influence on real activity. It turned out that this question can only be resolved within a specific context. Sims (1980a), for example, showed that the relationship between the growth rate of the money stock and changes in real activity depends on whether a short interest rate is accounted for in the empirical analysis or not. Another problem of the concept is that it is not unambiguously possible to infer a causal relationship just from the chronology of two variables as demonstrated by Tobin (1970). This and other conceptual issues (see Zellner (1979) and the discussion in the next chapter) and econometric problems (Geweke 1984) led to a decline in the practical importance of this concept.

We propose two econometric implementations of the causality concept. The first one is based on a VAR, the second one is non-parametric and uses the cross-correlations. In addition, we propose an interpretation in terms of the causal representation, respectively the Wold Decomposition Theorem (see Chap. 14)

15.1.1 VAR Approach

If one restricts oneself to linear least-squares forecasts, the above definition can be easily operationalized in the context of VAR models with only two variables (see also Sims 1972). Consider first a VAR(1) model. Then according to the explanations in Chap. 14 the one-period forecast is:

and therefore

If ϕ ₁₂ = 0 then the second variable does not contribute to the one-period forecast of the first variable and can therefore be omitted: $$\mathrm{MSE}_{1}(1) =\widetilde{\mathrm{ MSE}}_{1}(1)$$. Note that

where ∗ is a placeholder for an arbitrary number. Thus the second variable is not only irrelevant for the one-period forecast, but for any forecast horizon h ≥ 1. Thus, the second variable is not causal for the first variable in the sense of Wiener-Granger causality.

These arguments can be easily extended to VAR(p) models. According to Eq. (14.​4) we have that

where ϕ ij (k) denotes (i, j)-th element, i = 1, 2, of the matrix $$\Phi _{k}$$, k = 1, …, p. In order for the second variable to have no influence on the forecast of the first one, we must have that $$\phi _{12}^{(1)} =\phi _{ 12}^{(2)} =\ldots =\phi _{ 12}^{(p)} = 0$$. This implies that all matrices $$\Phi _{k}$$, k = 1, …, p, must be lower triangular, i.e. they must be of the form $$\left (\begin{array}{*{10}c} {\ast}&0\\ {\ast} &{\ast}\\ \end{array} \right )$$. As the multiplication and addition of lower triangular matrices is again a lower triangular matrix, the second variable is irrelevant in forecasting the first one at any forecast horizon. This can be seen by computing the corresponding forecast function recursively as in Chap. 14.

Based on this insight it is straightforward to test the null hypothesis that the second variable does not cause the first one within the VAR(p) context:

In terms of the VAR model this hypothesis can be stated as:

The alternative hypothesis is that the null hypothesis is violated. As the method of least-squares estimation leads under quite general conditions to asymptotically normal distributed coefficient estimates, it is straightforward to test the above hypothesis by a Wald-test (F-test). In the context of a VAR(1) model a simple t-test is also possible.

If more than two variables are involved the concept of Wiener-Granger causality is no longer so easy to implement. Consider for expositional purposes a VAR(1) model in three variables with coefficient matrix:

The one-period forecast function of the first variable then is

Thus, the third variable X 3T is irrelevant for the one-period forecast of the first variable. However, as the third variable has an influence on the second variable, ϕ ₂₃ ≠ 0, and because the second variable has an influence on the first variable, ϕ ₁₂ ≠ 0, the third variable will provide indirectly useful information for the forecast of the first variable for forecasting horizons h ≥ 2. Consequently, the concept of causality cannot immediately be extended from two to more than two variables.

It is, however, possible to merge variables one and two, or variables two and three, into groups and discuss the hypothesis that the third variable does not cause the first two variables, seen as a group; likewise that the second and third variable, seen as a group, does not cause the first variable. The corresponding null hypotheses then are:

Under these null hypotheses we get again lower (block-) triangular matrices:

Each of these hypotheses can again be checked by a Wald-test (F-test).

15.1.2 Wiener-Granger Causality and Causal Representation

We can get further insights into the concept of causality by considering a bivariate VAR, $$\Phi (\mathrm{L})X_{t} = Z_{t}$$, with causal representation $$X_{t} = \Psi (\mathrm{L})Z_{t}$$. Partitioning the matrices according to the two variables {X 1t } and {X 2t }, Theorem 12.​1 of Sect. 12.​3 implies that

where the polynomials $$\Phi _{12}(z)$$ and $$\Phi _{21}(z)$$ have no constant terms. The hypothesis that the second variable does not cause the first one is equivalent in this framework to the hypothesis that $$\Phi _{12}(z) = 0$$. Multiplying out the above expression leads to the condition

Because $$\Phi _{11}(z)$$ involves a constant term, the above equation implies that $$\Psi _{12}(z) = 0$$. Thus the causal representation is lower triangular. This means that the first variable is composed of the first shock, {Z 1t } only whereas the second variable involves both shocks {Z 1t } and {Z 2t }. The univariate causal representation of {X 1t } is therefore the same as the bivariate one.³ Finally, note the similarity to the issue of the identification of shocks discussed in subsequent sections.

15.1.3 Cross-Correlation Approach

In the case of two variables we also examine the cross-correlations to test for causality. This non-parametric test has the advantage that one does not have to rely on an explicit VAR model. This advantage becomes particularly relevant, if a VMA model must be approximated by a high order AR model. Consider the cross-correlations

If ρ ₁₂(h) ≠ 0 for h > 0, we can say that the past values of the second variable are useful for forecasting the first variable such that the second variable causes the first one in the sense of Wiener and Granger. Another terminology is that the second variable is a leading indicator for the first one. If in addition, ρ ₁₂(h) ≠ 0, for h < 0, so that the past values of the first variable help to forecast the second one, we have causality in both directions.

As the distribution of the cross-correlations of two independent variables depends on the autocorrelation of each variable, see Theorem 11.​4, Haugh (1976) and Pierce and Haugh (1977) propose a test based on the filtered time series. Analogously to the test for independence (see Sect. 11.​2), we proceed in two steps:

Step 1:

Estimate in the first step a univariate AR(p) model for each of the two time series {X 1t } and {X 2t }. Thereby chose p such that the corresponding residuals $$\{\hat{Z}_{1t}\}$$ and $$\{\hat{Z}_{2t}\}$$ are white noise. Note that although $$\{\hat{Z}_{1t}\}$$ and $$\{\hat{Z}_{2t}\}$$ are both not autocorrelated, the cross-correlations $$\rho _{Z_{1},Z_{2}}(h)$$ may still be non-zero for arbitrary orders h.

Step 2:

As $$\{\hat{Z}_{1t}\}$$ and $$\{\hat{Z}_{2t}\}$$ are the forecast errors based on forecasts which rely only on the own past, the concept of causality carries over from the original variables to the residuals. The null hypothesis that the second variable does not cause the first variable in the sense of Wiener and Granger can then be checked by the Haugh-Pierce statistic:

(15.1)

Thereby $$\hat{\rho }_{Z_{1},Z_{2}}^{2}(h)$$, h = 1, 2, …, denotes the squared estimated cross-correlation coefficients between $$\{\hat{Z}_{1t}\}$$ and $$\{\hat{Z}_{2t}\}$$. Under the null hypothesis that the second variable does not cause the first one, this test statistic is distributed as a χ ² distribution with L degrees of freedom.

15.2 Structural and Reduced Form

15.2.1 A Prototypical Example

The discussion in the previous section showed that the relation between VAR models and economic models is ambiguous. In order to better understand the quintessence of the problem, we first analyze a simple macroeconomic example. Let {y t } and {m t } denote the output and the money supply of an economy⁴ and suppose that the relation between the two variables is represented by the following simultaneous equation system:

Note the contemporaneous dependence of y t on m t in the AD-curve and a corresponding dependence of m t on y t in the policy reaction curve. These equations are typically derived from economic reasoning and may characterize a model explicitly derived from economic theory. In statistical terms the simultaneous equation system is called the structural form. The error terms {v yt } and {v mt } are interpreted as demand shocks and money supply shocks, respectively. They are called structural shocks and are assumed to follow a multivariate white noise process:

Note that the two structural shocks are assumed to be contemporaneously uncorrelated which is reflected in the assumption that $$\Omega $$ is a diagonal matrix. This assumption in the literature is uncontroversial. Otherwise, there would remain some unexplained relationship between them. The structural shocks can be interpreted as the statistical analog of an experiment in the natural sciences. The experiment corresponds in this case to a shift of the AD-curve due to, for example, a temporary non-anticipated change in government expenditures or money supply. The goal of the analysis is then to trace the reaction of the economy, in our case represented by the two variables {y t } and {m t }, to these isolated and autonomous changes in aggregate demand and money supply. The structural equations imply that the reaction is not restricted to contemporaneous effects, but is spread out over time. We thus represent this reaction by the impulse response function.

We can write the system more compactly in matrix notation:

or

where $$A = \left (\begin{array}{*{10}c} 1 &-a_{1} \\ -a_{2} & 1\\ \end{array} \right )$$, $$\Gamma = \left (\begin{array}{*{10}c} \gamma _{11} & \gamma _{12}\\ \gamma _{ 21} & \gamma _{22}\\ \end{array} \right )$$ and $$B = \left (\begin{array}{*{10}c} 1&0\\ 0 &1\\ \end{array} \right )$$.

Assuming that a ₁ a ₂ ≠ 1, we can solve the above simultaneous equation system for the two endogenous variables y t and m t to get the reduced form of the model:

Thus, the reduced form has the structure of a VAR(1) model with error term {Z t } = { (Z 1t , Z 2t )′}. The reduced form can also be expressed in matrix notation as:

where

Whereas the structural form represents the inner economic relations between the variables (economic model), the reduced form given by the VAR model summarizes their outer directly observable characteristics. As there is no unambiguous relation between the reduced and structural form, it is impossible to infer the inner economic relationships from the observations alone. This is known in statistics as the identification problem. Typically, a whole family of structural models is compatible with a particular reduced form. The models of the family are thus observationally equivalent to each other as they imply the same distribution for {X t }. The identification problem can be overcome if one is willing to make additional a priori assumptions. The nature and the type of these assumption and their interpretation is subject of the rest of this chapter.

In our example, the parameters characterizing the structural and the reduced form are

and

As there are eight parameters in the structural form, but only seven parameters in the reduced form, there is no one-to-one relation between structural and reduced form. The VAR(1) model delivers estimates for the seven reduced form parameters, but there is no way to infer from these estimates the parameters of the structural form. Thus, there is a fundamental identification problem.

The simple counting of the number of parameters in each form tells us that we need at least one additional restriction on the parameters of the structural form. The simplest restriction is a zero restriction. Suppose that a ₂ equals zero, i.e. that the central bank does not react immediately to current output. This seems reasonable because national accounting figures are usually released with some delay. With this assumption, we can infer the structural parameters from the reduced ones:

Remark 15.1.

Note that, because $$Z_{t} = A^{-1}BV _{t}$$, the reduced form disturbances Z t are a linear combination of the structural disturbances, in our case the demand disturbance v yt and the money supply disturbance v mt . In each period t the endogenous variables output y t and money supply m t are therefore hit simultaneously by both shocks. It is thus not possible without further assumptions to assign the movements in Z t and consequently in X t to corresponding changes in the fundamental structural shocks v yt and v mt .

Remark 15.2.

As Cooley and LeRoy (1985) already pointed out, the statement “money supply is not causal in the sense of Wiener and Granger for real economic activity”, which, in our example is equivalent to ϕ ₁₂ = 0, is not equivalent to the statement “money supply does not influence real economic activity“ because ϕ ₁₂ can be zero without a ₁ being zero. Thus, the notion of causality is not very meaningful in inferring the inner (structural) relationships between variables.

15.2.2 Identification: The General Case

We now present the general identification problem in the context of VAR.⁵ The starting point of the analysis consists of a linear model, derived ideally from economic theory, in its structural form:

(15.2)

where V t are the structural disturbances. These disturbances usually have an economic interpretation, for example as demand or supply shocks. A is a n × n matrix which is normalized such that the diagonal consists of ones only. The matrix B is also normalized such that its diagonal contains only ones. The process of structural disturbances, {V t }, is assumed to be a multivariate white noise process with a diagonal covariance matrix $$\Omega $$:

The assumption that the structural disturbance are uncorrelated with each other is not viewed as controversial as otherwise there would be unexplained relationships between them. In the literature one encounters an alternative completely equivalent normalization which leaves the coefficients in B unrestricted but assumes the covariance matrix of V t , $$\Omega $$, to be equal to the identity matrix I n .

The reduced form is obtained by solving the equation system with respect to X t . Assuming that A is nonsingular, the premultiplication of Eq. (15.2) by A ⁻¹ leads to the reduced form which corresponds to a VAR(p) model:

(15.3)

The relation between the structural disturbances V t and the reduced form disturbances Z t is in the form of a simultaneous equation system:

(15.4)

While the structural disturbances are not directly observed, the reduced form disturbances are given as the residuals of the VAR and can thus be considered as given. The relation between the lagged variables is simply

Consequently, once A and B have been identified, not only the coefficients of the lagged variables in the structural form are identified, but also the impulse response functions (see Sect. 15.4.1). We can therefore concentrate our analysis of the identification problem on Eq. (15.4).

With these preliminaries, it is now possible to state the identification problem more precisely. Equation (15.4) shows that the structural form is completely determined by the parameters $$(A,B,\Omega )$$. Taking the normalization of A and B into account, these parameters can be viewed as points in $$\mathbb{R}^{n(2n-1)}$$. These parameters determine the distribution of $$Z_{t} = A^{-1}BV _{t}$$ which is completely characterized by the covariance matrix of Z t , $$\Sigma $$, as the mean is equal to zero.⁶ Thus, the parameters of the reduced form, i.e. the independent elements of $$\Sigma $$ taking the symmetry into account, are points in $$\mathbb{R}^{n(n+1)/2}$$. The relation between structural and reduced form can therefore be described by a function $$g: \mathbb{R}^{n(2n-1)} \rightarrow \mathbb{R}^{n(n+1)/2}$$:

(15.5)

Ideally, one would want to find the inverse of this function and retrieve, in this way, the structural parameters $$(A,B,\Omega )$$ from $$\Sigma $$. This is, however, in general not possible because the dimension of the domain space of g, n(2n − 1), is strictly greater, for n ≥ 2, than the dimension of its range space, $$n(n + 1)/2$$. This discrepancy between the dimensions of the domain and the range space of g is known as the identification problem. To put it in another way, there are only $$n(n + 1)/2$$ (nonlinear) equations for n(2n − 1) unknowns.⁷

To overcome the identification problem, we have to bring in additional information. A customary approach is to impose a priori assumptions on the structural parameters. The Implicit Function Theorem tells us that we need

(15.6)

such restrictions, so-called identifying restrictions, to be able to invert the function g. Note that this is only a necessary condition and that the identification problem becomes more severe as the dimension of the VAR increases because the number of restrictions grows at a rate proportional to n ².

This result can also be obtained by noting that the function g in Eq. (15.5) is invariant under the following transformation h:

where R, Q and D are arbitrary invertible matrices such that R respects the normalization of A and B, Q is an orthogonal matrix, and D is a diagonal matrix. It can be verified that

The dimensions of the matrices R, Q, and D are n ² − 2n, $$n(n - 1)/2$$, and n, respectively. Summing up gives $$3n(n - 1)/2 = n^{2} - 2n + n(n - 1)/2 + n$$ degrees of freedom as before⁸.

The empirical economics literature proposed several alternative identification schemes:

(i)

Short-run restrictions place restrictions, usually zero restrictions, on the immediate impact of structural shocks (among many others, Sims 1980b; Blanchard 1989; Blanchard and Watson 1986; Christiano et al. 1999). See Sect. 15.3 for details.

(ii)

Long-run restrictions place restrictions, usually zero-restrictions, on the long–run impact structural shocks (Blanchard and Quah 1989; Galí 1992). See Sect. 15.5 for details.

(iii)

Maximization of the contribution to the forecast error variance of some variable at some horizon with respect to a particular shock (Faust 1998; Uhlig 2004; Francis et al. 2014, Uhlig, 2003, What drives real GNP? unpublished). This method has seen an interesting application in the identification of news shocks (see Barsky and Sims 2011). Further details will discussed in Sect. 15.4.2.

(iv)

Sign restrictions restrict the set of possible impulse response functions (see Sect. 15.4.1) to follow a given sign pattern (Faust 1998; Uhlig 2005; Fry and Pagan 2011; Kilian and Murphy 2012; Rubio-Ramírez et al. 2010; Arias et al. 2014; Baumeister and Hamilton 2015). This approach is complementary to the two previous identification schemes and will be discussed in Sect. 15.6.

(v)

Identification through heteroskedasticity (Rigobon 2003)

(vi)

Restrictions derived from a dynamic stochastic general equilibrium (DSGE) model. These restrictions often come as nonlinear cross-equation restrictions and are viewed as the hallmark of rational expectations models (Hansen and Sargent 1980). Typically, the identification issue is overcome by imposing a priori restrictions via a Bayesian approach (Negro and Schorfheide (2004) among many others).

(vii)

Identification using information on global versus idiosyncratic shocks in the context of multi-country or multi-region VAR models (Canova and Ciccarelli 2008; Dees et al. 2007)

(viii)

Instead of identifying all parameters, researchers may be interested in identifying only one equation or a subset of equations. This case is known as partial identification. The schemes presented above can be extended in a straightforward manner to the partial identification case.

These schemes are not mutually exclusive, but can be combined with each other. In the following we will only cover the identification through short- and long-run restrictions, because these are by far the most popular ones. The economic importance of these restrictions for the analysis of monetary policy has been emphasized by Christiano et al. (1999).

15.2.3 Identification: The Case n = 2

Before proceeding further, it is instructive to analyze the case n = 2 in more detail.⁹ Assume for simplicity A = I ₂, then the equation system (15.5) can be written explicitly as

with unknowns (B)₁₂, (B)₂₁, ω ₁ ², and ω ₂ ². Note that the assumption of $$\Sigma $$ being positive definite implies that (B)₁₂(B)₂₁ ≠ 1. Thus, we can solve out ω ₁ ² and ω ₂ ² and reduce the three equations to only one:

(15.7)

where b ₂₁ and b ₁₂ denote the least-squares regression coefficients of Z 2t on Z 1t , respectively of Z 1t on Z 2t , i.e. $$b_{21} =\sigma _{12}/\sigma _{1}^{2}$$ and $$b_{12} =\sigma _{12}/\sigma _{2}^{2}$$. r ₁₂ is the correlation coefficient between Z 2t and Z 1t , i.e. $$r_{12} =\sigma _{12}/(\sigma _{1}\sigma _{2})$$. Note that imposing a zero restriction by setting (B)₁₂, for example, equal to zero, implies that (B)₂₁ equals b ₂₁; and vice versa, setting (B)₂₁ = 0, implies (B)₁₂ = b ₁₂. As a final remark, the right hand side of Eq. (15.7) is always positive as the inverse of the squared correlation coefficient is bigger than one. This implies both product terms must be of the same sign.

Equation (15.7) delineates all possible combinations of (B)₁₂ and (B)₂₁ which are compatible with a given covariance matrix $$\Sigma $$. Its graph represents a rectangular hyperbola in the parameter space ((B)₁₂, (B)₂₁) with center $$C = (b_{21}^{-1},b_{12}^{-1})$$ and asymptotes $$(B)_{12} = b_{21}^{-1}$$ and $$(B)_{21} = b_{12}^{-1}$$ and is plotted in Fig. 15.1.¹⁰ The hyperbola consist of two disconnected branches with a pole at the center $$C = (b_{21}^{-1},b_{12}^{-1})$$. At this point, the relation between the two parameters changes sign. The figure also indicates the two possible zero restrictions (B)₁₂ = 0 and (B)₂₁ = 0, called short-run restrictions. These two restrictions are connected and its path completely falls within one quadrant. Thus, along this path the sign of the parameters remain unchanged.

Fig. 15.1

Identification in a two-dimensional structural VAR with $$\sigma _{12} > 0$$

Suppose that instead of fixing a particular parameter, we only want to restrict its sign. Assuming that (B)₁₂ ≥ 0 implies that (B)₂₁ must lie in one of the two disconnected intervals $$(-\infty,b_{21}]$$ and $$(b_{12}^{-1},+\infty )$$.¹¹ Although not very explicit, some economic consequences of this topological particularity are discussed in Fry and Pagan (2011). Alternatively, assuming (B)₁₂ ≤ 0 implies (B)₂₁ ∈ [b ₂₁, b ₁₂ ⁻¹). Thus, (B)₂₁ is unambiguously positive. Sign restrictions for (B)₂₁ can be discussed in a similar manner. Section 15.6 discuses sign restrictions more explicitly.

15.3 Identification via Short-Run Restrictions

Short-run restrictions represent the most common identification scheme encountered in practice. They impose direct linear restrictions on the structural parameters A and B and restrict in this way the contemporaneous effect of the structural shocks on the variables of the system. The most common type of such restrictions are zero restrictions which set certain coefficients a priori to zero. These zero restrictions are either derived from an explicit economic theory or are based on some ad hoc arguments. As explained above, it is necessary to have at least $$3n(n - 1)/2$$ restrictions at hand. If there are more restrictions, we have an overidentified system. This is, however, rarely the case in practice because the number of necessary restrictions grows at a rate proportional to n ². The case of overidentification is, thus, not often encountered and as such is not treated.¹² The way to find appropriate restrictions in a relatively large system is documented in Sect. 15.4.5. If the number of restrictions equals $$3n(n - 1)/2$$, we say that the system is exactly identified.

Given the necessary number of a priori restrictions on the coefficients A and B, there are two ways to infer A, B, and $$\Omega $$. The first one views the relation (15.4) as a simultaneous equation system in Z t with error terms V t and to estimate the coefficients by instrumental variables as in Blanchard and Watson (1986).¹³ The second way relies on the method of moments and solves the nonlinear equation system (15.5) as in Bernanke (1986). In the case of exact identification both methods are numerically equivalent.

In our example treated of Sect. 15.2.1, we had n = 2 so that three restrictions were necessary (six parameters, but only three equations). These three restrictions were obtained by setting B = I ₂ which gives two restrictions (i.e. $$b_{12} = b_{21} = 0$$). The third restriction was to set the immediate reaction of money supply to a demand shock to zero, i.e. to set a ₂ = 0. We then showed that these three restrictions are also sufficient to solve the nonlinear equation system (15.5).

Sims (1980b) proposed in his seminal article the VAR approach as an adequate alternative to then popular structural simultaneous approach. In particular, he suggested a simple recursive identification scheme. This scheme takes A = I n so that the equation system (15.5) simplifies to:

Next we assume that B is a lower triangular matrix:

where ∗ is just a placeholder. The matrices B and $$\Omega $$ are uniquely determined by the Cholesky decomposition of the matrix $$\Sigma $$. The Cholesky decomposition factorizes a positive-definite matrix $$\Sigma $$ uniquely into the product $$B\Omega B'$$ where B is a lower triangular matrix with ones on the diagonal and a diagonal matrix $$\Omega $$ with strictly positive diagonal entries (Meyer 2000). As Z t  = BV t , Sims’ identification gives rise to the following interpretation. v 1t is the only structural shock which has an effect on X 1t in period t. All other shocks have no contemporaneous effect. Moreover, Z 1t  = v 1t so that the residual from the first equation is just equal to the first structural shock and that $$\sigma _{1}^{2} =\omega _{ 1}^{2}$$. The second variable X 2t is contemporaneously only affected by v 1t and v 2t , and not by the remaining shocks v 3t , …, v nt . In particular, $$Z_{2t} = b_{21}v_{1t} + v_{2t}$$ so that b ₂₁ can be retrieved from the equation $$\sigma _{21} = b_{21}\omega _{1}^{2}$$. This identifies the second structural shock v 2t and ω ₂ ². Due to the triangular nature of B, the system is recursive and all structural shocks and parameters can be identified successively. The application of the Cholesky decomposition as an identification scheme rests crucially on the ordering of the variables (X 1t , X 2t , …, X nt )′ in the system.

Sims’ scheme, although easy to implement, becomes less plausible as the number of variables in the system increases. For this reason the more general scheme with A ≠ I n and B not necessarily lower triangular are more popular. However, even for medium sized systems such as n = 5, 30 restrictions are necessary which stresses the imagination even of brilliant economists as the estimation of Blanchard’s model in Sect. 15.4.5 shows.

Focusing on the identification of the matrices A and B brings also an advantage in terms of estimation. As shown in Chap. 13, the OLS-estimator of the VAR coefficient matrices $$\Phi _{1},\ldots,\Phi _{p}$$ equals the GLS-estimator independently of $$\Sigma $$. Thus, the estimation of the structural parameters can be broken down into two steps. In the first step, the coefficient matrices $$\Phi _{1},\ldots,\Phi _{p}$$ are estimated using OLS. The residuals are then used to estimate $$\Sigma $$ which leads to an estimate of the covariance matrix (see Eq. (13.​6)). In the second step, the coefficients of A, B, and $$\Omega $$ are then estimated given the estimate of $$\Sigma $$, $$\widehat{\Sigma }$$, by solving the nonlinear equation system (15.5) taking the specific identification scheme into account. Thereby $$\Sigma $$ is replaced by its estimate $$\widehat{\Sigma }$$. As $$\sqrt{T}\left (\mathrm{vech}(\widehat{\Sigma }) -\mathrm{ vech}(\Sigma )\right )$$ converges in distribution to a normal distribution with mean zero, $$\hat{A}$$, $$\hat{B}$$ and $$\widehat{\Omega }$$ are also asymptotically normal because they are obtained by a one-to-one mapping from $$\widehat{\Sigma }$$.¹⁴ The Continuous Mapping Theorem further implies that $$\hat{A}$$, $$\hat{B}$$ and $$\widehat{\Omega }$$ converge to their true means and that their asymptotic covariance matrix can be obtained by an application of the delta-method (see Theorem E.1 in the Appendix E). Further details can be found in Bernanke (1986), Blanchard and Watson (1986), Giannini (1991), Hamilton (1994b), and Sims (1986).

15.4 Interpretation of VAR Models

15.4.1 Impulse Response Functions

The direct interpretation of VAR models is rather difficult because it is composed of many coefficients so that it becomes difficult to understand the dynamic interactions between the variables. It is therefore advantageous to simulate the dynamic effects of the different structural shocks by computing the impulse response functions. They show the effect over time of the structural shocks on the variables at issue. These effects can often be related to the underlying economic model and are thus at the heart of the VAR analysis. The examples in Sect. 15.4.4 and 15.4.5 provide some illustrations of this statement.

The impulse response functions are derived from the causal representation¹⁵ of the VAR process (see Sect. 12.​3):

(15.8)

The effect of the j-th structural disturbance on the i-th variable after h periods, denoted by $$\frac{\partial X_{i,t+h}} {\partial v_{jt}}$$ is thus given by the (i, j)-th element of the matrix $$\Psi _{h}A^{-1}B$$:

Clearly, the impulse response functions depends on the identification scheme chosen. There are n ² impulse response functions if the system consists of n variables. Usually, the impulse response functions are represented graphically as a plot against h.

15.4.2 Forecast Error Variance Decomposition

Another instrument for the interpretation of VAR models is the forecast error variance decomposition (“FEVD”) or variance decomposition for short which decomposes the total forecast error variance of a variable into the variances of the structural shocks. It is again based on the causal representation of the VAR(p) model. According to Eq. (14.​3) in Chap. 14 the variance of the forecast error or mean squared error (MSE) is given by:

Given a specific identification scheme and estimates of the structural parameters, it is possible to attribute the MSE to the variance of the structural disturbances. Thereby it is customary to write the contribution of each disturbance as a percentage of the total variance. For this purpose let us write the MSE(h) explicitly as

Our interest lies exclusively on the variances, m ii (h), i = 1, …, n, so that we represent the uninteresting covariance terms by the placeholder ∗. These variances can be seen as a linear combination of the ω i ²’s because the covariance matrix of the structural disturbances $$\Omega =\mathrm{ diag}\left (\omega _{1}^{2},\ldots,\omega _{n}^{2}\right )$$ is a diagonal matrix:

or in matrix form

where the vector e i has entries equal to zero, except for the i-th entry which is equal to one. Given the positive definiteness of $$\Sigma $$, the weights d ij (h), i, j = 1, …, n and h = 1, 2, …, are strictly positive. They can be computed as

In order to arrive at the percentage value of the contribution of the j-th disturbance to the MSE of the i-th variable at forecast horizon h, denoted by f ij (h), we divide each summand in the above expression by the total sum:

The corresponding matrix expression is

Usually, these numbers are multiplied by 100 to give percentages and are either displayed graphically as a plot against h or in table form (see the example in Sect. 15.4.5). The forecast error variance f ij (h) thus shows which percentage of the forecast variance of variable i at horizon h can be attributed to the j-th structural shock and thus measures the contribution of each of these shocks to the overall fluctuations of the variables in question.

The FEVD can be used as an alternative identification scheme, sometimes called max share identification. Assume for the ease of exposition that A = I n . The VAR disturbances and the structural shocks are then simply related as Z t  = BV t (compare Eq. (15.4)). Moreover, take $$\Omega = I_{n}$$, but leave B unrestricted. This corresponds to a different, but equivalent normalization which economizes on the notation. Then the j-th structural disturbance can be identified by assuming that it maximizes the forecast error variance share with respect to variable i. Noting that, given $$\Sigma $$, B can be written as B = RQ with R being the unique Cholesky factor of $$\Sigma $$ and Q being an orthogonal matrix, this optimization problem can be casted as

where q j is the j-th column of Q, i.e. q j  = Qe j . The constraint q j ′q j  = 1 normalizes the vector to have length 1. From Z t  = BV t it then follows that corresponding structural disturbance is equal to $$V _{jt} = q_{j}'R^{-1}Z_{t}$$. Because $$e_{j}'Q'R^{-1}\Sigma R'^{-1}Qe_{k} = 0$$ for j ≠ k, this shock is orthogonal to the other structural disturbances. For practical applications it is advisable for reasons of numerical stability to transform to optimization problem into an equivalent eigenvalue problem (see Faust 1998, appendix).

15.4.3 Confidence Intervals

The impulse response functions and the variance decomposition are the most important tools for the analysis and interpretation of VAR models. It is, therefore, of importance not only to estimate these entities, but also to provide corresponding confidence intervals to underpin the interpretation from a statistical perspective. In the literature two approaches have been established: an analytic and a bootstrap approach. The analytic approach relies on the fact that the coefficient matrices $$\Psi _{h}$$, h = 1, 2, …, are continuously differentiable functions of the estimated VAR coefficients: $$\mathrm{vec}(\Psi _{h}) = F_{h}(\beta )$$ where β denotes as in Chap. 13 the vectorized form of the VAR coefficient matrices $$\Phi _{1},\ldots,\Phi _{p}$$.¹⁶ The relation between VAR coefficients and the causal representation (MA($$\infty $$) representation) was established in Sect. 12.​3. This discussion shows that the functions $$F_{h}: \mathbb{R}^{pn^{2} }\longrightarrow \mathbb{R}^{n^{2} }$$ are highly nonlinear. In Chap. 13 it was shown that $$\sqrt{ T}\left (\hat{\beta }-\beta \right )\mathop{\longrightarrow}\limits_{}^{\mathrm{\quad d\quad }}\mathrm{N}\left (0,\Sigma \otimes \Gamma _{p}^{-1}\right )$$ so that we can apply the Continuous Mapping Theorem (see Theorem E.1 or Serfling (1980, 122–124)), sometimes also called the Delta method to get

where in practice the covariance matrix is estimated by replacing β and $$\Sigma $$ by their estimate. The computation of the gradient matrices $$\frac{\partial F_{h}(\beta )} {\partial \beta '}$$ is rather involved, especially when h becomes large. Details can be found in Lütkepohl (1990, 2006) and Mittnik and Zadrozny (1993).

The use of this asymptotic approximation has two problems. First, the complexity of the relationship between the $$\Phi _{i}$$’s and the $$\Psi _{h}$$’s augments with h so that the quality of the approximation diminishes with h for any given sample size. This is true even when $$\hat{\beta }$$ is exactly normally distributed. Second, the distribution of $$\hat{\beta }$$ is approximated poorly by the normal distribution. This is particularly relevant when the roots of $$\Phi (\mathrm{L})$$ are near the unit circle. In this case, the bias towards zero can become substantial (see the discussion in Sect. 7.​2). These two problems become especially relevant as h increases. For these reasons the analytic approach has becomes less popular.

The bootstrap approach (Monte Carlo or Simulation approach), as advocated by Runkle (1987), Kilian (1998) and Sims (1999), has become the most favored approach. This is partly due to the development of powerful computer algorithms, and the increased speed in computations. The so-called naive bootstrap approach consists of several steps.¹⁷

First step:

Using a random number generator new disturbances are created. This can be done in two ways. The first one assumes a particular distribution for V t : $$V _{t} \sim \mathrm{ N}(0,\widehat{\Omega })$$, for example. The realizations V ₁, …, V T are then independent draws from this distribution. The second one, takes random draws with replacement from the identified realizations $$\hat{V }_{1},\ldots,\hat{V }_{T}$$.¹⁸ The second way has the advantage that non explicit distributional assumption is made which results in a better approximation of the true distribution of $$\hat{V }_{t}$$.

Second step:

Given the fixed starting values $$X_{-p+1},\ldots,X_{0}$$, the estimated coefficients matrices $$\widehat{\Phi }_{1},\ldots,\widehat{\Phi }_{p}$$ and the new disturbances drawn in step one, a new realization of the time series for {X t } is generated.

Third step:

Estimate the VAR model, given the newly generated realizations for {X t }, to obtain new estimates for the coefficient matrices.

Fourth step:

Generate a new set of impulse response functions given the new estimates, taking the identification scheme as fixed.

The steps one to four are repeated several times to generate a whole family of impulse response functions which form the basis for the computation of the confidence bands. In many applications, these confidence bands are constructed in a naive fashion by connecting the confidence intervals for individual impulse responses at different horizons. This, however, ignores the fact that the impulse responses at different horizons are correlated which implies that the true coverage probability of the confidence band is different from the presumed one. Thus, the joint probability distribution of the impulse responses should serve as the basis of the computation of the confidence bands. Recently, several alternatives have been proposed which take this feature in account. Lütkepohl et al. (2013) provides a comparison of several methods.

The number of repetitions should be at least 500. The method can be refined somewhat if the bias towards zero of the estimates of the $$\Phi $$’s is taken into account. This bias can again be determined through simulation methods (Kilian 1998). A critical appraisal of the bootstrap can be found in Sims (1999) where additional improvements are discussed. The bootstrap of the variance decomposition works in similar way.

15.4.4 Example 1: Advertisement and Sales

In this example we will analyze the dynamic relationship between advertisement expenditures and sales by a VAR approach. The data we will use are the famous data from the Lydia E. Pinkham Medicine Company which cover yearly observations from 1907 to 1960. These data were among the first ones which have been used to quantify the effect of advertisement expenditures on sales. The data are taken from Berndt (1991, Chapter 8) where details on the specificities of the data and a summary of the literature can be found.

We denote the two-dimensional logged time series of advertisement expenditures and sales by $$\{X_{t}\} =\{ (\ln (\mathrm{advertisement}_{t}),\ln (\mathrm{sales}_{t}))'\}$$. We consider VAR models of order one to six. For each VAR(p), p = 1, 2, …, 6, we compute the corresponding information criteria AIC and BIC (see Sect. 14.​3). Whereas AIC favors a model of order two, BIC proposes the more parsimonious model of order one. To be on the safe side, we work with a VAR model of order two whose estimates are reported below:

where the estimated standard deviations of the coefficients are reported in parenthesis. The estimate $$\widehat{\Sigma }$$ of the covariance matrix $$\Sigma $$ is

The estimated VAR(2) model is taken to be the reduced form model. The structural model contains two structural shocks: a shock to advertisement expenditures, V At , and a shock to sales, V St . The disturbance vector of the structural shock is thus {V t } = { (V At , V St )′}. It is related to Z t via relation (15.4), i.e. AZ t  = BV t . To identify the model we thus need 3 restrictions.¹⁹ We will first assume that A = I ₂ which gives two restrictions. A plausible further assumption is that shocks to sales have no contemporaneous effects on advertisement expenditures. This zero restriction seems justified because advertisement campaigns have to be planed in advance. They cannot be produced and carried out immediately. This argument then delivers the third restriction as it implies that B is a lower triangular. This lower triangular matrix can be obtained from the Cholesky decomposition of $$\widehat{\Sigma }$$:

The identifying assumptions then imply the impulse response functions plotted in Fig. 15.2.

Fig. 15.2

Impulse response functions for advertisement expenditures and sales with 95-% confidence intervals computed using the bootstrap procedure

The upper left figure shows the response of a sudden transitory increase in advertisement expenditures by 1 % (i.e. of a 1-% increase of V At ) to itself. This shock is positively propagated to the future years, but is statistically zero after four years. After four years the shock even changes to negative, but statistically insignificant, expenditures. The same shock produces an increase of sales by 0.3 % in the current and next year as shown in the lower left figure. The effect then deteriorates and becomes even negative after three years. The right hand figures display the reaction of a sudden transitory increase of sales by 1 %. Again, we see that the shock is positively propagated. Thereby the largest effect is reached after two years and then declines monotonically. The reaction of advertisement expenditures is initially equal to zero by construction as it corresponds to the identifying assumption with regard to B. Then, the effect starts to increase and reaches a maximum after three years and then declines monotonically. After 15 years the effect is practically zero. The 95-% confidence intervals are rather large so that all effects are no longer statistically significant after a few number of years.

15.4.5 Example 2: IS-LM Model with Phillips Curve

In this example we replicate the study of Blanchard (1989) which investigates the US business cycle within a traditional IS-LM model with Phillips curve.²⁰ The starting point of his analysis is the VAR(p) model:

where {X t } is a five-dimensional time series X t  = (Y t , U t , P t , W t , M t )′. The individual elements of X t denote the following variables:

The VAR has attached to it a disturbance term Z t  = (Z yt , Z ut , Z pt , Z wt , Z mt )′. Finally, {D t } denotes the deterministic variables of the model such as a constant, time trend or dummy variables. In the following, we assume that all variables are stationary.

The business cycle is seen as the result of five structural shocks which impinge on the economy:

We will use the IS-LM model to rationalize the restrictions so that we will be able to identify the structural form from the estimated VAR model. The disturbance of the structural and the reduced form models are related by the simultaneous equation system:

where V t  = (V yt , V st , V pt , V wt , V mt )′ and where A and B are 5 × 5 matrices with ones on the diagonal. Blanchard (1989) proposes the following specification:

In matrix notation the above simultaneous equation system becomes:

The first equation is interpreted as an aggregate demand (AD) equation where the disturbance term related to GDP growth, Z yt , depends on the demand shock V dt and the supply shock V st . The second equation is related to Okun’s law (OL) which relates the unemployment disturbance Z ut to the demand disturbance and the supply shock. Thereby an increase in GDP growth reduces unemployment in the same period by a ₂₁ whereas a supply shock increases it. The third and the fourth equation represent a price (PS) and wage setting (WS) system where wages and prices interact simultaneously. Finally, the fifth equation (MR) is supposed to determine the money shock (MR). No distinction is made between money supply and money demand shocks. A detailed interpretation of these equations is found in the original article by Blanchard (1989).

Given that the dimension of the system is five (i.e. n = 5), formula (15.6) instructs us that we need $$3 \times (5 \times 4)/2 = 30$$ restrictions. Counting the number of zero restrictions implemented above, we see that we only have 28 zeros. Thus we lack two additional restrictions. We can reach the same conclusion by counting the number of coefficients and the number of equations. The coefficients are a ₂₁, a ₃₁, a ₃₄, a ₄₂, a ₄₃, a ₅₁, a ₅₂, a ₅₃, a ₅₄, b ₁₂, b ₃₂, b ₄₂ and the diagonal elements of $$\Omega $$, the covariance matrix of V t . We therefore have to determine 17 unknown coefficients out of $$(5 \times 6)/2 = 15$$ equations. Thus we find again that we are short of two restrictions. Blanchard discusses several possibilities among which the restrictions b ₁₂ = 1. 0 and a ₃₄ = 0. 1 seem most plausible.

The sample period runs through the second quarter in 1959 to the second quarter in 2004 encompassing 181 observations. Following Blanchard, we include a constant in combination with a linear time trend in the model. Whereas BIC suggests a model of order one, AIC favors a model of order two. As a model of order one seems rather restrictive, we stick to the VAR(2) model whose estimated coefficients are reported below²¹:

The first column of $$\widehat{C}$$ relates to the constants, whereas the second column gives the coefficients of the time trend. From these estimates and given the identifying restrictions established above, the equation $$\widehat{\Sigma } = A^{-1}B\Omega B'A'^{-1}$$ uniquely determines the matrices A, B and $$\Omega $$:

In order to give better interpretation of the results we have plotted the impulse response functions and their 95-% confidence bands in Fig. 15.3. The results show that a positive demand shock has only a positive and statistically significant effect on GDP growth in the first three quarters, after that the effect becomes even slightly negative and vanishes after sixteen quarters. The positive demand shock reduces unemployment significantly for almost fifteen quarters. The maximal effect is achieved after three to four quarters. Although the initial effect is negative, the positive demand shock also drives inflation up which then pushes up wage growth. The supply shock also has a positive effect on GDP growth, but it takes more than four quarters before the effect reaches its peak. In the short-run the positive supply shock even reduces GDP growth. In contrast to the demand shock, the positive supply shock increases unemployment in the short-run. The effect will only reduce unemployment in the medium- to long-run. The effect on price and wage inflation is negative.

Fig. 15.3

Impulse response functions for the IS-LM model with Phillips curve with 95-% confidence intervals computed using the bootstrap procedure (compare with Blanchard (1989))

Finally, we compute the forecast error variance decomposition according to Sect. 15.4.2. The results are reported in Table 15.1. In the short-run, the identifying restrictions play an important role as reflected by the plain zeros. The demand shock accounts for almost all the variance of GDP growth in the short-run. The value of 99. 62 % for forecast horizon of one quarter, however, diminishes as h increases to 40 quarters, but still remains with a value 86. 13 very high. The supply shock on the contrary does not explain much of the variation in GDP growth. Even for a horizon of 40 quarters, it contributes only 6.11 %. The supply shock is, however, important for the variation in the unemployment rate, especially in the short-run. It explains more than 50 % whereas demand shocks account for only 42.22 %. Its contribution diminishes with the increase of the forecast horizon giving room for price and wage shocks. The variance of the inflation rate is explained in the short-run almost exclusively by price shocks. However, as the forecast horizon is increased supply and wage shocks become relatively important. The money growth rate does not interact much with the other variables. Its variation is almost exclusively explained by money shocks.

Table 15.1

Forecast error variance decomposition (FEVD) in terms of demand, supply, price, wage, and money shocks (percentages)

Horizon

Demand

Supply

Price

Wage

Money

Growth rate of real GDP

1

99.62

0.38

0

0

0

2

98.13

0.94

0.02

0.87

0.04

4

93.85

1.59

2.13

1.86

0.57

8

88.27

4.83

3.36

2.43

0.61

40

86.13

6.11

4.29

2.58

0.89

Unemployment rate

1

42.22

57.78

0

0

0

2

52.03

47.57

0.04

v0.01

0.00

4

64.74

33.17

1.80

0.13

0.16

8

66.05

21.32

10.01

1.99

0.63

40

39.09

16.81

31.92

10.73

0.89

Inflation rate

1

0.86

4.18

89.80

5.15

0

2

0.63

13.12

77.24

8.56

0.45

4

0.72

16.79

68.15

13.36

0.97

8

1.79

19.34

60.69

16.07

2.11

40

2.83

20.48

55.84

17.12

3.74

Growth rate of wages

1

1.18

0.10

0.97

97.75

0

2

1.40

0.10

4.30

93.50

0.69

4

2.18

2.75

9.78

84.49

0.80

8

3.80

6.74

13.40

74.72

1.33

40

5.11

8.44

14.19

70.14

2.13

Growth rate of money stock

1

0.10

0.43

0.00

0.84

98.63

2

1.45

0.44

0.02

1.02

97.06

4

4.22

1.09

0.04

1.90

92.75

8

8.31

1.55

0.81

2.65

86.68

40

8.47

2.64

5.77

4.55

78.57

15.5 Identification via Long-Run Restrictions

15.5.1 A Prototypical Example

Besides short-run restrictions, essentially zero restrictions on the coefficients of A and/or B, Blanchard and Quah (1989) proposed long-run restrictions as an alternative option. These long-run restrictions have to be seen as complementary to the short-run ones as they can be combined. Long-run restrictions constrain the long-run effect of structural shocks. This technique makes only sense if some integrated variables are involved, because in the stationary case the effects of all shocks vanish eventually. To explain this, we discuss the two-dimensional example given by Blanchard and Quah (1989).

They analyze a two-variable system consisting of logged real GDP denoted by {Y t } and the unemployment rate {U t }. Logged GDP is typically integrated of order one (see Sect. 7.​3.​4 for an analysis for Swiss GDP) whereas U t is considered to be stationary. Thus they apply the VAR approach to the stationary process $$\{X_{t}\} =\{ (\Delta Y _{t},U_{t})'\}$$. Assuming that {X t } is already demeaned and follows a causal VAR process, we have the following representations:

For simplicity, we assume that A = I ₂, so that

where $$V _{t} = (v_{dt},v_{st})' \sim \mathrm{ WN}(0,\Omega )$$ with $$\Omega =\mathrm{ diag}(\omega _{d}^{2},\omega _{s}^{2})$$. Thereby {v dt } and {v st } denote demand and supply shocks, respectively. The causal representation of {X t } implies that the effect of a demand shock in period t on GDP growth in period t + h is given by:

where $$[\Psi _{h}B]_{11}$$ denotes the upper left hand element of the matrix $$\Psi _{h}B$$. Y t+h can be written as $$Y _{t+h} = \Delta Y _{t+h} + \Delta Y _{t+h-1} +\ldots +\Delta Y _{t+1} + Y _{t}$$ so that the effect of the demand shock on the level of logged GDP is given by:

Blanchard and Quah (1989) propose, in accordance with conventional economic theory, to restrict the long-run effect of the demand shock on the level of logged GDP to zero:

This implies that

where ∗ is a placeholder. This restriction is sufficient to infer b ₂₁ from the relation $$[\Psi (1)]_{11}b_{11} + b_{21}[\Psi (1)]_{12} = 0$$ and the normalization b ₁₁ = 1:

The second part of the equation follows from the identity $$\Phi (z)\Psi (z) = I_{2}$$ which gives $$\Psi (1) = \Phi (1)^{-1}$$ for z = 1. The long-run effect of the supply shock on Y t is left unrestricted and is therefore in general nonzero. Note that the implied value of b ₂₁ depends on $$\Phi (1)$$, and thus on $$\Phi _{1},\ldots,\Phi _{p}$$. The results are therefore, in contrast to short-run restrictions, much more sensitive to correct specification of the VAR.

The relation Z t  = BV t implies that

or more explicitely

Taking b ₂₁ as already given from above, this equation system has three equations and three unknowns b ₁₂, ω d ², ω s ² which is a necessary condition for a solution to exist.

Analytic Solution of the System

Because b ₂₁ is already determined from the long-run restriction, we rewrite the equation system²² explicitly in terms of b ₂₁:

Using the last two equations, we can express ω d ² and ω s ² as functions of b ₁₂:

These expressions are only valid if b ₂₁ ≠ 0 and b ₁₂ b ₂₁ ≠ 1. The case b ₂₁ = 0 is not interesting with regard to content. It would simplify the original equation system strongly and would results in $$b_{12} =\sigma _{12}/\sigma _{2}^{2}$$, $$\omega _{d}^{2} = (\sigma _{1}^{2}\sigma _{2}^{2} -\sigma _{12}^{2})/\sigma _{2}^{2} > 0$$ and $$\omega _{s}^{2} =\sigma _{ 2}^{2} > 0$$. The case b ₁₂ b ₂₁ = 1 contradicts the assumption that $$\Sigma $$ is a positive-definite matrix and can therefore be disregarded.²³

Inserting the solutions of ω d ² and ω s ² into the first equation, we obtain a quadratic equation in b ₁₂:

The discriminant $$\Delta $$ of this equation is:

The positivity of the discriminant implies that the quadratic equation has two real solutions b ₁₂ ⁽¹⁾ and b ₁₂ ⁽²⁾:

The first solution b ₁₂ ⁽¹⁾ can be excluded because it violates the assumption b ₁₂ b ₂₁ ≠ 1 which stands in contradiction to the positive-definiteness of the covariance matrix $$\Sigma $$. Inserting the second solution back into the solution for ω d ² and ω s ², we finally get:

Because $$\Sigma $$ is a symmetric positive-definite matrix, $$\sigma _{1}^{2}\sigma _{2}^{2} -\sigma _{12}^{2}$$ and the denominator $$b_{21}^{2}\sigma _{1}^{2} - 2b_{21}\sigma _{12} +\sigma _{ 2}^{2}$$ are strictly positive. Thus, both solutions yield positive variances and we have found the unique admissible solution.

15.5.2 The General Approach

The general case of long-run restrictions has a structure similar to the case of short-run restrictions. Take as a starting point the structural VAR (15.2) from Sect. 15.2.2:

where {X t } is stationary and causal with respect to {V t }. As before the matrix A is normalized to have ones on its diagonal and is assumed to be invertible, $$\Omega $$ is a diagonal matrix with $$\Omega =\mathrm{ diag}(\omega _{1}^{2},\ldots,\omega _{n}^{2})$$, and B is a matrix with ones on the diagonal. The matrix polynomial A(L) is defined as $$A\mathrm{(L}) = A - \Gamma _{1}\mathrm{L} -\ldots -\Gamma _{p}\mathrm{L}^{p}$$. The reduced form is given by

where AZ t  = BV t and $$A\Phi _{j} = \Gamma _{j}$$, j = 1, …, p, respectively $$A\Phi (\mathrm{L}) = A\mathrm{(L})$$.

The long-run variance of {X t }, J, (see Eq. (11.​1) in Chap. 11) can be derived from the reduced as well as from the structural form, which gives the following expressions:

where $$X_{t} = \Psi (\mathrm{L})Z_{t}$$ denotes the causal representation of {X t }. The long-run variance J can be estimated by adapting the methods in Sect. 4.​4 to the multivariate case. Thus, the above equation system has a similar structure as the system (15.5) which underlies the case of short-run restrictions. As before, we get $$n(n + 1)/2$$ equations with 2n ² − n unknowns. The nonlinear equation system is therefore undetermined for n ≥ 2. Therefore, $$3n(n - 1)/2$$ additional equations or restrictions are necessary to achieve identification. Hence, conceptually we are in a similar situation as in the case of short-run restrictions.²⁴

In practice, it is customary to achieve identification through zero restrictions where some elements of $$\Psi (1)A^{-1}B$$, respectively $$\Phi (1)^{-1}A^{-1}B$$, are set a priori to zero. Setting the ij-th element $$[\Psi (1)A^{-1}B]_{ij} = [\Phi (1)^{-1}A^{-1}B]_{ij}$$ equal to zero amounts to set the cumulative effect of the j-th structural disturbance V jt on the i-th variable equal to zero. If the i-th variable enters X t in first differences, as was the case for Y t in the previous example, this zero restriction restrains the long-run effect on the level of that variable.

An interesting simplification arises if one assumes that A = I n and that $$\Psi (1)B = \Phi (1)^{-1}B$$ is a lower triangular matrix. In this case, B and $$\Omega $$ can be estimated from the Cholesky decomposition of the estimated long-run variance $$\hat{J }$$. Let $$\hat{J }=\hat{ L}\hat{D}\hat{L}'$$ be the Cholesky decomposition where $$\hat{L}$$ is lower triangular matrix with ones on the diagonal and $$\hat{D}$$ is a diagonal matrix with strictly positive diagonal entries. As $$\hat{J }=\hat{ \Phi }(1)^{-1}B\hat{\Omega }B'\hat{\Phi }(1)^{-1}{}'$$, the matrix of structural coefficients can then be estimated as $$\hat{B} =\hat{ \Phi }(1)\hat{L}\hat{U}^{-1}$$. The multiplication by the inverse of the diagonal matrix $$U =\mathrm{ diag}(\hat{\Phi }(1)\hat{L})$$ is necessary to guarantee that the normalization of $$\hat{B}$$ (diagonal elements equal to one) is respected. $$\Omega $$ is then estimated as $$\hat{\Omega } =\hat{ U}\hat{D}\hat{U}$$.

Instead of using a method of moments approach, it is possible to use an instrumental variable (IV) approach. For this purpose we rewrite the reduced form of X t in the Dickey-Fuller form (see Eqs. (7.​1) and (16.​4)):

where $$\widetilde{\Phi }_{j} = -\sum _{i=j+1}^{p}\Phi _{i}$$, $$j = 1,2,\ldots,p - 1$$. For the ease of exposition, we assume B = I n so that AZ t  = V t . Multiplying this equation by A yields:

(15.9)

Consider for simplicity the case that $$A\Phi (1)$$ is a lower triangular matrix. This implies that the structural shocks V 2t , V 3t , …, V nt have no long-run impact on the first variable X 1t . It is, therefore, possible to estimate the coefficients A ₁₂, A ₁₃, …, A 1n by instrumental variables taking $$X_{2,t-1},X_{3,t-1},\ldots,X_{n,t-1}$$ as instruments.

For n = 2 the Dickey-Fuller form of the equation system (15.9) is:

respectively

Thereby $$\Delta \tilde{X}_{1t}$$ and $$\Delta \tilde{X}_{2t}$$ denote the OLS residuals from a regression of $$\Delta X_{1t}$$, respectively $$\Delta X_{2t}$$ on $$(\Delta X_{1,t-1},\Delta X_{2,t-1},\ldots,\Delta X_{1,t-p+1},\Delta X_{2,t-p+1})$$. $$\tilde{X}_{2,t-1}$$ is a valid instrument for $$\Delta \tilde{X}_{2t}$$ because this variable does not appear in the first equation. Thus, A ₁₂ can be consistently estimated by the IV-approach. For the estimation of A ₂₁, we can use the residuals from the first equation as instruments because V 1t and V 2t are assumed to be uncorrelated. From this example, it is easy to see how this recursive method can be generalized to more than two variables. Note also that the IV-approach can also be used in the context of short-run restrictions.

The issue whether a technology shock leads to a reduction of hours worked in the short-run, led to a vivid discussion on the usefulness of long-run restrictions for structural models (Galí 1999; Christiano et al. 2003, 2006; Chari et al. 2008). From an econometric point of view, it turned out, on the one hand, that the estimation of $$\Phi (1)$$ is critical for the method of moments approach. The IV-approach, on the other hand, depends on the strength or weakness of the instrument used (Pagan and Robertson 1998; Gospodinov 2010).

It is, of course, possible to combine both short- and long-run restrictions simultaneously. An interesting application of both techniques was presented by Galí (1992). In doing so, one must be aware that both type of restrictions are consistent with each other and that counting the number of restrictions gives only a necessary condition.

Example 3: Identifying Aggregate Demand and Supply Shocks

In this example, we follow Blanchard and Quah (1989) and investigate the behavior of the growth rate of real GDP and the unemployment rate for the US over the period from the first quarter 1979 to the second quarter 2004 (102 observations). The AIC and the BIC suggest models of order two and one, respectively. Because some coefficients of $$\widehat{\Phi }_{2}$$ are significant at the 10 % level, we prefer to use the VAR(2) model which results in the following estimates²⁵:

These results can be used to estimate $$\Phi (1) = I_{2} - \Phi _{1} - \Phi _{2}$$ and consequently also $$\Psi (1) = \Phi (1)^{-1}$$:

Assuming that Z t  = BV t and following the argument in Sect. 15.5.1 that the demand shock has no long -run impact on the level of real GDP, we can retrieve an estimate for b ₂₁:

The solution of the quadratic equation for b ₁₂ are − 8. 894 and 43.285. As the first solution results in a negative variance for ω ₂ ², we can disregard this solution and stick to the second one. The second solution makes also sense economically, because a positive supply shock leads to positive effects on GDP. Setting b ₁₂ = 43. 285 gives the following estimates for covariance matrix of the structural shocks $$\Omega $$:

The big difference in the variance of both shocks clearly shows the greater importance of demand shocks for business cycle movements.

Figure 15.4 shows the impulse response functions of the VAR(2) identified by the long-run restriction. Each figure displays the dynamic effect of a demand and a supply shock on real GDP and the unemployment rate, respectively, where the size of the initial shock corresponds to one standard deviation. The result conforms well with standard economic reasoning. A positive demand shock increases real GDP and lowers the unemployment rate in the short-run. The effect is even amplified for some quarters before it declines monotonically. After 30 quarters the effect of the demand has practically vanished so that its long-run effect becomes zero as imposed by the restriction. The supply shock has a similar short-run effect on real GDP, but initially increases the unemployment rate. Only when the effect on GDP becomes stronger after some quarters will the unemployment rate start to decline. In the long-run, the supply shock has a positive effect on real GDP but no effect on unemployment. Interestingly, only the short-run effects of the demand shock are statistically significant at the 95-% level.

Fig. 15.4

Impulse response functions of the Blanchard-Quah model (Blanchard and Quah 1989) with 95-% confidence intervals computed using the bootstrap procedure

15.6 Sign Restrictions

In recent years the use of sign restrictions has attracted a lot of attention. Pioneering contributions have been provided by Faust (1998), Canova and De Nicoló (2002), and Uhlig (2005). Since then the literature has abounded by many applications in many contexts. Sign restrictions try to identify the impact of the structural shocks by requiring that the signs of the impulse response coefficients have to follow a given pattern. The motivation behind this development is that economists are often more confident about the sign of an economic relationship than about its exact magnitude. This seems to be true also for zero restrictions, whether they are short- or long-run restrictions. This insight already led Samuelson (1947) to advocate a calculus of qualitative relations in economics. Unfortunately, this approach has been forgotten in the progression of economic analysis.²⁶ With the rise in popularity of sign restrictions his ideas may see a revival.

To make the notion of sign restrictions precise, we introduce a language based on the following notation and definitions. Sign restrictions will be specified as sign pattern matrices. These matrices make use of the sign function of a real number x, sgn(x), defined as

Definition 15.1.

A sign pattern matrix or pattern for short is a matrix whose elements are solely from the set {1,−1,0}. Given a sign pattern matrix S, the sign pattern class of S is defined by

where M(n) is the set of all n × n matrices and where sgn is applied elementwise to M. Clearly, $$\mathcal{S}(S) = \mathcal{S}(\mathrm{sgn}(S))$$ .

Remark 15.1.

Often the set $$\{+,-,0\}$$ is used instead of $$\{+1,-1,0\}$$ to denote the sign patterns.

Remark 15.2.

In some instances we do not want to restrict all signs but only a subset of it. In this case, the elements of the sign pattern matrix S may come from the larger set { − 1, 0, 1, #} where # stands for an unspecified sign. S is then called a generalized sign pattern matrix or a generalized pattern. The addition + and the multiplication × of (generalized) sign pattern matrices are defined in a natural way.

In order not to overload the discussion, we set A = I n and focus on the case where Z t  = BV t .²⁷ Moreover, we use a different, but completely equivalent normalization. In particular, we relax, on the one hand, the assumption that B has only ones on its diagonal, but assume on the other hand that $$\Omega = I_{n}$$. Note that $$\Sigma = \mathbb{E}Z_{t}Z_{t}' = BB'$$ is a strictly positive definite matrix. Assuming that a causal representation of {X t } in terms of {Z t } and thus also in terms of {V t } exists, we can represent X t as

Denote by $$\mathcal{B}(\Sigma )$$ the set of invertible matrices B such that $$\Sigma = BB'$$, i.e. $$\mathcal{B}(\Sigma ) =\{ B \in \mathrm{ GL}(n)\mid BB' = \Sigma \}$$.²⁸ Thus, $$\mathcal{B}(\Sigma )$$ is the set of all feasible structural factorizations (models).

Sign restrictions on the impulse responses $$[\Psi _{h}B]_{i,j}$$ can then be defined in terms of a sequence of (generalized) sign pattern matrices {S h }, h = 0, 1, 2, …

Definition 15.2 (Sign Restrictions).

A causal VAR allows a sequence of (generalized) sign pattern matrices {S h } if and only if there exists $$B \in \mathcal{B}(\Sigma )$$ such that

(15.10)

Remark 15.3.

As $$\|\Psi _{j}B\|$$ converges to zero for $$j \rightarrow \infty $$, it seems reasonable to impose sign restrictions only up to some horizon $$h^{\max } < \infty $$. In this case, S h , $$h > h^{\max }$$, is equal to the generalized sign pattern matrix whose elements consist exclusively of #’s. A case of particular interest is given by $$h^{\max } = 0$$. In this case, we drop the index 0 and say that the VAR allows (generalized) sign pattern matrix S.

Remark 15.4.

With this notation we can also represent (short-run) zero restrictions if the sign patterns are restricted to 0 and #.

A natural question to ask is how restrictive a prescribed sign pattern is. This amounts to the question whether a given VAR can be compatible with any sign pattern. As is already clear from the discussion of the two-dimensional case in Sect. 15.2.3, this is not the case. As the set of feasible parameters can be represented by a rectangular hyperbola, there will always be one quadrant with no intersection with the branches of the hyperbola. In the example plotted in Fig. 15.1, this is quadrant III. Thus, configurations with (B)₂₁ < 0 and (B)₁₂ < 0 are not feasible given $$(\Sigma )_{12} > 0$$. This argument can be easily extended to models of higher dimensions. Thus, there always exist sign patterns which are incompatible with a given $$\Sigma $$.

As pointed out in Sect. 15.3 there always exists a unique lower triangular matrix R, called the Cholesky factor of $$\Sigma $$, such that $$\Sigma = RR'$$. Thus, $$\mathcal{B}(\Sigma )\neq \emptyset$$ because $$R \in \mathcal{B}(\Sigma )$$.

Lemma 15.1.

Let the Cholesky factor of $$\Sigma $$ be R, then

Proof.

Suppose B = RQ with $$Q \in \mathcal{O}(n)$$, then $$BB' = RQQ'R' = \Sigma $$. Thus, $$B \in \mathcal{B}(\Sigma )$$. If $$B \in \mathcal{B}(\Sigma )$$, define $$Q = R^{-1}B$$. Then, $$QQ' = R^{-1}BB'(R')^{-1} = R^{-1}\Sigma (R')^{-1} = R^{-1}RR'(R')^{-1} = I_{n}$$. Thus, $$Q \in \mathcal{O}(n)$$. □ 

This lemma establishes that there is a one-to-one function $$\varphi _{\Sigma }$$ from the group of orthogonal matrices $$\mathcal{O}(n)$$ onto the set of feasible structural factorizations $$\mathcal{B}(\Sigma )$$. From the proof we see that $$\varphi _{\Sigma }(Q) = RQ$$ and $$\varphi _{\Sigma }^{-1}(B) = R^{-1}B$$. Moreover, for any two matrices B ₁ and B ₂ in $$\mathcal{B}(\Sigma )$$ with B ₁ = RQ ₁ and B ₂ = RQ ₂, there exists an orthogonal matrix Q equal to Q ₂′Q ₁ such that B ₁ = B ₂ Q. As $$\varphi _{\Sigma }$$ and $$\varphi _{\Sigma }^{-1}$$ are clearly continuous, $$\varphi _{\Sigma }$$ is a homeomorphism. See Neusser (2016) for more details and further implications.

To make the presentation more transparent, we focus on sign restrictions only and disregard zero restrictions. Arias et al. (2014) show how sign and zero restrictions can be treated simultaneously. Thus, the entries of {S h } are elements of $$\{-1,+1,\#\}$$ only. Assume that a VAR allows sign patterns {S h }, $$h = 0,1,\ldots,h^{\max }$$. Then according to Definition 15.2, there exists $$B \in \mathcal{B}(\Sigma )$$ such that $$\Psi _{h}B \in \mathcal{S}(S_{h})$$ for all $$h = 0,1,\ldots,h^{\max }$$. As the (strict) inequality restrictions delineate an open subset of $$B \in \mathcal{B}(\Sigma )$$, there exist other nearby matrices which also fulfill the sign restrictions. Sign restrictions therefore do not identify one impulse response sequence, but a whole set. Thus, the impulse responses are called set identified.

This set is usually difficult to characterize algebraically so that one has to rely on computer simulations. Conditional on the estimated VAR, thus conditional on $$\{\widehat{\Psi }_{j}\}$$ and $$\widehat{\Sigma }$$, Lemma 15.1 suggests a simple and straightforward algorithm (see Rubio-Ramírez et al. 2010; Arias et al. 2014, for further details):

Step 1:

Draw at random an element Q from the uniform distribution on the set of orthogonal matrices $$\mathcal{O}(n)$$.

Step 2:

Convert Q into a random element of $$\mathcal{B}(\widehat{\Sigma })$$ by applying $$\varphi _{\widehat{\Sigma }}$$ to Q. As $$\varphi _{\widehat{\Sigma }}$$ is a homeomorphism this introduces a uniform distribution on $$\mathcal{B}(\Sigma )$$.

Step 3:

Compute the impulse responses with respect to $$\varphi _{\widehat{\Sigma }}(Q)$$, i.e. compute $$\widehat{\Psi }_{h}\,\varphi _{\widehat{\Sigma }}(Q)$$.

Step 4:

Keep those models with impulse response functions which satisfy the prescribed sign restrictions $$\widehat{\Psi }_{h}\,\varphi _{\widehat{\Sigma }}(Q) \in \mathcal{S}(S_{h})$$, $$h = 0,1,\ldots,h^{\max }$$.

Step 5:

Repeat steps 1–4 until a satisfactory number of feasible structural models with impulse responses obeying the sign restrictions have been obtained.

The implementation of this algorithm requires a way to generate random draws Q from the uniform distribution on $$\mathcal{O}(n)$$.²⁹ This is not a straightforward task because the elements of Q are interdependent as they must ensure the orthonormality of the columns of Q. Edelman and Rao (2005) proposes the following efficient two-step procedure. First, draw n × n matrices X such that $$X \sim \mathrm{ N}(0,I_{n} \otimes I_{n})$$, i.e. each element of X is drawn independently from a standard normal distribution. Second, perform the QR-decomposition which factorizes each matrix X into the product of an orthogonal matrix Q and an upper triangular matrix R normalized to have positive diagonal entries.³⁰

As the impulse responses are only set identified, the way to report the results and how to conduct inference becomes a matter of discussion. Several methods have been proposed in the literature:

(i)

One straightforward possibility consists in reporting, for each horizon h, the median of the impulse responses. Although simple to compute, this method presents some disadvantages. The median responses will not correspond to any of the structural models. Moreover, the orthogonality of the structural shocks will be lost. Fry and Pagan (2011) propose the median–target method as an ad hoc remedy to this shortage. They advocate to search for the admissible structural model whose impulse responses come closest to the median ones.

(ii)

Another possibility is to search for the admissible structural model which maximizes the share of the forecast error variance at some horizon of a given variable after a particular shock.³¹ An early application of this method can be found in Faust (1998). This method remains, however, uninformative about the relative explanatory power of alternative admissible structural models.

(iii)

The penalty function approach by Mountford and Uhlig (2009) does not accept or reject particular impulse responses depending on whether it is in accordance with the sign restrictions (see step 4 in the above algorithm). Instead, it associates for each possible impulse response function and every sign restriction a value which rewards a “correct” sign and penalizes a “wrong” sign. Mountford and Uhlig (2009) propose the following ad hoc penalty function: f(x) = 100x if sgn(x) is wrong and f(x) = x if sgn(x) is correct. The impulse response function which minimizes the total (standardized) penalty is then reported.

(iv)

The exposition becomes more coherent if viewed from a Bayesian perspective. From this perspective, the uniform distribution on $$\mathcal{O}(n)$$, respectively on $$\mathcal{B}(\widehat{\Sigma })$$, is interpreted as diffuse or uninformative prior distribution.³² The admissible structural models which have been retained in step 5 of the algorithm are then seen as draws from the corresponding posterior distribution. The most likely model is then given by the model which corresponds to the mode of the posterior distribution. This model is associated an impulse response function which can then be reported. This method also allows the construction of 100(1 −α)-% highest posterior density credible sets (see Inoue and Kilian 2013, for details). As shown by Moon and Schorfheide (2012) these sets cannot, even in a large sample context, be interpreted as approximate frequentist confidence intervals. Recently, however, Moon et al. (2013) proposed a frequentist approach to the construction of error bands for sign identified impulse responses.

References

Arias JE, Rubio-Ramírez J, Waggoner DF (2014) Inference based on SVARs identified with sign and zero restrictions: Theory and applications. FRB Atlanta Working Paper 2014–1, Federal Reserve Bank of Atlanta

Barsky R, Sims E (2011) News shocks and business cycles. J Monet Econ 58:273–289CrossRef

Baumeister C, Hamilton JD (2015) Sign restrictions, structural vector autoregressions, and useful prior information. Econometrica 83:1963–1999CrossRef

Bernanke BS (1986) Alternative explanations of money-income correlation. In: Brunner K, Meltzer A (eds) Real business cycles, real exchange rates, and actual policies, no. 25 in Carnegie-Rochester Conference Series on Public Policy. North-Holland, Amsterdam, pp 49–100

Berndt ER (1991) The practice of econometrics. Addison Wesley, Reading,

Blanchard OJ (1989) A traditional interpretation of macroeconomic fluctuations. Am Econ Rev 79:1146–1164

Blanchard OJ, Quah D (1989) The dynamic effects of aggregate demand and supply disturbances. Am Econ Rev 79:655–673

Blanchard OJ, Watson MW (1986) Are business cycles all alike? In: Gordon R (ed) The American business cycle: continuity and change. University of Chicago Press, Chicago, pp 123–179

Brualdi RA, Shader BL (1995) Matrices of sign-solvable linear systems. No. 116 in Cambridge tracts in mathematics. Cambridge University Press, Cambridge

Canova F, Ciccarelli M (2008) Estimating multi-country VAR models. Working Paper 603, European Central Bank

Canova F, De Nicoló G (2002) Monetary disturbances matter for business fluctuations in the G–7. J Monet Econ 49:1131–1159

Chari VV, Kehoe PJ, McGrattan ER (2008) Are structural VARs with long-run restrictions useful in developing business cycle theory? J Monet Econ 55:1337–1352CrossRef

Christiano LJ, Eichenbaum M, Evans CL (1999) Monetary policy shocks: what have we learned and to what end? Handbook of macroeconomics, vol 1A, chap 2. North-Holland, Amsterdam, pp 65–148

Christiano LJ, Eichenbaum M, Vigfusson RJ (2003) What happens after a technology shock? Working Paper No. 9819, NBER

Christiano LJ, Eichenbaum M, Vigfusson RJ (2006) Assessing structural VARs. International Finance Discussion Papers No. 866, Board of Governors of the Federal Reserve System

Cooley TF, LeRoy SF (1985) Atheoretical macroeconometrics - a critique. J Monet Econ 16:283–308CrossRef

Dees S, Mauro FD, Pesaran MH, Smith V (2007) Exploring the international linkages of the Euro area: A global VAR analysis. J Appl Econ 22:1–38CrossRef

Edelman A, Rao NR (2005) Random matrix theory. Acta Numerica 14:1–65CrossRef

Efron B, Tibshirani RJ (1993) An introduction to the bootstrap. Monographs on statistics and applied probability, vol 57. Chapman & Hall/CRC, Boca RatonCrossRef

Faust J (1998) The robustness of identified VAR conclusions about money. Carn-Roch Conf Ser Publis Policy 49:207–244CrossRef

Francis N, Owyang MT, Roush JE, DiCecio R (2014) A flexible finite-horizon alternative to long-run restrictions with an application to technology shocks. Rev Econ Stat 96:638–647CrossRef

Fry RA, Pagan AR (2011) Sign restrictions in structural vector autoregressions: A critical review. J Econ Lit 49:938–960CrossRef

Galí J (1992) How well does the IS-LM model fit postwar U.S. data? Q J Econ 107(2):709–738CrossRef

Galí J (1999) Technology, employment, and the business cycle: Do technology shocks explain aggregate fluctuations? Am Econ Rev 89:249–271CrossRef

Geweke JF (1984) Inference and causality in economic time series models. In: Griliches Z, Intriligator MD (eds) Handbook of econometrics, vol II. Elsevier, Amsterdam, pp 1101–1144

Giannini C (1991) Topics in structural VAR econometrics. Quaderni di Ricerca 21, Università degli Studi di Anacona, Dipartimento di Economia

Gospodinov N (2010) Inference in nearly nonstationary SVAR models with long-run identifying restrictions. J Bus Econ Stat 28:1–12CrossRef

Granger CWJ (1969) Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37(3):424–438CrossRef

Hall FJ, Li Z (2014) Sign pattern matrices. In: Hogben L (ed) Handbook of linear algebra, 2nd edn, chap 42. Chapman & Hall/CRC, Boca Raton, pp 1–32

Hamilton JD (1994b) Time series analysis. Princeton University Press, Princeton

Hansen LP, Sargent TJ (1980) Formulating and estimating dynamic linear rational expectations models. J Econ Dyn Control 2:7–46CrossRef

Haugh LD (1976) Checking the independence of two covariance stationary time series: A univariate residual cross-correlation approach. J Am Stat Assoc 71:378–385

Inoue A, Kilian L (2013) Inference on impulse response functions in structural VAR models. J Econ 177:1–13

Kilian L (1998) Small-sample confidence intervals for impulse response functions. Rev Econ Stat 80:186–201CrossRef

Kilian L, Murphy DP (2012) Why agnostic sign restrictions are not enough: Understanding the dynamics of oil market VAR models. J Eur Econ Assoc 10:1166–1188CrossRef

Leamer EE (1981) Is it a demand curve, or is it a supply curve? Partial identification through inequality constraints. Rev Econ Stat 63:319–327CrossRef

Lütkepohl H (1990) Asymptotic distributions of impulse response functions and forecast error variance decomposition. Rev Econ Stat 72:116–125CrossRef

Lütkepohl H (2006) New introduction to multiple time series analysis. Springer, Berlin

Lütkepohl H, Staszewska-Bystrova A, Winker P (2013) Comparison of methods for constructing joint confidence bands for impulse response functions. Discussion Paper 1292, DIW Berlin

Meyer CD (2000) Matrix analysis and applied linear algebra. Society for Industrial and Applied Mathematics, PhiladelphiaCrossRef

Mittnik S, Zadrozny PA (1993) Asymptotic distributions of impulse responses, step responses and variance decompositions of estimated linear dynamic models. Econometrica 61:857–870CrossRef

Moon HR, Schorfheide F (2012) Bayesian and frequentist inference in partially identified models. Econometrica 80:755–782CrossRef

Moon HR, Schorfheide F, Granziera E (2013) Inference for VARs identified with sign restrictions, working paper

Mountford A, Uhlig H (2009) What are the effects of fiscal policy shocks? J Appl Econ 24:960–992CrossRef

Negro Md, Schorfheide F (2004) Priors from general equilibrium models for VARs. Int Econ Rev 45:643–673CrossRef

Neusser K (2016) A topological view on the identification of structural vector autoregressions. Economics Letters forthcoming

Pagan AR, Robertson OC (1998) Structural models of the liquidity effect. Rev Econ Stat 80:202–217CrossRef

Pierce DA, Haugh LD (1977) Causality in temporal systems - characterization and survey. J Econ 5:265–293CrossRef

Rigobon R (2003) Identification through heteroskedasticity. Rev Econ Stat 85:777–792CrossRef

Rothenberg TJ (1971) Identification of parametric models. Econometrica 39:577–591CrossRef

Rubio-Ramírez JF, Waggoner DF, Zha T (2010) Structural vector autoregressions: Theory of identification and algorithms for inference. Rev Econ Stud 77:665–696CrossRef

Runkle D (1987) Vector autoregressions and reality. J Bus Econ Stat 5:437–442

Samuelson PA (1947) Foundations of economic analysis. Harvard University Press, Cambridge

Serfling RJ (1980) Approximation theorems of mathematical statistics. Wiley, New YorkCrossRef

Sims CA (1972) Money, income, and causality. Am Econ Rev 62:540–552

Sims CA (1980a) Comparison of interwar and postwar business cycles: Monetarism reconsidered. Am Econ Rev 70(2):250–257

Sims CA (1980b) Macroeconomics and reality. Econometrica 48:1–45

Sims CA (1986) Are forecasting models usable for policy analysis. Federal Reserve Bank of Minneapolis Q Rev 10(1):2–16

Sims CA (1999) Error bands for impulse responses. Econometrica 67:1113–1155CrossRef

Tobin J (1970) Money and income: Post hoc ergo propter hoc? Q J Econ 84:310–317

Uhlig H (2004) Do technology shocks lead to a fall in total hours worked? J Eur Econ Assoc 2:361–371CrossRef

Uhlig H (2005) What are the effects of monetary policy on output? results from an agnostic identification procedure. J Monet Econ 52:381–419CrossRef

Wiener N (1956) The theory of prediction. In: Beckenbach EF (ed) Modern mathematics for engineers. McGraw-Hill, New York, Series 1

Zellner A (1979) Causality and econometrics. In: Brunner K, Meltzer A (eds) Three aspects of policymaking: knowledge, data and institution, Carnegie-Rochester conference series on public policy. North-Holland, Amsterdam, pp 9–54

Footnotes

1

Compare this to the concept of a causal representation developed in Sects. 12.​3 and 2.​3

2

Sometimes also the concept of contemporaneous causality is considered. This concept is, however, controversial and has therefore not gained much success in practice and will, therefore, not be pursued.

3

As we are working with causal VAR’s, the above arguments also hold with respect to the Wold Decomposition.

4

If one is working with actual data, the variables are usually expressed in log-differences to achieve stationarity.

5

A thorough treatment of the identification problem in econometrics can be found in Rothenberg (1971), and for the VAR context in Rubio-Ramírez et al. (2010).

6

As usual, we concentrate on the first two moments only.

7

Note also that our discussion of the identification problem focuses on local identification, i.e. the invertibility of g in an open neighborhood of $$\Sigma $$. See Rothenberg (1971) and Rubio-Ramírez et al. (2010) for details on the distinction between local and global identification.

8

See Neusser (2016) for further implications of viewing the identification problem from an invariance perspective.

9

The exposition is inspired by Leamer (1981).

10

Moon et al. (2013, section 2) provided an alternative geometric representation.

11

That these two intervals are disconnected follows from the fact that $$\Sigma $$ is positive definite.

12

The case of overidentification is, for example, treated in Bernanke (1986).

13

A version of the instrumental variable (IV) approach is discussed in Sect. 15.5.2.

14

The vech operator transforms a symmetric n × n matrix $$\Sigma $$ into a $$\tfrac{1} {2}n(n + 1)$$ vector by stacking the columns of $$\Sigma $$ such that each element is listed only once.

15

Sometimes the causal representation is called the $$\mathrm{MA}(\infty $$) representation.

16

Recall that the vec operator stacks the columns of a n × m matrix to get one nm vector.

17

The bootstrap is a resampling method. Efron and Tibshirani (1993) provide a general introduction to the bootstrap.

18

The draws can also be done blockwise. This has the advantage that possible remaining temporal dependences are taken in account.

19

Formula (15.6) for n = 2 gives 3 restrictions.

20

The results do not match exactly those of Blanchard (1989), but are qualitatively similar.

21

To save space, the estimated standard errors of the coefficients are not reported.

22

This equation system is similar to the one analyzed in Sect. 15.2.3.

23

If b ₁₂ b ₂₁ = 1, $$\det \Sigma =\sigma _{ 1}^{2}\sigma _{2}^{2} -\sigma _{12}^{2} = 0$$. This implies that Z 1t and Z 2t are perfectly correlated, i.e. $$\rho _{Z_{1t},Z_{2t}}^{2} =\sigma _{ 12}^{2}/(\sigma _{1}^{2}\sigma _{2}^{2}) = 1$$.

24

See Rubio-Ramírez et al. (2010) for a unified treatment of both type of restrictions.

25

The results for the constants are suppressed to save space.

26

It is interesting to note that Samuelson’s ideas have fallen on fruitful grounds in areas like computer science or combinatorics (see Brualdi and Shader 1995; Hall and Li 2014).

27

The case with general A matrices can be treated analogously.

28

GL(n) is known as the general linear group. It is the set of all invertible n × n matrices.

29

It can be shown that this probability measure is the unique measure μ on $$\mathcal{O}(n)$$ which satisfies the normalization $$\mu (\mathcal{O}(n)) = 1$$ and the (left)-invariance property $$\mu (Q\mathfrak{Q}) =\mu (\mathfrak{Q})$$ for every $$\mathfrak{Q} \subseteq \mathcal{O}(n)$$ and $$Q \in \mathcal{O}(n)$$. In economics, this probability measure is often wrongly referred to as the Haar measure. The Haar measure is not normalized and is, thus, unique only up to a proportionality factor.

30

Given a value of n, the corresponding MATLAB commands are [Q,R]=qr(randn(n)); Q = Q*diag(sign(diag(R))); (see Edelman and Rao 2005).

31

The minimization of the forecast error variance share have also been applied as an identification device outside the realm of sign restrictions. See Sect. 15.4.2.

32

Whether this distribution is always the “natural” choice in economics has recently been disputed by Baumeister and Hamilton (2015).

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_16

16. Cointegration

Klaus Neusser1 

(1)

Bern, Switzerland

As already mentioned in Chap. 7, many raw economic time series are nonstationary and become stationary only after some transformation. The most common of these transformations is the formation of differences, perhaps after having taken logs. In most cases first differences are sufficient to achieve stationarity. The stationarized series can then be analyzed in the context of VAR models as explained in the previous chapters. However, many economic theories are formalized in terms of the original series so that we may want to use the VAR methodology to infer also the behavior of the untransformed series. Yet, by taking first differences we loose probably important information on the levels. Thus, it seems worthwhile to develop an approach which allows us to take the information on the levels into account and at the same time take care of the nonstationary character of the variables. The concept of cointegration tries to achieve this double requirement.

In the following we will focus our analysis on variables which are integrated of order one, i.e. on time series which become stationary after having taken first differences. However, as we have already mentioned in Sect. 7.​5.​1, a regression between integrated variables may lead to spurious correlations which make statistical inferences and interpretations of the estimated coefficients a delicate issue (see Sect. 7.​5.​3). A way out of this dilemma is presented by the theory of cointegrated processes. Loosely speaking, a multivariate process is cointegrated if there exists a linear combination of the processes which is stationary although each process taken individually may be integrated. In many cases, this linear combination can be directly related to economic theory which has made the analysis of cointegrated processes an important research topic. In the bivariate case, already been dealt with in Sect. 7.​5.​2, the cointegrating relation can be immediately read off from the cointegrating regression and the cointegration test boils down to a unit root test for the residuals of the cointegrating regression. However, if more than two variables are involved, the single equation residual based test is, as explained in Sect. 7.​5.​2, no longer satisfactory. Thus, a genuine multivariate is desirable.

The concept of cointegration goes back to the work of Engle and Granger (1987) which is itself based on the precursor study of Davidson et al. (1978). In the meantime the literature has grown tremendously. Good introductions can be found in Banerjee et al. (1993), Watson (1994) or Lütkepohl (2006). For the more statistically inclined reader Johansen (1995) is a good reference.

16.1 A Theoretical Example

Before we present the general theory of cointegration within the VAR context, it is instructive to introduce the concept in the well-known class of present discounted value models. These models relate some variable X t to present discounted value of another variable Y t :

where $$u_{t} \sim \mathrm{ WN}(0,\sigma _{u}^{2})$$ designates a preference shock. Thereby, β denotes the subjective discount factor and γ is some unspecified parameter. The present discounted value model states that the variable X t is proportional to the sum of future Y t+j , j = 0, 1, 2, …, discounted by the factor β. We can interpret X t and Y t as the price and the dividend of a share, as the interest rate on long- and short-term bonds, or as consumption and income. In order to operationalize this model, we will assume that forecasts are computed as linear mean-squared error forecasts. The corresponding forecast operator is denoted by $$\mathbb{P}_{t}$$. Furthermore, we will assume that the forecaster observes Y t and its past $$Y _{t-1},Y _{t-2},\ldots$$ The goal of the analysis is the investigation of the properties of the bivariate process {(X t , Y t )′}. The analysis of this important class models presented below is based on Campbell and Shiller (1987).¹

The model is closed by assuming some specific time series model for {Y t }. In this example, we will assume that {Y t } is an integrated process of order one (see Definition 7.​1 in Sect. 7.​1) such that $$\{\Delta Y _{t}\}$$ follows an AR(1) process:

This specification of the {Y t } process implies that $$\mathbb{P}_{t}\Delta Y _{t+h} =\mu (1 -\phi ^{h}) +\phi ^{h}\Delta Y _{t}$$. Because $$\mathbb{P}_{t}Y _{t+h} = \mathbb{P}_{t}\Delta Y _{t+h} +\ldots +\mathbb{P}_{t}\Delta Y _{t+1} + Y _{t}$$, h = 0, 1, 2, …, the present discounted value model can be manipulated to give:

This expression shows that the integratedness of {Y t } is transferred to {X t }. Bringing Y t to the left we get the following expression:

The variable S t is occasionally referred to as the spread. If γ is greater than zero, expected increases in $$\Delta Y _{t+j}$$, j ≥ 1, have a positive impact on the spread today. For γ = 1, S t can denote the log of the price-dividend ratio of a share, or minus the logged savings ratio as in the permanent income model of Campbell (1987). If investors expect positive (negative) change in the dividends tomorrow, they want to buy (sell) the share thereby increasing (decreasing) its price already today. In the context of the permanent income hypothesis expected positive income changes lead to a reduction in today’s saving rate. If, on the contrary, households expect negative income changes to occur in the future, they will save already today (“saving for the rainy days”).

Inserting for $$\mathbb{P}_{t}\Delta Y _{t+j}$$, j = 0, 1, …, the corresponding forecast equation $$\mathbb{P}_{t}\Delta Y _{t+h} =\mu (1 -\phi ^{h}) +\phi ^{h}\Delta Y _{t}$$, we get:

The remarkable feature about this relation is that {S t } is a stationary process because both $$\{\Delta Y _{t}\}$$ and {u t } are stationary, despite the fact that {Y t } and {X t } are both integrated processes of order one. The mean of S t is:

From the relation between S t and $$\Delta Y _{t}$$ and the AR(1) representation of $$\{\Delta Y _{t}\}$$ we can deduce a VAR representation of the joint process $$\{(S_{t},\Delta Y _{t})'\}$$:

Further algebraic transformation lead to a VAR representation of order two for the level variables {(X t , Y t )′}:

Next we want to check whether this stochastic difference equation possesses a stationary solution. For this purpose, we must locate the roots of the equation $$\det \Phi (z) =\det (I_{2} - \Phi _{1}z - \Phi _{2}z^{2}) = 0$$ (see Theorem 12.​1). As

the roots are $$z_{1} = 1/\phi$$ and z ₂ = 1. Thus, only the root z ₁ lies outside the unit circle whereas the root z ₂ lies on the unit circle. The existence of a unit root precludes the existence of a stationary solution. Note that we have just one unit root, although each of the two processes taken by themselves are integrated of order one.

The above VAR representation can be further transformed to yield a representation of process in first differences $$\{(\Delta X_{t},\Delta Y _{t})'\}$$:

This representation can be considered as a generalization of the Dickey-Fuller regression in first difference form (see Eq. (7.​1)). In the multivariate case, it is known as the vector error correction model (VECM) or vector error correction representation. In this representation the matrix

is of special importance. This matrix is singular and of rank one. This is not an implication which is special to this specification of the present discounted value model, but arises generally as shown in Campbell (1987) and Campbell and Shiller (1987). In the VECM representation all variables except $$(X_{t-1},Y _{t-1})'$$ are stationary by construction. This implies that $$-\Pi (X_{t-1},Y _{t-1})'$$ must be stationary too, despite the fact that {(X t , Y t )′} is not stationary as shown above. Multiplying $$-\Pi (X_{t-1},Y _{t-1})'$$ out, one obtains two linear combinations which define stationary processes. However, as $$\Pi $$ has only rank one, there is just one linearly independent combination. The first one is $$X_{t-1} -\gamma Y _{t-1}$$ and equals S t−1 which was already shown to be stationary. The second one is degenerate because it yields zero. The phenomenon is called cointegration.

Because $$\Pi $$ has rank one, it can be written as the product of two vectors α and β:

Clearly, this decomposition of $$\Pi $$ is not unique because $$\tilde{\alpha }= a\alpha$$ and $$\tilde{\beta }= a^{-1}\beta$$, a ≠ 0, would also qualify for such a decomposition as $$\Pi =\tilde{\alpha }\tilde{\beta } '$$. The vector β is called a cointegration vector. It has the property that {β′(X t , Y t )′} defines a stationary process despite the fact that {(X t , Y t )′} is non-stationary. The cointegration vector thus defines a linear combination of X t and Y t which is stationary. The matrix α, here only a vector, is called the loading matrix and its elements the loading coefficients.

The VAR and the VECM representations are both well suited for estimation. However, if we want to compute the impulse responses, we need a causal representation. Such a causal representation does not exist due to the unit root in the VAR process for {(X t , Y t )′} (see Theorem 12.​1). To circumvent this problem we split the matrix $$\Phi (z)$$ into the product of two matrices M(z) and V (z). M(z) is a diagonal matrix which encompasses all unit roots on its diagonal. V (z) has all its roots outside the unit circle so that V ⁻¹(z) exists for | z |  < 1. In our example, we get:

Multiplying $$\Phi (z)$$ with $$\widetilde{M}(z) = \left (\begin{array}{*{10}c} 1 - z&0\\ 0 &1\\ \end{array} \right )$$ from the left, we find:

The application of this result to the VAR representation of {(X t , Y t )} leads to a causal representation of $$\{(\Delta X_{t},\Delta Y _{t})\}$$:

The polynomial matrix $$\Psi (\mathrm{L})$$ can be recovered by the method of undetermined coefficients from the relation between V (L) and $$\Psi (\mathrm{L})$$:

In this exposition, we abstain from the explicit computation of V ⁻¹(L) and $$\Psi (\mathrm{L})$$. However, the following relation holds:

Implying that

The cointegration vector $$\beta = (1,-\gamma )'$$ and loading matrix $$\alpha = (-1,0)'$$ therefore have the following properties:

Like in the univariate case (see Theorem 7.​1 in Sect. 7.​1.​4), we can also construct the Beveridge-Nelson decomposition in the multivariate case. For this purpose, we decompose $$\Psi (\mathrm{L})$$ as follows:

with $$\widetilde{\Psi }_{j} =\sum _{ i=j+1}^{\infty }\Psi _{i}$$. This result can be used to derive the multivariate Beveridge-Nelson decomposition (see Theorem 16.1 in Sect. 16.2.3):

(16.1)

The Beveridge-Nelson decomposition represents the bivariate integrated process {(X t , Y t )′} as a sum of three components: a linear trend, a multivariate random walk and a stationary process. Multiplying the Beveridge-Nelson decomposition from the left by the cointegration vector $$\beta = (1,-\gamma )'$$, we see that both the trend and the random walk component are eliminated and that only the stationary component remains.

Because the first column of $$\Psi (1)$$ consists of zeros, only the second structural shock, namely {v t }, will have a long-run (permanent) effect. The long-run effect is $$\gamma /(1-\phi )$$ for the first variable, X t , and $$1/(1-\phi )$$ for the second variable, Y t . The first structural shock (preference shock) {u t } has non long-run effect, its impact is of a transitory nature only. This decomposition into permanent and transitory shocks is not typical for this model, but can be done in general as part of the so-called common trend representation (see Sect. 16.2.4).

Finally, we will simulate the reaction of the system to a unit valued shock in v t . Although this shock only has a temporary influence on $$\Delta Y _{t}$$, it will have a permanent effect on the level Y t . Taking ϕ = 0. 8, we get long-run effect (persistence) of $$1/(1-\phi ) = 5$$ as explained in Sect. 7.​1.​3. The present discounted value model then implies that this shock will also have a permanent effect on X t too. Setting γ = 1, this long-run effect is given by $$\gamma (1-\beta )\sum _{j=0}^{\infty }\beta ^{j}(1-\phi )^{-1} =\gamma /(1-\phi ) = 5$$. Because this long-run effect is anticipated in period t, the period of the occurrence of the shock, X t will increase by more than one. The spread turns, therefore, into positive. The error correction mechanism will then dampen the effect on future changes of X t so that the spread will return steadily to zero. The corresponding impulse responses of both variables are displayed in Fig. 16.1.

Fig. 16.1

Impulse response functions of the present discounted value model after a unit shock to Y t ($$\gamma = 1,\beta = 0.9,\phi = 0.8$$)

Figure 16.2 displays the trajectories of both variables after a stochastic simulation where both shocks {u t } and {v t } are drawn from a standard normal distribution. One can clearly discern the non-stationary character of both series. However, as it is typically for cointegrated series, they move more or less in parallel to each other. This parallel movement is ensured by the error correction mechanism. The difference between both series which is equal to the spread under this parameter constellation is mean reverting around zero.

Fig. 16.2

Stochastic simulation of the present discounted value model under standard normally distributed shocks ($$\gamma = 1,\beta = 0.9,\phi = 0.8$$)

16.2 Definition and Representation of Cointegrated Processes

16.2.1 Definition

We now want to make the concepts introduced earlier more precise and give a general definition of cointegrated processes and derive the different representations we have seen in the previous section. Given an arbitrary regular (purely non-deterministic) stationary process $$\{U_{t}\}_{t\in \mathbb{Z}}$$ of dimension n, n ≥ 1, with mean zero and some distribution for the starting random variable X ₀, we can define recursively a process {X t }, t = 0, 1, 2, … as follows:

Thereby, μ denotes an arbitrary constant vector of dimension n. If $$U_{t} \sim \mathrm{ WN}(0,\Sigma )$$, then {X t } is a multivariate random walk with drift μ. In general, however, {U t } is autocorrelated and possesses a Wold representation $$U_{t} = \Psi (L)Z_{t}$$ (see Sect. 14.​1.​1) such that

(16.2)

where $$Z_{t} \sim \mathrm{ WN}(0,\Sigma )$$ and $$\sum _{j=0}^{\infty }\|\Psi _{j}\|^{2} < \infty $$ with $$\Psi _{0} = I_{n}$$. We now introduce the following definitions.

Definition 16.1.

A regular stationary process {U t } with mean zero is integrated of order zero, I(0), if and only if it can be represented as

such that $$Z_{t} \sim \mathrm{ WN}(0,\Sigma )$$ , $$\sum _{j=0}^{\infty }j\|\Psi _{j}\| < \infty $$ , and $$\Psi (1) =\sum _{ j=0}^{\infty }\Psi _{j}\neq 0$$ .

Definition 16.2.

A stochastic process {X t } is integrated of order d, I(d), d = 0, 1,2,…, if and only if $$\Delta ^{d}(X_{t} - \mathbb{E}(X_{t}))$$ is integrated of order zero.

In the following we concentrate on I(1) processes. The definition of an I(1) process implies that {X t } equals $$X_{t} = X_{0} +\mu t +\sum _{ j=1}^{t}U_{j}$$ and is thus non-stationary even if μ = 0. The condition $$\Psi (1)\neq 0$$ corresponds to the one in the univariate case (compare Definition 7.​1 in Sect. 7.​1). On the one hand, it precludes the case that a trend-stationary process is classified as an integrated process. On the other hand, it implies that {X t } is in fact non-stationary. Indeed, if the condition is violated so that $$\Psi (1) = 0$$, we could express $$\Psi (\mathrm{L})$$ as $$(1 -\mathrm{ L})\widetilde{\Psi }(\mathrm{L})$$. Thus we could cancel 1 − L on both sides of Eq. (16.2) to obtain a stationary representation of {X t }, given some initial distribution for X ₀. This would then contradict our primal assumption that {X t } is non-stationary. The condition $$\sum _{j=0}^{\infty }j\|\Psi _{j}\| < \infty $$ is stronger than $$\sum _{j=0}^{\infty }\|\Psi _{j}\|^{2} < \infty $$ which follows from the Wold’s Theorem. It guarantees the existence of the Beveridge-Nelson decomposition (see Theorem 16.1 below).² In particular, the condition is fulfilled if {U t } is a causal ARMA process which is the prototypical case.

Like in the univariate case, we can decompose an I(1) process additively into several components.

Theorem 16.1 (Beveridge-Nelson Decomposition).

If {X t } is an integrated process of order one, it can be decomposed as

where $$V _{t} =\widetilde{ \Psi }(\mathrm{L})Z_{0} -\widetilde{ \Psi }(\mathrm{L})Z_{t}$$ with $$\widetilde{\Psi }_{j} =\sum _{ i=j+1}^{\infty }\Psi _{i}$$ , j = 0,1,2,… and {V t } stationary.

Proof.

Following the proof of the univariate case (see Sect. 7.​1.​4):

with $$\widetilde{\Psi }_{j} =\sum _{ i=j+1}^{\infty }\Psi _{i}$$. Thus,

The only point left is to show that $$\widetilde{\Psi }(\mathrm{L})Z_{0} -\widetilde{ \Psi }(\mathrm{L})Z_{t}$$ is stationary. Based on Theorem 10.​2, it is sufficient to show that the coefficient matrices are absolutely summable. This can be derived by applying the triangular inequality and the condition for integrated processes:

 □ 

The process {X t } can therefore be viewed as the sum of a linear trend, X ₀ +μ t, with stochastic intercept, a multivariate random walk, $$\Psi (1)\sum _{j=0}^{t}Z_{t}$$, and a stationary process {V t }. Based on this representation, we can then define the notion of cointegration (Engle and Granger 1987).

Definition 16.3 (Cointegration).

A multivariate stochastic process {X t } is called cointegrated if {X t } is integrated of order one and if there exists a vector $$\beta \in \mathbb{R}^{n}$$ , β ≠ 0, such that {β′X t }, is integrated of order zero, given a corresponding distribution for the initial random variable X ₀ . β is called the cointegrating or cointegration vector . The cointegrating rank is the maximal number, r, of linearly independent cointegrating vectors β ₁ ,…,β r . These vectors span a linear space called the cointegration space .

The Beveridge-Nelson decomposition implies that β is a cointegrating vector if and only if $$\beta '\Psi (1) = 0$$. In this case the random walk component $$\sum _{j=1}^{t}Z_{j}$$ is annihilated and only the deterministic and the stationary component remain.³ For some issues it is of interest whether the cointegration vector β also eliminates the trend component. This would be the case if β′μ = 0. See Sect. 16.3 for details.

The cointegration vectors are determined only up to some basis transformations. If β ₁, …, β r is a basis for the cointegration space then (β ₁, …, β r )R is also a basis for the cointegration space for any nonsingular r × r matrix R because $$((\beta _{1},\ldots,\beta _{r})R)'\Psi (1) = 0$$.

16.2.2 Vector Autoregressive (VAR) and Vector Error Correction Models (VECM)

Although the Beveridge-Nelson decomposition is very useful from a theoretical point of view, in practice it is often more convenient to work with alternative representations. Most empirical investigations of integrated processes start from a VAR(p) model which has the big advantage that it can be easily estimated:

(16.3)

where $$\Phi (\mathrm{L}) = I_{n} - \Phi _{1}\mathrm{L} -\ldots -\Phi _{p}\mathrm{L}^{p}$$ and c is an arbitrary constant. Subtracting X t−1 on both sides of the difference equation, the VAR model can be rewritten as:

(16.4)

where $$\Pi = -\Phi (1) = -I_{n} + \Phi _{1} +\ldots \Phi _{p}$$ and $$\Gamma _{i} = -\sum _{j=i+1}^{p}\Phi _{j}$$. We will make the following assumptions:

(i)

All roots of the polynomial $$\det \Phi (z)$$ are outside the unit circle or equal to one, i.e.

(ii)

The matrix $$\Pi $$ is singular with rank r, 1 ≤ r < n.

(iii)

$$\mathrm{Rank}(\Pi ) =\mathrm{ Rank}(\Pi ^{2})$$.

Assumption (i) makes sure that {X t } is an integrated process with order of integration d ≥ 1. Moreover, it precludes other roots on the unit circles than one. The case of seasonal unit roots is treated in Hylleberg et al. (1990) and Johansen and Schaumburg (1998).⁴ Assumption (ii) implies that there exists at least n − r unit roots and two n × r matrices α and β with full column rank r such that

The columns of β thereby represent the cointegration vectors whereas α denotes the so-called loading matrix. The decomposition of $$\Pi $$ in the product of α and β′ is not unique. For every non-singular r × r matrix R we can generate an alternative decomposition $$\Pi =\alpha \beta ' = (\alpha R'^{-1})(\beta R)'$$. Finally, assumption (iii) implies that the order of integration is exactly one and not greater. The number of unit roots is therefore exactly n − r.⁵ This has the implication that $$\Phi (z)$$ can be written as

where the roots of the matrix polynomials U(z) and V (z) are all outside the unit circle and where M(z) equals

This representation of $$\Phi (z)$$ is a special form of the Smith-McMillan factorization of polynomial matrices (see Kailath (1980) and Yoo (1987)). This factorization isolates the unit roots in one simple matrix so that the system can be analyzed more easily.

These assumptions will allow us to derive from the VAR(p) model several representations where each of them brings with it a particular interpretation. Replacing $$\Pi $$ by α β′ in Eq. (16.4), we obtain the vector error correction representation or vector error correction model (VECM):

(16.5)

Multiplying both sides of the equation by (α′α)⁻¹ α′ and solving for β′X t−1, we get:

α has full column rank r so that α′α is a non-singular r × r matrix. As the right hand side of the equation represents a stationary process, also the left hand side must be stationary. This means that the r-dimensional process {β′X t−1} is stationary despite the fact that {X t } is integrated and has potentially a unit root with multiplicity n.

The term error correction was coined by Davidson et al. (1978). They interpret the mean of β′X t , $$\mu ^{{\ast}} = \mathbb{E}\beta 'X_{t}$$, as the long-run equilibrium or steady state around which the system fluctuates. The deviation from equilibrium (error) is therefore given by $$\beta 'X_{t-1} -\mu ^{{\ast}}$$. The coefficients of the loading matrix α should then guarantee that deviations from the equilibrium are corrected over time by appropriate changes (corrections) in X t .

An Illustration

To illustrate the concept of the error correction model, we consider the following simple system with α = (α ₁, α ₂)′, α ₁ ≠ α ₂, and $$\beta = (1,-1)'$$. For simplicity, we assume that the long-run equilibrium μ ∗ is zero. Ignoring higher order lags, we consider the system:

The autoregressive polynomial of this system is:

The determinant of this polynomial is $$\det \Phi (z) = 1 - (2 +\alpha _{1} -\alpha _{2})z + (1 +\alpha _{1} -\alpha _{2})z^{2}$$ with roots equal to z = 1 and $$z = 1/(1 +\alpha _{1} -\alpha _{2})$$. This shows that assumption (i) is fulfilled. As $$\Pi = \left (\begin{array}{*{10}c} \alpha _{1} & -\alpha _{1} \\ \alpha _{2} & -\alpha _{2}\\ \end{array} \right )$$, the rank of $$\Pi $$ equals one which implies assumption (ii). Finally,

Thus, the rank of $$\Pi ^{2}$$ is also one because α ₁ ≠ α ₂. Hence, assumption (iii) is also fulfilled.

We can gain an additional insight into the system by subtracting the second equation from the first one to obtain:

The process $$\beta 'X_{t} = X_{1t} - X_{2t}$$ is stationary and causal with respect to Z 1t − Z 2t if and only if $$\vert 1 +\alpha _{1} -\alpha _{2}\vert < 1$$, or equivalently if and only if $$-2 <\alpha _{1} -\alpha _{2} < 0$$. Note the importance of the assumption that α ₁ ≠ α ₂. It prevents that X 1t − X 2t becomes a random walk and thus a non-stationary (integrated) process. A sufficient condition is that − 1 < α ₁ < 0 and 0 < α ₂ < 1 which imply that a positive (negative) error, i.e. $$X_{1,t-1} - X_{2,t-1} > 0(< 0)$$, is corrected by a negative (positive) change in X 1t and a positive (negative) change in X 2t . Although the shocks Z 1t and Z 2t push X 1t − X 2t time and again away from its long-run equilibrium, the error correction mechanism ensures that the variables are adjusted in such a way that the system moves back to its long-run equilibrium.

16.2.3 The Beveridge-Nelson Decomposition

We next want to derive from the VAR representation a causal representation or MA$$(\infty )$$ representation for $$\{\Delta X_{t}\}$$. In contrast to a normal causal VAR model, the presence of unit roots precludes the simple application of the method of undetermined coefficients, but requires an additional effort. Multiplying the VAR representation in Eq. (16.3), $$\Phi (\mathrm{L})X_{t} = U(\mathrm{L})M(\mathrm{L})V (\mathrm{L})X_{t} = c + Z_{t}$$, from the left by U ⁻¹(L) we obtain:

Multiplying this equation by $$\widetilde{M}(\mathrm{L}) = \left (\begin{array}{*{10}c} I_{n-r}& 0 \\ 0 &(1 -\mathrm{ L})I_{r}\\ \end{array} \right )$$ leads to:

which finally leads to

This is the MA$$(\infty )$$ representation of $$\{\Delta X_{t}\}$$ and corresponds to Eq. (16.2).

Because $$\Pi = -\Phi (1) = -U(1)M(1)V (1)$$, the following relation holds for the partitioned matrices:

This implies that we can define α and β as

U(1) and V (1) are non-singular so that α and β have full column rank r. Based on this derivation we can formulate the following lemma.

Lemma 16.1.

The columns of the so-defined matrix β are the cointegration vectors for the process {X t }. The corresponding matrix of loading coefficients is α which fulfills $$\Psi (1)\alpha = 0$$ .

Proof.

We must show that $$\beta '\Psi (1) = 0$$ which is the defining property of cointegration vectors. Denoting by (V (ij)(1)) i, j = 1, 2 the appropriately partitioned matrix of V (1)⁻¹, we obtain:

where the last equality is a consequence of the property of the inverse matrix.

With the same arguments, we can show that $$\Psi (1)\alpha = 0$$. □ 

The equivalence between the VEC and the MA representation is known as Granger’s representation theorem in the literature. Granger’s representation theorem immediately implies the Beveridge-Nelson decomposition:

(16.6)

(16.7)

where the stochastic process {V t } is stationary and defined as $$V _{t} =\widetilde{ \Psi }(\mathrm{L})Z_{0} -\widetilde{ \Psi }(\mathrm{L})Z_{t}$$ with $$\widetilde{\Psi }_{j} =\sum _{ i=j+1}^{\infty }\Psi _{i}$$ and $$\Psi (\mathrm{L}) = V ^{-1}(\mathrm{L})\widetilde{M}(\mathrm{L})U^{-1}(\mathrm{L})$$. As $$\beta '\Psi (1) =\beta 'V ^{-1}(1)\widetilde{M}(1)U^{-1}(1) = 0$$, β eliminates the stochastic trend (random walk), $$\sum _{j=1}^{t}Z_{t}$$, as well as the deterministic linear trend $$\mu t = V ^{-1}(1)\widetilde{M}(1)U^{-1}(1)c\:t$$.

An interesting special case is obtained when the constant c is a linear combination of the columns of α, i.e. if there exists a vector g such that c = α g. Under this circumstance, $$\Psi (1)c = \Psi (1)\alpha g = 0$$ and the linear trend vanishes and we have $$\mathbb{E}\Delta X_{t} = 0$$. In this case, the data will exhibit no trend although the VAR model contains a constant. A similar consideration can be made if the VAR model is specified to contain a constant and a linear time trend d t. The Beveridge-Nelson decomposition would then imply that the data should follow a quadratic trend. However, in the special case that d is a linear combination of the columns of α, the quadratic trend disappears and only the linear remains because of the constant.

16.2.4 Common Trend and Triangular Representation

The $$\Psi (1)$$ in the Beveridge-Nelson decomposition is singular. This implies that the multivariate random walk $$\Psi (1)\sum _{j=1}^{\infty }Z_{j}$$ does not consist of n independent univariate random walks. Instead only n − r independent random walks make up the stochastic trend so that {X t } is driven by n − r stochastic trends. In order to emphasize this fact, we derive from the Beveridge-Nelson decomposition the so-called common trend representation (Stock and Watson 1988a).

As $$\Psi (1)$$ has rank n − r, there exists a n × r matrix γ such that $$\Psi (1)\gamma = 0$$. Denote by γ  ⊥  the n × (n − r) matrix whose columns are orthogonal to γ, i.e. γ′γ  ⊥  = 0. The Beveridge-Nelson decomposition can then be rewritten as:

where $$\tilde{c} = \left (\begin{array}{*{10}c} \gamma ^{\perp }&\vdots&\gamma \end{array} \right )^{-1}c$$ and $$\widetilde{Z}_{j} = \left (\begin{array}{*{10}c} \gamma ^{\perp }&\vdots&\gamma \end{array} \right )^{-1}Z_{j}$$. Therefore, only the first n − r elements of the vector $$\tilde{c}$$ are relevant for the deterministic linear trend. The remaining elements are multiplied by zero and are thus irrelevant. Similarly, for the multivariate random walk only the first n − r elements of the process $$\{\widetilde{Z}_{t}\}$$ are responsible for the stochastic trend. The remaining elements of $$\widetilde{Z}_{t}$$ are multiplied by zero and thus have no permanent, but only a transitory influence. The above representation decomposes the shocks orthogonally into permanent and transitory ones (Gonzalo and Ng 2001). The previous lemma shows that one can choose for γ the matrix of loading coefficients α.

Summarizing the first n − r elements of $$\tilde{c}$$ and $$\widetilde{Z}_{t}$$ to $$\tilde{c}_{1}$$ and $$\widetilde{Z}_{1t}$$, respectively, we arrive at the common trend representation:

where the n × (n − r) matrix B is equal to $$\Psi (1)\gamma ^{\perp }$$.

Applying these results to our introductory example, we arrive at

This again demonstrates that the trend, the linear as well as the stochastic trend, are exclusively stemming from the nonstationary variables {Y t } (compare with Eq. (16.1)).

Finally, we want to present a triangular representation which is well suited to deal with the nonparametric estimation approach advocated by Phillips (1991) and Phillips and Hansen (1990) (see Sect. 16.4). In this representation we normalize the cointegration vector such $$\beta = (I_{r},-b')'$$. In addition, we partition the vector X t into X 1t and X 2t such that X 1t contains the first r and X 2t the last n − r elements. X t  = (X 1t ′, X 2t )′ can then be expressed as:

(16.8a)

(16.8b)

where D t summarizes the deterministic components such as constant and linear time trend. {u 1t } and {u 2t } denote potentially autocorrelated and cross-correlated stationary time series.

16.3 Johansen’s Test for Cointegration

In Sect. 7.​5.​2 we have already discussed a regression based test for cointegration among two variables. It was based on a unit root of the residuals from a bivariate regression of one variable against the other. In this regression, it turned out to be irrelevant which of the two variables was chosen as the regressor and which one as the regressand. This method can, in principle, be extended to more than two variables. However, with more than two variables, the choice of the regressand becomes more crucial as not all variables may be part of the cointegrating relation. Moreover, more than one independent cointegrating relation may exist. For these reasons, it is advantageous to use a method which treats all variables symmetrically. The cointegration test developed by Johansen fulfills this criterion because it is based on a VAR which does not single out a particular variable. This test has received wide recognition and is most often used in practice. The test serves two purposes. First, we want to determine the number r of cointegrating relationships. Second, we want to test properties of the cointegration vector β and the loading matrix α.

The exposition of the Johansen test follows closely the work of Johansen where the derivations and additional details can be found (Johansen 1988, 1991, 1995). We start with a VAR(p) model with constant c in VEC form (see Eq. (16.4)):

(16.9)

where $$Z_{t} \sim \mathrm{ IIDN}(0,\Sigma )$$ and given starting values X ₀ = x ₀, …, $$X_{-p+1} = x_{-p+1}$$. The problem can be simplified by regressing $$\Delta X_{t}$$ as well as X t−1 against $$c,\Delta X_{t-1},\ldots,\Delta X_{t-p+1}$$ and working with the residuals from these regressions.⁶ This simplification results in a VAR model of order one. We therefore start our analysis without loss of generality with a VAR(1) model without constant term:

where $$Z_{t} \sim \mathrm{ IIDN}(0,\Sigma )$$.⁷

The phenomenon of cointegration manifests itself in the singularity of the matrix $$\Pi $$. In particular, we want to determine the rank of $$\Pi $$ which gives the number of linearly independent cointegrating relationships. Denoting by r, the rank of $$\Pi $$, 0 ≤ r ≤ n,, we can formulate a sequence of hypotheses:

Hypothesis H(r), thus, implies that there exists at most r linearly independent cointegrating vectors. The sequence of hypotheses is nested in the sense that H(r) implies H(r + 1):

The hypothesis H(0) means that $$\mathrm{rank}(\Pi ) = 0$$. In this case, $$\Pi = 0$$ and there are no cointegration vectors. {X t } is thus driven by n independent random walks and the VAR can be transformed into a VAR model for $$\{\Delta X_{t}\}$$ which in our simplified version just means that $$\Delta X_{t} = Z_{t} \sim \mathrm{ IIDN}(0,\Sigma )$$. The hypothesis H(n) places no restriction on $$\Pi $$ and includes in this way the case that the level of {X t } is already stationary. Of particular interest are the hypotheses between these two extreme ones where non-degenerate cointegrating vectors are possible. In the following, we not only want to test for the number of linearly independent cointegrating vectors, r, but we also want to test hypotheses about the structure of the cointegrating vectors summarized in β.

Johansen’s test is conceived as a likelihood-ratio test. This means that we must determine the likelihood function for a sample X ₁, X ₂, …, X T where T denotes the sample size. For this purpose, we assume that $$\{Z_{t}\} \sim \mathrm{ IIDN}(0,\Sigma )$$ so that logged likelihood function of the parameters α, β, and $$\Sigma $$ conditional on the starting values is given by:

where $$\Pi =\alpha \beta '$$. For a fixed given β, α can be estimated by a regression of $$\Delta X_{t}$$ on β′X t−1:

where the moment matrices S ₀₀, S ₁₁, S ₀₁ and S ₁₀ are defined as:

The covariance matrix of the residuals then becomes:

Using these results, we can concentrate the log-likelihood function to obtain:

(16.10)

The expression $$\frac{Tn} {2}$$ in the above equation is derived as follows:

The log-likelihood function is thus maximized if

is minimized over β.⁸ The minimum is obtained by solving the following generalized eigenvalue problem (Johansen 1995):

This eigenvalue problem delivers n eigenvalues

with corresponding n eigenvectors $$\hat{\beta }_{1},\ldots,\hat{\beta }_{n}$$. These eigenvectors are normalized such that $$\hat{\beta }'S_{11}\hat{\beta } = I_{n}$$. Therefore we have that

Remark 16.1.

In the case of cointegration, $$\Pi $$ is singular with $$\mathrm{rank}\widehat{\Pi } = r$$. To estimate r, it seems natural to investigate the number of nonzero eigenvalues of $$\widehat{\Pi } = S_{01}S_{11}^{-1}$$. However, because eigenvalues may be complex, it is advantageous not to investigate the eigenvalues of $$\widehat{\Pi }$$ but those of $$\widehat{\Pi }'\widehat{\Pi }$$ which are all real and positive due to the symmetry of $$\widehat{\Pi }'\widehat{\Pi }$$. These eigenvalues are called the singular values of $$\widehat{\Pi }$$.⁹ Noting that

the generalized eigenvalue problem above therefore just determines the singular values of $$S_{00}^{-1/2}S_{01}S_{11}^{-1/2} = S_{00}^{-1/2}\widehat{\Pi }S_{11}^{1/2}$$.

Remark 16.2.

Based on the observation that, for n = 1, $$\lambda = \frac{S_{01}S_{10}} {S_{11}S_{00}}$$ equals the squared empirical correlation coefficient between $$\Delta X_{t}$$ and X t−1, we find that the eigenvalues $$\lambda _{j}$$, j = 1, …, n, are nothing but the squared canonical correlation coefficients (see Johansen 1995; Reinsel 1993). Thereby, the largest eigenvalue, $$\hat{\lambda }_{1}$$, corresponds to the largest squared correlation coefficient that can be achieved between linear combinations of $$\Delta X_{1}$$ and X t−1. Thus β ₁ gives the linear combination of the integrated variable X t−1 which comes closest in the sense of correlation to the stationary variable $$\{\Delta X_{t}\}$$. The second eigenvalue $$\lambda _{2}$$ corresponds to the maximal squared correlation coefficient between linear combinations of $$\Delta X_{t}$$ and X t−1 which are orthogonal to the linear combination corresponding to $$\lambda _{1}$$. The remaining squared canonical correlation coefficients are obtained by iterating this procedure n times.

If the dimension of the cointegrating space is r then $$\hat{\beta }$$ consists of those eigenvectors which correspond to the r largest eigenvalues $$\hat{\lambda }_{1},\ldots,\hat{\lambda }_{r}$$. The remaining eigenvalues $$\lambda _{r+1},\ldots,\lambda _{n}$$ should be zero. Under the null hypothesis H(r), the log-likelihood function (16.10) can be finally expressed as:

The expression for the optimized likelihood function can now be used to construct the Johansen likelihood-ratio test. There are two versions of the test depending on the alternative hypothesis:

The corresponding likelihood ratio test statistics are therefore:

In practice it is useful to adopt a sequential test strategy based on the trace test. Given some significance level, we test in a first step the null hypothesis H(0) against H(n). If, on the one hand, the test does not reject the null hypothesis, we conclude that r = 0 and that there is no cointegrating relation. If, on the other hand, the test rejects the null hypothesis, we conclude that there is at least one cointegrating relation. We then test in a second step the null hypothesis H(1) against H(n). If the test does not reject the null hypothesis, we conclude that there exists one cointegrating relation, i.e. that r = 1. If the test rejects the null hypothesis, we examine the next hypothesis H(2), and so on. In this way we obtain a test sequence. If in this sequence, the null hypothesis H(r) is not rejected, but H(r + 1) is, we conclude that exist r linearly independent cointegrating relations as explained in the diagram below.

If in this sequence we do not reject H(r) for some r, it is useful to perform the max test H(r) against H(r + 1) as a robustness check. The asymptotic distributions of the test statistics are, like in the Dickey-Fuller unit root test, nonstandard and depend on the specification of the deterministic components.

16.3.1 Specification of the Deterministic Components

As mentioned previously, the asymptotic distribution of Johansen’s test depends on the specification of the deterministic components. Thus, some care must be devoted to this issue. We illustrate this point by decomposing the model additively into a linear deterministic and a stochastic component in vector error correction form (see Johansen (1995, 80–84) and Lütkepohl (2006, section 6.​4)):

(16.11)

(16.12)

For the ease of exposition, we have omit the autoregressive corrections. Eliminating Y t using $$Y _{t} = X_{t} -\mu _{0} -\mu _{1}t$$ and $$\Delta Y _{t} = \Delta X_{t} -\mu _{1}$$ leads to

This equation can be rewritten as

(16.13)

with $$c_{0} =\mu _{1} -\alpha \beta '\mu _{0}$$ and $$c_{1} = -\alpha \beta '\mu _{1}$$

(16.14)

where X t ⁰ = (X t ′, t)′. Equation (16.13) is just the vector error correction model (16.4) augmented by the linear trend term c ₁ t. If the term c ₁ would be left unrestricted arguments similar to those in Sect. 16.2.3 would show that X t exhibits a deterministic quadratic trend with coefficient vector $$\Psi (1)c_{1}$$. This, however, contradicts the specification in Eq. (16.11). However, if we recognize that c ₁ in Eq. (16.13) is actually restricted to lie in the span of α, i.e. that c ₁ = α γ ₁ with $$\gamma _{1} = -\beta '\mu _{1}$$, no quadratic trend would emerge in the levels because $$\Psi (1)\alpha = 0$$ by Granger’s representation Theorem 16.1. Alternatively, one may view the time trend as showing up in the error correction term, respectively being part of the cointegrating relation, as in Eq. (16.14).

Similarly, one may consider the case that X t has a constant mean μ ₀, i.e. that μ ₁ = 0 in Eq. (16.11). This leads to the same error correction specification (16.13), but without the term c ₁ t. Leaving the constant c ₀ unrestricted, this will generate a linear trend $$\Psi (1)c_{0}t$$ as shown in Sect. 16.2.3. In order to reconcile this with the assumption of a constant mean, we must recognize that c ₀ = α γ ₀ with $$\gamma _{0} = -\beta '\mu _{0}$$.

Based on these arguments, we can summarize the discussion by distinguishing five different cases displayed in Table 16.1.¹⁰ This table also shows the implications for $$\mathbb{E}\Delta X_{t}$$ and $$\mathbb{E}(\beta 'X_{t})$$. These can read off from Eqs. (16.13) and (16.14).

Table 16.1

Trend specifications in vector error correction models

Case

Deterministic term in VECM or VAR

Restriction

Trend in X t

I

None

–

Zero

Zero

Zero

II

c ₀

c ₀ = α γ ₀

Constant

Zero

Constant

III

c ₀

None

Linear

Constant

Constant

IV

c ₀ + c ₁ t

c ₁ = α γ ₁

Linear

Constant

Linear

V

c ₀ + c ₁ t

None

Quadratic

Linear

Linear

Table inspired by Johansen (2007)

The corresponding asymptotic distributions of the trace as well as the max test statistic in these five cases are tabulated in Johansen (1995), MacKinnon et al. (1999), and Osterwald-Lenum (1992). ¹¹ The finite sample properties of theses tests can be quite poor. Thus, more recently, bootstrap methods have been proven to provide a successful alternative in practice (Cavaliere et al. 2012).

16.3.2 Testing Hypotheses on Cointegrating Vectors

As mentioned previously, the cointegrating vectors are not unique, only the cointegrating space is. This makes the cointegrating vectors often difficult to interpret economically, despite some basis transformation. It is therefore of interest to see whether the space spanned by the cointegrating vectors summarized in the columns of $$\hat{\beta }$$ can be viewed as a subspace spanned by some hypothetical vectors H = (h ₁, …, h s ), r ≤ s < n. If this hypothesis is true, the cointegrating vectors should be linear combinations of the columns of H so that the null hypothesis can be formulated as

(16.15)

for some s × r matrix $$\varphi$$. Under this null hypothesis, this amounts to solve an analogous general eigenvalue problem:

The solution of this problem is given by the eigenvalues $$1 >\tilde{\lambda } _{1} \geq \tilde{\lambda }_{2} \geq \ldots \tilde{\lambda }_{s} > 0$$ with corresponding normalized eigenvectors. The likelihood ratio test statistic for this hypothesis is then

This test statistic is asymptotically distributed as a χ ² distribution with r(n − s) degrees of freedom.

With similar arguments it is possible to construct a test of the null hypothesis that the cointegrating space spanned by the columns of $$\hat{\beta }$$ contains some hypothetical vectors K = (h ₁, …, h s ), 1 ≤ s ≤ r. The null hypothesis can then be formulated as

(16.16)

for some s × r matrix $$\varphi$$. Like in the previous case, this hypothesis can also be tested by the corresponding likelihood ratio test statistic which is asymptotically distributed as a χ ² distribution with s(n − r) degrees of freedom. Similarly, it is possible to test hypotheses on α and joint hypotheses on α and β (see Johansen 1995; Kunst and Neusser 1990; Lütkepohl 2006).

16.4 Estimation and Testing of Cointegrating Relationships

Johansen’s approach has become very popular because it presents an integrated framework for testing and estimating cointegrating relationships based on the maximum likelihood method. However, it requires the specification of a concrete VAR model. This proves sometimes difficult it practice, especially when the true data generating process is not purely autoregressive. Similar to the Phillips-Perron test discussed in Sect. 7.​3.​2, Phillips and Hansen (1990) propose a nonparametric approach for the estimation and hypothesis testing of cointegrating relationships. This approach is especially appropriate if the long-run relationships are the prime objective of the investigation as f.e. in Neusser and Kugler (1998).

The Phillips and Hansen approach is based on the triangular representation of cointegrated processes given in the equation system (16.8). Thereby the r cointegration vectors are normalized such that $$\beta = (I_{r},-b')'$$ where b is the regression coefficient matrix from a regression of X 1t on X 2t controlling for deterministic components D t (see Eq. (16.8a)).¹² The least-squares estimate of b is (super) consistent as already noted in Sect. 7.​5.​2. However, the estimator is not directly suitable for hypothesis testing because the conventional test statistics do not have the usual asymptotic distributions. The idea of Phillips and Hansen (1990) is to correct the conventional least-squares estimates to account for serial correlation and for the endogeneity arising from the cointegrating relationship. This leads to the fully-modified ordinary least-squares estimator (FMOLS estimator). As the endogeneity shows up in the long-run correlation between the variables, the proposed modification uses of the long-run variance J of u t  = (u 1t ′, u 2t ′)′. According to Sect. 11.​1 this entity defined as:

where

The fully-modified ordinary least-squares estimator of (b, π ₁) is then constructed as follows. Estimate the Eqs. (16.8a) and (16.8b) by ordinary least-squares to compute the residuals $$\hat{u}_{t} = (\hat{u}_{1t}',\hat{u}_{2t}')'$$. From these residuals estimate $$\Sigma $$ as $$\widehat{\Sigma } =\sum _{ t=1}^{T}u_{t}u_{t}'$$ and the long-run variance J and its one-sided counterpart $$\Lambda $$. Estimates of J and $$\Lambda $$, denoted by $$\hat{J }$$ and $$\hat{\Lambda }$$, can be obtained by applying a kernel estimator as explained in Sect. 4.​4, respectively Sect. 11.​1. The estimators $$\widehat{\Sigma }$$, $$\hat{J }$$ and $$\hat{\Lambda }$$ are consistent because ordinary least-squares is. The corresponding estimates are then used to correct the data for X 1t and to construct the bias correction term $$\hat{\Lambda }_{21}^{(+)}$$:

The fully-modified ordinary least-squares estimator (FMOLS estimator) is then given by

It turns out that this estimator is asymptotically equivalent to full maximum likelihood with limiting distributions free of nuisance parameters.

The main advantage of the FMOLS estimator is that conventional Wald test statistics, appropriately modified, have limiting χ ² distributions. This brings statistical inference back to the realm of traditional econometric analysis. Consider testing the null hypothesis

where q is a vector of dimension g and R selects the appropriate elements of $$\mathop{\mathrm{vec}}b$$. Thus, in effect we are considering hypothesis of the form H₀: b = b ₀. The hypothesis b = 0 is thereby of particular interest. The Wald test statistic is then defined as

where $$\hat{J } _{11.2} = \hat{J } _{11} -\hat{J } _{12}\hat{J } _{22}^{-1}\hat{J } _{21}$$. It can be shown that the so defined modified Wald test statistic is asymptotically distributed as χ ² with g degrees of freedom (see Phillips and Hansen 1990; Hansen 1992).

16.5 An Example

This example reproduces the study by Neusser (1991) with actualized data for the United States over the period first quarter 1950 to fourth quarter 2005. The starting point is a VAR model which consists of four variables: real gross domestic product (Y), real private consumption (C), real gross investment (I), and the ex-post real interest rate (R). All variables, except the real interest rate, are in logs. First, we identify a VAR model for these variables where the order is determined by Akaike’s (AIC), Schwarz’ (BIC) or Hannan-Quinn’ (HQ) information criteria. The AIC suggests seven lags whereas the other criteria propose a VAR of order two. As the VAR(7) consists of many statistically insignificant coefficients, we prefer the more parsimonious VAR(2) model which produces the following estimates:

where the estimated standard errors of the corresponding coefficients are reported in parenthesis. The estimate covariance matrix $$\Sigma $$, $$\widehat{\Sigma }$$, is

The sequence of the hypotheses starts with H(0) which states that there exists no cointegrating relation. The alternative hypothesis is always H(n) which says that there are n cointegrating relations. According to Table 16.2 the value of the trace test statistic is 111.772 which is clearly larger than the 5 % critical value of 47.856. Thus, the null hypothesis H(0) is rejected and we consider next the hypothesis H(1). This hypothesis is again clearly rejected so that we move on to the hypothesis H(2). Because H(3) is not rejected, we conclude that there exists 3 cointegrating relations. To check this result, we test the hypothesis H(2) against H(3) using the max test. As this test also rejects H(2), we can be pretty confident that there are three cointegrating relations given as:

This matrix is actually the outcome from the EVIEWS econometrics software package. It should be noted that EVIEWS, like other packages, chooses the normalization mechanically. This can become a problem if the variable on which the cointegration vectors are normalized is not part of the cointegrating relation.

Table 16.2

Evaluation of the results of Johansen’s cointegration test

Trace statistic

Max statistic

Null hypothesis

Eigenvalue

Test statistic

Critical value

Test statistic

Critical value

H(0): r = 0

0.190

111.772

47.856

47.194

27.584

H(1): r ≤ 1

0.179

64.578

29.797

44.075

21.132

H(2): r ≤ 2

0.081

20.503

15.495

18.983

14.265

H(3): r ≤ 3

0.007

1.520

3.841

1.520

3.841

Critical 5 % values are taken from MacKinnon et al. (1999)

In this form, the cointegrating vectors are economically difficult to interpret. We therefore ask whether they are compatible with the following hypotheses:

These hypotheses state that the log-difference (ratio) between consumption and GDP, the log-difference (ratio) between investment and GDP, and the real interest rate are stationary. They can be rationalized in the context of the neoclassical growth model (see King et al. 1991; Neusser 1991). Each of them can be brought into the form of Eq. (16.16) where β is replaced by its estimate $$\hat{\beta }$$. The corresponding test statistics for each of the three cointegrating relations is distributed as a χ ² distribution with one degree of freedom,¹³ which gives a critical value of 3.84 at the 5 % significance level. The corresponding values for the test statistic are 12.69, 15.05 and 0.45, respectively. This implies that we must reject the first two hypotheses β C and β I . However, the conjecture that the real interest is stationary, cannot be rejected. Finally, we can investigate the joint hypothesis β ₀ = (β C , β I , β R ) which can be represented in the form (16.15). In this case the value of the test statistic is 41.20 which is clearly above the critical value of 7.81 inferred from the χ ₃ ² distribution.¹⁴ Thus, we must reject this joint hypothesis.

As a matter of comparison, we perform a similar investigation using the fully-modified approach of Phillips and Hansen (1990). For this purpose we restrict the analysis to Y t , C t , and I t because the real interest rate cannot be classified unambiguously as being stationary, respectively integrated of order one. The long-run variance J and its one-sided counterpart $$\Lambda $$ are estimated using the quadratic spectral kernel with VAR(1) prewhitening as advocated by Andrews and Monahan (1992) (see Sect. 4.​4). Assuming two cointegrating relations and taking Y t and C t as the left hand side variables in the cointegrating regression (Eq. (16.8a)), the following results are obtained:

where the estimated standard deviations are reported in parenthesis. The specification allows for a constant and a deterministic trend as well as a drift in the equation for $$\Delta I_{t}$$ (Eq. (16.8b), not shown).

Given these results we can test a number of hypotheses to get a better understanding of the cointegrating relations. First we test the hypothesis of no cointegration of Y t , respectively C t with I t . Thus, we test $$\mathrm{H}_{0}: b(1) = b(2) = 0$$. The value of the corresponding Wald test statistic is equal to 2.386 which is considerably less than the 5 % critical value of 5.992. Therefore we can not reject the null hypothesis no cointegration Another interesting hypothesis is H₀: b(1) = b(2) which would mean that Y t and C t are cointegrated with cointegration vector (1, −1). As the corresponding Wald statistic is equal to 0.315, this hypothesis can not be rejected at the 5 % critical value of 3.842. This suggests a long-run relation between Y t and C t .

Repeating the analysis with C t and I t as the left hand side variables leads to the following results:

As before first the hypothesis $$H_{0}: b(1) = b(2) = 0$$ is tested. The corresponding value of the test statistic is 137.984 which is clearly above the 5 % critical value. Thus, the null hypothesis of no cointegration is rejected. Next, the hypothesis $$H_{0}: b(1) = b(2) = 1$$ is tested. This hypothesis is rejected as the value 7.717 of the test statistic is above the critical value. If these hypotheses are not tested jointly, but individually, the null hypothesis b(1) = 1 can be rejected, but b(2) = 1 can not. These findings conform reasonably well with those based on the Johansen approach.

The diverse result between the two specifications demonstrates that the sensitivity of cointegration analysis with respect to the normalization. It is important that the variable on which the cointegrating vector is normalized is indeed in the cointegrating space. Otherwise, insensible results may be obtained.

References

Andrews DWK, Monahan JC (1992) An improved heteroskedasticity and autocorrelation consistent covariance matrix estimator. Econometrica 60:953–966CrossRef

Banerjee A, Dolado J, Galbraith JW, Hendry DF (1993) Co-integration, error-correction, and the econometric analysis of non-stationary data. Oxford University Press, OxfordCrossRef

Bauer D, Wagner M (2003) A canonical form for unit root processes on the state space framework, diskussionsschrift 3-12, Volkswirtschaftliches Institut, Universität Bern

Beaudry P, Portier F (2006) Stock prices, news and economic fluctuations. Am Econ Rev 96(4):1293–1307CrossRef

Campbell JY (1987) Does saving anticipate declining labor income? An alternative test of the permanent income hypothesis. Econometrica 55:1249–1273CrossRef

Campbell JY, Shiller RJ (1987) Cointegration and tests of present value models. J Polit Econ 95:1062–1088CrossRef

Cavaliere G, Rahbek A, Taylor AMR (2012) Bootstrap determination of the co-integration rank in vector autoregressive models. Econometrica 80:1721–1740CrossRef

Davidson JEH, Hendry DF, Srba F, Yeo S (1978) Econometric modelling of the aggregate time-series relationship between consumers’ expenditure and income in the United Kingdom. Econ J 88:661–692CrossRef

Davidson R, MacKinnon JG (1993) Estimation and inference in econometrics. Oxford University Press, Oxford

Engle RF, Granger CWJ (1987) Co-integration and error correction: Representation, estimation and testing. Econometrica 55:251–276CrossRef

Gonzalo J, Ng S (2001) A systematic framework for analyzing the dynamic effects of permanent and transitory schocks. J Econ Dyn Control 25:1527–1546CrossRef

Hansen BE (1992) Efficient estimation and testing of cointegrating vectors in the presence of deterministic trends. J Econ 53:321–335CrossRef

Hylleberg S, Engle RF, Granger CWJ, Yoo S (1990) Seasonal integration and cointegration. J Econ 44:215–238CrossRef

Johansen S (1988) Statistical analysis of cointegration vectors. J Econ Dyn Control 12:231–254CrossRef

Johansen S (1991) Estimation and hypothesis testing of cointegration vectors in Gaussian vector autoregressive models. Econometrica 59:1551–1580CrossRef

Johansen S (1995) Likelihood-based inference in cointegrated vector autoregressive models. Oxford University Press, OxfordCrossRef

Johansen S (2007) Cointegration: A survey. In: Mills TC, Patterson K (eds) Econometric theory, palgrave handbook of econometrics, vol 1. Palgrave MacMillan, Basingstoke and New York, pp 540–577

Johansen S, Schaumburg E (1998) Likelihood analysis of seasonal cointegration. J Econ 88:301–339CrossRef

Kailath T (1980) Linear systems. Prentice Hall, Englewood Cliffs

King RG, Plosser CI, Stock JH, Watson MW (1991) Stochastic trends and economic fluctuations. Am Econ Rev 81:819–840

Kunst R, Neusser K (1990) Cointegration in a macroeconomic system. J Appl Econ 5:351–365CrossRef

Lütkepohl H (2006) New introduction to multiple time series analysis. Springer, Berlin

MacKinnon JG, Haug A, Michelis L (1999) Numerical distribution functions of the likelihood ratio test for cointegration. J Appl Econ 14:563–577CrossRef

Meyer CD (2000) Matrix analysis and applied linear algebra. Society for Industrial and Applied Mathematics, PhiladelphiaCrossRef

Neusser K (1991) Testing the long-run implications of the neoclassical growth model. J Monet Econ 27:3–37CrossRef

Neusser K (2000) An algebraic interpretation of cointegration. Econ Lett 67:273–281CrossRef

Neusser K, Kugler M (1998) Manufacturing growth and financial development: Evidence from oecd countries. Rev Econ Stat 80:638–646CrossRef

Osterwald-Lenum M (1992) A note with quantiles of the asymptotic distribution of the maximum likelihood cointegration rank test statistics. Oxford Bull Econ Stat 54:461–471CrossRef

Phillips PCB (1991) Optimal inference in cointegrating systems. Econometrica 59:283–306CrossRef

Phillips PCB, Hansen BE (1990) Statistical inference in instrumental variables regresssion with I(1) processes. Rev Econ Stud 57:99–125CrossRef

Phillips PCB, Solo V (1992) Asymptotics for linear processes. Ann Stat 20:971–1001CrossRef

Reinsel GC (1993) Elements of multivariate time series analysis. Springer Series in statistics. Springer, New YorkCrossRef

Stock JH, Watson MW (1988a) Testing for common trends. J Am Stat Assoc 83:1097–1107

Strang G (1988) Linear algebra and its applications, 3rd edn. Harcourt Brace Jovanovich, San Diego

Watson MW (1994) Vector autoregressions and cointegration. Handbook of econometrics, vol 4, chap 47. North-Holland, Amsterdam, pp 2843–2915

Yoo BS (1987) Multi-cointegrated time series and a generalized error correction model. Ph.D., University of California, San Diego

Footnotes

1

A more recent interesting application of this model is given by the work of Beaudry and Portier (2006).

2

This condition could be relaxed and replaced by the condition $$\sum _{j=0}^{\infty }j^{2}\|\Psi _{j}\|^{2} < \infty $$. In addition, this condition is an important assumption for the application of the law of large numbers and for the derivation of the asymptotic distribution (Phillips and Solo 1992).

3

The distribution of X ₀ is thereby chosen such that $$\beta 'X_{0} =\beta '\widetilde{\Psi }(\mathrm{L})Z_{0}$$.

4

The seasonal unit roots are the roots of $$z^{s} - 1 = 0$$ where s denotes the number of seasons. These roots can be expressed as $$\cos (2k\pi /s) +\iota \sin (2k\pi /s)$$, $$k = 0, 1,\ldots,s - 1$$.

5

For details see Johansen (1995), Neusser (2000) and Bauer and Wagner (2003).

6

If the VAR model (16.9) contains further deterministic components besides the constant, these components have to be accounted for in these regressions.

7

This two-stage least-squares procedure is also known as partial regression and is part of the Frisch-Waugh-Lowell Theorem (Davidson and MacKinnon 1993, 19–24).

8

Thereby we make use of the following equality for partitioned matrices:

where A ₁₁ and A ₂₂ are invertible matrices (see for example Meyer 2000, p. 475).

9

An appraisal of the singular values of a matrix can be found in Strang (1988) or Meyer (2000).

10

It is instructive to compare theses cases to those of the unit root test (see Sect. 7.​3.​1).

11

The tables by MacKinnon et al. (1999) allow for the possibility of exogenous integrated variables.

12

The choice of the variables used for normalization turns out to be important in practice. See the application in Sect. 16.5.

13

The degrees of freedom are computed according to the formula: $$s(n - r) = 1(4 - 3) = 1$$.

14

The degrees of freedom are computed according to the formula: $$r(n - s) = 3(4 - 3) = 3$$.

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_17

17. State-Space Models and the Kalman Filter

Klaus Neusser1 

(1)

Bern, Switzerland

The state space representation is a flexible technique originally developed in automatic control engineering to represent, model, and control dynamic systems. Thereby we summarize the unobserved or partially observed state of the system in period t by an m-dimensional vector X t . The evolution of the state is then described by a VAR of order one usually called the state equation. A second equation describes the connection between the state and the observations given by a n-dimensional vector Y t . Despite its simple structure, state space models encompass a large variety of model classes: VARMA, respectively VARIMA models,¹ unobserved-component models, factor models, structural time series models which decompose a given time series into a trend, a seasonal, and a cyclical component, models with measurement errors, VAR models with time-varying parameters, etc.

From a technical point of view, the main advantage of state space modeling is the unified treatment of estimation, forecasting, and smoothing. At the center of the analysis stands the Kalman-filter named after its inventor Rudolf Emil Kálmán (Kalman 1960, 1963). He developed a projection based algorithm which recursively produces a statistically optimal estimate of the state. The versatility and the ease of implementation have made the Kalman filter an increasingly popular tool also in the economically oriented times series literature. Here we present just an introduction to the subject and refer to Anderson and Moore (1979), Brockwell and Davis (1991, Chapter 12), Brockwell and Davis (1996, Chapter 8), Hamilton (1994b, Chapter 13), Hamilton (1994a), Hannan and Deistler (1988), or Harvey (1989), and in particular to Durbin and Koopman (2011) and Kim and Nelson (1999) for extensive reviews and further details.

17.1 The State Space Model

We consider a dynamical system whose state at each point in time t is determined by a vector X t . The evolution of the system over time is then described by a state equation. The state is, however, unobserved or only partly observed to the outside observer. Thus, a second equation, called the observation equation, is needed to describe the connection of the state to the observations. This relation may be subject to measurement errors. The equation system consisting of state and observation equation is called a state space model which is visualized in Fig. 17.1. The state equation typically consists of a VAR model of order one whereas the observation equation has the structure of multivariate linear regression model.² Despite the simplicity of each of these two components, their combination is very versatile and able to represent a great variety of models.

Fig. 17.1

State space model

In the case of time invariant coefficients³ we can set up these two equations as follows:

(17.1)

(17.2)

Thereby X t denotes an m-dimensional vector which describes the state of the system in period t. The evolution of the state is represented as a vector autoregressive model of order one with coefficient matrix F and disturbances V t+1.⁴ As we assume that the state X t is unobservable or at least partly unobservable, we need a second equation which relates the state to the observations. In particular, we assume that there is a linear time-invariant relation given by A and G of the n-dimensional vector of observations, Y t , to the state X t . This relation may be contaminated by measurement errors W t . The system is initialized in period t = 1.

We make the following simplifying assumption of the state space model represented by Eqs. (17.1) and (17.2).

(i)

$$\{V _{t}\} \sim \mathrm{ WN}(0,Q)$$ where Q is a constant nonnegative definite m × m matrix.

(ii)

$$\{W_{t}\} \sim \mathrm{ WN}(0,R)$$ where R is a constant nonnegative definite n × n matrix.

(iii)

The two disturbances are uncorrelated with each other at all leads and lags, i.e.:

(iv)

V t and W t are multivariate normally distributed.

(v)

X ₁ is uncorrelated with V t as well as with W t , t = 1, 2, …

Remark 17.1.

In a more general context, we can make both covariance matrices Q and R time-varying and allow for contemporaneous correlations between V t and W t (see example Sect. 17.4.1).

Remark 17.2.

As both the state and the observation equation may include identities, the covariance matrices need not be positive definite. They can be non-negative definite.

Remark 17.3.

Neither {X t } nor {Y t } are assumed to be stationary.

Remark 17.4.

The specification of the state equation and the normality assumption imply that the sequence {X ₁, V ₁, V ₂, …} is independent so that the conditional distribution X t+1 given X t , X t−1, …, X ₁ equals the conditional distribution of X t+1 given X t . Thus, the process {X t } satisfies the Markov property. As the dimension of the state vector X t is arbitrary, it can be expanded in such a way as to encompass every component X t−1 for any t (see, for example, the state space representation of a VAR(p) model with p > 1). However, there remains the problem of the smallest dimension of the state vector (see Sect. 17.3.2).

Remark 17.5.

The state space representation is not unique. Defining, for example, a new state vector $$\tilde{X}_{t}$$ by multiplying X t with an invertible matrix P, i.e. $$\tilde{X}_{t} = PX_{t}$$, all properties of the system remain unchanged. Naturally, we must redefine all the system matrices accordingly: $$\tilde{F} = PFP^{-1}$$, $$\tilde{Q} = PQP'$$, $$\tilde{G} = GP^{-1}$$.

Given X ₁, we can iterate the state equation forward to arrive at:

The state equation is called stable or causal if all eigenvalues of F are inside the unit circle which is equivalent that all roots of $$\det (\mathrm{I}_{m} - Fz) = 0$$ are outside the unit circle (see Sect. 12.​3). In this case the state equation has a unique stationary solution:

(17.3)

The process {Y t } is therefore also stationary and we have:

(17.4)

In the case of a stationary state space model, we may do without an initialization period and take $$t \in \mathbb{Z}$$.

In the case of a stable state equation, we can easily deduce the covariance function for {X t }, $$\Gamma _{X}(h)$$, h = 0, 1, 2, … According to Sect. 12.​4 it holds that:

where $$\Gamma _{X}(0)$$ is uniquely determined given the stability assumption. Similarly, we can derive the covariance function for the observation vector, $$\Gamma _{Y }(h)$$, h = 0, 1, 2, …:

17.1.1 Examples

The following examples should illustrate the versatility of the state space model and demonstrate how many economically relevant models can be represented in this form.

VAR(p) Process

Suppose that {Y t } follows a n-dimensional VAR(p) process given by $$\Phi (L)Y _{t} = Z_{t}$$, respectively by $$Y _{t} = \Phi _{1}Y _{t-1} +\ldots +\Phi _{p}Y _{t-p} + Z_{t}$$, with $$Z_{t} \sim \mathrm{ WN}(0,\Sigma )$$. Then the companion form of the VAR(p) process (see Sect. 12.​2) just represents the state equation (17.1):

with $$V _{t+1} = (Z_{t+1}',0,0,\ldots,0)'$$ and $$Q = \left (\begin{array}{*{10}c} \Sigma &0\\ 0 &0\\ \end{array} \right ).$$ The observation equation is just an identity because all components of X t are observable:

Thus, G = (I n , 0, 0, …, 0) and R = 0. Assuming that X t is already mean adjusted, A = 0.

ARMA(1,1) Process

The representation of ARMA processes as a state space model is more involved when moving-average terms are involved. Let {Y t } be an ARMA(1,1) process defined by the stochastic difference equation $$Y _{t} =\phi Y _{t-1} + Z_{t} +\theta Z_{t-1}$$ with $$Z_{t} \sim \mathrm{ WN}(0,\sigma ^{2})$$ and $$\phi \theta \neq 0$$.

Define {X t } as the AR(1) process defined by the stochastic difference equation $$X_{t} -\phi X_{t-1} = Z_{t}$$ and $$\mathbf{X}_{t} = (X_{t},X_{t-1})'$$ as the state vector, then we can write the observation equation as:

with R = 0. The state equation is then

where $$Q = \left (\begin{array}{*{10}c} \sigma ^{2} & 0 \\ 0&0\\ \end{array} \right )$$. It is easy to verify that the so defined process {Y t } satisfies the stochastic difference equation $$Y _{t} =\phi Y _{t-1} + Z_{t} +\theta Z_{t-1}$$. Indeed .

If | ϕ |  < 1, the state equation defines a causal process {X t } so that the unique stationary solution is given by Eq. (17.3). This implies a stationary solution for {Y t } too. It is thus easy to verify if this solution equals the unique solution of the ARMA stochastic difference equation.

The state space representation of an ARMA model is not unique. An alternative representation in the case of a causal system is given by:

Note that in this representation the dimension of the state vector is reduced from two to one. Moreover, the two disturbances $$V _{t+1} = (\phi +\theta )Z_{t}$$ and W t  = Z t are perfectly correlated.

ARMA(p,q) Process

It is straightforward to extend the above representation to ARMA(p,q) models.⁵ Let {Y t } be defined by the following stochastic difference equation:

Define r as $$r =\max \{ p,q + 1\}$$ and set ϕ j  = 0 for j > p and $$\theta _{j} = 0$$ for j > q. Then, we can set up the following state space representation with state vector X t and observation equation

where the state vector equals $$\mathbf{X}_{t} = (X_{t},\ldots,X_{t-r+2},X_{t-r+1})'$$ and where {X t } follows an AR(p) process $$\Phi (\mathrm{L})X_{t} = Z_{t}$$. The AR(p) process can be transformed into companion form to arrive at the state equation:

Missing Observations

The state space approach is best suited to deal with missing observations. However, in this situation the coefficient matrices are no longer constant, but time-varying. Consider the following simple example of an AR(1) process for which we have only observations for the periods t = 1, …, 100 and t = 102, …, 200, but not for period t = 101 which is missing. This situation can be represented in state space form as follows:

This means that W t  = 0 and that Y t  = X t for all t except for t = 101. For the missing observation, we have $$G_{101} = Y _{101} = 0$$. The variance for this observation is set to R ₁₀₁ = c > 0.

The same idea can be used to obtain quarterly data when only yearly data are available. This problem typically arises in statistical offices which have to produce, for example, quarterly GDP data from yearly observations incorporating quarterly information from indicator variables (see Sect. 17.4.1). More detailed analysis for the case of missing data can be found in Harvey and Pierce (1984) and Brockwell and Davis (1991, Chapter 12.3).

Time-Varying Coefficients

Consider the regression model with time-varying parameter vector β t :

(17.5)

where Y t is an observed dependent variable, x t is a K-vector of exogenous regressors, and W t is a white noise error term. Depending on the specification of the evolution of β t several models have been proposed in the literature:

where v t , v 1t , and v 2t are white noise error terms. In the first specification, proposed originally proposed by Hildreth and Houck (1968), the parameter vector is in each period just a random from a distribution with mean $$\bar{\beta }$$ and variance given by the variance of v t . Departures from the mean are seen as being only of a transitory nature. In the specification by Harvey and Phillips (1982), assuming that all eigenvalues of F are strictly smaller than one in absolute value, the parameter vector is a mean reverting VAR of order one. In this case, the departures from the mean can have a longer duration depending on the eigenvalues of F. The last specification due to Cooley and Prescott (1973, 1976) views the parameter vector as being subject to transitory and permanent shifts. Whereas shifts in v 1t have only a transitory effect on β t , movements in v 2t result in permanent effects.

In the Cooley-Prescott specification, for example, the state is given by X t  = (β t ′, β t p ′)′ and the state equation can be written as:

The observation equation then becomes:

Thus, A = 0 and G t  = (x t ′, 0). Note that this an example of a state space model with time-varying coefficients. In Sect. 18.​2, we will discuss time-varying coefficient models in the context of VAR models.

Structural Time Series Analysis

An important application of the state space representation in economics is the decomposition of a given time series into several components: trend, cycle, season and irregular component. This type of analysis is usually coined structural time series analysis (See Harvey 1989; Mills 2003). Consider, for example, the additive decomposition of a time series {Y t } into a trend T t , a seasonal component S t , a cyclical component {C t }, and an irregular or cyclical component W t :

The above equation relates the observed time series to its unobserved components and is called the basic structural model (BSM) (Harvey 1989).

The state space representation is derived in several steps. Consider first the case with no seasonal and no cyclical component. The trend is typically viewed as a random walk with time-varying drift δ t−1:

The second equation models the drift as a random walk. The two disturbances $$\{\varepsilon _{t}\}$$ and $$\{\xi _{t}\}$$ are assumed to be uncorrelated with each other and with {W t }. Defining the state vector X t (T) as X t (T) = (T t , δ t )′, the state and the observation equations become:

with $$W_{t} \sim \mathrm{ WN}(0,\sigma _{W}^{2})$$. This representation is called the local linear trend (LLT) model and implies that {Y t } follows an ARIMA(0,2,2) process (see Exercise 17.5.1).

In the special case of a constant drift equal to δ, $$\sigma _{\xi }^{2} = 0$$ and we have that $$\Delta Y _{t} =\delta +\varepsilon _{t} + W_{t} - W_{t-1}$$. $$\{\Delta Y _{t}\}$$ therefore follows a MA(1) process with $$\rho (1) = -\sigma _{W}^{2}/(\sigma _{\varepsilon }^{2} + 2\sigma _{W}^{2}) = -(2+\kappa )^{-1}$$ where $$\kappa =\sigma _{ \varepsilon }^{2}/\sigma _{W}^{2}$$ is called the signal-to-noise ratio. Note that the first order autocorrelation is necessarily negative. Thus, this model is not suited for time series with positive first order autocorrelation in its first differences.

The seasonal component is characterized by two conditions $$S_{t} = S_{t-d}$$ and $$\sum _{t=1}^{d}S_{t} = 0$$ where d denotes the frequency of the data.⁶ Given starting values $$S_{1},S_{0},S_{-1},\ldots,S_{-d+3}$$, the subsequent values can be computed recursively as:

where a noise $$\eta _{t} \sim \mathrm{ WN}(0,\sigma _{\eta }^{2})$$ is taken into account.⁷ The state vector related to the seasonal component, X t (S), is defined as $$X_{t}^{(S)} = (S_{t},S_{t-1},\ldots,S_{t-d+2})'$$ which gives the state equation

with $$Q^{(S)} =\mathrm{ diag}(\sigma _{\eta }^{2},0,\ldots,0)$$.

Combining the trend and the seasonal model to an overall model with state vector X t given by $$X_{t} = \left (X_{t}^{}{(T)}',X_{t}^{}{(S)}'\right )'$$, we arrive at the state equation:

with $$Q =\mathrm{ diag}(\sigma _{\varepsilon }^{2},\sigma _{\delta }^{2},\sigma _{\eta }^{2},0,\ldots,0)$$. The observation equation then is:

with $$R =\sigma _{ W}^{2}$$.

Finally, we can add a cyclical component {C t } which is modeled as a harmonic process (see Sect. 6.​2) with frequency $$\lambda _{C}$$, respectively periodicity $$2\pi /\lambda _{C}$$:

Following Harvey (1989, p.39), we let the parameters A and B evolve over time by introducing the recursion

where C ₀ = A and C ₀ ∗ = B and where {C t ∗} is an auxiliary process. The dampening factor ρ allows for additional flexibility in the specification. The processes {V 1, t (C)} and {V 2, t (C)} are two mutually uncorrelated white noise processes. It is instructive to examine the spectral density (see Sect. 6.​1) of the cyclical component in Fig. 17.2. It can be shown (see Exercise 17.5.2) that {C t } follows an ARMA(2,1) process.

Fig. 17.2

Spectral density of the cyclical component for different values of $$\lambda _{C}$$ and ρ

The cyclical component can be incorporated into the state space model above by augmenting the state vector X t+1 by the cyclical components C t+1 and C t+1 ∗ and the error term V t+1 by {V 1, t+1 (C)} and {V 2, t+1 (C)}. The observations equation has to be amended accordingly. Section 17.4.2 presents an empirical application of this approach.

Dynamic Factor Models

Dynamic factor models are an interesting approach when it comes to modeling simultaneously a large cross-section of times series. The concept was introduced into macroeconomics by Sargent and Sims (1977) and was then developed further and popularized by Quah and Sargent (1993), Reichlin (2003) and Breitung and Eickmeier (2006), among others. The idea is to view each time series Y it , i = 1, …, n, as the sum of a linear combination of some joint unobserved factors f t  = (f 1t , …, f rt )′ and an idiosyncratic component {W it }, i = 1, … n. Dynamic factor models are particularly effective when the number of factors r is small compared to the number of time series n. In practice, several hundred time series are related to a handful factors. In matrix notation we can write the observation equation for the dynamic factor model as follows:

where $$\Lambda _{i}$$, i = 0, 1, …, q, are n × r matrices. The state vector X t equals (f t ′, …, f t−q ′)′ if we assume that the idiosyncratic component is white noise, i.e. $$W_{t} = (W_{1t},\ldots,W_{nt})' \sim \mathrm{ WN}(0,R)$$. The observation equation can then be written compactly as:

where $$G = (\Lambda _{0},\Lambda _{1},\ldots,\Lambda _{q})$$. Usually, we assume that R is a diagonal matrix. The correlation between the different time series is captured exclusively by the joint factors.

The state equation depends on the assumed dynamics of the factors. One possibility is to model {f t } as a VAR(p) process with $$\Phi (\mathrm{L})f_{t} = e_{t}$$, $$e_{t} \sim \mathrm{ WN}(0,\Sigma )$$, and p ≤ q + 1, so we can use the state space representation of the VAR(p) process from above. For the case p = 2 and q = 2 we get:

and $$Q =\mathrm{ diag}(\Sigma,0,0)$$. This scheme can be easily generalized to the case p > q + 1 or to allow for autocorrelated idiosyncratic components, assuming for example that they follow autoregressive processes.

The dimension of these models can be considerably reduced by an appropriate re–parametrization or by collapsing the state space adequately (Bräuning and Koopman 2014). Such a reduction can considerably increase the efficiency of the estimation.

Real Business Cycle Model (RBC Model)

State space models are becoming increasingly popular in macroeconomics, especially in the context of dynamic stochastic general equilibrium (DSGE) models. These models can be seen as generalizations of the real business cycle (RBC) models.⁸ In these models a representative consumer is supposed to maximize the utility of his consumption stream over his infinite life time. Thereby, the consumer has the choice to consume part of his income or to invest his savings (part of his income which is not consumed) at the market rate of interest. These savings can be used as a mean to finance investment projects which increase the economy wide capital stock. The increased capital stock then allows for increased production in the future. The production process itself is subject to a random shocks called technology shocks.

The solution of this optimization problem is a nonlinear dynamic system which determines the capital stock and consumption in every period. Its local behavior can be investigated by linearizing the system around its steady state. This equation can then be interpreted as the state equation of the system. The parameters of this equation F and Q are related, typically in a nonlinear way, to the parameters describing the utility and the production function as well as the process of technology shocks. Thus, the state equation summarizes the behavior of the theoretical model.

The parameters of the state equation can then be estimated by relating the state vector, given by the capital stock and the state of the technology, via the observation equation to some observable variables, like real GDP, consumption, investment, or the interest rate. This then completes the state space representation of the model which can be analyzed and estimated using the tools presented in Sect. 17.3.⁹

17.2 Filtering and Smoothing

As we have seen, the state space model provides a very flexible framework for a wide array of applications. We therefore want to develop a set of tools to handle this kind of models in terms of interpretation and estimation. In this section we will analyze the problem of inferring the unobserved state from the data given the parameters of the model. In Sect. 17.3 we will then investigate the estimation of the parameters by maximum likelihood.

In many cases the state of the system is not or only partially observable. It is therefore of interest to infer from the data Y ₁, Y ₂, …, Y T the state vector X t . We can distinguish three types of problems depending on the information used:

(i)

estimation of X t from Y ₁, …, Y t−1, known as the prediction problem;

(ii)

estimation of X t from Y ₁, …, Y t , known as the filtering problem;

(iii)

estimation of X t from Y ₁, …, Y T , known as the smoothing problem.

For the ease of exposition, we will assume that the disturbances V t and W t are normally distributed. The recursive nature of the state equation implies that $$X_{t} = F^{t-1}X_{1} +\sum _{ j=0}^{t-2}F^{j}V _{t-j}$$. Therefore, X t is also normally distributed for all t, if X ₁ is normally distributed. From the observation equation we can infer also that Y t is normally distributed, because it is the sum of two normally distributed random variables A + GX t and W t . Thus, under these assumptions, the vector (X ₁′, …, X T ′, Y ₁′, …, Y T ′)′ is jointly normally distributed:

where the covariance matrices $$\Gamma _{X},\Gamma _{Y X},\Gamma _{XY }$$ and $$\Gamma _{Y }$$ can be retrieved from the model given the parameters.

For the understanding of the rest of this section, the following theorem is essential (see standard textbooks, like Amemiya 1994; Greene 2008).

Theorem 17.1.

Let Z be a n-dimensional normally distributed random variable with $$Z \sim \mathrm{ N}(\mu,\Sigma )$$ . Consider the partitioned vector Z = (Z ₁ ′,Z ₂ ′)′ where Z ₁ and Z ₂ are of dimensions n ₁ ≥ 1 and n ₂ ≥ 1, $$n = n_{1} + n_{2}$$ , respectively. The corresponding partitioning of the covariance matrix $$\Sigma $$ is

where $$\Sigma _{11} = \mathbb{V}Z_{1}$$ , $$\Sigma _{22} = \mathbb{V}Z_{2}$$ , and $$\Sigma _{12} = \Sigma _{21}' =\mathrm{ cov}(Z_{1},Z_{2}) = \mathbb{E}(Z_{1} - \mathbb{E}Z_{1})'(Z_{2} - \mathbb{E}Z_{2})$$ . Then the partitioned vectors Z ₁ and Z ₂ are normally distributed. Moreover, the conditional distribution of Z ₁ given Z ₂ is also normal with mean and variance

This formula can be directly applied to figure out the mean and the variance of the state vector given the observations. Thus, setting Z ₁ = (X ₁′, …, X t ′)′ and $$Z_{2} = (Y _{1}',\ldots,Y _{t-1}')'$$, we get the predicted values; setting Z ₁ = (X ₁′, …, X t ′)′ and Z ₂ = (Y ₁′, …, Y t ′)′, we get the filtered values; setting Z ₁ = (X ₁′, …, X t ′)′ and Z ₂ = (Y ₁′, …, Y T ′)′, we get the smoothed values.

AR(1) Process with Measurement Errors

We illustrate the above ideas by analyzing a univariate AR(1) process with measurement errors¹⁰ :

For simplicity, we assume | ϕ |  < 1. Suppose that we only have observations Y ₁ and Y ₂ at our disposal. The joint distribution of (X ₁, X ₂, Y ₁, Y ₂)′ is normal. The covariances can be computed by applying the methods discussed in Chap. 2:

The smoothed values are obtained by applying the formula from Theorem 17.1:

Note that for the last observation, Y ₂ in our case, the filtered and the smoothed values are the same. For X ₁ the filtered value is

An intuition for this result can be obtained by considering some special cases. For ϕ = 0, the observations are not correlated over time. The filtered value for X ₁ therefore corresponds to the smoothed one. This value lies between zero, the unconditional mean of X ₁, and Y ₁ with the variance ratio $$\sigma _{w}^{2}/\sigma _{v}^{2}$$ delivering the weights: the smaller the variance of the measurement error the closer the filtered value is to Y ₁. This conclusion holds also in general. If the variance of the measurement error is relatively large, the observations do not deliver much information so that the filtered and the smoothed values are close to the unconditional mean.

For large systems the method suggested by Theorem 17.1 may run into numerical problems due to the inversion of the covariance matrix of Y, $$\Sigma _{22}$$. This matrix can become rather large as it is of dimension nT × nT. Fortunately, there exist recursive solutions to this problem known as the Kalman filter, and also the Kalman smoother.

17.2.1 The Kalman Filter

The Kalman filter circumvents the problem of inverting a large nT × nT matrix by making use of the Markov property of the system (see Remark 17.4). The distribution of X t given the observations up to period t can thereby be computed recursively from the distribution of the state in period t − 1 given the information available up to period t − 1. Starting from some initial distribution in period 0, we can in this way obtain in T steps the distribution of all states. In each step only an n × n matrix must be inverted. To describe the procedure in detail, we introduce the following notation:

Suppose, we have already determined the distribution of X t conditional on the observations Y ₁, …, Y t . Because we are operating in a framework of normally distributed random variables, the distribution is completely characterized by its conditional mean X t | t and variance P t | t . The goal is to carry forward these entities to obtain $$X_{t+1\vert t+1}$$ and $$P_{t+1\vert t+1}$$ having observed an additional data point Y t+1. This problem can be decomposed into a forecasting and an updating step.

Step 1: Forecasting Step The state equation and the assumption about the disturbance term V t+1 imply:

(17.6)

The observation equation then allows to compute a forecast of Y t+1 where we assume for simplicity that A = 0:

(17.7)

Step 2: Updating Step In this step the additional information coming from the additional observation Y t+1 is processed to update the conditional distribution of the state vector. The joint conditional distribution of $$(X_{t+1}',Y _{t+1}')'$$ given Y ₁, …, Y t is

As all elements of the distribution are available from the forecasting step, we can again apply Theorem 17.1 to get the distribution of the filtered state vector at time t + 1:

(17.8)

(17.9)

where we replace X t+1 | t , P t+1 | t , and Y t+1 | t by FX t | t , FP t | t F′ + Q, and GFX t | t , respectively, which have been obtained from the forecasting step.

Starting from given values for X 0 | 0 and P 0 | 0, we can therefore iteratively compute X t | t and P t | t for all t = 1, 2, …, T. Only the information from the last period is necessary at each step. Inserting Eq. (17.8) into Eq. (17.6) we obtain as a forecasting equation:

where the matrix

is know as the (Kalman) gain matrix. It prescribes how the innovation $$Y _{t} - Y _{t\vert t-1} = Y _{t} - GX_{t\vert t-1}$$ leads to an update of the predicted state.

Initializing the Algorithm It remains to determine how to initialize the recursion. In particular, how to set the starting values for X 0 | 0 and P 0 | 0. If X t is stationary and causal with respect to V t , the state equation has the solution $$X_{0} =\sum _{ j=0}^{\infty }F^{j}V _{t-j}$$. Thus,

where P 0 | 0 solves the equation (see Sect. 12.​4)

According to Eq. (12.​4), the solution of the above matrix equation is:

If the process is not stationary, we can set X 0 | 0 to zero and P 0 | 0 to infinity. In practice, a very large number is sufficient.

17.2.2 The Kalman Smoother

The Kalman filter determines the distribution of the state at time t given the information available up to this time. In many instances, we want, however, make an optimal forecast of the state given all the information available, i.e. the whole sample. Thus, we want to determine X t | T and P t | T . The Kalman filter determines the smoothed distribution for t = T, i.e. X T | T and P T | T . The idea of the Kalman smoother is again to determine the smoothed distribution in a recursive manner. For this purpose, we let the recursion run backwards. Starting with the last observation in period t = T, we proceed back in time by letting t take successively the values $$t = T - 1,T - 2,\ldots$$ until the first observation in period t = 1.

Using again the linearity of the equations and the normality assumption, we get:

This implies that

The above mean is only conditional on all information available up to time t and on the information at time t + 1. The Markov property implies that this mean also incorporates the information from the observations Y t+1, …, Y T . Thus, we have:

Applying the law of iterated expectations or means (see, f.e.  Amemiya 1994, p. 78), we can derive X t | T :

(17.10)

The algorithm can now be implemented as follows. In the first step compute X T−1 | T according to Eq. (17.10) as

All entities on the right hand side can readily be computed by applying the Kalman filter. Having found X T−1 | T , we can again use Eq. (17.10) for $$t = T - 2$$ to evaluate X T−2 | T :

Proceeding backward through the sample we can derive a complete sequence of smoothed states $$X_{T\vert T},X_{T-1\vert T},X_{T-2\vert T},\ldots,X_{1\vert T}$$. These calculations are based on the computations of X t | t , X t+1 | t , P t | t , and P t+1 | t which have already been obtained from the Kalman filter. The smoothed covariance matrix P t | T is given as (see Hamilton 1994b, Section 13.6):

Thus, we can compute also the smoothed variance with the aid of the values already determined by the Kalman filter.

AR(1) Process with Measurement Errors (Continued)

We continue our illustrative example of an AR(1) process with measurement errors and just two observations. First, we determine the filtered values for the state vector with the aid of the Kalman filter. To initialize the process, we have to assign a distribution to X ₀. For simplicity, we assume that | ϕ |  < 1 so that it makes sense to assign the stationary distribution of the process as the distribution for X ₀:

Then we compute the forecasting step as the first step of the filter (see Eq. (17.6)):

P 1 | 0 was computed by the recursive formula from the previous section, but is, of course, equal to the unconditional variance. For the updating step, we get from Eqs. (17.8) and (17.9):

These two results are then used to calculate the next iteration of the algorithm. This will give the filtered values for t = 2 which would correspond to the smoothed values because we just have two observations. The forecasting step is:

Next we perform the updating step to calculate X 2 | 2 and P 2 | 2. It is easy to verify that this leads to the same results as in the first part of this example.

An interesting special case is obtained when we assume that ϕ = 1 so that the state variable is a simple random walk. In this case the unconditional variance of X t and consequently also of Y t are no longer finite. As mentioned previously, we can initialize the Kalman filter by X 0 | 0 = 0 and $$P_{0\vert 0} = \infty $$. This implies:

Inserting this result in the updating Eqs. (17.8) and (17.9), we arrive at:

Letting P 0 | 0 go to infinity, leads to:

This shows that the filtered variance is finite for t = 1 although P 1 | 0 was infinite.

17.3 Estimation of State Space Models

Up to now we have assumed that the parameters of the system are known and that only the state is unknown. In most economic applications, however, also the parameters are unknown and have therefore to be estimated from the data. One big advantage of the state space models is that they provide an integrated approach to forecasting, smoothing and estimation. In particular, the Kalman filter turns out to be an efficient and quick way to compute the likelihood function. Thus, it seems natural to estimate the parameters of state space models by the method of maximum likelihood. Kim and Nelson (1999) and Durbin and Koopman (2011) provide excellent and extensive reviews of the estimation of state space models using the Kalman filter.

More recently, due to advances in computational methods, in particular with respect to sparse matrix programming, other approaches can be implemented. For example, by giving the states a matrix representation Chan and Jeliazkov (2009) derive a viable and efficient method for the estimation of state space models.

17.3.1 The Likelihood Function

The joint unconditional density of the observations (Y ₁′, …, Y T ′)′ can be factorized into the product of conditional densities as follows:

Each conditional density is the density of a normal distribution and is therefore given by:

where $$\Delta _{t} = GP_{t\vert t-1}G' + R$$. The Gaussian likelihood function L is therefore equal to:

Note that all the entities necessary to evaluate the likelihood function are provided by the Kalman filter. Thus, the evaluation of the likelihood function is a byproduct of the Kalman filter. The maximum likelihood estimator (MLE) is then given by the maximizer of the likelihood function, or more conveniently the log-likelihood function. Usually, there is no analytic solution available so that one must resort to numerical methods. An estimation of the asymptotic covariance matrix can be obtained by evaluating the Hessian matrix at the optimum. Under the usual assumptions, the MLE is consistent and delivers asymptotically normally distributed estimates (Greene 2008; Amemiya 1994).

The direct maximization of the likelihood function is often not easy in practice, especially for large systems involving many parameters. The expectation-maximization algorithm, EM algorithm for short, represents a valid, though slower alternative. As the name indicates, it consists of two steps which have to be carried out iteratively. Based on some starting values for the parameters, the first step (expectation step) computes estimates, X t | T , of the unobserved state vector X t using the Kalman smoother. In the second step (maximization step), the likelihood function is maximized taking the estimates of X t , X t | T , as additional observations. The treatment of X t | T as additional observations, allows to reduce the maximization step to a simple multivariate regression. Indeed, by treating X t | T as if they were known, the state equation becomes a simple VAR(1) which can be readily estimated by linear least-squares to obtain the parameters F and Q. The parameters A, G and R are also easily retrieved from a regression of Y t on X t | T . Based on these new parameter estimates, we go back to step one and derive new estimates for X t | T which are then used in the maximization step. One can show that this procedure maximizes the original likelihood function (see Dempster et al. 1977; Wu 1983). A more detailed analysis of the EM algorithm in the time series context is provided by Brockwell and Davis (1996).¹¹

Sometimes it is of interest not only to compute parameter estimates and to derive from them estimates for the state vector via the Kalman filter or smoother, but also to find confidence intervals for the estimated state vector to take the uncertainty into account. If the parameters are known, the methods outlined previously showed how to obtain these confidence intervals. If, however, the parameters have to be estimated, there is a double uncertainty: the uncertainty from the filter and the uncertainty arising from the parameter estimates. One way to account for this additional uncertainty is by the use of simulations. Thereby, we draw a given number of parameter vectors from the asymptotic distribution and compute for each of these draws the corresponding estimates for the state vector. The variation in these estimates is then a measure of the uncertainty arising from the estimation of the parameters (see Hamilton 1994b, Section 13.7).

17.3.2 Identification

As emphasized in Remark 17.5 of Sect. 17.1, the state space representations are not unique. See, for example, the two alternative representations of the ARMA(1,1) model in Sect. 17.1. This non-uniqueness of state space models poses an identification problem because different specifications may give rise to observationally equivalent models.¹² This problem is especially serious if all states are unobservable. In practice, the identification problem gives rise to difficulties in the numerical maximization of the likelihood function. For example, one may obtain large differences for small variations in the starting values; or one may encounter difficulties in the inversion of the matrix of second derivatives.

The identification of state space models can be checked by transforming them into VARMA models and by investigating the issue in this reparameterized setting (Hannan and Deistler 1988). Exercise 17.5.6 invites the reader to apply this method to the AR(1) model with measurement errors. System identification is a special field in systems theory and will not be pursued further here. A systematic treatment can be found in the textbook by Ljung (1999).

17.4 Examples

17.4.1 Disaggregating Yearly Data into Quarterly Ones

The official data for quarterly GDP are released in Switzerland by the State Secretariat for Economic Affairs (SECO). They estimate these data taking the yearly values provided by the Federal Statistical Office (FSO) as given. This division of tasks is not uncommon in many countries. One of the most popular methods for disaggregation of yearly data into quarterly ones was proposed by Chow and Lin (1971).¹³ It is a regression based method which can take additional information in the form of indicator variables (i.e. variables which are measured at the higher frequency and correlated at the lower frequency with the variable of interest) into account. This procedure is, however, rather rigid. The state space framework is much more flexible and ideally suited to deal with missing observations. Applications of this framework to the problem of disaggregation were provided by Bernanke et al. (1997:1) and Cuche and Hess (2000), among others. We will illustrate this approach below.

Starting point of the analysis are the yearly growth rates of GDP and indicator variables which are recorded at the quarterly frequency and which are correlated with GDP growth. In our application, we will consider the growth of industrial production (IP) and the index on consumer sentiment (C) as indicators. Both variables are available on a quarterly basis from 1990 onward. For simplicity, we assume that the annualized quarterly growth rate of GDP, {Q t }, follows an AR(1) process with mean μ:

In addition, we assume that GDP is related to industrial production and consumer sentiment by the following two equations:

where the residuals v IP, t and v C, t are uncorrelated. Finally, we define the relation between quarterly and yearly GDP growth as:

We can now bring these equations into state space form. Thereby the observation equation is given by

with observation and state vectors

and time-varying coefficient matrices

The state equation becomes:

where

On my homepage http://​www.​neusser.​ch/​ you will find a MATLAB code which maximizes the corresponding likelihood function numerically. Figure 17.3 plots the different estimates of GDP growth and compares them with the data released by State Secretariat for Economic Affairs (SECO).

Fig. 17.3

Estimates of quarterly GDP growth rates for Switzerland

17.4.2 Structural Time Series Analysis

A customary practice in business cycle analysis is to decompose a time series into several components. As an example, we estimate a structural time series model which decomposes a times series additively into a local linear trend, a business cycle component, a seasonal component, and an irregular component. This is the specification studied as the basic structural model (BSM) in Sect. 17.1.1.6. We carry over the specification explained there to apply it to quarterly real GDP of Switzerland. Figure 17.4 shows the smoothed estimates of the various components. In the left upper panel the demeaned logged original series (see Fig. 17.4a) is plotted. One clearly discern the trend and the seasonal variations. The right upper panel shows the local linear trend (LLT). As one can see the trend is not a straight line, but exhibits pronounced waves of low frequency. The business cycle component showed in Fig. 17.4c is much more volatile. The large drop of about 2.5 % in 2008/09 corresponds to the financial markets. The lower right panel plots the seasonal component (see Fig. 17.4d). From a visual inspections, one can infer that the volatility of the seasonal component is much larger than the cyclical component (compare the scale of the two components) so that movements in GDP are dominated by seasonal fluctuations.¹⁴ Moreover, the seasonal component changes its character over time.

Fig. 17.4

Components of the basic structural model (BSM) for real GDP of Switzerland. (a ) Logged Swiss GDP (demeaned). (b ) Local linear trend (LLT). (c ) Business cycle component. (d ) Seasonal component

17.5 Exercises

Exercise 17.5.1.

Consider the basic structural time series model for {Y t }:

where the error terms W t , $$\varepsilon _{t}$$ and $$\xi _{t}$$ are all uncorrelated with other at all leads and lags.

(i)

Show that {Y t } follows an ARIMA(0,2,2) process.

(ii)

Compute the autocorrelation function of $$\{\Delta ^{2}Y _{t}\}$$ .

Exercise 17.5.2.

If the cyclical component of the basic structural model for {Y t } is:

where {V 1,t (C) } and {V 2,t (C) } are mutually uncorrelated white-noise processes.

(i)

Show that {C t } follows an ARMA(2,1) process with ACF given by $$\gamma _{h}(h) =\rho ^{h}\cos \lambda _{C}h$$ .

Exercise 17.5.3.

Write the ARMA(p,q) process $$Y _{t} =\phi _{1}X_{t-1} +\ldots +\phi _{p}X_{t-p} + Z_{t} +\theta _{1}Z_{t-1} +\ldots +\theta _{q}Z_{t-q}$$ as a state space model such that the state vector X t is given by:

Exercise 17.5.4.

Show that X t and Y t have a unique stationary and causal solution if all eigenvalues of F are absolutely strictly smaller than one. Use the results from Sect.  12.3 .

Exercise 17.5.5.

Find the Kalman filter equations for the following system:

where $$\lambda$$ and ϕ are scalars and where

Exercise 17.5.6.

Consider the state space model of an AR(1) process with measurement error analyzed in Sect.  17.2:

For simplicity assume that |ϕ| < 1.

(i)

Show that {Y t } is an ARMA(1,1) process given by $$Y _{t} -\phi Y _{t-1} = Z_{t} +\theta Z_{t-1}$$ with $$Z_{t} \sim \mathrm{ WN}(0,\sigma _{Z}^{2})$$ .

(ii)

Show that the parameters of the state space, $$\phi,\sigma _{v}^{2},\sigma _{w}^{2}$$ and those of the ARMA(1,1) model are related by the equation

(iii)

Why is there an identification problem?

References

Amemiya T (1994) Introduction to statistics and econometrics. Harvard University Press, Cambridge

An S, Schorfheide F (2007) Bayesian analysis of DSGE models. Econ Rev 26:113–172CrossRef

Anderson BDO, Moore JB (1979) Optimal filtering. Electrical Engeneering Series. Prentice-Hall, Englewood Cliffs

Bernanke BS, Gertler M, Watson MW (1997:1) Systematic monetary policy and the effects of oil price shocks. Brook Pap Econ Act 91–142

Bräuning F, Koopman SJ (2014) Forecasting macroeconomic variables using collapsed dynamic factor analysis. Int J Forecast 30:572–584CrossRef

Breitung J, Eickmeier S (2006) Dynamic factor models. In: Hübler O, Frohn J (eds) Modern econometric analysis, chap 3. Springer, Berlin, pp 25–40CrossRef

Brockwell PJ, Davis RA (1991) Time series: theory and methods, 2nd edn. Springer, New YorkCrossRef

Brockwell PJ, Davis RA (1996) Introduction to time series and forecasting. Springer, New YorkCrossRef

Canova F (2007) Methods for applied macroeconomic research. Princeton University Press, Princeton

Chan JCC, Jeliazkov I (2009) Efficient simulation and integrated likelihood estimation in state space models. Int J Math Model Numer Optim 1:101–120

Chow GC, Lin A (1971) Best linear unbiased interpolation, distribution, and extrapolation of time series by related series. Rev Econ Stat 53:372–375CrossRef

Cooley TF, Prescott EC (1973) Varying parameter regression. A theory and some applications. Ann Econ Soc Meas 2:463–474

Cooley TF, Prescott EC (1976) Estimation in the presence of stochastic parameter variation. Econometrica 44:167–184CrossRef

Cuche NA, Hess MA (2000) Estimating monthly GDP in a general Kalman filter framework: Evidence from Switzerland. Econ Financ Model 7:153–194

Dejong DN, Dave C (2007) Structural macroeconomics. Princeton University Press, Princeton

Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood estimation from incomplete data via the EM algorithm. J R Stat Soc Ser B 39:1–38

Durbin J, Koopman SJ (2011) Time series analysis by state space methods, 2nd edn. Oxford University Press, Oxford

Fernandez-Villaverde J, Rubio-Ramírez JF, Sargent TJ, Watson MW (2007) ABCs and (Ds) of understanding VARs. Am Econ Rev 97:1021–1026CrossRef

Geweke JF (2005) Contemporary bayesian econometrics and statistics. Wiley series in probability and statistics. Wiley, New YorkCrossRef

Greene WH (2008) Econometric anlysis, 7th edn. Prentice Hall, New Jersey

Hamilton JD (1994a) State-Space models. In: Engle RF, McFadden DL (eds) Handbook of econometrics, vol 4, chap 50. Elsevier, Amsterdam, pp 3039–3080

Hamilton JD (1994b) Time series analysis. Princeton University Press, Princeton

Hannan EJ, Deistler M (1988) The statistical theory of linear systems. Wiley, New York

Harvey AC (1989) Forecasting, structural time series models and the kalman filter. Cambridge University Press, Cambridge

Harvey AC, Phillips GD (1982) Estimation of regression models with time varying parameters. In: Deistler M, Fürst E, Schödiauer G (eds) Games, economic dynamics and time series analysis. Physica-Verlag, Wien-Würzburg, pp 306–321CrossRef

Harvey AC, Pierce RG (1984) Estimating missing observations in economic time series. J Am Stat Assoc 79:125–131CrossRef

Hildreth C, Houck JP (1968) Some estimators for a linear model with random coefficients. J Am Stat Assoc 63:584–595

Hylleberg S (1986) Seasonality in regression. Academic Press, Orlando, FL

Kalman RE (1960) A new approach to linear filtering and prediction problems. J Basic Eng Trans ASME Ser D 82:35–45CrossRef

Kalman RE (1963) New methods in Wiener filtering theory. In: Bogdanoff JL, Kozin F (eds) Proceedings of the first symposium of engeneering applications of random function theory and probability. Wiley, New York, pp 270–388

Kim CJ, Nelson CR (1999) State-space models with regime-switching: classical and Gibbs-sampling approaches with applications. MIT Press, Cambridge

King RG, Plosser CI, Rebelo ST (1988) Production, growth, and business cycles: I. The basic neoclassical model. J Monet Econ 21:195–232CrossRef

Ljung L (1999) System identification: theory for the user, 2nd edn. Prentice Hall, Englewood Cliffs

Mills TC (2003) Modelling trends and cycles in economic time series. Palgrave texts in econometrics. Palgrave Macmillan, HampshireCrossRef

Quah D, Sargent TJ (1993) A dynamic index model for large cross sections. In: Stock JH, Watson MW (eds) Business cycles, indicators, and forecasting, Chapter 7. University of Chicago Press, Chicago

Reichlin L (2003) Factor models in large cross sections of time series. In: Dewatripont M, Hansen LP, JTurnovsky S (eds) Advances in econometrics, theory and applications, econometric society monographs, vol III. Cambridge University Press, Cambridge, pp 47-86

Sargent TJ (1989) Two models of measurements and the investment accelerator. J Polit Econ 97:251–87CrossRef

Sargent TJ (2004) Recursive macroeconomic theory, 2nd edn. MIT Press, Cambridge

Sargent TJ, Sims CA (1977) Business cycle modelling without pretending to have too much a priori economic theory. In: Sims CA (ed) New Methods in business cycle research. Federal Reserve Bank of Minneapolis, Minneapolis, pp 45–109

Woodford M (2003) Interest and prices: foundations of a theory of monetary policy. Princeton University Press, Princeton

Wu CFJ (1983) On the convergence of the EM algorithm. Ann Stat 11:95–103CrossRef

Footnotes

1

VARIMA models stand for vector autoregressive integrated moving-average models.

2

We will focus on linear dynamic models only. With the availability of fast and cheap computing facilities, non-linear approaches have gained some popularity. See Durbin and Koopman (2011) for an exposition.

3

For the ease of exposition, we will present first the time-invariant case and analyze the case of time-varying coefficients later.

4

In control theory the state equation (17.1) is amended by an additional term HU t which represents the effect of control variables U t . These exogenous controls are used to regulate the system.

5

See also Exercise 17.2.

6

Four in the case of quarterly and twelve in the case of monthly observations.

7

Alternative seasonal models can be found in Harvey (1989) and Hylleberg (1986).

8

Prototypical models can be found in King et al. (1988) or Woodford (2003). Canova (2007) and Dejong and Dave (2007) present a good introduction to the analysis of DSGE models.

9

See Sargent (2004) or Fernandez-Villaverde et al. (2007) for systematic treatment of state space models in the context of macroeconomic models. In this literature the use of Bayesian methods is widespread (see An and Schorfheide 2007; Dejong and Dave 2007).

10

Sargent (1989) provides an interesting application showing the implications of measurement errors in macroeconomic models.

11

The analogue to the EM algorithm in the Bayesian context is given by the Gibbs sampler. In contrast to the EM algorithm, we compute in the first step not the expected value of the states, but we draw a state vector from the distribution of state vectors given the parameters. In the second step, we do not maximize the likelihood function, but draw a parameter from the distribution of parameters given the state vector drawn previously. Going back and forth between these two steps, we get a Markov chain in the parameters and the states whose stationary distribution is exactly the distribution of parameters and states given the data. A detailed description of Bayesian methods and the Gibbs sampler can be found in Geweke (2005). Kim and Nelson (1999) discuss this method in the context of state space models.

12

Remember that, in our context, two representations are equivalent if they generate the same mean and covariance function for {Y t }.

13

Similarly, one may envisage the disaggregation of yearly data into monthly ones or other forms of disaggregation.

14

The irregular component which is not shown has only very small variance.

© Springer International Publishing Switzerland 2016

Klaus NeusserTime Series EconometricsSpringer Texts in Business and Economics10.1007/978-3-319-32862-1_18

18. Generalizations of Linear Time Series Models

Klaus Neusser1 

(1)

Bern, Switzerland

Autoregressive moving-average models have become the predominant approach in the analysis of economic, especially macroeconomic time series. The success of these parametric models is due to a mature and by now well-understood statistical theory which has been the subject of this book. The main assumption behind this theory is its linear structure. Although convenient, the assumption of a constant linear structure turned out to be unrealistic in many empirical applications. The evolution of economies and the economic dynamics are often not fully captured by constant coefficient linear models. Many time series are subject to structural breaks which manifest themselves as a sudden change in the model coefficients by going from one period to another. The detection and dating of such structural breaks is the subject of Sect. 18.1. Alternatively, one may think of the model coefficients as varying over time. Such models have proven to be very flexible and able to generate a variety of non-linear features. We present in Sects. 18.2 and 18.3 two variants of such models. In the first one, the model parameters vary in a systematic way with time. They are, for example, following an autoregressive process. In the second one, the parameters switch between a finite number of states according to a hidden Markov chain. These states are often identified as regimes which have a particular economic meaning, for example as booms and recessions. Further parametric and nonparametric methods for modeling and analyzing nonlinear time series can be found in Fan and Yao (2003).

18.1 Structural Breaks

There is a extensive literature dealing with the detection and dating of structural breaks in the context of time series. This literature is comprehensibly summarized in Perron (2006), among others. A compact account can also be found in Aue and Horváth (2011) where additional testing procedures, like the CUSUM test, are presented. In this short exposition we follow Bai et al. (1998) and focus on Chow type test procedures. For the technical details the interested reader is referred to these papers.

18.1.1 Methodology

Consider, for the ease of exposition, a VAR(1) process which allows for a structural break at some known date t b :

(18.1)

where

Thus, before time t b the coefficients of the VAR process are given by c ⁽¹⁾ and $$ \Phi ^{(1)} $$ whereas after t b they are given by c ⁽²⁾ and $$ \Phi ^{(2)} $$. The error process {Z t } is assumed to be $$ \mathrm{IID}(0,\Sigma ) $$ with $$ \Sigma $$ positive definite.¹ Suppose further that the roots of $$ \Phi ^{(1)}(z) $$ as well as those of $$ \Phi ^{(2)}(z) $$ are outside the unit circle. The process therefore is stationary and admits a causal representation with respect to {Z t } before and after date t b .

The assumption of a structural break at some known date t b can then be investigated by testing the hypothesis

The standard way to test such a hypothesis is via the F-statistic. Given a sample ranging from period 0 to period T, the strategy is to partition all variables and matrices along the break date t b . Following the notation and the spirit of Sect. 13.​2, define $$ Y =\mathop{ \mathrm{vec}}(Y ^{(1)},Y ^{(2)}) $$ where $$ Y ^{(1)} = (X_{1},X_{2},\ldots,X_{t_{b}}) $$ and $$ Y ^{(2)} = (X_{t_{b}+1},X_{t_{b}+2},\ldots,X_{T}) $$, Z = (Z ₁, Z ₂, …, Z T ), and

then the model (18.1) can be written as

The least-squares estimator becomes

This amounts to estimate the model separately over the two sample periods. Note that as in Sect. 13.​2 the GLS estimator is numerically identical to the OLS estimator because the same regressors are used for each equation. The corresponding Wald-test can be implemented by defining $$ R = (I_{n^{2}+n},-I_{n^{2}+n}) $$ and computing the F-statistic

(18.2)

where $$ \hat{\beta }=\mathop{ \mathrm{vec}}(\hat{c}^{(1)},\widehat{\Phi }^{(1)},\hat{c}^{(2)},\widehat{\Phi }^{(2)}) $$ and where $$ \widehat{\Sigma }_{t_{b}} $$ is computed from the least-squares residuals $$ \widehat{Z}_{t} $$ as $$ \widehat{\Sigma }_{t_{b}} = \frac{1} {T}\sum _{t=1}^{T}\widehat{Z}_{ t}\widehat{Z}_{t}' $$ given break date t b . Under the standard assumptions made in Sect. 13.​2, the test statistic F(t b )∕(n ² + n) converges for $$ T \rightarrow \infty $$ to a chi-square distribution with n ² + n degrees of freedom.² This test is known in the literature as the Chow test.

The previous analysis assumed that the potential break date t b is known. This assumption often turns out to be unrealistic in practice. The question then arises how to determine a potential break date. Quandt (1960) proposed a simple procedure: compute the Chow-test for all possible break dates and take as a candidate break date the date where the F-statistic reaches its maximal value. Despite its simplicity, Quandt’s procedure could not be implemented coherently because it was not clear which distribution to use for the construction of the critical values. This problem remained open for more than thirty years until the contribution of Andrews (1993).³ Denote by ⌊x⌋ the value of x rounded to the nearest integer towards minus infinity, then the maximum Wald statistic and the logarithm of the Andrews and Ploberger (1994) exponential Wald statistic can be written as follows:

where τ ∗ denotes the percentage of the sample which is trimmed. Usually, τ ∗ takes the value of 0. 15 or 0. 10. Critical values for low degrees of freedom are tabulated in Andrews (1993, 2003) and Stock and Watson (2011). It is possible to construct an asymptotic confidence interval for the break date. The corresponding formulas can be found in Bai et al. (1998, p. 401–402).

18.1.2 An Example

The use of the structural break test is demonstrated using historical data for the United Kingdom. The data consist of logged per capita real GDP, logged per capita real government expenditures, logged per capita real government revenues, the inflation based on the consumer price index, and a long-term interest rate over a sample period from 1830 to 2003. The basis for the analysis consists of a five variable VAR(2) model including a constant term and a linear trend. Three alternative structural break modes are investigated: break in the intercept, break in the intercept and the time trend, and break in all coefficients, including the VAR coefficients. The corresponding F-statistics are plotted in Fig. 18.1 against all possible break dates allowing for a trimming value of 10 %. The horizontal lines show for all three alternative break modes the corresponding critical values for the supF test given 5 % significance levels. These critical values have been obtained from Monte Carlo simulations as in Andrews (1993, 2003) and are given as 18.87, 28.09, and 97.39.⁴

Fig. 18.1

Analysis of breaks dates with the $$ \sup \mathrm{F} $$ test statistic for historical UK time series

Figure 18.1 shows that for all three modes a significant structural break occurs. The corresponding values of the supF statistics are 78.06, 104.75, and 285.22. If only the deterministic parts are allowed to change, the break date is located in 1913. If all coefficients are allowed to change, the break is dated in 1968. However, all three F-statistics show a steep increase in 1913. Thus, if only one break is allowed 1913 seems to be the most likely one.⁵ The breaks are quite precisely dated. The corresponding standard errors are estimated to be two years for the break in the intercept only and one year for the other two break modes.

18.2 Time-Varying Parameters

This section discusses time-varying coefficient vector autoregressive models (TVC-VAR models). This model class retains the flavor of VAR models but assumes that they are only valid locally. Consider for this purpose a VAR(1) model with time-varying autoregressive coefficients $$ \Phi _{t} $$:

(18.3)

This model can be easily generalized to higher order VAR’s (see below) or, alternatively, one may think of Eq. (18.3) as a higher order VAR in companion form. The autoregressive coefficient matrix is assumed to be stochastic. Thus, $$ \Phi _{t} $$ is a random n × n matrix. Models of this type have been widely discussed in the probabilistic literature because they arise in many diverse contexts. In economics, Eq. (18.3) can be interpreted as the probabilistic version describing the value of a perpetuity, i.e. the present discounted value of a permanent commitment to pay a certain sum each period. Thereby Z t denotes the random periodic payments and $$ \Phi _{t} $$ the random cumulative discount factors. The model also plays an important role in the characterization of the properties of volatility models as we have seen in Sect. 8.​1 (see in particular the proofs of Theorems 8.​1 and 8.​3). In this presentation, the above model is interpreted as a locally valid VAR process.

A natural question to ask is under which conditions Eq. (18.3) admits a stationary solution. An answer to this question can be found by iterating the equation backwards in time:

where it is understood that $$ \prod _{i=1}^{0} = I_{n} $$. This suggests as a solution candidate

(18.4)

Based on results obtained by Brandt (1986) and extended by Bougerol and Picard (1992b), we can cite the following theorem.

Theorem 18.1 (Solution TVC-VAR(1)).

Let $$ \{(\Phi _{t},Z_{t})\} $$ be a strictly stationary ergodic process such that

(i)

$$ \mathbb{E}(\log ^{+}\|\Phi _{t}\|) < \infty $$ and $$ \mathbb{E}(\log ^{+}\|Z_{t}\|) < \infty $$ where x ⁺ denotes $$ \max \{x,0\} $$ ;

(ii)

the top Lyapounov exponent γ defined as

is strictly negative.

Then X t as defined in Eq. (18.4) converges a.s. and {X t } is the unique strictly stationary solution of equation (18.3).

Remark 18.1.

The Lyapunov exponent measures the rate of separation of nearly trajectories in a dynamic system. The top Lyapunov exponent gives the largest of these rates. It is used to characterize the stability of a dynamic system (see Colonius and Kliemann (2014)).

Remark 18.2.

Although Theorem 18.1 states only sufficient conditions, these assumptions can hardly be relaxed.

The solution (18.4), if it exists, is similar to a causal representation. The matrix sequence $$ \{\prod _{i=1}^{h}\Phi _{t+h-i}\}_{h=0,1,2,\ldots } =\{ I_{n},\Phi _{t},\Phi _{t+1}\Phi _{t},\Phi _{t+2}\Phi _{t+1}\Phi _{t},\ldots \} $$ represents the effect of an impulse in period t to X t+h , h = 0, 1, 2, … and can therefore be interpreted as impulse response functions. In contrast to the impulse response functions studied so far, they are clearly random and time-dependent because the effect of Z t depends on future coefficients. In particular, the effect of Z t on X t+h , h ≥ 1, is not the same as the effect of Z t−h on X t . Nevertheless it is possible to construct meaningful impulse response functions by Monte Carlo simulations. One may then report the mean of the impulse responses or some quantiles for different time periods.⁶ Alternatively, one may ignore the randomness and time-dependency and define “local” impulse responses as $$ \Phi _{t}^{h} $$, h = 0, 1, 2, …. Note, however, that the impulse responses so defined still vary with time. Irrespectively how the impulse responses are constructed, they can be interpreted in the same way as in the case of constant coefficients. In particular, we may use some of the identification schemes discussed in Chap. 15 and compute the impulse responses with respect to structural shocks. Similar arguments apply to the forecast error variance decomposition (FEVD).

The model is closed by fixing the law of motion for $$ \Phi _{t} $$. As already mentioned in Sect. 17.​1.​1.​5 there are several possibilities. In this presentation we adopt the following flexible autoregressive specification:

(18.5)

where $$ \beta _{t} =\mathop{ \mathrm{vec}}\Phi _{t} $$ denotes the n ² vector of stacked coefficients. Q is assumed to be fixed and is, usually, specified as a diagonal matrix. If the eigenvalues of F are inside the unit circle, the autoregressive model is mean-reverting and $$ \bar{\beta } $$ can be interpreted as the average coefficient vector. The formulation in Eq. (18.5) is, however, not restricted to this case and allows explicitly the possibility that {β t } follows a random walk. This specification has become very popular in the empirical macroeconomic literature and was initially adopted by Cogley and Sargent (2001) to analyze the dynamics of inflation across different policy regimes.⁷

The model consisting of Eqs. (18.3) and (18.5) can be easily reformulated as a state space model by defining $$ \xi _{t} =\beta _{t}-\bar{\beta } $$ as the state vector. The state and the measurement equation can then be written as:

(18.6)

(18.7)

Conditional on initial values for the coefficients and their covariances, the state space model can be estimated by maximum likelihood by applying the Kalman filter (see Sect. 17.​3 and Kim and Nelson (1999)). One possibility to initialize the Kalman filter is to estimate the model for some initial sample period assuming fixed coefficients and extract from these estimates the corresponding starting values.

As it turns out, allowing time-variation only in the coefficients of the VAR model overstates the role attributed to structural changes. We therefore generalize the model to allow for time-varying volatility. More specifically, we also allow $$ \Sigma $$ in Eq. (18.3) to vary with time. The modeling of the time-variation in $$ \Sigma $$ is, however, not a straightforward task because we must ensure that in each period $$ \Sigma _{t} $$ is a symmetric positive definite matrix. One approach is to specify a process especially designed for modeling the dynamics of covariance matrices. This so-called Wishart autoregressive process was first introduced to economics by Gouriroux et al. (2009) and successfully applied by Burren and Neusser (2013). It leads to a nonlinear state space system which can be estimated with the particle filter, a generalization of the Kalman filter.

Another more popular approach was initiated by Cogley and Sargent (2005) and Primiceri (2005). It is based on the Cholesky factorization of the time-varying covariance matrix $$ \Sigma _{t} $$. Using the same notation as in Sect. 15.​3 $$ \Sigma _{t} $$ is decomposed as

(18.8)

where B t is a time-varying lower triangular matrix with ones on the diagonal and $$ \Omega _{t} $$ a time-varying diagonal matrix with strictly positive diagonal elements.⁸ The logged diagonal elements of $$ \Omega _{t} $$ are then assumed to evolve as independent univariate random walks. This specification can be written in matrix terms as

(18.9)

where D t is a diagonal matrix with $$ \mathrm{diag}(D_{t}) \sim \mathrm{ WN}(0,\Omega _{D}) $$. In the above formulation $$ \exp $$ denotes the matrix exponential.⁹ Taking the matrix logarithm, we get exactly the formulation of Cogley and Sargent (2005) and Primiceri (2005). For the time evolution of B t we propose a similar specification:

(18.10)

where C t is a strictly lower triangular matrix, i.e. C t is a lower triangular matrix with zeros on the diagonal. The non-zero entries of C t , denoted by [C t ] i > j , are assumed to follow a multivariate white noise process with diagonal covariance matrix $$ \Sigma _{B} $$, i.e. $$ [C_{t}]_{i>j} \sim \mathrm{ WN}(0,\Sigma _{B}) $$. It can be shown that the matrix exponential of strictly lower triangular matrices are triangular matrices with ones on the diagonal. As the set of triangular matrices with ones on the diagonal form a group, called the unipotent group and denoted by SLT n , the above specification is well-defined. Moreover, this formulation is a very natural one as the set of strictly lower triangular matrices is the tangent space of SLT n at the identity (see Baker 2002, for details). Thus, Eq. (18.10) can be interpreted as a log-linearized version of B t . The technique proposed for the evolution of B t in Eq. (18.10) departs from Primiceri (2005) who models each element of the inverse of B t and therefore misses a coherent system theoretic approach. See Neusser (2016) for details.

Although this TVC-VAR model with time-varying volatility can in principle also be estimated by maximum likelihood, this technique can hardly be implemented successfully in practice. The main reason is that the likelihood function of such a model, even when the dimension and the order of the VAR is low, is a very high dimensional nonlinear object with probably many local maxima. Moreover, as the variances governing the time-variation are small, at least for some of the coefficients, the likelihood function is flat in some regions of the parameter space. These features make maximization of the likelihood function a very difficult, if not impossible, task in practice. For these reasons, Bayesian techniques have been used almost exclusively. There is, however, also a conceptional issue involved. As the Bayesian approach does not strictly distinguish between fixed “true” parameters and random samples, it is better suited to handle TVC-VAR models which treat the parameters as random. In this monograph, we will not tackle the Bayesian approach but refer to the relevant literature. See for example Primiceri (2005), Negro and Primiceri (2015), Cogley and Sargent (2005), Canova (2007) and Koop and Korobilis (2009) among others.

The Minnesota Prior

Although, we will not discuss the Bayesian approach to VAR modeling, it is nevertheless instructive to portray the so-called Minnesota prior applied by Doan et al. (1984) to TVC-VAR models. This prior has gained some reputation in connection to forecasting with VAR models and as a way to specify the initial distribution for the Kalman filter in time-varying models. The combination of the prior distribution with the likelihood function delivers via Bayes’ rule a posterior distribution of the parameters which can then be analyzed using simulation methods.

The Minnesota prior is based on the a priori belief that each variable follows a random walk with no interaction among the variables nor among the coefficients of the VAR equations. We expose one version of the Minnesota prior in the general context of a TVC-VAR model of order p with time-varying constant term c t :

(18.11)

This model can be written compactly as

(18.12)

where X t−1 = (1, X t−1′, …, X t−p ′)′, $$ \Phi _{t-1} = (c_{t},\Phi _{t-1}^{(1)},\ldots,\Phi _{t-1}^{(p)}) $$, and $$ \beta _{t} =\mathop{ \mathrm{vec}}\Phi _{t} $$. Assuming for β t the same autoregressive form as in Eq. (18.5), the state space representation (18.6) and (18.7) also applies to the TVC-VAR(p) model with X t−1 replaced by X t−1. Note that the dimension of the state equation can become very high because β t is a n + n ² p vector.

Taking date 0 as the initial date, the prior distribution of the autoregressive parameters is supposed to be normal:

where $$ \bar{\beta }=\mathop{ \mathrm{vec}}(0,I_{n},0,\ldots,0) $$. This implies that the mean for all coefficients, including the constant term, is assumed to be zero except for the own lag coefficients of order one $$ [\Phi _{0}^{(1)}]_{ii} $$, i = 1, …, n, which are assumed to be one. The covariance matrix P 0 | 0 is taken as being diagonal so that there is no correlation across coefficients. Thus, the prior specification amounts to assuming that each variable follows a random walk with no interaction with other variables.

The strength of this belief is governed by a number of so-called hyperparameters which regulate the diagonal elements of P 0 | 0. The first one, γ ², controls the confidence placed on the assumption that $$ [\Phi _{0}^{(1)}]_{ii} = 1 $$:

A small (large) value of γ ² thus means more (less) confidence. As the lag order increases more confidence is placed on the assumption $$ [\Phi _{0}^{(h)}]_{ii} = 0 $$:

Instead of the harmonic decline other schemes have been proposed. For h = 1, …, p the off-diagonal elements of $$ \Phi _{0}^{(h)} $$ are assumed to have prior distribution

Thereby $$ \hat{\tau }_{i}^{2}/\hat{\tau }_{j}^{2} $$ represents a correction factor which accounts for the magnitudes of X it relative to X jt . Specifically, $$ \hat{\tau }_{i}^{2} $$ is the residual variance of a univariate AR(1) model. The hyperparameter w ² is assumed to be strictly smaller than one. This represents the belief that X j, t−h is less likely to be important as an explanation for X i, t , i ≠ j, than the own lag X i, t−h . Finally, the strength of the belief that the constant terms are zero is

This completes the specification for the prior belief on β ₀. Combining all elements we can write P 0 | 0 as a block diagonal matrix with diagonal blocks:

where $$ P_{0\vert 0}^{(c)} = g \times \mathrm{ diag}(\hat{\tau }_{1},\ldots,\hat{\tau }_{n}) $$ and $$ P_{0\vert 0}^{(\phi )} =\mathrm{ diag}(\mathop{\mathrm{vec}}(G \otimes \Upsilon )) $$. Thereby, G and $$ \Upsilon $$ are defined as

According to Doan et al. (1984) the preferred values for the three hyperparameters are g = 700, γ ² = 0. 07, and w ² = 0. 01.

Thus, for a bivariate TVC-VAR(2) model the mean vector is given by $$ \bar{\beta }= (0,0,1,0,0,1,0,0,0,0)' $$ with diagonal covariance matrix P 0 | 0:

with

and

Next we specify the parameters of the state transition equation (18.5). Following Doan et al. (1984), $$ F =\pi _{F}I_{n+pn^{2}} $$ with π F  = 0. 999 and Q = π Q P 0 | 0 with π Q  = 10⁻⁷. The proportionality factor does, however, not apply to the constant terms. For these terms, the corresponding diagonal elements of Q, [Q] ii , i = 1, …, n, are set to π Q [P 0 | 0] i(n+1), i(n+1), i = 1, …, n. The reason for this correction is that the prior put on the constants is rather loose as expressed by the high value of g. The final component is a specification for $$ \Sigma $$, the variance of Z t . This matrix is believed to be diagonal with $$ \Sigma =\pi _{\Sigma }\mathrm{diag}(\hat{\tau }_{1}^{2},\ldots,\hat{\tau }_{n}^{2}) $$ and $$ \pi _{\Sigma } = 0.9 $$.

With these ingredients the state space model is completely specified. Given observations X ₁, …, X t , the Kalman filter produces a sequence of β t+1 | t , t = 1, 2, … and one-period ahead forecasts X t+1 | t computed as

Doan et al. (1984) suggest to compute an approximate h period ahead forecast by treating the forecast from the previous periods as if they were actual observations.

18.3 Regime Switching Models

The regime switching model is similar to the time-varying model discussed in the previous section. The difference is that the time-varying parameters are governed by a hidden Markov chain with a finite state space $$ \mathbb{S} =\{ 1,2,\ldots,k\} $$. Usually, the number of states k is small and is equal in practice to two or maximal three. The states have usually an economic connotation. For example, if k equals two, state 1 might correspond to a boom phase whereas state 2 to a recession. Such models have a long tradition in economics and have therefore been used extensively. Seminal references include Goldfeld and Quandt (1973, 1976), Hamilton (1994b), Kim and Nelson (1999), Krolzig (1997), and Maddala (1986). Frühwirt-Schnatter (2006) presents a detailed statistical analysis of regime switching models.

The starting point of our presentation of the regime switching model is again the TVC-VAR(1) as given in Eq. (18.3). We associate to each state $$ j \in \mathbb{S} $$ a coefficient matrix $$ \Phi ^{(j)} $$. Thus, in the regime switching model the coefficients $$ \Phi _{t} $$ can only assume a finite number values $$ \Phi ^{(1)},\ldots,\Phi ^{(k)} $$ depending on the state of the Markov chain. The actual value assigned to $$ \Phi _{t} $$ is governed by a Markov chain defined through a fixed but unknown transition probability matrix P where

(18.13)

Thus, [P] ij is the probability that $$ \Phi _{t} $$ assumes value $$ \Phi ^{(j)} $$ given that it assumed in the previous period the value $$ \Phi ^{(i)} $$. The probability that $$ \Phi _{t+h} $$ is in state j given that $$ \Phi _{t} $$ was in state i is therefore [P h ] ij . The definition of the transition matrix in Eq. (18.13) implies that P is a stochastic matrix, i.e. that [P] ij  ≥ 0 and $$ \sum _{j=1}^{k}[P]_{ij} = 1 $$. Moreover, we assume that the chain is regular meaning that it is ergodic (irreducible) and aperiodic.¹⁰ This is equivalent to the existence of a fixed integer m > 0 such that P m has only strictly positive entries (see Berman and Plemmons 1994, Chapter 8). Regular Markov chains have a unique ergodic (stationary) distribution vector π with strictly positive entries and determined by π′P = π′. This distribution is approached from any initial distribution vector π ₀, i.e. $$ \lim _{t\rightarrow \infty }\pi _{0}'P^{t} =\pi ' $$. Moreover, $$ \lim _{t\rightarrow \infty }P^{t} = P^{\infty } $$ where $$ P^{\infty } $$ is a transition matrix with all rows equal to π′.

Given this setup we can again invoke Theorem 18.1 and claim that a (strictly) stationary solution of the form of Eq. (18.4) exists if all the autoregressive matrices $$ \Phi ^{(j)} $$, j = 1, 2, …, k, have eigenvalues strictly smaller than one.

Given observations $$ x_{T},x_{T-1},\ldots,x_{1},x_{0} $$ for $$ X_{T},X_{T-1},\ldots,X_{1},X_{0} $$, a maximum likelihood approach can be set up to estimate the unknown parameters $$ \Phi ^{(1)},\ldots,\Phi ^{(k)},\Sigma,P $$.¹¹ Collect these parameters into a vector $$ \theta $$ and denote by $$ s_{t} \in \mathbb{S} $$ the state of the Markov chain in period t and by $$ \mathcal{X}_{t} = (x_{t},x_{t-1},\ldots,x_{1},x_{0}) $$ the information available up to period t. Write the conditional density of x t given s t  = j and observations $$ \mathcal{X}_{t-1} $$ as

The joint density of (x t , s t  = j) is

where in analogy to the Kalman filter the expressions $$ P(s_{t} = j\vert \mathcal{X}_{t-1};\theta ) $$, $$ j = 1,\ldots,k $$, are called the predicted transition probabilities. The conditional marginal density of x t then becomes

In the case of $$ Z_{t} \sim IID\,N(0,\Sigma ) $$ the above density is a finite mixture of Gaussian distributions (see Frühwirt-Schnatter 2006, for details). The (conditional) log likelihood function, finally, is therefore given by

In order to evaluate the likelihood function note that the joint density of (x t , s t  = j) may also be factored as

Combining these expressions one obtains an expression for the filtered transition probabilities $$ P(s_{t} = j\vert \mathcal{X}_{t};\theta ) $$:

(18.14)

Next period’s predicted transition probabilities are then obtained by multiplication with the transition matrix:

(18.15)

Given initial probabilities $$ P(s_{1} = j\vert \mathcal{X}_{0};\theta ) $$, $$ j = 1,\ldots,k $$, and a fixed value for $$ \theta $$, Eqs. (18.14) and (18.15) can be iterated forward to produce a sequence of predicted transition probabilities $$ (P(s_{t} = 1\vert \mathcal{X}_{t-1};\theta ),\ldots,P(s_{t} = k\vert \mathcal{X}_{t-1};\theta ))' $$, $$ t = 1,2,\ldots,T $$ which can be used to evaluate the Gaussian likelihood function. Numerical procedures must then be used for the maximization of the likelihood function. This task is not without challenge because the likelihood function of Gaussian mixture models typically has singularities and many local maxima. Kiefer (1978) showed that there exists a bounded local maximum which yields a consistent and asymptotically normal estimate for $$ \theta $$ for which standard errors can be constructed in the usual way. In practice, problems encountered during the maximization can be alleviated by experimentation with alternative starting values. Thereby the initial probability $$ (P(s_{1} = 1\vert \mathcal{X}_{0};\theta ),\ldots,P(s_{1} = k\vert \mathcal{X}_{0};\theta ) $$ could either be treated as additional parameters as in Goldfeld and Quandt (1973) or set to the uniform distribution. For technical details and alternative estimation strategies, like the EM algorithm, see Hamilton (1994b, chapter 22) and in particular Frühwirt-Schnatter (2006).

By reversing the above recursion it is possible to compute smoothed transition probabilities $$ P(s_{t} = j\vert \mathcal{X}_{T};\theta ) $$ (see Kim 1994):

The iteration is initialized with $$ P(s_{T} = j\vert \mathcal{X}_{T};\theta ) $$ which has been computed in the forward recursion.

The basic model can and has been generalized in several dimensions. The most obvious one is the inclusion of additional lags beyond the first one. The second one concerns the possibility of a regime switching covariance matrix $$ \Sigma $$. These modifications can be accommodated using the methods outlined above. Thirdly, one may envision time-varying transition probabilities to account for duration dependence. In business cycle analysis, for example, the probability of moving out of a recession may depend on how long the economy has been in the recession regime. This idea can be implemented by modeling the transition probabilities via a logit specification:

where z t includes a constant and a set of additional variables. These additional variables can be some exogenous variables, but more interestingly may include some lagged variables x t−d (Krolzig 1997). Note that the transition probabilities do not only depend on z t , but also on the state. The resulting model has some features shared with the smooth transition autoregressive model of Granger and Teräsvirta (1993). Early economic applications of regime switching models with time-varying transition probabilities can be found in Diebold et al. (1994), Filardo (1994), and Filardo and F.Gordon (1998).

An important aspect in practice is the determination of the number of regimes. Unfortunately, there is no direct test available for the null hypothesis k = m against the alternative k = m + 1. The reason is that the likelihood contains parameters which are only present under the alternative. The parameters describing the m + 1-th state are unidentified under the null hypothesis. The problem has been analyzed by Andrews and Ploberger (1994) in a general theoretical context. Alternatively, one may estimate the model under the null hypothesis and conduct a series of specification tests as proposed by Hamilton (1996). It has also been suggested to use the information criteria like AIC and BIC to determine the number of regimes (Frühwirt-Schnatter 2006, p. 346–347):

where k(k − 1) are the free parameters in the transition matrix P.

References

Andrews DWK (1993) Tests for parameter instability and structural change with unknown change point. Econometrica 61:821–856CrossRef

Andrews DWK (2003) Tests for parameter instability and structural change with unknown change point: A corrigendum. Econometrica 71:395–397CrossRef

Andrews DWK, Ploberger W (1994) Optimal tests when a nuisance parameter is present only under the alternative. Econometrica 62:1383–1414CrossRef

Aue A, Horváth L (2011) Structural breaks in time series. J Time Ser Anal 34:1–16CrossRef

Bai J (2000) Vector autoregressive models with structural changes in regression coefficients and in variance-covariance matrices. Ann Econ Finance 1:303–339

Bai J, Lumsdaine RL, Stock JH (1998) Testing for and dating common breaks in multivariate time series. Rev Econ Stud 65:395–432CrossRef

Baker A (2002) Matrix groups – an introduction to Lie Group theory. Springer, London

Berman A, Plemmons RJ (1994) Nonnegative matrices in the mathematical sciences. No. 9 in Classics in Applied Mathematics, Society of Industrial and Applied Mathematics, Philadelphia

Bougerol P, Picard N (1992b) Strict stationarity of generalized autoregressive processes. Ann Probab 20:1714–1730

Brandt A (1986) The stochastic equation y n+1 = a n y n + b n with stationary coefficients. Adv Appl Probab 18:211–220

Burren D, Neusser K (2013) The role of sectoral shifts in the decline of real GDP volatility. Macroecon Dyn 17:477–500CrossRef

Canova F (2007) Methods for applied macroeconomic research. Princeton University Press, Princeton

Cogley T, Sargent TJ (2001) Evolving post-world war II U.S. inflation dynamics. In: Bernanke BS, Rogoff K (eds) NBER macroeconomics annual, vol 16. MIT Press, Cambridge, pp 331–373

Cogley T, Sargent TJ (2005) Drift and volatilities: Monetary policies and outcomes in the post WWII U.S. Rev Econ Dyn 8:262–302CrossRef

Colonius F, Kliemann W (2014) Dynamical Systems and Linear Algebra. Graduate Studies in Mathematics, Vol. 158. American Mathematical Society, Providence, Rhode Island

Diebold FX, Lee JH, Weinbach GC (1994) Regime switching with time-varying transition probabilities. In: Hargreaves CP (ed) Nonstationary time series analysis and cointegration. Oxford Universiy Press, Oxford, pp 283–302

Doan T, Litterman RB, Sims CA (1984) Forecasting and conditional projection using realistic prior distributions. Econ Rev 3:1–100CrossRef

Fan J, Yao Q (2003) Nonlinear time series. Springer, New York

Filardo AJ (1994) Business-cycle phases and their transitional dynamics. J Bus Econ Anal 12:299–308

Filardo AJ, FGordon S (1998) Business cycle duration. J Econ 85:99–123CrossRef

Frühwirt-Schnatter S (2006) Finite mixture and markov switching models. Springer Science + Business Media LLC, New York

Goldfeld SM, Quandt RE (1973) A Markov model for switching regressions. J Econ 1:3–15CrossRef

Goldfeld SM, Quandt RE (1976) Studies in nonlinear estimation. Ballinger Publishing, Cambridge

Gouriroux C, Jasiak J, Sufana R (2009) The Wishart autoregressive process of multivariate stochastic volatility. J Econ 150:167–181CrossRef

Granger CWJ, Teräsvirta T (1993) Modelling nonlinear economic relationships. Oxford University Press, Oxford

Hamilton JD (1994b) Time series analysis. Princeton University Press, Princeton

Hamilton JD (1996) Specification testing in Markov-switching time-series models. J Econ 70:127–157CrossRef

Kiefer NM (1978) Discrete parameter variation: Efficient estimation of a switching regression model. Econometrica 46:427–434CrossRef

Kim CJ (1994) Dynamic linear models with Markov-switching. J Econ 60:1–22CrossRef

Kim CJ, Nelson CR (1999) State-space models with regime-switching: classical and Gibbs-sampling approaches with applications. MIT Press, Cambridge

Koop G, Korobilis D (2009) Bayesian multivariate time series methods for empirical macroeconomics. Found Trends Econ 3:267–358CrossRef

Krolzig HM (1997) Markov–switching vector autoregressions. Modelling, statistical inference, and application to business cycle analysis. Springer, Berlin

Maddala GS (1986) Disequilibrium, self-selection, and switching models. In: Griliches Z, Intriligator MD (eds) Handbook of econometrics, vol III. North-Holland, Amsterdam

Negro Md, Primiceri GE (2015) Time varying structural vector autoregressions and monetary polic: A corrigendum. Rev Econ Stud, forthcoming

Neusser K (2016) A topological view on the identification of structural vector autoregressions. Economics Letters forthcoming

Norris JR (1998) Markov chains. Cambridge series in statistical and probabilistic mathematics. Cambridge University Press, Cambridge

Perron P (2006) Dealing with structural breaks. In: Hassani H, Mills TC, Patterson K (eds) Palgrave handbook in econometrics, vol 1. Econometric theory. Palgrave Macmillan, Hampshire, pp 278–352

Potter SM (2000) Nonlinear impulse response functions. J Econ Dyn Control 24:1425–1446CrossRef

Primiceri GE (2005) Time varying structural vector autoregressions and monetary policy. Rev Econ Stud 72:821–852CrossRef

Quandt RE (1960) Tests of the hypothesis that a linear regression system obeys two separate regimes. J Am Stat Assoc 55:324–330CrossRef

Stock JH, Watson MW (2011) Introduction to econometrics, 3rd edn. Addison Wesley, Longman

Footnotes

1

Generalization to higher order VAR models is straightforward. For changes in the covariance matrix $$ \Sigma $$ see Bai (2000). For the technical details the reader is referred to the relevant literature.

2

As the asymptotic theory requires that t b ∕T does not go to zero, one has to assume that both the number of periods before and after the break go to infinity.

3

A textbook version of the test can be found in Stock and Watson (2011).

4

Assuming a trimming value of 0. 10 Andrews (2003, table I) reports critical values of 18.86 for p = 5 which corresponds to changes in the intercept only and 27.27 for p = 10 which corresponds to changes in intercept and time trend.

5

See Perron (2006) for a discussion of multiple breaks.

6

Potter (2000) discusses the primal problems of defining impulse responses in a nonlinear context.

7

They allow for a correlation between V t and Z t .

8

It is possible to consider other short-run type identification schemes (see Sect. 15.​3) than the Cholesky factorization.

9

The matrix exponential of a matrix A is defined as $$ \exp (A) =\sum _{ i=0}^{\infty }\tfrac{1} {i!}A^{i} $$ where A is any matrix. Its inverse $$ \log (A) $$ is defined only for $$ \|A\| < 1 $$ and is given by $$ \log (A) =\sum _{ i=1}^{\infty }\tfrac{(-1)^{i-1}} {i} A^{i} $$.

10

A chain is called ergodic or irreducible if for every states i and j there is a strictly positive probability that the chain moves from state i to state j in finitely many steps. A chain is called aperiodic if it can return to any state i at irregular times. See, among others, Norris (1998) and Berman and Plemmons (1994) for an introduction to Markov chains and its terminology.

11

The presentation of the maximum likelihood approach follows closely the exposition by Hamilton (1994b, chapter 22) where more details can be found.

A Complex Numbers

The simple quadratic equation $$x^{2} + 1 = 0$$ has no solution in the field of real numbers, $$\mathbb{R}$$ . Thus, it is necessary to envisage the larger field of complex numbers $$\mathbb{C}$$ . A complex number z is an ordered pair ( a ,  b ) of real numbers where ordered means that we regard ( a ,  b ) and ( b ,  a ) as distinct if a ≠ b . Let x  = ( a ,  b ) and y  = ( c ,  d ) be two complex numbers. Then we endow the set of complex numbers with an addition and a multiplication in the following way:

These two operations will turn $$\mathbb{C}$$ into a field where (0, 0) and (1, 0) play the role of 0 and 1. ¹ The real numbers $$\mathbb{R}$$ are embedded into $$\mathbb{C}$$ because we identify any $$a \in \mathbb{R}$$ with $$(a,0) \in \mathbb{C}$$ .

The number $$\imath = (0,1)$$ is of special interest. It solves the equation $$x^{2} + 1 = 0$$ , i.e.  $$\imath^{2} = -1$$ . The other solution being $$-\imath = (0,-1)$$ . Thus any complex number ( a ,  b ) may be written as $$(a,b) = a+\imath{b}$$ where a ,  b are arbitrary real numbers. ²

An element z in this field can be represented in two ways:

In the representation in Cartesian coordinates $$a =\mathrm{ Re}(z) = \mathfrak{R}(z)$$ is called the real part whereas $$b =\mathrm{ Im}(z) = \mathfrak{I}(z)$$ is called the imaginary part of z .

A complex number z can be viewed as a point in the two-dimensional Cartesian coordinate system with coordinates ( a ,  b ). This geometric interpretation is represented in Fig.  A.1 .

The absolute value or modulus of z , denoted by |  z  | , is given by $$r = \sqrt{a^{2 } + b^{2}}$$ . Thus, the absolute value is nothing but the distance of z viewed as a point in the complex plane (the two-dimensional Cartesian coordinate system) to the origin (see Fig.  A.1 ). $$\theta$$ denotes the angle to the positive real axis (x-axis) measured in radians. It is denoted by $$\theta =\arg z$$ . It holds that $$\tan \theta = \frac{b} {a}$$ . Finally, the conjugate of z , denoted by $$\bar{z}$$ , is defined by $$\bar{z} = a-\imath{b}$$ .

Setting r  = 1 and $$\theta =\pi$$ , gives the following famous formula:

This formula relates the most famous numbers in mathematics.

Fig. A.1

Representation of a complex number

From the definition of complex numbers in polar coordinates, we immediately derive the following implications:

Further implications are de Moivre’s formula and Pythagoras’ theorem (see Fig.  A.1 ):

From Pythagoras’ theorem it follows that $$r^{2} = a^{2} + b^{2}$$ . The representation in polar coordinates allows to derive many trigonometric formulas.

Consider the polynomial $$\Phi (z) =\phi _{0} -\phi _{1}z -\phi _{2}z^{2} -\ldots -\phi _{p}z^{p}$$ of order p  ≥ 1 with ϕ ₀  = 1. ³ The fundamental theorem of algebra then states that every polynomial of order p  ≥ 1 has exactly p roots in the field of complex numbers. Thus, the field of complex numbers is algebraically complete. Denote these roots by $$\lambda _{1},\ldots,\lambda _{p}$$ , allowing that some roots may appear several times. The polynomial can then be factorized as

This expression is well-defined because the assumption of a nonzero constant ( ϕ ₀  = 1 ≠ 0) excludes the possibility of roots equal to zero. If we assume that the coefficients ϕ j , j  = 0,  … ,  p , are real numbers, the complex roots appear in conjugate pairs. Thus if $$z = a+\imath{b}$$ , b ≠ 0, is a root then $$\bar{z} = a-\imath{b}$$ is also a root.

B Linear Difference Equations

Linear difference equations play an important role in time series analysis. We therefore summarize the most important results. ⁴ Consider the following linear difference equation of order p with constant coefficients. This equation is defined by the recursion:

Thereby { X t } represents a sequence of real numbers and ϕ ₁ ,  … ,  ϕ p are p constant coefficients. The above difference equation is called homogeneous because it involves no other variable than X t . A solution to this equation is a function $$F: \mathbb{Z} \rightarrow \mathbb{R}$$ such that its values F ( t ) or F t reduce the difference equation to an identity.

It is easy to see that if { X t ⁽¹⁾ } and { X t ⁽²⁾ } are two solutions than { c ₁ X t ⁽¹⁾ + c ₂ X t ⁽²⁾ }, for any $$c_{1},c_{2} \in \mathbb{R}$$ , is also a solution. The set of solutions is therefore a linear space (vector space).

Definition B.1.

A set of solutions {{X t ⁽¹⁾ },…,{X t (m) }}, m ≤ p, is called linearly independent if

implies that $$c_{1} =\ldots = c_{m} = 0$$ . Otherwise we call the set linearly dependent .

Given arbitrary starting values x ₀ ,  … ,  x p −1 for X ₀ ,  … ,  X p −1 , the difference equation determines all further through the recursion:

Similarly for X t mit $$t = -1,-2,\ldots$$ . Suppose we have p linearly independent solutions {{ X t ⁽¹⁾ },  … , { X t ( p ) }} then there exists exactly p numbers c ₁ ,  … ,  c p such that the solution

is compatible with arbitrary starting values x ₀ ,  … ,  x p −1 . These starting values then determine uniquely all values of the sequence { X t }. Thus { X t } is the only solution compatible with starting values. The goal therefore consists in finding p linearly independent solutions.

We guess that the solutions are of the form $$X_{t} = z^{-t}$$ where z may be a complex number. If this guess is right then we must have for t  = 0:

This equation is called the characteristic equation . ⁵ Thus z must be a root of the polynomial $$\Phi (z) = 1 -\phi _{1}z -\ldots -\phi _{p}z^{p}$$ . From the fundamental theorem of algebra we know that there are exactly p roots in the field of complex numbers. Denote these roots by z ₁ ,  … ,  z p .

Suppose that these roots are different from each other. In this case $$\{\{z_{1}^{-t}\},\ldots,\{z_{p}^{-t}\}\}$$ constitutes a set of p linearly independent solutions. To show this it is sufficient to verify that the determinant of the matrix

is different from zero. This determinant is known as Vandermonde’s determinant and is equal to $$\det W =\prod _{1\leq i<j\leq p}(z_{i} - z_{j})$$ . This determinant is clearly different from zero because the roots are different from each other. The general solution to the difference equation therefore is

(B.1)

where the constants c ₁ ,  … ,  c p are determined from the starting values (initial conditions).

In the case where some roots of the characteristic polynomial are equal, the general solution becomes more involved. Let z ₁ ,  … ,  z r , r  <  p , be the roots which are different from each other and denote their corresponding multiplicities by m ₁ ,  … ,  m r . It holds that $$\sum _{j=1}^{r} = p$$ . The general solution is then given by

(B.2)

where the constants c ji are again determined from the starting values (initial conditions).

C Stochastic Convergence

This appendix presents the relevant concepts and theorems from probability theory. The reader interested in more details should consult corresponding textbooks, for example Billingsley ( 1986 ), Brockwell and Davis ( 1991 ), Hogg and Craig ( 1995 ), or Kallenberg ( 2002 ) among many others.

In the following, all real random variables or random vectors X are defined with respect to some probability space $$(\Omega,\mathfrak{A},\mathbf{P})$$ . Thereby, $$\Omega $$ denotes an arbitrary space with $$\sigma$$ -field $$\mathfrak{A}$$ and probability measure P . A random variable, respectively random vector, X is then defined as a measurable function from $$\Omega $$ to $$\mathbb{R}$$ , respectively $$\mathbb{R}^{n}$$ . The probability space $$\Omega $$ plays no role as it is introduced just for the sake of mathematical rigor. The interest rather focuses on the distributions induced by P ∘ X ⁻¹ .

We will make use of the following important inequalities.

Theorem C.1 (Cauchy-Bunyakovskii-Schwarz Inequality).

For any two random variables X and Y,

The equality holds if and only if $$X = \tfrac{\mathbb{E}(XY )} {\mathbb{E}(Y ^{2})} Y$$ .

Theorem C.2 (Minkowski’s Inequality).

Let X and Y be two random variables with $$\mathbb{E}\vert X\vert ^{2} < \infty $$ and $$\mathbb{E}\vert Y \vert ^{2} < \infty $$ , then

Theorem C.3 (Chebyschev’s Inequality).

If $$\mathbb{E}\vert X\vert ^{r} < \infty $$ for r ≥ 0 then for every r ≥ 0 and any $$\varepsilon > 0$$

Theorem C.4 (Borel-Cantelli Lemma).

Let $$A_{1},A_{2},\ldots \in \mathfrak{A}$$ be an infinite sequence of events in some probability space $$(\Omega,\mathfrak{A},\mathbf{P})$$ such that $$\sum _{k=1}^{\infty }\mathbf{P}(A_{k}) < \infty $$ . Then, P {A k  i.o.} = 0. The event {A k  i.o.} is defined by $$\{A_{k}\:i.o.\} =\limsup _{k}\{A_{k}\} =\bigcap _{ k=1}^{\infty }\bigcup _{j=k}^{\infty }A_{j}$$ where i.o. stands for infinitely often.

On several occasions it is necessary to evaluate the limit of a sequence of random variables. In probability theory several concepts of convergence are discussed: almost sure convergence , convergence in probability , convergence in r-th mean ( convergence in quadratic mean ), convergence in distribution . We only give definitions and the most important theorems leaving an in-depth discussion to the relevant literature. Although not explicitly mentioned, many of the theorem below also hold in an analogous way in a multidimensional context.

Definition C.1 (Almost Sure Convergence).

For random variables X and {X t } defined on the same probability space $$(\Omega,\mathfrak{A},\mathbf{P})$$ , we say that {X t } converges almost surely or with probability one to X if

This fact is denoted by $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;a.s.\;}X$$ or $$\lim X_{t} = X\mathrm{a.s.}$$

Theorem C.5 (Kolmogorov’s Strong Law of Large Numbers (SLLN)).

Let X,X ₁ ,X ₂ ,… be identically and independently distributed random variables. Then, the arithmetic average $$\overline{X}_{T} = \tfrac{1} {T}\sum _{t=1}^{T}X_{ t}$$ converges almost surely to $$\mathbb{E}X$$ if and only if $$\mathbb{E}\vert X\vert < \infty $$ .

Definition C.2 (Convergence in Probability).

For random variables X and {X t } defined on the same probability space, we say that {X t } converges in probability to X if

This fact is denoted by $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;p\;}X$$ or $$\mathop{\mathrm{plim}}X_{t} = X$$ .

Remark C.1.

If X and { X t } are real valued random vectors, we replace the absolute value in the definition above by the Euclidean norm $$\|.\|$$ . This is, however, equivalent to saying that every component X it converges in probability to X i , the i -th component of X .

Definition C.3 (Convergence in r -th Mean).

A sequence {X t } of random variables converges in r-th mean to a random variable X if

We denote this fact by $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;r\;}X$$ . If r = 1 we say that the sequence converges absolutely; and if r = 2 we say that the sequence converges in mean square which is denoted by $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;m.s.\;}X$$ .

Remark C.2.

In the case r  = 2, the corresponding definition for random vectors is

Theorem C.6 (Riesz-Fisher).

Let {X t } be a sequence of random variables such $$\sup _{t}\mathbb{E}\vert X_{t}\vert ^{2} < \infty $$ . Then there exists a random variable X with $$\mathbb{E}\vert X\vert ^{2} < \infty $$ such that

This version of the Riesz-Fisher theorem provides a condition, known as the Cauchy criterion, which is often easier to verify when the limit is unknown.

Definition C.4 (Convergence in Distribution).

A sequence {X t } of random vectors with corresponding distribution functions $$\{F_{X_{t}}\}$$ converges in distribution , if there exists an random vector X with distribution function F X such that

where $$\mathcal{C}$$ denotes the set of points for which F X (x) is continuous. We denote this fact by $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;d\;}X$$ .

Note that, in contrast to the previously mentioned modes of convergence, convergence in distribution does not require that all random vectors are defined on the same probability space. The convergence in distribution states that, for large enough t , the distribution of X t can be approximated by the distribution of X .

The following Theorem relates the four convergence concepts.

Theorem C.7.

(i)

If $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;a.s.\;}X$$ then $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;p\;}X$$ .

(ii)

If $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;p\;}X$$ then there exists a subsequence $$\{X_{t_{n}}\}$$ such that $$X_{t_{n}}\mathop{\longrightarrow}\limits_{}^{\;a.s.\;}X$$ .

(iii)

If $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;r\;}X$$ then $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;p\;}X$$ by Chebyschev’s inequality (Theorem  C.3 ).

(iv)

If $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;p\;}X$$ then $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;d\;}X$$ .

(v)

If X is a fixed constant, then $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;d\;}X$$ implies $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;p\;}X$$ . Thus, the two concepts are equivalent under this assumption.

These facts can be summarized graphically:

A further useful theorem is:

Theorem C.8.

If $$\mathbb{E}X_{t}\longrightarrow \mu$$ and $$\mathbb{V}X_{t}\longrightarrow 0$$ then $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;m.s.\;}\mu$$ and consequently $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;p\;}\mu$$ .

Theorem C.9 (Continuous Mapping Theorem).

For any continuous function $$f: \mathbb{R}^{n}\longrightarrow \mathbb{R}^{m}$$ and random vectors {X t } and X defined on some probability space, the following implications hold:

(i)

$$X_{t}\mathop{\longrightarrow}\limits_{}^{\;a.s.\;}X$$ implies $$f(X_{t})\mathop{\longrightarrow}\limits_{}^{\;a.s.\;}f(X)$$ .

(ii)

$$X_{t}\mathop{\longrightarrow}\limits_{}^{\;p\;}X$$ implies $$f(X_{t})\mathop{\longrightarrow}\limits_{}^{\;p\;}f(X)$$ .

(iii)

$$X_{t}\mathop{\longrightarrow}\limits_{}^{\;d\;}X$$ implies $$f(X_{t})\mathop{\longrightarrow}\limits_{}^{\;d\;}f(X)$$ .

An important application of the Continuous Mapping Theorem is the so-called Delta method which can be used to approximate the distribution of f ( X t ) (see Appendix E).

A further useful result is given by:

Theorem C.10 (Slutzky’s Lemma).

Let {X t } and {Y t } be two sequences of random vectors such that $$X_{t}\mathop{\longrightarrow}\limits_{}^{\;d\;}X$$ and $$Y _{t}\mathop{\longrightarrow}\limits_{}^{\;d\;}c$$ , c constant, then

(i)

$$X_{t} + Y _{t}\mathop{\longrightarrow}\limits_{}^{\quad \text{d}\quad }X + c$$ ,

(ii)

$$Y _{t}'X_{t}\mathop{\longrightarrow}\limits_{}^{\quad \text{d}\quad }c'X$$ .

(iii)

$$X_{t}/Y _{t}\mathop{\longrightarrow}\limits_{}^{\quad d\quad }X/c$$ if c is a nonzero scalar.

Like the (cumulative) distribution function, the characteristic function provides an alternative way to describe a random variable.

Definition C.5 (Characteristic Function).

The characteristic function of a real random vector X, denoted by $$\varphi _{X}$$ , is defined by

where $$\imath$$ is the imaginary unit.

If, for example, $$X \sim \mathrm{ N}(\mu,\sigma ^{2})$$ , then $$\varphi _{X}(s) =\exp (\imath{s}\mu -\frac{1} {2}\sigma ^{2}s^{2})$$ . The characteristic function uniquely determines the distribution of X . Thus, if two random variables have the same characteristic function, they have the same distribution. Moreover, convergence in distribution is equivalent to convergence of the corresponding characteristic functions.

Theorem C.11 (Convergence of Characteristic Functions, Lévy).

Let {X t } be a sequence of real random variables with corresponding characteristic functions $$\varphi _{X_{t}}$$ then

In many cases the limiting distribution is a normal distribution. In which case one refers to the asymptotic normality.

Definition C.6 (Asymptotic Normality).

A sequence of random variables {X t } with “means” μ t and “variances” $$\sigma _{t}^{2} > 0$$ is said to be asymptotically normally distributed if

Note that the definition does not require that $$\mu _{t} = \mathbb{E}X_{t}$$ nor that $$\sigma _{t}^{2} = \mathbb{V}(X_{t})$$ . Asymptotic normality is obtained if the X t ’s are identically and independently distributed with constant mean and variance. In this case the Central Limit Theorem (CLT) holds.

Theorem C.12 (Central Limit Theorem).

Let {X t } be a sequence of identically and independently distributed random variables with constant mean μ and constant variance $$\sigma ^{2}$$ then

where $$\overline{X}_{T} = T^{-1}\sum _{t=1}^{T}X_{t}$$ is the arithmetic average.

It is possible to relax the assumption of identically distributed variables in various ways so that there exists a variety of CLT’s in the literature. For our purpose it is especially important to relax the independence assumption. A natural way to do this is by the notion of m- dependence .

Definition C.7 (m-Dependence).

A strictly stationary random process {X t } is called m-dependent for some nonnegative integer m if and only if the two sets of random variables {X τ ,τ ≤ t} and $$\{X_{\tau },\tau \geq t + m + 1\}$$ are independent.

Note that for such processes $$\Gamma (j) = 0$$ for j  >  m . This type of dependence allows to proof the following generalized Central Limit Theorem (see Brockwell and Davis  1991 ).

Theorem C.13 (CLT for m-Dependent Processes).

Let {X t } be a strictly stationary mean zero m-dependent process with autocovariance function $$\Gamma (h)$$ such that $$\mathbf{V}_{m} =\sum _{ h=-m}^{m}\Gamma (h)\neq 0$$ then

(i)

$$\lim _{T\rightarrow \infty }T\mathbb{V}(\overline{X}_{T}) = \mathbf{V}_{m}$$ and

(ii)

$$\sqrt{T}\overline{X}_{T}$$ is asymptotically normal N (0, V m ).

Often it is difficult to derive the asymptotic distribution of { X t } directly. This situation can be handled by approximating the original process { X t } by a process { X t ( m ) } which is easier to handle in terms of its asymptotic distribution and where the precision of the approximation can be “tuned” by the parameter m .

Theorem C.14 (Basis Approximation Theorem).

Let {X t } and {X t (m) } be two random vectors process such that

(i)

$$X_{t}^{(m)}\mathop{\longrightarrow}\limits_{}^{\quad \text{d}\quad }X^{(m)}$$ as $$t \rightarrow \infty $$ for each m = 1,2,…,

(ii)

$$X^{(m)}\mathop{\longrightarrow}\limits_{}^{\quad \text{d}\quad }X$$ as $$m \rightarrow \infty $$ , and

(iii)

$$\lim _{m\rightarrow \infty }\limsup _{t\rightarrow \infty }\mathbf{P}[\vert X_{t} - X_{t}^{(m)}\vert >\epsilon ] = 0$$ for every ε > 0.

Then

D Beveridge-Nelson Decomposition

The Beveridge-Nelson decomposition proves to be an indispensable tool. Based on the seminal paper by Phillips and Solo ( 1992 ), we proof the following Theorem for matrix polynomials where $$\|.\|$$ denotes the matrix norm (see Definition  10.​6 in Chapter  10 ). The univariate version is then a special case with the absolute value replacing the norm.

Theorem D.1.

Any a lag polynomial $$\Psi (\mathrm{L}) =\sum _{ j=0}^{\infty }\Psi _{j}\mathrm{L}^{j}$$ where $$\Psi _{j}$$ are n × n matrices with $$\Psi _{0} = I_{n}$$ can be represented by

(D.1)

where $$\widetilde{\Psi }(\mathrm{L}) =\sum _{ j=0}^{\infty }\widetilde{\Psi }_{j}\mathrm{L}$$ with $$\widetilde{\Psi }_{j} =\sum _{ i=j+1}^{\infty }\Psi _{i}$$ . Moreover,

Proof.

The first part of the Theorem is obtained by the algebraic manipulations below:

Taking any δ  ∈ (1∕2, 1), the second part of the Theorem follows from

The first inequality follows from the triangular inequality for the norm. The second inequality is Hölder’s inequality (see, for example, Naylor and Sell  1982 , p. 548) with $$p = q = 2$$ . The third and the fourth inequality follow from the Lemma below. The last inequality, finally, follows from the assumption.

The last assertion follows from

The last inequality is again a consequence of Hölder’s inequality. The summability assumption then guarantees the convergence of the first term in the product. Cauchy’s condensation test finally establishes the convergence of the last term. □ 

Lemma D.1.

The following results are useful:

(i)

For any b > 0, $$\sum _{i=j+1}^{\infty }i^{-1-b} \leq b^{-1}j^{-b}$$ .

(ii)

For any c ∈ (0,1), $$\sum _{j=1}^{i}j^{c-1} \leq c^{-1}i^{c}$$ .

Proof.

Let k be a number greater than j , then $$k^{-1-b} \leq j^{-1-b}$$ and

This implies that $$\sum _{k=j+1}^{\infty }k^{-1-b} \leq b^{-1}j^{-b}$$ . This proves part (i) by changing the summation index back from k to j . Similarly, $$k^{c-1} \leq j^{c-1}$$ and

Therefore $$\sum _{k=1}^{i}k^{c-1} \leq c^{-1}i^{c}$$ which proves part (ii) by changing the summation index back from k to j . □ 

Remark D.1.

An alternative common assumption is $$\sum _{j=1}^{\infty }j\|\Psi _{j}\| < \infty $$ . It is, however, easy to see that this assumption is more restrictive as it implies the one assumed in the Theorem, but not vice versa. See Phillips and Solo ( 1992 ) for more details.

E The Delta Method

It is often the case that it is possible to obtain an estimate $$\hat{\beta }_{T}$$ of some parameter β , but that one is really interested in a function f of β . The Continuous Mapping Theorem then suggests to estimate f ( β ) by $$f(\hat{\beta }_{T})$$ . But then the question arises how the distribution of $$\hat{\beta }_{T}$$ is related to the distribution of $$f(\hat{\beta }_{T})$$ .

Expanding the function into a first order Taylor approximation allows to derive the following theorem.

Theorem E.1.

Let $$\{\hat{\beta }_{T}\}$$ be a K-dimensional sequence of random variables with the property $$\sqrt{ T}(\hat{\beta }_{T}-\beta )\mathop{\longrightarrow}\limits_{}^{\quad \mathrm{d}\quad }N(0,\Sigma )$$ then

where $$f: \mathbb{R}^{K}\longrightarrow \mathbb{R}^{J}$$ is a continuously differentiable function with Jacobian matrix (matrix of first order partial derivatives) $$\nabla f(\beta ) = \partial f(\beta )/\partial \beta '$$ .

Proof.

See Serfling (Serfling  1980 , 122–124). □ 

Remark E.1.

In the one-dimensional case where $$\sqrt{T}(\hat{\beta }_{T}-\beta )\mathop{\longrightarrow}\limits_{}^{\quad \mathrm{d}\quad }\mathrm{N}(0,\sigma ^{2})$$ and $$f: \mathbb{R}\longrightarrow \mathbb{R}$$ the above theorem becomes:

where f ′( β ) is the first derivative evaluated at β .

Remark E.2.

The J × K Jacobian matrix of first order partial derivatives is defined as

Remark E.3.

In most applications β is not known so that one evaluates the Jacobian matrix at $$\hat{\beta }_{T}$$ .

Example: Univariate

Suppose we have obtained an estimate of β equal to $$\hat{\beta }= 0.6$$ together with an estimate for its variance $$\hat{\sigma }_{\hat{\beta }}^{2} = 0.2$$ . We can then approximate the variance of $$f(\hat{\beta }) = 1/\hat{\beta } = 1.667$$ by

Example: Multivariate

In the process of computing the impulse response function of a VAR(1) model with $$\Phi = \left (\begin{array}{*{10}c} \phi _{11} & \phi _{12}\\ \phi _{ 21} & \phi _{22}\\ \end{array} \right )$$ one hasto calculate $$\Psi _{2} = \Phi ^{2}$$ . If we stack all coefficients of $$\Phi $$ into a vector $$\beta =\mathrm{ vec}(\Phi ) = (\phi _{11},\phi _{21},\phi _{12},\phi _{22})'$$ then we get:

where $$\Psi _{2} = \left [\psi _{ij}^{(2)}\right ]$$ . The Jacobian matrix then becomes:

In Section  15.​4.​4 we obtained the following estimate for a VAR(1) model for $$\{X_{t}\} =\{ (\ln (\mathrm{A}_{t}),\ln (\mathrm{S}_{t}))'\}$$ :

The estimated covariance matrix of $$\mathrm{vec}\,\hat{\Phi }$$ , $$\hat{\mathrm{V}}(\mathrm{vec}\,\hat{\Phi })$$ , was:

We can then approximate the variance of $$f(\hat{\beta }) =\mathrm{ vec}(\Phi ^{2})$$ by

This leads:

References

Abraham B, Ledolter J (1983) Statistical methods for forecasting. Wiley, New York

Adelman I, Adelman FL (1959) The dynamic properties of the Klein-Goldberger model. Econometrica 27:596–625

Agarwal RP (2000) Difference equations and inequalities, 2nd edn. Marcel Dekker, New York

Akaike H (1969) Fitting autoregressive models for prediction. Ann Inst Stat Math 21:243–247

Amemiya T (1994) Introduction to statistics and econometrics. Harvard University Press, Cambridge

An S, Schorfheide F (2007) Bayesian analysis of DSGE models. Econ Rev 26:113–172

Anderson BDO, Moore JB (1979) Optimal filtering. Electrical Engeneering Series. Prentice-Hall, Englewood Cliffs

Andrews DWK (1991) Heteroskedasticity and autocorrelation consistent covariance matrix estimation. Econometrica 59:817–858

Andrews DWK (1993) Tests for parameter instability and structural change with unknown change point. Econometrica 61:821–856

Andrews DWK (2003) Tests for parameter instability and structural change with unknown change point: A corrigendum. Econometrica 71:395–397

Andrews DWK, Monahan JC (1992) An improved heteroskedasticity and autocorrelation consistent covariance matrix estimator. Econometrica 60:953–966

Andrews DWK, Ploberger W (1994) Optimal tests when a nuisance parameter is present only under the alternative. Econometrica 62:1383–1414

Arias JE, Rubio-Ramírez J, Waggoner DF (2014) Inference based on SVARs identified with sign and zero restrictions: Theory and applications. FRB Atlanta Working Paper 2014–1, Federal Reserve Bank of Atlanta

Ashley R, Granger CWJ, Schmalensee R (1980) Advertising and aggregate consumption: An analysis of causality. Econometrica 48(5):1149–1168

Aue A, Horváth L (2011) Structural breaks in time series. J Time Ser Anal 34:1–16

Bai J (2000) Vector autoregressive models with structural changes in regression coefficients and in variance-covariance matrices. Ann Econ Finance 1:303–339

Bai J, Lumsdaine RL, Stock JH (1998) Testing for and dating common breaks in multivariate time series. Rev Econ Stud 65:395–432

Baker A (2002) Matrix groups – an introduction to Lie Group theory. Springer, London

Banbura M, Giannone D, Reichlin L (2010) Large Bayesian vector autoregressions. J Appl Econ 25:71–92

Banerjee A, Dolado J, Galbraith JW, Hendry DF (1993) Co-integration, error-correction, and the econometric analysis of non-stationary data. Oxford University Press, Oxford

Barsky R, Sims E (2011) News shocks and business cycles. J Monet Econ 58:273–289

Bauer D, Wagner M (2003) A canonical form for unit root processes on the state space framework, diskussionsschrift 3-12, Volkswirtschaftliches Institut, Universität Bern

Baumeister C, Hamilton JD (2015) Sign restrictions, structural vector autoregressions, and useful prior information. Econometrica 83:1963–1999

Beaudry P, Portier F (2006) Stock prices, news and economic fluctuations. Am Econ Rev 96(4):1293–1307

Berman A, Plemmons RJ (1994) Nonnegative matrices in the mathematical sciences. No. 9 in Classics in Applied Mathematics, Society of Industrial and Applied Mathematics, Philadelphia

Bernanke BS (1986) Alternative explanations of money-income correlation. In: Brunner K, Meltzer A (eds) Real business cycles, real exchange rates, and actual policies, no. 25 in Carnegie-Rochester Conference Series on Public Policy. North-Holland, Amsterdam, pp 49–100

Bernanke BS, Gertler M, Watson MW (1997:1) Systematic monetary policy and the effects of oil price shocks. Brook Pap Econ Act 91–142

Berndt ER (1991) The practice of econometrics. Addison Wesley, Reading,

Bhansali RJ (1999) Parameter estimation and model selection for multistep prediction of a time series: A review. In: Gosh S (ed) Asymptotics, Nonparametrics, and Time Series. Marcel Dekker, New York, pp 201–225

Billingsley P (1986) Probability and measure, 2nd edn. Wiley, New York

Blanchard OJ (1989) A traditional interpretation of macroeconomic fluctuations. Am Econ Rev 79:1146–1164

Blanchard OJ, Quah D (1989) The dynamic effects of aggregate demand and supply disturbances. Am Econ Rev 79:655–673

Blanchard OJ, Watson MW (1986) Are business cycles all alike? In: Gordon R (ed) The American business cycle: continuity and change. University of Chicago Press, Chicago, pp 123–179

Bollerslev T (1986) Generalized autoregressive conditional heteroskedasticity. J Econ 31:307–327

Bollerslev T (1988) On the correlation structure for the generalized autoregressive conditional heteroskedastic process. J Time Ser Anal 9:121–131

Bollerslev T, Engle RF, Nelson DB (1994) ARCH models. In: Engle RF, McFadden DL (eds) Handbook of econometrics, vol IV. Elsevier Science B.V., Amsterdam, pp 2959–3038

Bougerol P, Picard N (1992a) Stationarity of GARCH processes and some nonnegative time series. J Econ 52:115–127

Bougerol P, Picard N (1992b) Strict stationarity of generalized autoregressive processes. Ann Probab 20:1714–1730

Box GEP, Jenkins GM (1976) Time series analysis: forecasting and control, revised edn. Holden-Day, San Francisco

Brandner P, Neusser K (1992) Business cycles in open economies: Stylized facts for Austria and Germany. Weltwirtschaftliches Arch 128:67–87

Brandt A (1986) The stochastic equation $$y_{n+1} = a_{n}y_{n} + b_{n}$$ with stationary coefficients. Adv Appl Probab 18:211–220

Bräuning F, Koopman SJ (2014) Forecasting macroeconomic variables using collapsed dynamic factor analysis. Int J Forecast 30:572–584

Breitung J, Eickmeier S (2006) Dynamic factor models. In: Hübler O, Frohn J (eds) Modern econometric analysis, chap 3. Springer, Berlin, pp 25–40

Brockwell PJ, Davis RA (1991) Time series: theory and methods, 2nd edn. Springer, New York

Brockwell PJ, Davis RA (1996) Introduction to time series and forecasting. Springer, New York

Brualdi RA, Shader BL (1995) Matrices of sign-solvable linear systems. No. 116 in Cambridge tracts in mathematics. Cambridge University Press, Cambridge

Burren D, Neusser K (2013) The role of sectoral shifts in the decline of real GDP volatility. Macroecon Dyn 17:477–500

Campbell JY (1987) Does saving anticipate declining labor income? An alternative test of the permanent income hypothesis. Econometrica 55:1249–1273

Campbell JY, Mankiw NG (1987) Are output fluctuations transitory? Q J Econ 102:857–880

Campbell JY, Perron P (1991) Pitfalls and opportunities: What macroeconomists should know about unit roots. In: Blanchard OJ, Fischer S (eds) Macroeconomics annual 1991, vol 6. MIT Press, Cambridge, pp 141–201

Campbell JY, Shiller RJ (1987) Cointegration and tests of present value models. J Polit Econ 95:1062–1088

Campbell JY, Lo AW, MacKinlay AC (1997) The econometrics of financial markets. Princeton University Press, Princeton

Canova F (2007) Methods for applied macroeconomic research. Princeton University Press, Princeton

Canova F, Ciccarelli M (2008) Estimating multi-country VAR models. Working Paper 603, European Central Bank

Canova F, De Nicoló G (2002) Monetary disturbances matter for business fluctuations in the G–7. J Monet Econ 49:1131–1159

Cavaliere G, Rahbek A, Taylor AMR (2012) Bootstrap determination of the co-integration rank in vector autoregressive models. Econometrica 80:1721–1740

Chan JCC, Jeliazkov I (2009) Efficient simulation and integrated likelihood estimation in state space models. Int J Math Model Numer Optim 1:101–120

Chari VV, Kehoe PJ, McGrattan ER (2008) Are structural VARs with long-run restrictions useful in developing business cycle theory? J Monet Econ 55:1337–1352

Chow GC, Lin A (1971) Best linear unbiased interpolation, distribution, and extrapolation of time series by related series. Rev Econ Stat 53:372–375

Christiano LJ, Eichenbaum M (1990) Unit roots in real GNP: Do we know and do care? Carn-Roch Conf Ser Public Pol 32:7–62

Christiano LJ, Eichenbaum M, Evans CL (1999) Monetary policy shocks: what have we learned and to what end? Handbook of macroeconomics, vol 1A, chap 2. North-Holland, Amsterdam, pp 65–148

Christiano LJ, Eichenbaum M, Vigfusson RJ (2003) What happens after a technology shock? Working Paper No. 9819, NBER

Christiano LJ, Eichenbaum M, Vigfusson RJ (2006) Assessing structural VARs. International Finance Discussion Papers No. 866, Board of Governors of the Federal Reserve System

Christoffersen PF (1998) Evaluating interval forecasts. Int Econ Rev 39:841–862

Clements MP, Hendry DF (1996) Intercept corrections and structural change. J Appl Econ 11:475–494

Clements MP, Hendry DF (2006) Forecasting with breaks. In: Handbook of economic forecasting, vol 1, Elsevier, Amsterdam, pp 605–657

Cochrane JH (1988) How big is the random walk in GNP? J Polit Econ 96(5):893–920

Cogley T, Sargent TJ (2001) Evolving post-world war II U.S. inflation dynamics. In: Bernanke BS, Rogoff K (eds) NBER macroeconomics annual, vol 16. MIT Press, Cambridge, pp 331–373

Cogley T, Sargent TJ (2005) Drift and volatilities: Monetary policies and outcomes in the post WWII U.S. Rev Econ Dyn 8:262–302

Colonius F, Kliemann W (2014) Dynamical systems and linear algebra. Graduate studies in mathematics, vol 158. American Mathematical Society, Providence

Colonius F, Kliemann W (2014) Dynamical Systems and Linear Algebra. Graduate Studies in Mathematics, Vol. 158. American Mathematical Society, Providence, Rhode Island

Cooley TF, LeRoy SF (1985) Atheoretical macroeconometrics - a critique. J Monet Econ 16:283–308

Cooley TF, Prescott EC (1973) Varying parameter regression. A theory and some applications. Ann Econ Soc Meas 2:463–474

Cooley TF, Prescott EC (1976) Estimation in the presence of stochastic parameter variation. Econometrica 44:167–184

Corradi V, Swanson NR (2006) Predictive density evaluation. In: Handbook of economic forecasting, vol 1, Elsevier, Amsterdam, pp 197–284

Cuche NA, Hess MA (2000) Estimating monthly GDP in a general Kalman filter framework: Evidence from Switzerland. Econ Financ Model 7:153–194

Davidson JEH, Hendry DF, Srba F, Yeo S (1978) Econometric modelling of the aggregate time-series relationship between consumers’ expenditure and income in the United Kingdom. Econ J 88:661–692

Davidson R, MacKinnon JG (1993) Estimation and inference in econometrics. Oxford University Press, Oxford

Dees S, Mauro FD, Pesaran MH, Smith V (2007) Exploring the international linkages of the Euro area: A global VAR analysis. J Appl Econ 22:1–38

Deistler M, Neusser K (2012) Prognosen uni- und multivariater Zeitreihen. In: Mertens P (ed) Prognoserechnung. Physica-Verlag, Heidelberg, pp 225–256

Dejong DN, Dave C (2007) Structural macroeconomics. Princeton University Press, Princeton

Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood estimation from incomplete data via the EM algorithm. J R Stat Soc Ser B 39:1–38

Dhrymes PJ (1978) Introductory econometrics. Springer, New York

Dickey D, Fuller WA (1976) Distribution of the estimators for autoregressive time series with a unit root. J Am Stat Assoc 74:427–431

Dickey DA, Fuller WA (1981) Likelihood ratio statistics for autoregressive time series with a unit root. Econometrica 49:1057–1072

Diebold FX, Lee JH, Weinbach GC (1994) Regime switching with time-varying transition probabilities. In: Hargreaves CP (ed) Nonstationary time series analysis and cointegration. Oxford Universiy Press, Oxford, pp 283–302

Diebold FX, Gunther TA, Tay AS (1998) Evaluating density forecasts, with applications to financial risk management. Int Econ Rev 39:863–883

Doan T, Litterman RB, Sims CA (1984) Forecasting and conditional projection using realistic prior distributions. Econ Rev 3:1–100

Dufour JM (1985) Unbiasedness of predictions from estimated autoregressions. Econ Theory 1:387–402

Durbin J (1960) The fitting of time series models. Revue de l’Institut International de Statistique 28:233–244

Durbin J, Koopman SJ (2011) Time series analysis by state space methods, 2nd edn. Oxford University Press, Oxford

Edelman A, Rao NR (2005) Random matrix theory. Acta Numerica 14:1–65

Efron B, Tibshirani RJ (1993) An introduction to the bootstrap. Monographs on statistics and applied probability, vol 57. Chapman & Hall/CRC, Boca Raton

Elaydi S (2005) An introduction to difference equations, 3rd edn. Springer Science + Business Media, New York

Elder J, Kennedy PE (2001) Testing for unit roots: What should students be taught? J Econ Educ 32:137–146

Elliott G, Timmermann A (2008) Economic forecasting. J Econ Lit 46:3–56

Engle RF (1982) Autoregressive conditional heteroskedasticity with estimates of the variance of the United Kingdom inflation. Econometrica 50:987–1007

Engle RF (2004) Risk and volatility: Econometric models and financial practice. Am Econ Rev 94:405–420

Engle RF, Bollerslev T (1986) Modeling the persistence of conditional variances. Econ Rev 5:1–50

Engle RF, Granger CWJ (1987) Co-integration and error correction: Representation, estimation and testing. Econometrica 55:251–276

Engle RF, Lilien D, Robins R (1987) Estimating time varying risk premia in the term structure: The ARCH–M model. Econometrica 55:391–407

Engsteg T, Pedersen TQ (2014) Bias–correction in vectorautoregressive models: A simulation study. Econometrics 2:45–71

Epstein RJ (1987) A history of econometrics. North-Holland, Amsterdam

Eurostat (2009) ESS guidelines on seasonal adjustment. Eurostat, Luxembourg

Fan J, Yao Q (2003) Nonlinear time series. Springer, New York

Faust J (1998) The robustness of identified VAR conclusions about money. Carn-Roch Conf Ser Publis Policy 49:207–244

Fernandez-Villaverde J, Rubio-Ramírez JF, Sargent TJ, Watson MW (2007) ABCs and (Ds) of understanding VARs. Am Econ Rev 97:1021–1026

Filardo AJ (1994) Business-cycle phases and their transitional dynamics. J Bus Econ Anal 12:299–308

Filardo AJ, FGordon S (1998) Business cycle duration. J Econ 85:99–123

Francis N, Owyang MT, Roush JE, DiCecio R (2014) A flexible finite-horizon alternative to long-run restrictions with an application to technology shocks. Rev Econ Stat 96:638–647

Friedman M, Schwartz AJ (1963) A monetary history of the United States, 1867–1960. Priceton University Press, Princeton

Frisch R (1933) Propagation problems and impulse problems in dynamic economics. In: Economic essays in honour of Gustav cassel. Frank Cass, London, pp 171–205

Frühwirt-Schnatter S (2006) Finite mixture and markov switching models. Springer Science + Business Media LLC, New York

Fry RA, Pagan AR (2011) Sign restrictions in structural vector autoregressions: A critical review. J Econ Lit 49:938–960

Fuller WA (1976) Introduction to statistical time series. Wiley, New York

Galí J (1992) How well does the IS-LM model fit postwar U.S. data? Q J Econ 107(2):709–738

Galí J (1999) Technology, employment, and the business cycle: Do technology shocks explain aggregate fluctuations? Am Econ Rev 89:249–271

Geweke JF (1984) Inference and causality in economic time series models. In: Griliches Z, Intriligator MD (eds) Handbook of econometrics, vol II. Elsevier, Amsterdam, pp 1101–1144

Geweke JF (2005) Contemporary bayesian econometrics and statistics. Wiley series in probability and statistics. Wiley, New York

Ghysels E, Osborn DR (2001) The econometric analysis of seasonal time series. Cambridge University Press, Cambridge

Giannini C (1991) Topics in structural VAR econometrics. Quaderni di Ricerca 21, Università degli Studi di Anacona, Dipartimento di Economia

Giraitis L, Kokoszka P, Leipus R (2000) Stationary ARCH models: Dependence structure and central limit theorem. Econ Theory 16:3–22

Glosten LR, Jagannathan R, Runkle DE (1993) On the relation between expected value and the volatility of the nominal excess returns on stocks. J Finance 48:1779–1801

Gohberg I, Lancaster P, Rodman L (1982) Matrix polynomials. Academic Press, New York

Goldfeld SM, Quandt RE (1973) A Markov model for switching regressions. J Econ 1:3–15

Goldfeld SM, Quandt RE (1976) Studies in nonlinear estimation. Ballinger Publishing, Cambridge

Gómez V, Maravall A (1996) Programs TRAMO and SEATS. Instructions for the user (with some updates). Working Paper 9628, Servicio de Estudios, Banco de España

Gonzalo J, Ng S (2001) A systematic framework for analyzing the dynamic effects of permanent and transitory schocks. J Econ Dyn Control 25:1527–1546

Gospodinov N (2010) Inference in nearly nonstationary SVAR models with long-run identifying restrictions. J Bus Econ Stat 28:1–12

Gouriéroux C (1997) ARCH models and financial applications. Springer, New York

Gouriroux C, Jasiak J, Sufana R (2009) The Wishart autoregressive process of multivariate stochastic volatility. J Econ 150:167–181

Granger CWJ (1964) Spectral analysis of economic time series. Princeton University Press, Princeton

Granger CWJ (1966) The typical spectral shape of an economic variable. Econometrica 34:150–161

Granger CWJ (1969) Investigating causal relations by econometric models and cross-spectral methods. Econometrica 37(3):424–438

Granger CWJ, Newbold P (1974) Spurious regression in econometrics. J Econ 2:111–120

Granger CWJ, Teräsvirta T (1993) Modelling nonlinear economic relationships. Oxford University Press, Oxford

Greene WH (2008) Econometric anlysis, 7th edn. Prentice Hall, New Jersey

Haan WJ, Levin AT (1997) A practitioner’s guide to robust covariance matrix estimation. In: Maddala GS, Rao CR (eds) Handbook of statistics: robust inference, vol 15. Elsevier, New York, pp 299–342

Hall FJ, Li Z (2014) Sign pattern matrices. In: Hogben L (ed) Handbook of linear algebra, 2nd edn, chap 42. Chapman & Hall/CRC, Boca Raton, pp 1–32

Hall P, Yao Q (2003) Inference in ARCH and GARCH models with heavy-tailed errors. Econometrica 71(1):285–317

Hall RE (1978) Stochastic implications of the life cycle-permanent income hypothesis: Theory and evidence. J Polit Econ 86:971–987

Hamilton JD (1994a) State-Space models. In: Engle RF, McFadden DL (eds) Handbook of econometrics, vol 4, chap 50. Elsevier, Amsterdam, pp 3039–3080

Hamilton JD (1994b) Time series analysis. Princeton University Press, Princeton

Hamilton JD (1996) Specification testing in Markov-switching time-series models. J Econ 70:127–157

Hannan EJ, Deistler M (1988) The statistical theory of linear systems. Wiley, New York

Hansen BE (1992) Efficient estimation and testing of cointegrating vectors in the presence of deterministic trends. J Econ 53:321–335

Hansen LP, Sargent TJ (1980) Formulating and estimating dynamic linear rational expectations models. J Econ Dyn Control 2:7–46

Hansen LP, Sargent TJ (1991) Two difficulties in interpreting vector autoregressions. In: Hansen LP, Sargent TJ (eds) Rational expectations econometrics, underground classics in economics. Westview, Boulder, Colorado, pp 77–119

Hansen LP, Sargent TJ (1993) Saesonality and approximation errors in rational expectations models. J Econ 55:21–55

Harvey AC (1989) Forecasting, structural time series models and the kalman filter. Cambridge University Press, Cambridge

Harvey AC, Jaeger A (1993) Detrending, stylized facts and the business cycle. J Appl Econ 8:231–247

Harvey AC, Phillips GD (1982) Estimation of regression models with time varying parameters. In: Deistler M, Fürst E, Schödiauer G (eds) Games, economic dynamics and time series analysis. Physica-Verlag, Wien-Würzburg, pp 306–321

Harvey AC, Pierce RG (1984) Estimating missing observations in economic time series. J Am Stat Assoc 79:125–131

Haugh LD (1976) Checking the independence of two covariance stationary time series: A univariate residual cross-correlation approach. J Am Stat Assoc 71:378–385

Hauser MA, Pötscher BM, Reschenhofer E (1999) Measuring persistence in aggregate output: ARMA models, fractionally integrated ARMA models and nonparametric procedures. Empir Econ 24:243–269

Hildreth C, Houck JP (1968) Some estimators for a linear model with random coefficients. J Am Stat Assoc 63:584–595

Hodrick RJ, Prescott EC (1980) Post-war U.S. business cycles: An empirical investigation. Discussion Paper 451, Carnegie-Mellon University, Pittsburgh

Hogg RV, Craig AT (1995) Introduction to mathematical statistics, 5th edn. Prentice-Hall, Upper Saddle River

Hong EP (1991) The autocorrelation structure for the GARCH-M process. Econ Lett 37:129–132

Howrey EP (1968) A spectrum analysis of the long swing hypothesis. Int Econ Rev 9:228–252

Hylleberg S (1986) Seasonality in regression. Academic Press, Orlando, FL

Hylleberg S, Engle RF, Granger CWJ, Yoo S (1990) Seasonal integration and cointegration. J Econ 44:215–238

Inoue A, Kilian L (2013) Inference on impulse response functions in structural VAR models. J Econ 177:1–13

Jensen ST, Rahbek A (2004) Asymptotic normality for non-stationary, explosive GARCH. Econ Theory 20(6):1203–1226

Johansen S (1988) Statistical analysis of cointegration vectors. J Econ Dyn Control 12:231–254

Johansen S (1991) Estimation and hypothesis testing of cointegration vectors in Gaussian vector autoregressive models. Econometrica 59:1551–1580

Johansen S (1995) Likelihood-based inference in cointegrated vector autoregressive models. Oxford University Press, Oxford

Johansen S (2007) Cointegration: A survey. In: Mills TC, Patterson K (eds) Econometric theory, palgrave handbook of econometrics, vol 1. Palgrave MacMillan, Basingstoke and New York, pp 540–577

Johansen S, Schaumburg E (1998) Likelihood analysis of seasonal cointegration. J Econ 88:301–339

Kailath T (1980) Linear systems. Prentice Hall, Englewood Cliffs

Kallenberg O (2002) Foundations of modern probability, 2nd edn. Probability and its applications, Springer, New York

Kalman RE (1960) A new approach to linear filtering and prediction problems. J Basic Eng Trans ASME Ser D 82:35–45

Kalman RE (1963) New methods in Wiener filtering theory. In: Bogdanoff JL, Kozin F (eds) Proceedings of the first symposium of engeneering applications of random function theory and probability. Wiley, New York, pp 270–388

Kendall MG (1954) Note on the bias in the estimation of autocorelation. Biometrika 41:403–404

Kiefer NM (1978) Discrete parameter variation: Efficient estimation of a switching regression model. Econometrica 46:427–434

Kilian L (1998) Small-sample confidence intervals for impulse response functions. Rev Econ Stat 80:186–201

Kilian L (2013) Structural vector autoregressions. In: Hashimzade N, Thornton M (eds) Handbook of research methods and applications in empirical macroeconomics. Edward Elgar, Cheltenham, pp 515–554

Kilian L, Murphy DP (2012) Why agnostic sign restrictions are not enough: Understanding the dynamics of oil market VAR models. J Eur Econ Assoc 10:1166–1188

Kim CJ (1994) Dynamic linear models with Markov-switching. J Econ 60:1–22

Kim CJ, Nelson CR (1999) State-space models with regime-switching: classical and Gibbs-sampling approaches with applications. MIT Press, Cambridge

King RG, Rebelo ST (1993) Low frequency filtering and real business cycles. J Econ Dyn Control 17:201–237

King RG, Plosser CI, Rebelo ST (1988) Production, growth, and business cycles: I. The basic neoclassical model. J Monet Econ 21:195–232

King RG, Plosser CI, Stock JH, Watson MW (1991) Stochastic trends and economic fluctuations. Am Econ Rev 81:819–840

Klein LR (1950) Economic fluctuations in United States 1921–1941. Wiley, New York

Klein LR (1985) New developments in project LINK. Am Econ Rev 75:223–227

Klein LR, Goldberger AS (1955) An econometric model of the United States, 1929–1952. North Holland, Amsterdam

Klüppelberg C, Lindner A, Maller R (2004) A continuous time GARCH process driven by a Lévy process: Stationarity and second order behaviour. J Appl Probab 41:601–622

Koop G, Korobilis D (2009) Bayesian multivariate time series methods for empirical macroeconomics. Found Trends Econ 3:267–358

Kristensen D, Linton O (2006) A closed-form estimator for the GARCH(1,1)-model. Econ Theory 22:323–337

Krolzig HM (1997) Markov–switching vector autoregressions. Modelling, statistical inference, and application to business cycle analysis. Springer, Berlin

Kunst R, Neusser K (1986) A forecasting comparison of some var techniques. Int J Forecast 2:447–456

Kunst R, Neusser K (1990) Cointegration in a macroeconomic system. J Appl Econ 5:351–365

Kuznets S (1930) Secular movements in production and prices. Their nature and their bearing upon cyclical fluctuations. Houghton Mifflin, Boston

Kwiatkowski D, Phillips PC, Schmidt P, Shin Y (1992) Testing the null hypothesis of stationarity against the alternative of a unit root. How sure are we that economic time series have a unit root? J Econ 54:159–178

Leamer EE (1981) Is it a demand curve, or is it a supply curve? Partial identification through inequality constraints. Rev Econ Stat 63:319–327

Lee SW, Hansen BE (1994) Asymptotic theory for the GARCH(1,1) quasi-maximum likelihood estimator. Econ Theory 10:29–52

Lindner AM (2009) Stationarity, mixing, distributional properties and moments of GARCH(p,q)-processes. In: Anderson TG, Davis RA, Kreiß JP, Mikosch T (eds) Handbook of financial time series. Springer, Berlin

Lippi M, Reichlin L (1993) The dynamic effects of aggregate demand and supply disturbances: Comment. Am Econ Rev 83:644–652

Litterman RB (1986) Forecasting with Bayesian vector autoregressions: Five years of experience. J Bus Econ Stat 4:25–38

Ljung L (1999) System identification: theory for the user, 2nd edn. Prentice Hall, Englewood Cliffs

Lucas RE (1976) Econometric policy evaluation: A critique. In: Brunner K, Meltzer AH (eds) The phillips curve and labor markets. Carnegie-Rochester Conference Series on Public Policy, vol 1. North-Holland, Amsterdam, pp 19–46

Lumsdaine RL (1986) Consistency and asymptotic normality of the quasi-maximum likelihood estimator in IGARCH(1,1) and covariance stationary GARCH(1,1) models. Econometrica 64:575–596

Lütkepohl H (1990) Asymptotic distributions of impulse response functions and forecast error variance decomposition. Rev Econ Stat 72:116–125

Lütkepohl H (2006) New introduction to multiple time series analysis. Springer, Berlin

Lütkepohl H, Staszewska-Bystrova A, Winker P (2013) Comparison of methods for constructing joint confidence bands for impulse response functions. Discussion Paper 1292, DIW Berlin

MacKinnon JG (1991) Critical values for co-integration tests. In: Engle RF, Granger CWJ (eds) Long-run economic relationships. Oxford University Press, Oxford, pp 267–276

MacKinnon JG (1996) Numerical distribution functions for unit root and cointegration tests. J Appl Econ 11:601–618

MacKinnon JG, Smith AA Jr (1998) Approximate bias correction in econometrics. J Econ 85: 205–230

MacKinnon JG, Haug A, Michelis L (1999) Numerical distribution functions of the likelihood ratio test for cointegration. J Appl Econ 14:563–577

Maddala GS (1986) Disequilibrium, self-selection, and switching models. In: Griliches Z, Intriligator MD (eds) Handbook of econometrics, vol III. North-Holland, Amsterdam

Magnus JR, Neudecker H (1988) Matrix differential calculus with applications in statistics and econometrics. Wiley, Chichester

Marcellino M, Stock JH, Watson MW (2006) A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series. J Econ 135:499–526

Marriott FHC, Pope JA (1954) Bias in the estimation of autocorrelation. Biometrika 41:393–402

Mertens P, Rässler S (2005) Prognoserechnung - einführung und Überblick. In: Mertens P, Rässler S (eds) Prognoserechnung. Physica-Verlag, Heidelberg, pp 1–37

Meyer CD (2000) Matrix analysis and applied linear algebra. Society for Industrial and Applied Mathematics, Philadelphia

Mills TC (2003) Modelling trends and cycles in economic time series. Palgrave texts in econometrics. Palgrave Macmillan, Hampshire

Miron JA (1996) The economics of seasonal cycles. MIT Press, Cambridge

Mittnik S, Zadrozny PA (1993) Asymptotic distributions of impulse responses, step responses and variance decompositions of estimated linear dynamic models. Econometrica 61:857–870

Moon HR, Schorfheide F (2012) Bayesian and frequentist inference in partially identified models. Econometrica 80:755–782

Moon HR, Schorfheide F, Granziera E (2013) Inference for VARs identified with sign restrictions, working paper

Mountford A, Uhlig H (2009) What are the effects of fiscal policy shocks? J Appl Econ 24:960–992

Muth J (1960) Optimal properties of exponentially weighted forecasts. J Am Stat Assoc 55:299–306

Naylor AW, Sell GR (1982) Linear operator theory in engeneering and science, applied mathematical sciences, vol 40. Springer, New York

Negro Md, Primiceri GE (2015) Time varying structural vector autoregressions and monetary polic: A corrigendum. Rev Econ Stud, forthcoming

Negro Md, Schorfheide F (2004) Priors from general equilibrium models for VARs. Int Econ Rev 45:643–673

Nelson CR, Plosser CI (1982) Trends and random walks in macro-economic time series: Some evidence and implications. J Monet Econ 10:139–162

Nelson DB (1990) Stationarity and persistence in the GARCH(1,1) model. Econ Theory 6:318–334

Nelson DB (1991) Conditional heteroskedasticity in asset returns: A new approach. Econometrica 59:347–370

Neusser K (1991) Testing the long-run implications of the neoclassical growth model. J Monet Econ 27:3–37

Neusser K (2000) An algebraic interpretation of cointegration. Econ Lett 67:273–281

Neusser K (2009) Difference equations for economists. http://​www.​neusser.​ch/​downloads/​DifferenceEquati​ons.​pdf

Neusser K (2016) A topological view on the identification of structural vector autoregressions. Econ. Lett. (forthcoming)

Neusser K, Kugler M (1998) Manufacturing growth and financial development: Evidence from oecd countries. Rev Econ Stat 80:638–646

Newey WK, West KD (1994) Automatic lag selection in covariance matrix estimation. Rev Econ Stud 61:631–653

Ng S, Perron P (1995) Unit root tests in ARMA models with data dependent methods for the selection of the truncation lag. J Am Stat Assoc 90:268–281

Nicholls DF, Pagan AR (1984) Estimating predictions, prediction errors and their standard deviations using constructed variables. J Econ 24:293–310

Norris JR (1998) Markov chains. Cambridge series in statistical and probabilistic mathematics. Cambridge University Press, Cambridge

Ogaki M (1992) An introduction to the generalized method of moments. Working Paper No. 314, University of Rochester

Orcutt GH, Winokur HS Jr (1969) First order autoregression inference, estimation, and prediction. Econometrica 37:1–14

Osterwald-Lenum M (1992) A note with quantiles of the asymptotic distribution of the maximum likelihood cointegration rank test statistics. Oxford Bull Econ Stat 54:461–471

Pagan AR (1984) Econometric issues in the analysis of regressions with generated regressors. Int Econ Rev 25:183–209

Pagan AR, Robertson OC (1998) Structural models of the liquidity effect. Rev Econ Stat 80:202–217

Perron P (1989) The great crash, the oil price shock, and the unit root hypothesis. Econometrica 57:1361–1401

Perron P (2006) Dealing with structural breaks. In: Hassani H, Mills TC, Patterson K (eds) Palgrave handbook in econometrics, vol 1. Econometric theory. Palgrave Macmillan, Hampshire, pp 278–352

Phillips PC (1987) Time series regression with a unit root. Econometrica 55:277–301

Phillips PCB (1986) Understanding spurious regressions in econometrics. J Econ 33:311–340

Phillips PCB (1991) Optimal inference in cointegrating systems. Econometrica 59:283–306

Phillips PCB (2004) HAC estimation by automated regression. Cowles Foundation Discussion Paper 1470, Yale University

Phillips PCB, Hansen BE (1990) Statistical inference in instrumental variables regresssion with I(1) processes. Rev Econ Stud 57:99–125

Phillips PCB, Ouliaris S (1990) Asymptotic properties of residual based tests of cointegration. Econometrica 58(1):165–193

Phillips PCB, Perron P (1988) Testing for a unit root in time series regression. Biometrika 75:335–346

Phillips PCB, Solo V (1992) Asymptotics for linear processes. Ann Stat 20:971–1001

Phillips PCB, Sul D (2007) Some empirics on economic growth under heterogeneous technology. J Macroecon 29:455–469

Pierce DA, Haugh LD (1977) Causality in temporal systems - characterization and survey. J Econ 5:265–293

Potter SM (2000) Nonlinear impulse response functions. J Econ Dyn Control 24:1425–1446

Press H, Tukey JW (1956) Power spetral methods of analysis and their application to problems in airplane dynamics. In: Flight Test Manual, NATO Advisory Group for Aeronautical Research and Development, pp 1–41

Priestley MB (1981) Spectral analysis and time series, vol 1&2. Academic Press, London

Primiceri GE (2005) Time varying structural vector autoregressions and monetary policy. Rev Econ Stud 72:821–852

Quah D (1990) Permanent and transitory movements in labor income: An explanation for ‘excess smoothnes’ in consumption. J Polit Econ 98:449–475

Quah D, Sargent TJ (1993) A dynamic index model for large cross sections. In: Stock JH, Watson MW (eds) Business cycles, indicators, and forecasting, Chapter 7. University of Chicago Press, Chicago

Quandt RE (1960) Tests of the hypothesis that a linear regression system obeys two separate regimes. J Am Stat Assoc 55:324–330

Reichlin L (2003) Factor models in large cross sections of time series. In: Dewatripont M, Hansen LP, JTurnovsky S (eds) Advances in econometrics, theory and applications, econometric society monographs, vol III. Cambridge University Press, Cambridge, pp 47-86

Reinsel GC (1993) Elements of multivariate time series analysis. Springer Series in statistics. Springer, New York

Rigobon R (2003) Identification through heteroskedasticity. Rev Econ Stat 85:777–792

Robinson EA (1982) A historical perspective of spectrum estimation. Proc IEEE 70:885–907

Rosenblatt M (2000) Gaussian and Non-Gaussian linear time series and random fields. Springer, New York

Rothenberg TJ (1971) Identification of parametric models. Econometrica 39:577–591

Rubio-Ramírez JF, Waggoner DF, Zha T (2010) Structural vector autoregressions: Theory of identification and algorithms for inference. Rev Econ Stud 77:665–696

Rudin W (1976) Principles of mathematical analysis, 3rd edn. McGraw-Hill, New York

Rudin W (1987) Real and complex analysis, 3rd edn. McGraw-Hill, Boston

Runkle D (1987) Vector autoregressions and reality. J Bus Econ Stat 5:437–442

Said SE, Dickey DA (1984) Testing for unit roots in autoregressive-moving average models of unknown order. Biometrika 71:599–607

Samuelson PA (1947) Foundations of economic analysis. Harvard University Press, Cambridge

Samuelson PA (1965) Proof that properly anticipated prices fluctuate randomly. Ind Manage Rev 6:41–49

Sargent TJ (1987) Macroeconomic theory, 2nd edn. Academic Press, Orlando, FL

Sargent TJ (1989) Two models of measurements and the investment accelerator. J Polit Econ 97:251–87

Sargent TJ (2004) Recursive macroeconomic theory, 2nd edn. MIT Press, Cambridge

Sargent TJ, Sims CA (1977) Business cycle modelling without pretending to have too much a priori economic theory. In: Sims CA (ed) New Methods in business cycle research. Federal Reserve Bank of Minneapolis, Minneapolis, pp 45–109

Schorfheide F (2005) VAR forecasting under misspecification. J Econ 128:99–136

Serfling RJ (1980) Approximation theorems of mathematical statistics. Wiley, New York

Shaman P, Stine RA (1988) The bias of autoregressive coefficient estimators. J Am Stat Assoc 83:842–848

Silverman BW (1986) Density estimation. Chapman and Hall, London

Sims CA (1972) Money, income, and causality. Am Econ Rev 62:540–552

Sims CA (1974) Seasonality in regression. J Am Stat Assoc 69:618–626

Sims CA (1980a) Comparison of interwar and postwar business cycles: Monetarism reconsidered. Am Econ Rev 70(2):250–257

Sims CA (1980b) Macroeconomics and reality. Econometrica 48:1–45

Sims CA (1986) Are forecasting models usable for policy analysis. Federal Reserve Bank of Minneapolis Q Rev 10(1):2–16

Sims CA (1993) Rational expectations modeling with seasonally adjsuted data. J Econ 55:9–19

Sims CA (1999) Error bands for impulse responses. Econometrica 67:1113–1155

Sims CA, Stock JH, Watson MW (1990) Inference in linear time series with some unit roots. Econometrica 58:113–144

Slutzky E (1937) The summation of random causes as the source of cyclic processes. Econometrica 5:105–146

Stock JH (1994) Unit roots, structural breaks and trends. In: Engle RF, McFadden DL (eds) Handbook of econometrics, vol IV. Elsevier Science B.V., Amsterdam, pp 2739–2841

Stock JH, Watson MW (1988a) Testing for common trends. J Am Stat Assoc 83:1097–1107

Stock JH, Watson MW (1988b) Variable trends in economic time series. J Econ Perspect 2(3):147–174

Stock JH, Watson MW (2011) Introduction to econometrics, 3rd edn. Addison Wesley, Longman

Strang G (1988) Linear algebra and its applications, 3rd edn. Harcourt Brace Jovanovich, San Diego

Sul D, Phillips PCB, Choi CY (2005) Prewhitening bias in HAC estimation. Oxford Bull Econ Stat 67:517–546

Tay AS, Wallis KF (2000) Density forecasting: A Survey. J Forecast 19:235–254

Tinbergen J (1939) Statistical testing of business cycle theories. League of Nations, Genf

Tjøstheim D, Paulsen J (1983) Bias of some commonly-used time series estimates. Biometrika 70:389–399; corrigendum (1984), 71:656

Tobin J (1970) Money and income: Post hoc ergo propter hoc? Q J Econ 84:310–317

Uhlig H (2004) Do technology shocks lead to a fall in total hours worked? J Eur Econ Assoc 2:361–371

Uhlig H (2005) What are the effects of monetary policy on output? results from an agnostic identification procedure. J Monet Econ 52:381–419

Uhlig H, Ravn M (2002) On adjusting the HP-filter for the frequency of observations. Rev Econ Stat 84:371–376

Vogelsang TJ (1997) Wald-type tests for detecting breaks in the trend function of a dynamic time series. Econ Theory 13:818–849

Watson MW (1994) Vector autoregressions and cointegration. Handbook of econometrics, vol 4, chap 47. North-Holland, Amsterdam, pp 2843–2915

Weiss AA (1986) Asymptotic theory for ARCH models: Estimation and testing. Econ Theory 2:107–131

White H (1980) A heteroskedasticity consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica 48:817–838

Whittaker ET (1923) On a new method of graduation. Proc Edinbrough Math Soc 41:63–75

Wiener N (1956) The theory of prediction. In: Beckenbach EF (ed) Modern mathematics for engineers. McGraw-Hill, New York, Series 1

Woodford M (2003) Interest and prices: foundations of a theory of monetary policy. Princeton University Press, Princeton

Wu CFJ (1983) On the convergence of the EM algorithm. Ann Stat 11:95–103

Yoo BS (1987) Multi-cointegrated time series and a generalized error correction model. Ph.D., University of California, San Diego

Yule GU (1926) Why do we sometimes get nonsense correlations between time series? A study in sampling and the nature of time series. J R Stat Soc 89:1–64

Yule GU (1927) On a method of investigating periodicities in disturbed series, with special reference to Wolfer’ sunspot numbers. Philos Trans R Soc A 226:267–298

Zadrozny PA (2005) Necessary and sufficient restrictions for existence of a unique fourth moment of a univariate GARCH(p,q) process, cESifo Working Paper No.1505

Zakoïan JM (1994) Threshold heteroskedastic models. J Econ Dyn Control 18:931–955

Zellner A (1979) Causality and econometrics. In: Brunner K, Meltzer A (eds) Three aspects of policymaking: knowledge, data and institution, Carnegie-Rochester conference series on public policy. North-Holland, Amsterdam, pp 9–54

Zellner A, Palm F (1974) Time series analysis and simultaneous equation econometric models. J Econ 2:17–54

Zivot E, Andrews DW (1992) Further evidence on the great crash, the oil-price shock, and the unit-root hypothesis. J Bus Econ Stat 10:251–270

Index

A

ACF

See alsoAutocorrelation function

ADF-test

AIC

SeeInformation criterion, see Information criterion

AR process

autocorrelation function

autocovariance function

stationary solution

ARIMA process

ARMA model

estimation

identification

ARMA process

See alsoAutoregressive moving-average process

autocovariance function

causality

causality condition

estimation

invertibility

invertibility condition

maximum likelihood estimation

state space representation

Autocorrelation function

confidence interval

MA(q) process

AR(1) process

estimation

asymptotic distribution

Bartlett’s formula

confidence interval

interpretation

order

properties

random walk

univariate

Autocorrelation function, partial

AR process

estimation

interpretation

MA process

Autocovariance function

ARMA process

estimation

linear process

MA(1) process

multivariate

order

properties

random walk

univariate

Autoregressive conditional heteroskedasticity models

SeeVolatility

Autoregressive final form

Autoregressive moving-average process

Autoregressive moving-average prozess mean

B

Back-shift operator

See alsoLag operator

Bandwidth

Bartlett’s formula

Basic structural model

cylical component

local linear trend model

seasonal component

Bayesian VAR

Beveridge-Nelson decomposition

Bias proportion

Bias, small sample

correction

BIC

SeeInformation criterion, see Information criterion

Borel-Cantelli lemma

Box-Pierce statistic

BSM

SeeBasic structural model

C

Canonical correlation coefficients

Cauchy-Bunyakovskii-Schwarz inequality

Causal representation

Causality

See alsoWiener-Granger causality

Wiener-Granger causality

Central Limit Theorem m-dependence

Characteristic function

Chebyschev’s inequality

Chow test

Cointegration

Beveridge-Nelson decomposition,

bivariate

common trend representation

definition

fully-modified OLS

Granger’s representation theorem

normalization

order of integration

shocks, permanent and transitory

Smith-McMillan factorization

test

Johansen test

regression test

triangular representation

VAR model

assumptions

VECM

vector error correction

Wald test

Companion form

Convergence

Almost sure convergence

Convergence in r-th mean

Convergence in distribution

Convergence in probability

Correlation function

estimator

multivariate

Covariance function

estimator

properties

covariance function

Covariance proportion

Covariance, long-run

Cross-correlation

distribution, asymptotic

Cyclical component

D

Dickey-Fuller distribution

Durbin-Levinson algorithm

Dynamic factor model

Dynamic multiplier

SeeShocks, transitory

E

EM algorithm

Ergodicity

Estimation

ARMA model

order

Estimator

maximum likelihood estimator

method of moments GARCH(1,1) model

moment estimator

OLS estimator

process, integrated

Yule-Walker estimator

Example

AD-curve and Money Supply

advertisement and sales

ARMA processes

cointegration

fully-modified OLS

Johansen approach

consumption expenditure and

advertisement

demand and supply shocks

estimation of long-run variance

estimation of quarterly GDP

GDP and consumer sentiment index

growth model, neoclassical

inflation and short-term interest rate

IS-LM model with Phillips curve

modeling real GDP of Switzerland

present discounted value model

structural breaks

Swiss Market Index

term structure of interest rate

unit root test

Expectation, adaptive

Exponential smoothing

F

Factor model, dynamic

SeeDynamic factor model

FEVD

See alsoForecast error variance Decomposition

Filter

gain function

Gibbs phenomenon

high-pass

Hodrick-Prescott filter

HP-filter

Kuznets filter

low-pass

phase function

TRAMO-SEATS

transfer function

X-11 filter

X-12-Filter

Filter, time invariant

Filtering problem

Final form

SeeAutoregressive final form

FMOLS estimator

Wald test

Forecast error variance decomposition

Forecast evaluation

Bias proportion

Covariance proportion

Mean-absolute-error

Out-of-sample strategy

Root-mean-squared-error

Uncertainty

Variance proportion

Forecast function

AR(p) process

ARMA(1,1) process

forecast error

infinite past

linear

MA(q) process

variance of forecast error

Forecast, direct

Forecast, iterated

Fourier frequencies

Fourier transform, discrete

FPE

SeeInformation criterion

Frequency domain

Fully-modified ordinary least-squares

G

Gain function

Gauss Markov theorem

Growth component

H

HAC variance

See alsovariance,

heteroskedastic and autocorrelation consistent

Harmonic process

Hodrick-Prescott filter

HQC

SeeInformation criterion, see Information criterion

I

Identification

Box-Jenkins

Kalman filter

Identification problem

Impulse response function

Information criterion

AIC

BIC

Final prediction error

FPE

Hannan-Quinn

HQC

Schwarz

Innovation algorithm

Innovations

Integrated GARCH

Integrated process

Integrated regressors

rules of thumb

Integration, order of

Intercept correction

Invertibility

J

Johansen test

distribution, asymptotic

hypothesis tests over β

max test

specification of deterministic part

trace test

K

Kalman filter

application

basic structural model

estimation of quarterly GDP

AR(1) process

assumptions

causal

EM algorithm

filtering problem

forecasting step

gain matrix

identification

initialization

likelihood function

Markov property

measurement errors

observation equation

prediction problem

smoother

smoothing problem

stable

state equation

stationarity

updating step

Kalman smoother

Kernel function

bandwidth

optimal

rule of thumb

Bartlett

boxcar

Daniell

lag truncation parameter

optimal

quadratic spectral

Tukey-Hanning

L

Lag operator

calculation rules

definition

polynomial

Lag polynomial

Lag truncation parameter

Lag window

Lead operator

Leading indicator

Least-squares estimator

Likelihood function

ARMA process

Kalman filter

regime switching model

Ljung-Box statistic

Loading matrix

definition

Local linear trend model

Long-run identification

instrumental variables

M

m-dependence

MA process

autocorrelation function

autocovariance function

MAE

Markov chain

ergodic distribution

regular

Matrix norm

absolute summability

quadratic summability

submultiplicativity

Max share identification

See alsoVAR process, VAR process

Maximum likelihood estimator

ARMA(p,q) model

asymptotic distribution

AR process

ARMA(1,1) process

MA process

GARCH(p,q) model

Maximum likelihood method

Mean

asymptotic distribution

distribution, asymptotic

estimation

estimator

Mean reverting

Mean squared error matrix

estimated coefficients

known coefficients

Measurement errors

Median–target method

Memory, short

Minnesota prior

Missing observations

Mixture distributions

Model

N

Normal distribution, multivariate conditional

Normal equations

O

Observation equation

Observationally equivalent

OLS estimator

distribution, asymptotic

Order of integration

Ordinary-least-squares estimator

Oscillation length

Overfitting

P

PACF

See alsoAutocorrelation function, Partial

Partial autocorrelation function

computation

estimation

Particle filter

Penalty function approach

Period length

Periodogram

Perpetuity

Persistence

Phase function

Portmanteau test

PP-test

Prediction problem

Predictor

See alsoForecast function

Present discounted value model

Beveridge-Nelson decomposition

cointegration

spread

VAR representation

vector error correction model

Prewhitening

Process, ARIMA

Process, stochastic

ARMA process

branching process

deterministic

difference-stationary

finite memory

finite-range dependence

Gaussian process

harmonic process

integrated

Beveridge-Nelson decomposition,

forecast, long-run

impulse response function

OLS estimator

persistence

variance of forecast error

linear

linearly regular

memory

moving-average process

multivariate

purely non-deterministic

random walk

random walk with drift

singular

spectral representation

trend-stationary

forecast, long-run

impulse response function

variance of forecast error

white noise

R

Random walk

autocorrelation function

autocovariance function

Random walk with drift

Real business cycle model

Realization

Regime switching model

maximum likelihood estimation

Restrictions

long-run

short-run

sign restrictions

RMSE

S

Seasonal component

Set identified

Shocks

fundamental

permanent

structural

transitory

Short range dependence

See alsoMemory, short

Signal-to-noise ratio

Singular values

Smoothing

Smoothing problem

Smoothing, exponential

Spectral average estimator, discrete

Spectral decomposition

Spectral density

ARMA process

autocovariance function

estimator, direct

estimator, indirect

Fourier coefficients

spectral density, rational

variance, long-run

Spectral distribution function

Spectral representation

Spectral weighting function

Spectral window

Spectrum estimation

Spurious correlation

Spurious regression

State equation

State space

State space representation

ARMA processes

ARMA(1,1)

missing observations

stationarity

time-varying coefficients

Cooley-Prescott

Harvey-Phillips

Hildreth-Houck

VAR process

Stationarity

multivariate

strict

weak

Stationarity, strict

multivariate

Strong Law of Large Numbers

Structural breaks

Chow test

dating of breaks

tests

Structural change

Structural time series analysis

Structural time series model

basic structural model

Summability

absolute

quadratic

Summability Condition

Superconsistency

Swiss Market Index (SMI)

T

Test

autocorrelation, squared residuals

cointegration

regression test

Dickey-Fuller regression

Dickey-Fuller test

augmented

correction, autoregressive

heteroskedasticity

Engle’s Lagrange-multiplier test

Independence

Johansen test

correlation coefficients, canonical

distribution, asymptotic

eigenvalue problem

hypotheses

hypothesis tests over β

likelihood function

max test

singular values

trace test

Kwiatkowski-Phillips-Schmidt-Shin-test

Phillips-Perron test

stationarity

uncorrelatedness

unit root test

structural breaks

testing strategy

unit-root test

white noise

Box-Pierce statistic

Ljung-Box statistic

Portmanteau test

Time

Time domain

Time series model

Time-varying coefficients

Minnesota prior

regime switching model

Times series analysis, structural

Trajectory

Transfer function

Transfer function form

Transition probability matrix

U

Underfitting

V

Value-at-Risk

VaR

SeeValue-at-Risk

VAR process

Bayesian VAR

correlation function

covariance function

estimation

order of VAR

Yule-Walker estimator

forecast error variance decomposition

forecast function

mean squared error

form, reduced

form, structural

identification

forecast error variance share maximization

long-run identification

short-run identification

sign restrictions

zero restrictions

identification problem

Cholesky decomposition

impulse response function

bootstrap

confidence intervals

delta method

state space representation

Structural breaks

time-varying coefficients

VAR(1) process

stationarity

variance decomposition

confidence intervals

Variance proportion

Variance, heteroskedastic and autocorrelation consistent

Variance, long-run

estimation

prewhitening

multivariate

spectral density

VARMA process

causal representation

condition for causal representation

VECM

See alsoCointegration

Vector autoregressive moving-average process

See alsoVARMA process

Vector autoregressive process

See alsoVAR process

Volatility

ARCH(p) model

ARCH-in-mean model

EGARCH model

Forecasting

GARCH(1,1) model

GARCH(p,q) model

ARMA process

heavy-tail property

GARCH(p,q) model, asymmetric

heavy-tail property

IGARCH

models

TARCH(p,q) model

time-varying

Wishart autoregressive process

W

Weighting function

White noise

multivariate

univariate

Wiener-Granger causality

test

F-test

Haugh-Pierce test

Wishart autoregressive process

Wold Decomposition Theorem

multivariate

univariate

Y

Yule-Walker equations

multivariate

univariate

Yule-Walker estimator

AR(1) process

asymptotic distribution

MA process

Footnotes

1

Substraction and division can be defined accordingly:

2

A more detailed introduction of complex numbers can be found in Rudin ( 1976 ) or any other mathematics textbook.

3

The notation with “ − ϕ j z j ” instead of “ ϕ j z j ” was chosen to conform to the notation of AR-models.

4

For more detailed presentations see Agarwal ( 2000 ), Elaydi ( 2005 ) or Neusser ( 2009 ).

5

Sometimes one can find $$z^{p} -\phi _{1}z^{p-1} -\ldots -\phi _{p} = 0$$ as the characteristic equation. The roots are of the two characteristic equations are then reciprocal to each other.