Springer Texts in Business and Economics

More information about this series at http://​www.​springer.​com/​series/​10099

Uwe Hassler

Stochastic Processes and CalculusAn Elementary Introduction with Applications1st ed. 2016

Uwe Hassler

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

ISSN 2192-4333e-ISSN 2192-4341

ISBN 978-3-319-23427-4e-ISBN 978-3-319-23428-1

DOI 10.1007/978-3-319-23428-1

Springer Cham Heidelberg New York Dordrecht London

Library of Congress Control Number: 2015957196

© Springer International Publishing Switzerland 2016

Springer Texts in Business and Economics

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made.

Printed on acid-free paper

Springer International Publishing AG Switzerland is part of Springer Science+Business Media (www.springer.com)

I do not know what I may appear to the world, but to myself I seem to have been only like a boy playing on the sea-shore, and diverting myself in now and then finding a smoother pebble or a prettier shell than ordinary, whilst the great ocean of truth lay all undiscovered before me.

ISAAC NEWTON

Quoted from the novel Beyond Sleep by Willem Frederik Hermans

Preface

Over the past decades great importance has been placed on stochastic calculus and processes in mathematics, finance, and econometrics. This book addresses particularly readers from these fields, although students of other subjects as biology, engineering, or physics may find it useful, too.

Scope of the Book

By now there exist a number of books describing stochastic integrals and stochastic calculus in an accessible manner. Such introductory books, however, typically address an audience having previous knowledge about and interest in one of the following three fields exclusively: finance, econometrics, or mathematics. The textbook at hand attempts to provide an introduction into stochastic calculus and processes for students from each of these fields. Obviously, this can on no account be an exhaustive treatment. In the next chapter a survey of the topics covered is given. In particular, the book does neither deal with finance theory nor with statistical methods from the time series econometrician’s toolkit; it rather provides a mathematical background for those readers interested in these fields.

The first part of this book is dedicated to discrete-time processes for modeling temporal dependence in time series. We begin with some basic principles of stochastics enabling us to define stochastic processes as families of random variables in general. We discuss models for short memory (so-called ARMA models), for long memory (fractional integration), and for conditional heteroscedasticity (so-called ARCH models) in respective chapters. One further chapter is concerned with the so-called frequency domain or spectral analysis that is often neglected in introductory books. Here, however, we propose an approach that is not technically too demanding. Throughout, we restrict ourselves to the consideration of stochastic properties and interpretation. The statistical issues of parameter estimation, testing, and model specification are not addressed due to space limitations; instead, we refer to, e.g., Mills and Markellos (2008), Kirchgässner, Wolters, and Hassler (2013), or Tsay (2005).

The second part contains an introduction to stochastic integration. We start with elaborations on the Wiener process W ( t ) as we will define (almost) all integrals in terms of Wiener processes. In one chapter we consider Riemann integrals of the form $$\int f(t)W(t)dt$$ , where f is a deterministic function. In another chapter Stieltjes integrals are constructed as $$\int f(t)dW(t)$$ . More specifically, stochastic integrals as such result when a stochastic process is integrated with respect to the Wiener process, e.g., the Ito integral $$\int W(t)dW(t)$$ . Solving stochastic differential equations is one task of stochastic integration for which we will need to use Ito’s lemma. Our description aims at a similar compromise between concreteness and mathematical rigor as, e.g., Mikosch (1998). If the reader wants to address this matter more rigorously, we recommend Klebaner (2005) or Øksendal (2003).

The third part of the book applies previous results. The chapter on stochastic differential equations consists basically of applications of Ito’s lemma. Concrete differential equations, as they are used, e.g., when modeling interest rate dynamics, will be covered in a separate chapter. The second area of application concerns certain limiting distributions of time series econometrics. A separate chapter on the asymptotics of integrated processes covers weak convergence to Wiener processes. The final two chapters contain applications for nonstationary processes without cointegration on the one hand and for the analysis of cointegrated processes on the other. Further details regarding econometric application can be found in the books by Banerjee, Dolado, Galbraith and Hendry (1993), Hamilton (1994), or Tanaka (1996).

The exposition in this book is elementary in the sense that knowledge of measure theory is neither assumed nor used. Consequently, mathematical foundations cannot be treated rigorously which is why, e.g., proofs of existence are omitted. Rather I had two goals in mind when writing this book. On the one hand, I wanted to give a basic and illustrative presentation of the relevant topics without many “troublesome” derivations. On the other hand, in many parts a technically advanced level has been aimed at: procedures are not only presented in form of recipes but are to be understood as far as possible which means they are to be proven. In order to meet both requirements jointly, this book is equipped with a lot of challenging problems at the end of each chapter as well as with the corresponding detailed solutions. Thus the virtual text – augmented with more than 60 basic examples and 45 illustrative figures – is rather easy to read while a part of the technical arguments is transferred to the exercise problems and their solutions. This is why there are at least two possible ways to work with the book. For those who are merely interested in applying the methods introduced, the reading of the text is sufficient. However, for an in-depth knowledge of the theory and its application, the reader necessarily needs to study the problems and their solution extensively.

Note to Students and Instructors

I have taught the material collected here to master students (and diploma students in the old days) of economics and finance or students of mathematics with a minor in those fields. From my personal experience I may say that the material presented here is too vast to be treated in a course comprising 45 contact hours. I used the textbook at hand for four slightly differing courses corresponding to four slightly differing routes through the parts of the book. Each of these routes consists of three stages: time series models, stochastic integration, and applications. After Part I on time series modeling, the different routes separate.

The finance route: When teaching an audience with an exclusive interest in finance, one may simply drop the final three chapters. The second stage of the course then consists of Chaps.  7 , 8, 9, 10, and 11. This Part II on stochastic integration is finally applied to the solution of stochastic differential equations and interest rate modeling in Chaps.  12 and 13, respectively.

The mathematics route: There is a slight variant of the finance route for the mathematically inclined audience with an equal interest in finance or econometrics. One simply replaces Chap.  13 on interest rate modeling by Chap.  14 on weak convergence on function spaces, which is relevant for modern time series asymptotics.

The econometrics route: After Part I on time series modeling, the students from a class on time series econometrics should be exposed to Chaps.  7 , 8, 9, and 10 on Wiener processes and stochastic integrals. The three chapters (Chaps.  11 , 12, and 13) on Ito’s lemma and its applications may be skipped to conclude the course with the last three chapters (Chaps.  14 , 15, and 16) culminating in the topic of “cointegration.”

The nontechnical route: Finally, the entire content of the textbook at hand can still be covered in one single semester; however, this comes with the cost of omitting technical aspects for the most part. Each chapter contains a rather technical section which in principle can be skipped without leading to a loss in understanding. When omitting these potentially difficult sections, it is possible to go through all the chapters in a single course. The following sections should be skipped for a less technical route:

It has been mentioned that each chapter concludes with problems and solutions. Some of them are clearly too hard or lengthy to be dealt with in exams, while others are questions from former exams of my own or are representative of problems to be solved in my exams.

References

Banerjee, A., Dolado, J. J., Galbraith, J. W., & Hendry, D. F. (1993). Co-integration, error correction, and the econometric analysis of non-stationary data . Oxford/New York: Oxford University Press.

Hamilton, J. (1994). Time series analysis . Princeton: Princeton University Press.

Kirchgässner, G., Wolters, J., & Hassler, U. (2013). Introduction to modern time series analysis (2nd ed.). Berlin/New York: Springer.

Klebaner, F. C. (2005). Introduction to stochastic calculus with applications (2nd ed.). London: Imperical College Press.

Mikosch, Th. (1998). Elementary stochastic calculus with finance in view . Singapore: World Scientific Publishing.

Mills, T. C., & Markellos, R. N. (2008). The econometric modelling of financial time series (3rd ed.). Cambridge/New York: Cambridge University Press.

Øksendal, B. (2003). Stochastic differential equations: An introduction with applications (6th ed.). Berlin/New York: Springer.

Tanaka, K. (1996). Time series analysis: Nonstationary and noninvertible distribution theory . New York: Wiley.

Tsay, R. S. (2005). Analysis of financial time series (2nd ed.). New York: Wiley.

Uwe Hassler

Frankfurt, Germany

July 2015

Acknowledgments

This textbook grew out of lecture notes from which I taught over 15 years. Without my students’ thirst for knowledge and their critique, I would not even have started the project. In particular, I thank Balázs Cserna, Matei Demetrescu, Eduard Dubin, Mehdi Hosseinkouchack, Vladimir Kuzin, Maya Olivares, Marc Pohle, Adina Tarcolea, and Mu-Chun Wang who corrected numerous errors in the manuscript. Originally, large parts of this text had been written in German, and I thank Verena Werkmann for her help when translating into English. Last but not least I am indebted to Goethe University Frankfurt for allowing me to take sabbatical leave. Without this support I would not have been able to complete this book at a time when academics are under pressure to publish in the first place primary research.

Contents

1 Introduction 1

1.​1 Summary 1

1.​2 Finance 1

1.​3 Econometrics 3

1.​4 Mathematics 6

1.​5 Problems and Solutions 7

References 10

Part I Time Series Modeling

2 Basic Concepts from Probability Theory 13

2.​1 Summary 13

2.​2 Random Variables 13

2.​3 Joint and Conditional Distributions 22

2.​4 Stochastic Processes (SP) 29

2.​5 Problems and Solutions 35

References 42

3 Autoregressive Moving Average Processes (ARMA) 45

3.​1 Summary 45

3.​2 Moving Average Processes 45

3.​3 Lag Polynomials and Invertibility 51

3.​4 Autoregressive and Mixed Processes 56

3.​5 Problems and Solutions 68

References 75

4 Spectra of Stationary Processes 77

4.​1 Summary 77

4.​2 Definition and Interpretation 77

4.​3 Filtered Processes 84

4.​4 Examples of ARMA Spectra 89

4.​5 Problems and Solutions 95

References 101

5 Long Memory and Fractional Integration 103

5.​1 Summary 103

5.​2 Persistence and Long Memory 103

5.​3 Fractionally Integrated Noise 108

5.​4 Generalizations 113

5.​5 Problems and Solutions 118

References 125

6 Processes with Autoregressive Conditional Heteroskedastici​ty (ARCH) 127

6.​1 Summary 127

6.​2 Time-Dependent Heteroskedastici​ty 127

6.​3 ARCH Models 130

6.​4 Generalizations 135

6.​5 Problems and Solutions 142

References 148

Part II Stochastic Integrals

7 Wiener Processes (WP) 151

7.​1 Summary 151

7.​2 From Random Walk to Wiener Process 151

7.​3 Properties 157

7.​4 Functions of Wiener Processes 161

7.​5 Problems and Solutions 170

References 177

8 Riemann Integrals 179

8.​1 Summary 179

8.​2 Definition and Fubini’s Theorem 179

8.​3 Riemann Integration of Wiener Processes 183

8.​4 Convergence in Mean Square 186

8.​5 Problems and Solutions 190

References 197

9 Stieltjes Integrals 199

9.​1 Summary 199

9.​2 Definition and Partial Integration 199

9.​3 Gaussian Distribution and Autocovariances 202

9.​4 Standard Ornstein-Uhlenbeck Process 204

9.​5 Problems and Solutions 207

Reference 211

10 Ito Integrals 213

10.​1 Summary 213

10.​2 A Special Case 213

10.​3 General Ito Integrals 218

10.​4 (Quadratic) Variation 222

10.​5 Problems and Solutions 229

References 237

11 Ito’s Lemma 239

11.​1 Summary 239

11.​2 The Univariate Case 239

11.​3 Bivariate Diffusions with One WP 245

11.​4 Generalization for Independent WP 250

11.​5 Problems and Solutions 254

Reference 258

Part III Applications

12 Stochastic Differential Equations (SDE) 261

12.​1 Summary 261

12.​2 Definition and Existence 261

12.​3 Linear Stochastic Differential Equations 265

12.​4 Numerical Solutions 272

12.​5 Problems and Solutions 273

References 282

13 Interest Rate Models 285

13.​1 Summary 285

13.​2 Ornstein-Uhlenbeck Process (OUP) 285

13.​3 Positive Linear Interest Rate Models 288

13.​4 Nonlinear Models 292

13.​5 Problems and Solutions 296

References 302

14 Asymptotics of Integrated Processes 303

14.​1 Summary 303

14.​2 Limiting Distributions of Integrated Processes 303

14.​3 Weak Convergence of Functions 310

14.​4 Multivariate Limit Theory 317

14.​5 Problems and Solutions 321

References 329

15 Trends, Integration Tests and Nonsense Regressions 331

15.​1 Summary 331

15.​2 Trend Regressions 331

15.​3 Integration Tests 336

15.​4 Nonsense Regression 341

15.​5 Problems and Solutions 344

References 352

16 Cointegration Analysis 353

16.​1 Summary 353

16.​2 Error-Correction and Cointegration 353

16.​3 Cointegration Regressions 358

16.​4 Cointegration Testing 365

16.​5 Problems and Solutions 373

References 381

References383

Index389

List of Figures

Fig. 3.1 Simulated MA(1) processes47

Fig. 3.2 Simulated AR(1) processes59

Fig. 3.3 Stationarity triangle for AR(2) processes62

Fig. 3.4 Autocorrelograms for AR(2) processes63

Fig. 3.5 Autocorrelograms for ARMA(1,1) processes67

Fig. 4.1 Cosine cycle with different frequencies79

Fig. 4.2 Spectra of MA( S ) Processes 83

Fig. 4.3 Business Cycle84

Fig. 4.4 AR(1) spectra (2 π   f ( λ )) with positive autocorrelation 91

Fig. 4.5 AR(1) spectra91

Fig. 4.6 AR(2) spectra93

Fig. 4.7 ARMA(1,1) spectra94

Fig. 4.8 Spectra of multiplicative seasonal AR processes95

Fig. 5.1 Hyperbolic decay.105

Fig. 5.2 Exponential decay106

Fig. 5.3 Autocorrelogram of fractional noise110

Fig. 5.4 Spectrum of fractional noise113

Fig. 5.5 Simulated fractional noise114

Fig. 5.6 Nonstationary fractional noise118

Fig. 6.1 ARCH(1) with α ₀  = 1 and α ₁  = 0. 5 133

Fig. 6.2 ARCH(1) with α ₀  = 1 and α ₁  = 0. 9 133

Fig. 6.3 GARCH(1,1) with α ₀  = 1, α ₁  = 0. 3 and β ₁  = 0. 3 137

Fig. 6.4 GARCH(1,1) with α ₀  = 1, α ₁  = 0. 3 and β ₁  = 0. 5 138

Fig. 6.5 IGARCH(1,1)139

Fig. 6.6 GARCH(1,1)-M140

Fig. 6.7 EGARCH(1,1)142

Fig. 7.1 Step function on the interval [0,1]155

Fig. 7.2 Simulated paths of the WP158

Fig. 7.3 WP and Brownian motion162

Fig. 7.4 WP and Brownian motion with drift163

Fig. 7.5 WP and Brownian bridge164

Fig. 7.6 WP and reflected WP along with expectation165

Fig. 7.7 Geometric Brownian motion along with expectation166

Fig. 7.8 WP and geometric Brownian motion167

Fig. 7.9 WP and maximum process along with expectation168

Fig. 7.10 WP and integratedWP169

Fig. 9.1 Standard Ornstein-Uhlenbeck processes207

Fig. 10.1 Sine cycles of different frequencies225

Fig. 13.1 OUP with Starting Values X (0) = μ = 5 288

Fig. 13.2 OUP with Starting Value X (0) = 5.1 including Expected Value Function 289

Fig. 13.3 Interest Rate Dynamics According to Dothan291

Fig. 13.4 Interest Rate Dynamics According to Brennan-Schwartz292

Fig. 13.5 Interest Rate Dynamics According to CKLS294

Fig. 13.6 OUP and CIR295

Fig. 15.1 Linear Time Trend333

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_1

1. Introduction

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

1.1 Summary

Stochastic calculus is used in finance and econom(etr)ics for instance for solving stochastic differential equations and handling stochastic integrals. This requires stochastic processes. Although stemming from a rather recent area of mathematics, the methods of stochastic calculus have shortly come to be widely spread not only in finance and economics. Moreover, these techniques – along with methods of time series modeling – are central in the contemporary econometric tool box. In this introductory chapter some motivating questions are brought up being answered in the course of the book, thus providing a brief survey of the topics treated.

1.2 Finance

The names of two Nobel prize winners¹ dealing with finance are closely connected to one field of applications treated in the textbook at hand. The analysis and the modeling of stock prices and returns is central to this work.

Stock Prices

Let S(t), t ≥ 0, be the continuous stock price of a stock with return $$R(t) = S'(t)/S(t)$$ expressed as growth rate. We assume constant returns,

This differential equation for the stock price is usually also written as follows:

(1.1)

The corresponding solution is (see Problem 1.1)

(1.2)

i.e. if c > 0 the exponential process is explosive. The assumption of a deterministic stock price movement is of course unrealistic which is why a stochastic differential equation consistent with (1.1) is often assumed since Black and Scholes (1973) and Merton (1973),

(1.3)

where dW(t) are the increments of a so-called Wiener process W(t) (also referred to as Brownian motion, cf. Chap. 7). This is a stochastic process, i.e. a random process. Thus, for a fixed point in time t, S(t) is a random variable. How does this random variable behave on average? How do the parameters c and σ affect the expected value and the variance as time passes by? We will find answers to these questions in Chap. 12 on stochastic differential equations.

Interest Rates

Next, r(t) denotes an interest rate for t ≥ 0. Assume it is given by the differential equation

(1.4)

with $$c \in \mathbb{R}$$ or equivalently by

Expression (1.4) can alternatively be written as the following integral equation:

(1.5)

The solution to this reads (see Problem 1.2)

(1.6)

For c < 0 therefore it holds that the interest rate converges to μ as time goes by. Again, a deterministic movement is not realistic. This is why Vasicek (1977) specified a stochastic differential equation consistent with (1.4):

(1.7)

As aforementioned, dW(t) denotes the increments of a Wiener process. How is the interest rate movement (on average) affected by the parameter c? Which kind of stochastic process is described by (1.7)? The answers to these and similar questions will be obtained in Chap. 13 on interest rate models.

Empirical Returns

Looking at return time series one can observe that the variance (or volatility) fluctuates a lot as time passes by. Long quiet market phases characterized by only mild variation are followed by short periods characterized by extreme observations where extreme amplitudes again tend to entail extreme observations. Such a behavior is in conflict with the assumption of normally distributed data. It is an empirically well confirmed law (“stylized fact”) that financial market data in general and returns in particular produce “outliers” with larger probability than it would be expected under normality.

It is crucial, however, that extreme observations occur in clusters (volatility clusters). Even though returns are not correlated over time in efficient markets, they are not independent as there exists a systematic time dependence of volatility. Engle (1982) suggested the so-called ARCH model (see Chap. 6) in order to capture the outlined effects. His work constituted an entire field of research known nowadays under the keyword “financial econometrics”, and consequently he was awarded the Nobel prize in 2003.²

1.3 Econometrics

Clive Granger (1934–2009) was a British econometrician who created the concept of cointegration (Granger, 1981). He shared the Nobel prize “for methods of analyzing economic time series with common trends (cointegration)” (official statement of the Nobel Committee) with R.F. Engle. The leading example of trending time series he considered is the random walk.

Random Walks

In econometrics, we are often concerned with time series not fluctuating with constant variance around a fixed level. A widely-used model for accounting for this nonstationarity are so-called integrated processes. They form the basis for the cointegration approach that has become an integral part of common econometric methodology since Engle and Granger (1987). Let’s consider a special case – the random walk – as a preliminary model,

(1.8)

where $$\{\varepsilon _{t}\}$$ is a random process, i.e. $$\varepsilon _{t}$$ and $$\varepsilon _{s}$$, t ≠ s, are uncorrelated or even independent with zero expected value and constant variance σ ². For a random walk with zero starting value x ₀ = 0 it holds by definition that:

(1.9)

The increments can also be written using the difference operator Δ,

Regressing two stochastically independent random walks on each other, a statistically significant relationship is identified which is a statistical artefact and therefore nonsense (see Chap. 15). Two random walks following a common trend, however, are called cointegrated. In this case the regression on each other does not only give the consistent estimation of the true relationship but the estimator is even “superconsistent” (cf. Chap. 16).

Dickey-Fuller Distribution

If one wants to test whether a given time series indeed follows a random walk, then equation (1.9) suggests to estimate the regression

From this, the (ordinary) least squares (LS) estimator under the null hypothesis (1.9), i.e. under a = 1, is obtained as

This constitutes the basic ingredient for the test by Dickey and Fuller (1979). Under the null hypothesis of a random walk (a = 1) it holds asymptotically (n → ∞)

(1.10)

where “$$\stackrel{d}{\rightarrow }$$” stands for convergence in distribution and $$\mathcal{D}\mathcal{F}_{a}$$ denotes the so-called Dickey-Fuller distribution. Corresponding modes of convergence will be explained in Chap. 14. Since Phillips (1987) an elegant way for expressing the Dickey-Fuller distribution by stochastic integrals is known (again, W(t) denotes a Wiener process):

(1.11)

Note (and enjoy!) the formal correspondence of the sum of squares $$\sum _{t=1}^{n}x_{t-1}^{2}$$ in the denominator of $$\hat{a} - 1$$ and the integral over the squared Wiener process in the denominator of (1.11), $$\int _{0}^{1}W^{2}(t)\,dt$$ (this is a Riemann integral, cf. Chap. 8). Just as well the sum $$\sum _{t=1}^{n}x_{t-1}\,\varepsilon _{t} =\sum _{ t=1}^{n}x_{t-1}\,\varDelta x_{t}$$ resembles the so-called Ito integral $$\int _{0}^{1}W(t)\,dW(t)$$. But how are these integrals defined, what are they about? How is this distribution (and similar ones) attained? And why does there exist another equivalent representation,

(1.12)

of the Dickey-Fuller distribution? We concern ourselves with these questions in connection with Ito’s lemma in Chap. 11.

Autocorrelation

The assumption of the increments $$\varDelta x_{t} = x_{t} - x_{t-1}$$ of economic times series being free from serial (temporal) correlation – as it is true for the random walk – is too restrictive in practice. Thus, we have to learn how the Dickey-Fuller distribution is generalized with autocorrelated (i.e. serially correlated) increments. In practise, so-called ARMA models are used most frequently in order to model autocorrelation. This class of models will be discussed intuitively as well as rigorously in Chap. 3 The so-called spectral analysis translates autocorrelation patterns in oscillation patterns. In Chap. 4 we learn to determine which frequency’s or period’s oscillations add particularly intensely to a time series’ variation. Often economists are refused access to spectral analysis because of the extensive use of complex numbers. Therefore, we suggest an approach that avoids complex numbers. Finally, Chap. 5 introduces a model where the temporal dependence is particularly persistent such that the autocorrelations die out more slowly than in the ARMA case. Such a feature has been called “long memory” and is observed with many economic and financial series.

1.4 Mathematics

Stochastic calculus, which will be applied here, is a rather recent area in mathematics. It was pioneered by Kiyoshi Ito³ in a sequence of pathbreaking papers published in Japanese starting from the forties of the last century.⁴ The Ito integral as a special case of stochastic integration is introduced in Chap. 10.

Ito Integrals

The aforementioned interest rate model by Vasicek (1977) leads to a stochastic process given by an integral constructed as $$\int _{0}^{t}f(s)\,dW(s)$$ where f is a deterministic function and again dW denotes the increments of a Wiener process. Such integrals – being in a sense classical integrals – will be defined as Stieltjes integrals in Chap. 9 Ito integrals are a generalization of these. At first glance, the deterministic function f is replaced by a stochastic process X, $$\int _{0}^{t}X(s)\,dW(s)$$. Mathematically, this results in a considerably more complicated object, the definition thereof being a problem on its own, cf. Chap. 10.

Ito’s Lemma

At this point, the idea of Ito’s lemma is briefly conveyed. For the moment, assume a deterministic (differentiable) function f(t). Using the chain rule it holds for the derivative of the square f ²:

or rather

(1.13)

Thus, for the ordinary integral it follows

However, among other things, we will learn that the Wiener process is not a differentiable function with respect to time t. The ordinary chain rule does not apply and for the according Ito integral one obtains

(1.14)

This result follows from the famous and fundamental lemma by Ito being a kind of “stochastified chain rule” for Wiener processes in its simplest case. Instead of (1.13) for Wiener processes it holds that

(1.15)

Substantial generalizations and multivariate extensions will be discussed in Chap. 11. In particular, Ito’s lemma will enable us to solve stochastic differential equations in Chap. 12, and it will turn out that S(t) solving (1.3) is a so-called geometric Brownian motion. In Chap. 13 we will look in greater detail in models for interest rates as e.g. given by Eq. (1.7).

Starting point for all the considerations outlined is the Wiener process – often also called Brownian motion. Before turning to it and its properties, general stochastic processes need to be defined and classified beforehand. This is done – among other things – in the following chapter on basic concepts from probability theory.

1.5 Problems and Solutions

Problems

1.1 Solve the differential equation (1.1), i.e. obtain the solution (1.2).

1.2

Verify that r(t) from (1.6) solves the differential equation (1.4).

1.3 Consider a simple regression model,

with OLS estimator $$\hat{\beta }$$. Check that:

with arithmetic mean $$\overline{x}$$.

Hint: $$\sum _{i=1}^{n}\left (x_{i} -\overline{x}\right ) = 0$$.

1.4 Let f n denote n-th power of a function f with derivative f′, $$n \in \mathbb{N}$$. Show that:

Hint: Chain rule as in (1.13).

Solutions

1.1 Using equation (1.1) we get by integration

Since⁵

this implies

or

which is the required solution.

1.2 Taking the derivative of (1.6) yields:

where again the given form of r(t) was used. By purely symbolically multiplying by dt the equation (1.4) is obtained. Hence, the problem is already solved.

1.3 It is well known that the OLS estimator is given by “covariance divided by variance of the regressor”, i.e. it holds that:

Because of $$\sum _{i=1}^{n}(x_{i} -\bar{ x}) = 0$$ this simplifies to

Assuming the model to be correct and substituting $$y_{i} =\alpha +\beta \,x_{i} +\varepsilon _{i}$$, one obtains

Again applying the argument $$\sum _{i=1}^{n}(x_{i} -\bar{ x}) = 0$$ yields

This was exactly the claim.

1.4 We address the problem in a slightly more general way. Let g be a differentiable function with derivative g ′ . By the fundamental theorem of calculus it holds that⁶

or

If g describes a process over time, this last relation can be interpreted the following way: The value at time t is made up by the starting value g(0) plus the sum or integral over all changes occurring between 0 and t. Now, choosing in particular g(t) = f n (t) with

we obtain the required result.

References

Black, F., & Scholes, M. (1973). The pricing of options and corporate liabilities. The Journal of Political Economy, 81, 637–654.CrossRef

Dickey, D. A., & Fuller, W. A. (1979). Distribution of the estimators for autoregressive time series with a unit root. Journal of the American Statistical Association, 74, 427–431.

Engle, R. F. (1982). Autoregressive conditional heteroskedasticity with estimates of the variance of U.K. inflation. Econometrica, 50, 987–1008.

Engle, R. F., & Granger, C. W. J. (1987). Co-integration and error correction: Representation, estimation, and testing. Econometrica, 55, 251–276.CrossRef

Granger, C. W. J. (1981). Some properties of time series data and their use in econometric model specification. Journal of Econometrics, 16, 121–130.CrossRef

Merton, R. C. (1973). Theory of rational option pricing. The Bell Journal of Economics and Management Science, 4, 141–183.CrossRef

Phillips, P. C. B. (1987). Time series regression with a unit root. Econometrica, 55, 277–301.CrossRef

Tanaka, K. (1996). Time series analysis: Nonstationary and noninvertible distribution theory. New York: Wiley.

Trench, W. F. (2013). Introduction to real analysis. Free Hyperlinked Edition 2.04 December 2013. Downloaded on 10th May 2014 from http://​digitalcommons.​trinity.​edu/​mono/​7.

Vasicek, O. (1977). An equilibrium characterization of the term structure. Journal of Financial Economics, 5, 177–188.CrossRef

Footnotes

1

In 1997, R.C. Merton and M.S. Scholes were awarded the Nobel prize jointly, “for a new method to determine the value of derivatives” (according to the official statement of the Nobel Committee).

2

R.F. Engle shared the Nobel prize “for methods of analyzing economic time series with time-varying volatility (ARCH)” (official statement of the Nobel Committee) with C.W.J. Granger.

3

Alternative transcriptions of his name into the Latin alphabet, Itô or Itō, are frequently used in the literature and are equally accepted. In this textbook we follow the spelling of Ito’s compatriot (Tanaka, 1996).

4

In 2006, Ito received the inaugural Gauss Prize for Applied Mathematics by the International Mathematical Union, which is awarded every fourth year since then.

5

By “log” we denote the natural logarithm to the base e.

6

For an introduction to calculus we recommend Trench (2013); this book is available electronically for free as a textbook approved by the American Institute of Mathematics.

Part I
Time Series Modeling

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_2

2. Basic Concepts from Probability Theory

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

2.1 Summary

This chapter reviews some basic material. We collect some elementary concepts and properties in connection with random variables, expected values, multivariate and conditional distributions. Then we define stochastic processes, both discrete and continuous in time, and discuss some fundamental properties. For a successful study of the remainder of this book, the reader is required to be familiar with all of these principles.

2.2 Random Variables

Stochastic processes are defined as families of random variables. This is why related concepts will be recapitulated to facilitate the definition of random variables. Measure theoretical aspects, however, will not be touched.¹

Probability Space

We denote the possible set of outcomes of a random experiment by Ω. Subsets A, A ⊆ Ω, are called events . These events are assigned probabilities to. The probability is a mapping

which fulfills the axioms of probability,

P(A) ≥ 0,

P(Ω) = 1,

$$\mbox{ P}\left (\bigcup \limits _{i}A_{i}\right ) =\sum \limits _{i}\mbox{ P}(A_{i})$$ for $$A_{i} \cap A_{j} =\emptyset$$ with i ≠ j,

where {A i } may be a possibly infinite sequence of pairwise disjoint events. For a well-defined mapping, we do not consider every possible event but in particular only those being contained in σ-algebras. A $$\boldsymbol{\sigma }$$-algebra ² F of Ω is defined as a system of subsets containing

the empty set $$\emptyset$$,

the complement A c of every subset $$A \in \mathcal{F}$$ (this is the set Ω without A, $$A^{c} =\varOmega \setminus A$$),

and the union $$\bigcup \limits _{i}A_{i}$$ of a possibly infinite sequence of elements $$A_{i} \in \mathcal{F}$$.

Of course, a σ-algebra is not unique but can be constructed according to problems of interest. The interrelated triple of set of outcomes, σ-algebra and probability measure, $$\left (\varOmega,\,\mathcal{F},\,\mbox{ P}\right )$$, is also called a probability space .

Example 2.1 (Game of Dice)

Consider a fair hexagonal die with the set of outcomes

where each elementary event $$\{\omega \}\subseteq \varOmega$$ is assigned the same probability to:

When #(A) denotes the number of elements of A ⊆Ω, it holds in the example of the die that

The probability for the occurrence of A hence equals the number of outcomes leading to A divided by the number of possible outcomes. If one is only interested in the event whether an even or an odd number occurs,

then the σ-algebra obviously reads

If one is interested in all possible outcomes without any qualification, then the σ-algebra chosen will be the power set of Ω, $$\mathcal{P}(\varOmega )$$. This is the set of all subsets of Ω:

Systematic counting shows that $$\mathcal{P}(\varOmega )$$ contains exactly $$2^{\#(\varOmega )} = 2^{6} = 64$$ elements. With one and the same probability mapping one obtains for different σ-algebras different probability spaces:

 ■ 

Random Variable

Often not the events themselves are of interest but some values associated with them, that is to say random variables. A real-valued one-dimensional random variable X maps the set of outcomes Ω of the space $$\left (\varOmega,\,\mathcal{F},\,\mbox{ P}\right )$$ to the real numbers:

Again, however, not all such possible mappings can be considered. In particular, a random variable is required to have the property of measurability (more precisely: F-measurability). This implies the following: A subset $$B \subseteq \mathbb{R}$$ defines an event of Ω in such a way that:

This so-called inverse image $$X^{-1}(B) \subseteq \varOmega$$ of B contains exactly the very elements of Ω which are mapped by X to B. Let B be a family of sets consisting of subsets of $$\mathbb{R}$$. Then as measurability it is required from a random variable X that for all $$B \in \mathcal{B}$$ all inverse images are contained in the σ-algebra F: $$X^{-1}(B) \in \mathcal{F}$$. Thereby the probability measure P on F is conveyed to B, i.e. the probability function P x assigning values to X is induced as follows:

Thus, strictly speaking, X does not map from Ω to $$\mathbb{R}$$ but from one probability space to another:

where B now denotes a σ-algebra named after Emile Borel. This Borel algebra B is the smallest σ-algebra over $$\mathbb{R}$$ containing all real intervals. In particular, for $$x \in \mathbb{R}$$ the event X ≤ x has an induced probability leading to the distribution function of X defined as follows:

Example 2.2 (Game of Dice)

Let us continue the example of dice and let us define a random variable X assigning a gain of 50 monetary units to an even number and assigning a loss of 50 monetary units to an odd number,

The random variable X operates on the probability space $$(\varOmega,\mathcal{F}_{1},\mbox{ P})$$ known from Example 2.1. For arbitrary real intervals probabilities Px with $$\mathcal{F}_{1} =\{\emptyset,E,E^{c},\varOmega \}$$ are induced, e.g.:

Let a second random variable Y model the following gain or loss function:

As in this case each outcome leads to another value of the random variable, the probability space chosen is $$(\varOmega,\mathcal{F}_{2},\mbox{ P})$$ with the power set $$\mathcal{F}_{2} = \mathcal{P}(\varOmega )$$ being the σ-algebra. Then we obtain for Y for instance the following probabilities:

For another probability space the mapping Y is possibly not measurable and therefore it cannot be a random variable. E.g. Y is not $$\mathcal{F}_{1}$$-measurable. This is due to the fact that the image Y = 0 has the inverse image $$Y ^{-1}(0) =\{ 5\} \subseteq \varOmega$$ which is not contained in $$\mathcal{F}_{1}$$ as an elementary event: $$\{5\}\notin \mathcal{F}_{1}$$. ■

Continuous Random Variables

For most of all problems in practice we do not explicitly construct a random experiment with probability P in order to derive probabilities P x of a random variable X. Typically we start directly with the quantity of interest X modeling a probability distribution without inducing it. In particular, this is the case for so-called continuous variables. For a continuous random variable every value taken from a real interval is a possible realization. As a continuous random variable can therefore take uncountably many values it is not possible to calculate a probability $$\mbox{ P}(x_{1} <X \leq x_{2})$$ by summing up the individual probabilities. Instead, probabilities are calculated by integrating a probability density. We assume the function f(x) to be continuous (or at least Riemann-integrable) and to be nonnegative for all $$x \in \mathbb{R}$$. Then f is called (probability) density (or density function ) of X if it holds for arbitrary numbers $$x_{1} <x_{2}$$ that

The area beneath the density function therefore measures the probability with which the continuous random variable takes on values of the interval considered. In general, a density is defined by two properties:

1.

f(x) ≥ 0 ,

2.

$$\int _{-\infty }^{+\infty }f(x)\,dx = 1$$ .

Thus, the distribution function F(x) = P(X ≤ x) of a continuous random variable X is calculated as follows:

If there is the danger of a confusion, we sometimes subscript the distribution function, e.g. F x (0) = P(X ≤ 0).

Expected Value and Higher Moments

As is well known, the expected value E(X) (also called expectation) of a continuous random variable X with continuous density f is defined as follows:

For (measurable) mappings g, transformations g(X) are again random variables, and the expected value is given by:

In particular, for each power of X so-called moments are defined for k = 1, 2, …:

Note that this term represents integrals which are not necessarily finite (then one says: the respective moments do not exist). There are even random variables whose density f allows for very large observations in absolute value with such a high probability that even the expected value μ ₁ is not finite.³ If nothing else is suggested, we will always assume random variables with finite moments without pointing out explicitly.

Often we consider so-called centered moments where g(X) is chosen as (X −E(X)) k . For k = 2 the variance is obtained (often denoted by σ ²)⁴:

Elementarily, the following additive decomposition is shown:

(2.1)

Since Var(X) ≥ 0 by construction, this gives rise to the following inequality:

(2.2)

In addition to centering, for higher moments a standardization is typically considered. The following measures of skewness and kurtosis with k = 3 and k = 4, respectively, are widely used:

The skewness coefficient is used to measure deviations from symmetry. If X exhibits a density f which is symmetric around the expected value, it obviously follows that γ ₁ = 0. The interpretation of the kurtosis coefficient is more difficult. Generally, γ ₂ is taken as a measure for a distribution’s “peakedness”, or alternatively, for how probable extreme observations (“outliers”) are. Frequently, the normal distribution is taken as a reference. For every normal distribution (also called Gaussian distribution, see Example 2.4) it holds that the kurtosis takes the value 3. Furthermore, it can be shown that it holds always true that

which is verified in Problem 2.1.

Example 2.3 (Kurtosis of a Continuous Uniform Distribution)

The random variable X is assumed to be uniformly distributed on [0,b] with density

As is well known, it then holds that

In order to calculate the kurtosis γ ₂ we are interested in the fourth centered moment:

For this we determine (binomial theorem):

From this it is obtained that

and hence

The kurtosis coefficient is therefore determined as

It is obvious that the kurtosis is independent of b. The value 1.8 is clearly smaller than 3 indicating that the uniform distribution’s curve exhibits a flatter behavior than that of the normal distribution. ■

Markov’s and Chebyshev’s Inequality

Consider again the random variable X with variance σ ² = Var(X). Depending on σ ², Chebyshev’s inequality allows to bound the probability with which the random variable is distributed around its expected value. In fact, this result is a special case of the more general Markov’s inequality , see (2.3), which is established e.g. in Ross (2010, Sect. 8.​2). A proof of Chebyshev’s result given in (2.4) will be provided in Problem 2.3.

Lemma 2.1 (Markov’s and Chebyshev’s Inequality)

Let X be a random variable.

(a)

If X takes only nonnegative values, then it holds for any real constant a > 0:

(2.3)

(b)

with σ ² = Var(X) < ∞ it holds that

(2.4)

where $$\varepsilon> 0$$ is an arbitrary real constant.

Example 2.4 (Normal Distribution )

The density of a random variable X with normal or Gaussian distribution with parameters μ and σ > 0 goes back to Gauss⁵ and is, as is well known,

with

In symbols we also write $$X \sim \mathcal{N}(\mu,\sigma ^{2})$$. As the density function is symmetric around μ it follows that γ ₁ = 0. The kurtosis we adopt from the literature without calculation as γ ₂ = 3. Sometimes we use this result for determining the fourth centered moment. Under normality it holds that:

We want to use this example to show that Chebyshev’s inequality may be not very sharp. For example,

When using the standard normal distribution, however, one obtains a much smaller probability than the bound due to (2.4):

 ■ 

2.3 Joint and Conditional Distributions

In this section we first recapitulate some widely known results. At the end we introduce the more involved theory of conditional expectation.

Joint Distribution and Independence

In order to restrict the notational burden, we only consider the three-dimensional case of continuous random variables X, Y and Z with the joint density function f x, y, z mapping from $$\mathbb{R}^{3}$$ to $$\mathbb{R}$$. For arbitrary real numbers a, b and c, probabilities are defined as multiple (or iterated) integrals:

As long as f is a continuous function, the order of integration does not matter, i.e. one obtains e.g.

This reversibility is sometimes called Fubini’s theorem. ⁶

Univariate and bivariate marginal distributions arise from integrating the respective variable:

The variables are called stochastically independent if, for arbitrary arguments, the joint distribution is given as the product of the marginal densities:

which implies pairwise independence:

The joint probability

is, under independence, factorized to

Covariance

In particular for only two variables a generalization of the expectation operator is considered. Let h be a real-valued function of two variables, h: $$\mathbb{R}^{2} \rightarrow \mathbb{R}$$, then we define as a double integral:

Hence, the covariance between X and Y can be defined as follows:

where the finiteness of these integrals is again assumed tacitly. It can be easily shown that the independence of two variables implies their uncorrelatedness, i.e. Cov(X, Y ) = 0, whereas the reverse does not generally hold true. In particular, the covariance only measures the linear relation between two variables. In order to have the measure independent of the units, it is usually standardized as follows:

The correlation coefficient ρ xy is smaller than or equal to one in absolute value, see Problem 2.7.

Example 2.5 (Bivariate Normal Distribution)

Let X and Y be two Gaussian random variables,

with correlation coefficient ρ. We talk about a bivariate normal distribution if the joint density takes the following form:

with $$\varphi _{x,y}(x,y)$$ equal to

Symbolically, we denote the vector as

where μ is a vector and Σ stands for a symmetric matrix:

In general, the covariance matrix is defined as follows:

Note that in the case of uncorrelatedness (ρ = 0) it holds that

The joint density function is then determined as the product of the individual densities. Consequently, the random variables X and Y are independent. Therefore it follows, in particular for the normal distribution, that uncorrelatedness is equivalent to stochastic independence. Furthermore, bivariate Gaussian random variables have the property that each linear combination is univariate normally distributed. More precisely, it holds for $$\lambda \in \mathbb{R}^{2}$$ with⁷ $$\lambda ' = (\lambda _{1},\,\lambda _{2})$$ that:

Interesting special cases are obtained with λ′ = (1, 1) and $$\lambda ' = (1,\,-1)$$ for sums and differences. Note that furthermore for multivariate normal distributions necessarily all marginal distributions are normal (with λ′ = (1,0) and λ′ = (0,1)). The reverse does not hold. A bivariate example for Gaussian marginal distributions without joint normal distributions is given by Bickel and Doksum (2001, p. 533). ■

Cauchy-Schwarz Inequality

The inequality by Cauchy and Schwarz is the reason why | ρ xy  | ≤ 1 applies. The following statement is verified in Problem 2.6.

Lemma 2.2 (Cauchy-Schwarz Inequality )

For arbitrary random variables Y and Z it holds that

(2.5)

where finite moments are assumed.

We want to supplement the Cauchy-Schwarz inequality by an intermediate inequality, see (2.8). For this purpose we remember the so-called triangle inequality for two real numbers:

Obviously, this can be generalized to:

If the sequence is absolutely summable, it is allowed to set n = ∞. This suggests that an analogous inequality also applies for integrals. If the function g is continuous, this implies continuity of | g | and one obtains:

This implies for the expected value of a random variable X:

(2.6)

This relation resembles (2.2); in fact, both relations are special cases of Jensen’s inequality .⁸ A random variable is called integrable if E( | X | ) < ∞. Of course this implies a finite expected value. For integrability a finite second moment is sufficient, which follows from (2.5) with Y = | X | and Z = 1:

Now, if setting X = YZ in (2.6), it follows that: | E(YZ) | ≤ E( | Y | | Z | ). This is the bound added to (2.5):

(2.8)

The first inequality follows from (2.6). The second one will be verified in the problem section.

Conditional Distributions

Conditional distributions and densities, respectively, are defined as the ratio of the joint density and the “conditioning density”, i.e. they are defined by the following density functions (where positive denominators are assumed):

It should be clear that these conditional densities are in fact density functions. In case of independence it holds by definition that the conditional and the unconditional densities are equal, e.g.

This is very intuitive: In case of two independent random variables, one does not have any influence on the probability with which the other takes on values.

Conditional Expectation

If the random variables X and Y are not independent and if the realization of Y is known, Y = y, then the expectation of X will be affected:

Analogously, we define the conditional expectation of a random variable Z, Z = h(X, Y ), $$h: \mathbb{R}^{2} \rightarrow \mathbb{R}$$, given Y = y as:

In particular, for h(X, Y ) = X g(Y ) with $$g: \mathbb{R} \rightarrow \mathbb{R}$$ one therefore obtains

Here, the marginal density of X is replaced by the conditional density conditioned on the value Y = y.

Technically, we can calculate the density conditioned on the random variable Y instead of conditioned on a value⁹ Y = y:

By f x | Y (x) a transformation of the random variable Y and consequently a new random variable is obtained. This is also true for the related conditional expectations:

As this is about random variables, it is absolutely reasonable to determine the expected value over the conditional expectation. This calculation can be carried out applying a rule called the “law of iterated expectations (LIE)” in the literature; it is given in Proposition 2.1. In order to prevent confusion whether X or Y is integrated, it is advisable to subscript the expectation operator accordingly:

Although Y and g(Y ) are random variables, after conditioning on Y they can be treated as constants and in case of a multiplicative composition, they can be put in front of the expected value when integration is with respect to X. This is the second statement in the following proposition, also cf. Davidson (1994, Theorem 10.10). The first statement will be derived in Problem 2.9.

Proposition 2.1 (Conditional Expectation)

With the notation introduced above, it holds that:

(a)

$$\mbox{ E}_{y}[\mbox{ E}_{x}(X\vert Y )] = \mbox{ E}_{x}(X)$$ ,

(b)

$$\mbox{ E}_{h}(g(Y )X\vert Y ) = g(Y )\mbox{ E}_{x}(X\vert Y )$$ for h(X,Y ) = X g(Y ).

Frequently, we formulate these statements in a shorter way,

if there is no risk of misunderstanding.

2.4 Stochastic Processes (SP)

In this section stochastic processes are defined and classified. In the following chapters we will be confronted with concrete types of stochastic processes.

Definition

A univariate stochastic process (SP) is a family of (real-valued) random variables, $$\{X(t;\omega )\}_{t\in \mathbb{T}}$$, for a given index set $$\mathbb{T}$$:

The subscript $$t \in \mathbb{T}$$ is always to be interpreted as “time”. At a fixed point in time t ₀ the stochastic process is therefore simply a random variable,

A fixed ω ₀, however, results in a path, a trajectory or a realization of a process which is also often referred to as time series ,

In fact, a stochastic process is a rather complex object. In order to characterize it mathematically, random vectors of arbitrary, finite length n at arbitrary points in time $$t_{1} <\cdots <t_{n}$$ have to be considered:

The multivariate distribution of such an arbitrary random vector characterizes a stochastic process. In particular, certain minimal requirements for the finite-dimensional distribution of $$X_{n}(t_{i})$$ guarantee that a stochastic process exists at all.¹⁰

Depending on the countability or non-countability of the index set $$\mathbb{T}$$, discrete-time and continuous-time SPs are distinguished. In the case of sequences of random variables, we talk about discrete-time processes , where the index set consists of integers, $$\mathbb{T} \subseteq \mathbb{N}$$ or $$\mathbb{T} \subseteq \mathbb{Z}$$. For discrete-time processes we agree upon lower case letters as an abbreviation without explicitly denoting the dependence on ω,

For so-called continuous-time processes the index set $$\mathbb{T}$$ is a real interval, $$\mathbb{T} = [a,b] \subseteq \mathbb{R}$$, frequently $$\mathbb{T} = [0,T]$$ or $$\mathbb{T} = [0,1]$$, however, open intervals are also admitted. For continuous-time processes we also suppress the dependence on ω notationally and write in a shorter way¹¹

Stationary and Gaussian Processes

Consider again generally an arbitrary vector of the length n,

If $$X_{n}(t_{i})$$ is jointly normally distributed for all n and t i , then X(t; ω) is called a normal process (also: Gaussian process ). Furthermore, we talk about a strictly stationary process if the distribution is invariant over time. More precisely, $$X_{n}(t_{i})$$ follows the same distribution as a vector which is shifted by s units on the time axis.

The distributional properties of a strictly stationary process do not depend on the location on the time axis but only on how far the individual components X(t i ; ω) are apart from each other temporally. Strict stationarity therefore implies the expected value and the variance to be constant (assuming they are finite) and the autocovariance for two points in time to depend only on the temporal interval:

1.

$$\mbox{ E}\left (X(t;\omega )\right ) =\mu _{x}$$ for $$t \in \mathbb{T}$$,

2.

$$\mbox{ Cov}\left (X(t;\omega ),\,X(t + h;\omega )\right ) =\gamma _{x}(h)$$ for all $$t,t + h \in \mathbb{T}$$,

and therefore in particular

A process (with finite second moments $$\mbox{ E}[\left (X(t;\omega )\right )^{2}]$$) fulfilling these two conditions (without necessarily being strictly stationary) is also called weakly stationary (or: second-order stationary). Under stationarity, we define as autocorrelation coefficient also independent of t:

Synonymously to autocorrelation we also speak of serial or temporal correlation. For weak stationarity not necessarily the whole distribution is invariant over time, however, at least the expected value and the autocorrelation structure are constant.

In the following, the term “stationarity” always refers to the weak form unless stated otherwise.

Example 2.6 (White Noise Process)

In the following chapters, $$\{\varepsilon _{t}\}$$ often denotes a discrete-time process $$\left \{\varepsilon (t;\omega )\right \}$$ free of serial correlation. In addition we assume a mean of zero and a constant variance σ ² > 0, i.e.

By definition such a process is weakly stationary. We typically denote it as

The reason why such a process is called white noise will be provided in Chap. 4 ■

Example 2.7 (Pure Random Process)

Sometimes $$\{\varepsilon _{t}\}$$ from Example 2.6 will meet the stronger requirements of being identically and independently distributed. Identically distributed implies that the marginal distribution

does not vary over time. Independence means that the joint distribution of the vector

equals the product of the marginal distributions. As the marginal distributions are invariant with respect to time, this also holds for their product. Thus, $$\{\varepsilon _{t}\}$$ is strictly stationary. In the following it is furthermore assumed that $$\varepsilon _{t}$$ has zero expectation and the finite variance σ ². Symbolically, we also write¹²:

A stochastic process with these properties is frequently called a pure random process. Clearly, an iid (or pure random) process is white noise. ■

Markov Processes and Martingales

A SP is called a Markov process if all information of the past about its future behavior is entirely concentrated in the present. In order to capture this concept more rigorosly, the set of information about the past of the process available up to time t is denoted by $$\mathcal{I}_{t}$$. Frequently, the information set is also referred to as

because it is the smallest σ-algebra generated by the past and presence of the process X(r; ω) up to time t.¹³ The entire information about the process up to time t is contained in $$\mathcal{I}_{t}$$. A Markov process , so to speak, does not remember how it arrived at the present state: The probability that the process takes on a certain value at time t + s depends only on the value at time t (“present”) and does not depend on the past behavior. In terms of conditional probabilities, for s > 0 the corresponding property reads:

(2.9)

A process is called a martingale if the present value is the best prediction for the future. A martingale technically fulfills two properties. In the first place, it has to be (absolutely) integrable, i.e. it is required that (2.10) holds. Secondly, given all information $$\mathcal{I}_{t}$$, the conditional expectation only uses the information at time t. More precisely, the expected value for the future is equal to today’s value. Technically, this amounts to:

(2.10)

(2.11)

Note that the conditional expectation is a random variable. Therefore, strictly speaking, equation (2.11) only holds with probability one.

Martingale Differences

Now, let us focus on the discrete-time case. A discrete-time martingale is defined by the expectation at time t for t + 1 being given by the value at time t. This is equivalent to expecting a zero increment from t to t + 1. Therefore, this concept is frequently expressed in form of differences. We then talk about martingale differences. As we will see, in a sense, such a property is settled between uncorrelatedness and independence and is interesting from both an economic and a statistical point of view.

We again assume an integrable process, i.e. {x t } fulfills (2.10). It is called a martingale difference (or martingale difference sequences) if the conditional expectation (given its own past) is zero:

This condition states concretely that the past does not have any influence on predictions (conditional expectation); i.e. knowing the past does not lead to an improvement of the prediction, the forecast is always zero. Not surprisingly, this also applies if only one single past observation is known (see Proposition 2.2(a)). Two further conclusions for unconditional moments contained in the proposition can be verified,¹⁴ see Problem 2.10: martingale differences are zero on average and free of serial correlation. In spite of serial uncorrelatedness, martingale differences in general are on no account independent over time. What is more, they do not even have to be stationary as it is not ruled out that their variance function depends on t.

Proposition 2.2 (Martingale Differences)

For a martingale difference sequence {x t } with $$\mathcal{I}_{t} =\sigma (x_{s}\,,\ s \leq t)$$ it holds that

(a)

$$\mbox{ E}(x_{t}\vert x_{t-h}) = 0$$ for h > 0,

(b)

E(x t ) = 0,

(c)

$$\mbox{ Cov}(x_{t},x_{t+h}) = \mbox{ E}(x_{t}x_{t+h}) = 0$$ for h ≠ 0

for all $$t \in \mathbb{T}$$ .

Note that a stationary martingale difference sequence has a constant variance and is thus white noise by Proposition 2.2. The concept should be further clarified by means of an example.

Example 2.8 (Martingale Difference)

Consider the process given by

with $$x_{1} =\varepsilon _{1}\mbox{ and }\varepsilon _{0} = 1$$. From this it follows that $$x_{2} = x_{1}\,\frac{\varepsilon _{2}} {\varepsilon _{0}} =\varepsilon _{1}\varepsilon _{2}$$ and by continued substitution:

We want to show that this is a martingale difference sequence. Therefore, we note that the past of the pure random process can be reconstructed from the past of xt:

Therefore, the information set $$\mathcal{I}_{t}$$ constructed from $$\{x_{t},\ldots,x_{1}\}$$ contains not only the past values of xt+1, but also the ones of the iid process up to time t. Thus, it holds that

The first equality follows from the definition of the process. The second equality is accounted for by Proposition 2.1(b). The third step is due to the independence of $$\varepsilon _{t+1}$$ of the past up to t, that is why conditional and unconditional expectation coincide. Finally, by assumption, $$\varepsilon _{t+1}$$ is zero on average. All in all, by this the property of martingale differences is established. Therefore, {xt} is free of serial correlation, however, it is serially (i.e. temporally) dependent, which is obvious from the recursive definition. ■ 

A prominent class of martingale differences are the ARCH processes treated in Chap. 6

2.5 Problems and Solutions

Problems

2.1 Prove for the kurtosis coefficient: γ ₂ ≥ 1.

2.2 Let X follow a Pareto distribution with

Prove that X has finite k-th moments if and only if θ > k.

2.3 Prove Chebyshev’s inequality (2.4).

2.4 Consider a bivariate distribution with:

Prove that X and Y are stochastically independent.

2.5 Calculate the expected values, variances and the correlation of X and Y from Example 2.2.

2.6 Prove the second inequality from (2.8).

2.7 Prove for the correlation coefficient that | ρ xy  | ≤ 1.

2.8 Consider a bivariate logistic distribution function for X and Y:

where x and y from $$\mathbb{R}$$ are arbitrary. What does the conditional density function of X given Y = y look like?

2.9 Prove statement (a) from Proposition 2.1.

2.10 Derive the properties (b) and (c) from Proposition 2.2.Hint: Use statement (a).

Solutions

2.1 Assuming finite fourth moments we define for a random variable X with μ ₁ = E(X):

Consider the standardized random variable Z with expectation 0 and variance 1:

For this random variable, it holds that $$\gamma _{2} = \mbox{ E}(Z^{4})$$. Replacing X by Z ² in (2.2), it follows

which proves the claim.

2.2 For the k-th moment it holds:

1.

case: If θ ≠ k, then the antiderivative results in

The corresponding improper integral is defined as limit:

For θ > k it follows that

For θ < k, however, no finite value is obtained as M k−θ goes off to infinity.

2.

case: For θ = k the antiderivative takes on another form:

As the logarithm is unbounded, or log(M) → ∞ for M → ∞, one cannot obtain a finite expectation either, as the upper bound of integration is ∞.

Both cases jointly prove the claim.

2.3 We provide two proofs. The first one builds on the fact that (2.4) is a special case of (2.3). The second one is less abstract and more elementary, and hence instructive, too.

1.

Note that (X −μ)² is a nonnegative random variable. Therefore, (2.3) applies with $$a =\varepsilon ^{2}$$:

The event $$(X-\mu )^{2} \geq \varepsilon ^{2}$$, however, is equivalent to $$\vert X -\mu \vert \geq \varepsilon$$, which establishes (2.4).

2.

Elementarily, we prove the claim for the case that X is a continuous random variable with density function f; the discrete case can be accomplished analogously. Note the following sequence of inequalities:

The first inequality is of course due to the omittance of

The second one is accounted for by the fact that for the integrands of the respective integrals it holds that:

and

Up to this point, it is therefore shown that:

This is equivalent to the claim.

2.4 The marginal density is obtained as follows:

and f x (x) = 0 for $$x\notin [0,a]$$. It also holds that

Hence, one immediately obtains for all x and y:

which was to be proved.

2.5 Obviously, the expected value of X is zero,

Therefore, it holds for the variance that:

Also Y is zero on average:

Hence, the variance reads

For the covariance we obtain

In order to compute it, the entire joint probability distribution is to be established:

We may collect those numbers in a table:

Y = 

− 40

− 30

− 20

− 10

0

100

0

0

0

X = 50

0

0

0

Plugging in yields

Therefore one obtains for the correlation coefficient apart from rounding errors ρ xy  = 0. 286.

2.6 It only remains to be shown that:

In order to see that we use the binomial formula and obtain

Therefore, the expectation of the left hand side cannot become negative, which yields:

In particular, it can be observed that the expression is always positive except for the case Y = Z. Rearranging terms verifies the second inequality from (2.8).

2.7 Plugging in X −E(X) and Y −E(Y ) instead of Y and Z in (2.5) by Cauchy-Schwarz it follows that

which is the same as:

This verifies the claim.

2.8 Due to

f x, y is determined by taking the partial derivative of F x, y with respect to both arguments:

The marginal distribution of Y is determined by

The marginal density therefore reads

Division yields the conditional density:

2.9 The following sequence of equalities holds and will be justified in detail. The first two equations define exactly the corresponding (conditional) expectations. For the third equality, the order of integration is reversed; this is due to Fubini’s theorem. The fourth equation is again by definition (conditional density), whereas in the fifth equation only the density of Y is cancelled out. In the sixth equation, the influence of Y on the joint density is integrated out such that the marginal density of X remains. This again yields the expectation of X by definition. Therefore, it holds that

which was to be verified.

2.10 We use statement (a), $$\mbox{ E}(x_{t}\vert x_{t-h}) = 0$$ for h > 0, connected with the law of iterated expectations:

This proves (b), that martingale differences are also unconditionally zero on average. By applying both results of Proposition 2.1 for h > 0 again with (a), one arrives at:

Therefore, $$\mbox{ Cov}(x_{t},\,x_{t+h}) = 0$$ for h > 0. However, as the covariance function is symmetric in h, the result holds for arbitrary h ≠ 0 which was to be verified to show (c).

References

Bickel, P. J., & Doksum, K. A. (2001). Mathematical statistics: Basic ideas and selected topics, volume 1 (2nd ed.). Upper Saddle River: Prentice-Hall.

Billingsley, P. (1986). Probability and measure (2nd ed.). New York: Wiley.

Breiman, L. (1992). Probability (2nd ed.). Philadelphia: Society for Industrial and Applied Mathematics.

Brockwell, P. J., & Davis, R. A. (1991). Time series: Theory and methods (2nd ed.). New York: Springer.

Davidson, J. (1994). Stochastic limit theory: An introduction for econometricians. Oxford/New York: Oxford University Press.

Grimmett, G. R., & Stirzaker, D. R. (2001). Probability and random processes (3rd ed.). Oxford: Oxford University Press.

Klebaner, F. C. (2005). Introduction to stochastic calculus with applications (2nd ed.). London: Imperical College Press.

Ross, S. (2010). A first course in probability (8th ed.). Upper Saddle River: Prentice-Hall.

Rudin, W. (1976). Principles of mathematical analyis (3rd ed.). New York: McGraw-Hill.

Sydsæter, K., Strøm, A., & Berck, P. (1999). Economists’ mathematical manual (3rd ed.). Berlin/New York: Springer.

Trench, W. F. (2013). Introduction to real analysis. Free Hyperlinked Edition 2.04 December 2013. Downloaded on 10th May 2014 from http://​digitalcommons.​trinity.​edu/​mono/​7.

Footnotes

1

Ross (2010) provides a nice introduction to probability, and so do Grimmett and Stirzaker (2001) with a focus on stochastic processes. For a short reference and refreshing e.g. the shorter appendix in Bickel and Doksum (2001) is recommended.

2

Sometimes also called a σ-field, which motivates the symbol F.

3

An example for this is the Cauchy distribution, i.e. the t-distribution with one degree of freedom. For the Pareto distribution, as well, the existence of moments is dependent on the parameter value; this is shown in Problem 2.2.

4

Then σ describes the square root of Var(X) with positive sign.

5

The traditional German spelling is Gauß. Carl Friedrich Gauß lived from 1777 to 1855 and was a professor in Göttingen. His name is connected to many discoveries and inventions in theoretical and applied mathematics. His portrait and a graph of the density of the normal distribution decorated the 10-DM-bill in Germany prior to the Euro.

6

Cf. Sydsæter, Strøm, and Berck (1999, p. 53). A proof is contained e.g. in the classical textbook by Rudin (1976, Thm. 10.2), or in Trench (2013, Coro. 7.2.2); the latter book may be recommended since it is downloadable free of charge.

7

Up to this point a superscript prime at a function has denoted its derivative. In the rare cases in which we are concerned with matrices or vectors, the symbol will also be used to indicate transposition. Bearing in mind the respective context, there should not occur any ambiguity.

8

The general statement is: for a convex function g it holds

(2.7)

see e.g. Sydsæter et al. (1999, p. 181), while a proof is given e.g. in Davidson (1994, Ch. 9) or Ross (2010, p. 409).

9

This is not a really rigorous way of introducing expectations conditioned on random variables. A mathematically correct exposition, however, requires measure theoretical arguments not being available at this point; cf. for example Davidson (1994, Ch. 10), or Klebaner (2005, Ch. 2). More generally, one may define expectations conditioned on a σ-algebra, $$\mbox{ E}(X\vert \mathcal{G})$$, where $$\mathcal{G}$$ could be the σ-algebra generated by Y: $$\mathcal{G} =\sigma (Y )$$.

10

These “consistency” requirements due to Kolmogorov are found e.g. in Brockwell and Davis (1991, p. 11) or Grimmett and Stirzaker (2001, p. 372). A proof of Kolmogorov’s existence theorem can be found e.g. in Billingsley (1986, Sect. 36).

11

The convention of using upper case letters for continuous-time process is not universal.

12

The acronym stands for “independently identically distributed”.

13

By assumption, the information at an earlier point in time is contained in the information set at a subsequent point in time: $$\mathcal{I}_{t} \subseteq \mathcal{I}_{t+s}$$ for s ≥ 0. A family of such nested σ-algebras is also called “filtration”.

14

We cannot prove the first statement rigorously, which would require a generalization of Proposition 2.1(a). The more general statement taken e.g. from Breiman (1992, Prop. 4.20) or Davidson (1994, Thm. 10.26) reads in our setting as

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_3

3. Autoregressive Moving Average Processes (ARMA)

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

3.1 Summary

This chapter is concerned with the modelling of serial correlation (or autocorrelation) that is characteristic of many time series dynamics. To that end we cover a class of stochastic processes widely used in practice. They are discrete-time processes: $$\{x_{t}\}_{t\in \mathbb{T}}$$ with $$\mathbb{T} \subseteq \mathbb{Z}$$. Throughout this chapter, the innovations or shocks $$\{\varepsilon _{t}\}$$ behind {x t } are assumed to form a white noise sequence as defined in Example 2.​6. The next section treats the rather simple moving average structure. The third section addresses the inversion of lag polynomials at a general level. The fourth section breaks down the technical aspects to the application with ARMA processes.

3.2 Moving Average Processes

We define the moving average process of order q (MA(q)) as

(3.1)

In the following, we assume a white noise process for the so-called innovation $$\{\varepsilon _{t}\}$$ governing the processes considered. In general, we assume b q ≠ 0. Let us consider a special case before we enter the general discussion of the model.

MA(1)

We set μ to zero and q to one in (3.1) and thereby obtain

As the innovations are WN(0, σ ²), the moments are independent of time: Firstly, it holds that $$\mbox{ E}(x_{t}) =\mu = 0$$. Secondly, one obtains from

immediately

and

For the MA(1) process considered the autocorrelation function $$\rho (h) = \frac{\gamma (h)} {\gamma (0)}$$ therefore is:

This is why the process is always (weakly) stationary without any assumptions with respect to b or to the index set $$\mathbb{T}$$. Elementary curve sketching shows that ρ(1) considered with respect to b,

becomes extremal for b = ±1. For $$b = -1$$, ρ(1) takes the minimum $$-\frac{1} {2}$$, and for b = 1 it takes the maximum $$\frac{1} {2}$$ (see Problem 3.1).

In Fig. 3.1 realizations were simulated by means of so-called pseudo random number with each 50 observations of moving average processes. All three graphs were generated by the same realizations of $$\{\varepsilon _{t}\}$$. The first graph with $$b_{1} = b = 0$$ shows the simulation of a pure random process $$\{\varepsilon _{t}\}$$: The times series oscillates arbitrarily around the expected value zero. The third graph with $$b_{1} = b = 0.9$$ depicts the case of (strong) positive autocorrelation of first order: Positive values are followed by positive values whereas negative values tend to entail negative values, i.e. the zero line is less often crossed than in the first case. Finally, the graph in the middle lies in between both extreme cases as weak positive autocorrelation ($$b_{1} = b = 0.3$$) is present.

Fig. 3.1

Simulated MA(1) processes with σ ² = 1 and μ = 0

MA(q)

The results obtained for the MA(1) process can be generalized. Every MA process is – for all parameter values independent of starting value conditions – always stationary. For the MA(q) process, it holds that its autocorrelation sequence vanishes from the order q on (i.e. it becomes zero). The proof is elementary and is therefore omitted.

Proposition 3.1 (MA(q))

Assume an MA(q) process from (3.1) .

(a)

The process is stationary with expectation μ.

(b)

For the autocovariances it holds that,

and γ(h) = 0 for h > q.

Example 3.1 (Seasonal MA)

Let S denote the seasonality, e.g. S = 4 or S = 12 for quarterly or monthly data, or S = 5 for (work) daily observations. Hence, we define as a special MA process¹

In this context, q = S and bq = b hold and

Proposition 3.1 (b) yields:

and also

The case h = S, however, yields

For h > S, according to Proposition 3.1 it holds that γ(h) = 0. Hence, the process at hand is exclusively autocorrelated at lag S, which is why it is also called a seasonal MA process. ■

MA(∞) Processes

We now let q go off to infinity. For reasons that become obvious in the fourth section, the MA coefficients, however, are not denoted by b j anymore. Instead, consider the infinite real sequence {c j }, $$j \in \mathbb{N}$$, to define:

(3.2)

Sometimes the process from (3.2) is called “causal” as there are only past or contemporaneous random variables $$\varepsilon _{t-j}$$, j ≥ 0, entering the process at time t. The condition on the coefficients {c j } of being absolutely summable guarantees that $$\sum _{j=0}^{\infty }c_{j}\,\varepsilon _{t-j}$$ in (3.2) is a well-defined random variable, which then can be called x t , see e.g. Fuller (1996, Theorem 2.2.1). Absolute summability naturally implies square summability,

and, indeed, sometimes we define the MA(∞) process upon this weaker assumption. Square summability of {c j } is sufficient for stationarity with E(x t ) = μ and

see Fuller (1996, Theorem 2.2.3), which is the first result of the follow proposition. The second one is established in Problem 3.3.

Proposition 3.2 (Infinite MA)

Assume an MA(∞) process,

with $$\sum _{j=0}^{\infty }c_{j}^{2} <\infty$$.

(a)

The process is stationary with expected value μ, and for the autocovariances it holds that

(b)

Under absolute summability, $$\sum _{j=0}^{\infty }\vert c_{j}\vert <\infty$$ , the sequence of autocovariances is absolutely summable:

The fact that the sequence of autocovariances is absolutely summable under (3.2), see (b) of the proposition, has an immediate implication: the autocovariances tend to zero with growing lag:

This means that the correlation between x t and x t−h tends to decrease with growing lag h.

Note that x t from (3.2) or Proposition 3.2 is defined as a linear combination of $$\varepsilon _{t-j}$$, j ≥ 0. Therefore, one sometimes speaks of a linear process . Other authors reserve this label for the more restricted case of iid innovations, where all temporal dependence of {x t } arises exclusively from the MA coefficients {c j }. The results of Proposition 3.2, however, hold under the weaker assumption that $$\{\varepsilon _{t}\}$$ is white noise.

Impulse Responses

The MA(∞) coefficients are often called impulse responses , since they measure the effect of a shock j periods ago on x t :

Square summability implies that shocks are transient in that c j  → 0 as j → ∞. The speed at which the impulse responses converge to zero characterizes the dynamics. In particular, Campbell and Mankiw (1987) popularized the cumulated impulse responses as measure of persistence , where the cumulated effect is defined as

This measure quantifies the total effect up to J periods back, if there occurred a unit shock in each past period, including the present period at time t. Asymptotically, one obtains the so-called long-run effect, often called total multiplier in economics. This measure is defined as

(3.3)

provided that this quantity exists. Clearly, under absolute summability of the impulse responses, CIR is well defined. Andrews and Chen (1994) advocated CIR as being superior to alternative measures of persistence. We will return to the measurement and interpretation of persistence in the next chapter.

The MA(∞) process is of considerable generality. In fact, Wold (1938) showed that every stationary process with expected value zero can be decomposed into a square summable, purely non-deterministic MA(∞) component and an uncorrelated component, say {δ t }, which is deterministic in the sense that it is perfectly predictable from its past:²

(3.4)

where $$\mbox{ E}(\varepsilon _{t}) = \mbox{ E}(\delta _{t}) = 0$$. For details on this Wold decomposition see Fuller (1996, Theorem 2.10.2) or Brockwell and Davis (1991, Theorem 5.7.1).

In practice it is not reasonable to construct a model with infinitely many parameters c j as they cannot be estimated from a finite number of observations without further restrictions. In the fourth section of this chapter, however, we will learn that very simple, so-called autoregressive processes with a finite number of parameters possess an MA(∞) representation. In order to discuss autoregressive processes rigorously, we need to concern ourselves with polynomials in the lag operator and their invertibility.

3.3 Lag Polynomials and Invertibility

Frequently, time series models are written by means of the lag operator ³ L. The lag operator shifts the process {x t } by one unit in time: $$Lx_{t} = x_{t-1}$$. By the inverse operator L ⁻¹ the shift is just reversed, $$L^{-1}Lx_{t} = x_{t}$$, or $$L^{-1}x_{t} = x_{t+1}$$. Successive use of the operator is denoted by its power, $$L^{\,j}x_{t} = x_{t-j}$$, $$j \in \mathbb{Z}$$. The identity is described with L ⁰, $$L^{0}x_{t} = x_{t}$$. Applied to a constant c, the operator leaves the value unchanged, Lc = c.

Causal Linear Filters

Let us consider an input process {x t }, $$t \in \mathbb{T}$$, which is transferred into an output process {y t } by linear filtering,

(3.5)

with the real filter coefficients {w j } (which need not to add up to one). Therefore, y t is a linear combination of values of x t−j where we assume constant filters, i.e. the weights do not depend on t. In particular, the filter is called causal as y t is defined by past and contemporaneous values of {x t } only.

The general linear causal filter from (3.5) can be formulated as polynomial in the lag operator:

Occasionally, two filters are put in a row:

Then, the filter F(L) transforming {x t } into {z t } is defined by multiplying the filters, $$F(L) = F_{2}(L)F_{1}(L)$$, which is called a “convolution”:

This convolution is commutative:

By expressing filters by means of the lag operator, we can manipulate them just as ordinary (complex-valued) polynomials. As an example we consider so-called difference filters.

Example 3.2 (Difference Filters )

By means of the lag operator, filters can be constructed, for example the difference filter $$\varDelta = 1 - L$$ or the difference of the previous year for quarterly data $$\varDelta _{4} = 1 - L^{4}$$:

The seasonal difference filter for monthly observations (S = 12) as well as for daily observations (S = 5) are defined analogously:

Instead of extensively calculating the double difference,

we write in short by expanding (1 − L)²:

While the ordinary difference operator Δ eliminates a linear time trend from a time series, the second differences naturally remove a quadratic time trend:

The fact that the order of filtering is exchangeable is well demonstrated by means of the example of seasonal and ordinary differences:

 ■ 

Invertibility of Lag Polynomials

We define as a polynomial of degree p (also of order p) in the lag operator

(3.6)

with the real coefficients b ₁ to b p . For brevity, P(L) is also called lag polynomial.

Consider a first degree polynomial as a special case of (3.6),

where the reason for the negative sign will immediately be obvious. When and how can this polynomial be inverted? A comparison of coefficients (“method of undetermined coefficients”) results in the following series expansion (see Problem 3.4):

As is well known, it holds that (infinite geometric series, see Problem 3.2)

(3.7)

Hence, it holds that

if and only if | a | < 1. This condition of invertibility is frequently reformulated. In order to do this, we determine the so-called z-transform of the lag polynomial with z being an element of the complex numbers ($$z \in \mathbb{C}$$): $$A_{1}(z) = 1 - az$$. Now, | a | < 1 implies that the z-transform $$A_{1}(z) = 1 - az$$ exhibits only roots outside the unit circle, i.e. roots being greater than one in absolute value:

(3.8)

Causally Invertible Polynomials

This condition of invertibility for A ₁(L) from (3.8) is easily conveyed to a polynomial P(L) of the order p. We say P(L) is causally invertible if there exists a power series expansion with non-negative powers and absolutely summable coefficients:

The invertibility depends on the z-transform

(3.9)

or rather on the absolute value of its roots. The following condition of invertibility is adopted from Brockwell and Davis (1991, Thm. 3.1.1), and it is discussed as an exercise (Problem 3.5).

Proposition 3.3

(a)

The polynomial 1 − aL is causally invertible,

if and only if |a| < 1.

(b)

The polynomial P(L) from (3.6) is causally invertible, i.e. for (P(L)) ⁻¹ there exits the absolutely summable series expansion,

if and only if it holds for all roots of P(z) that they are greater than one in absolute value:

(3.10)

Example 3.3 (Invertible MA Processes)

The MA(1) process, $$x_{t} =\varepsilon _{t} + b\varepsilon _{t-1}$$, can now be formulated alternatively by applying the MA(1) polynomial $$B(L) = 1 + b\,L$$. Although not required for stationarity, it is usually assumed that |b| < 1. What for? This implies for the MA polynomial that:

According to Proposition 3.3 the condition of invertibility is fulfilled and there exists

where the coefficients {α j} are absolutely summable. The {α j} are obtained explicitly by comparison of coefficients in

yielding

or

Hence, the MA(1) process $$x_{t} = B(L)\,\varepsilon _{t}$$ can be reformulated as follows:

or

In other words: The invertible MA(1) process (for |b| < 1) can be expressed (in an absolutely summable manner) as a process depending on its own infinitely many lags. Such a process is called autoregressive (of infinite order). ■

Frequently, one is interested in the special case of a quadratic polynomial, p = 2. In this case, the so-called Schur criterion provides an equivalent reformulation of the condition of invertibility, rephrasing (3.10) in terms of the polynomial coefficients b i directly. These have to fulfill three conditions simultaneously. We take the corresponding statement from e.g. Sydsæter, Strøm, and Berck (1999, p. 58).

Corollary 3.1

For p = 2 the polynomial from (3.6) ,

is causally invertible with absolutely summable series expansion $$\left (P(L)\right )^{-1}$$ if and only if:

Instead of checking | z 1, 2 | > 1 for

it may in practice be simpler to check (i) through (iii) from Corollary 3.1.

3.4 Autoregressive and Mixed Processes

Let {x t } be given by the following stochastic difference equation ,

defining an autoregressive process of the order p, AR(p). The properties of the general AR process can be illustrated well at the example p = 1.

AR(1)

Particularly, let p = 1:

(3.11)

When replacing x t−1 by this defining equation, one obtains:

Hence, x t and x t−2 prove to be correlated. Continued substitution yields

or for any h ≥ 0:

Now, let us suppose that the index set $$\mathbb{T}$$ does not have a lower bound at zero but includes an infinite past, then h can be arbitrarily large and the substitution can be repeated ad infinitum. If it furthermore holds that

then the geometric series yields (h → ∞)

and $$a^{h}\,x_{t-h} \rightarrow 0$$ in a sense that can be made rigoros, see Brockwell and Davis (1991, p. 71) or Fuller (1996, p. 39). In this manner, it follows for h → ∞ under the aforementioned conditions that:

This way one obtains an infinite MA representation with geometrically decaying coefficients, c j  = a  j in (3.2). In fact, this representation can formally be obtained by inverting 1 − aL, see Proposition 3.3 (a). The process is therefore stationary with (see Proposition 3.2)

and

It follows

Note that these results are obtained for | a | < 1. Furthermore, stationarity also depends on the index set. That is to say, if it holds that $$\mathbb{T} =\{ 0,1,2,\ldots \}$$ then the above-mentioned repeated substitution cannot be performed infinitely often. From

the expected value follows as time-dependent:

In particular, this example shows that the stationarity behavior of a process can depend on the index set $$\mathbb{T}$$. Therefore, in general a stochastic process is not completely characterized without specifying $$\mathbb{T}$$.

Example 3.4 (AR(1))

Figure 3.2 displays 50 realizations each of AR(1) processes obtained by simulation. However, in the first case $$a_{1} = a = 0$$ such that the graph depicts the realizations of $$\{\varepsilon _{t}\}$$. On the right the theoretical autocorrelogram, ρ(h) = 0 for h > 0, is shown. In the second panel, the case of a positive autocorrelation ($$a_{1} = a = 0.75$$) is illustrated: Positive values tend to be followed by positive values (and vice versa for negative values), such that phases of positive realizations tend to alternate with phases of negative values. The corresponding autocorrelogram shows the geometrically decaying positive autocorrelations up to the order 10. In the last case, there is a negative autocorrelation ($$a_{1} = a = -0.75$$); consistently, negative values tend to be followed by positive ones and vice versa positive values tend to be followed by negative ones. Therefore, the zero line is more often crossed than in the first case of no serial correlation. The corresponding autocorrelogram is alternating: $$\rho (h) = \vert a\vert ^{h}(-1)^{h}$$ for $$a = -0.75$$. Qualitatively different patterns of autocorrelation cannot be generated by the simple AR(1) model. Note that the impulse responses or MA(∞) coefficients of the AR(1) model are cj = a j. Consequently, the cumulated effect defined in (3.3) becomes for |a| < 1:

The larger a, the larger is CIR, which hence quantifies the persistence described above for positive a. ■

Fig. 3.2

Simulated AR(1) processes with σ ² = 1 and ν = 0

We have seen that the stationarity of an AR(1) process depends essentially on the absolute value of a. For the Markov property , however, this value is irrelevant. We talk about a Markov process if the entire past information $$\mathcal{I}_{t}$$ up to time t is concentrated in the last observation x t :

for all s > 0 and $$x \in \mathbb{R}$$. Assuming a normal distribution, we show in Problem 3.7 that every AR(1) process is a Markov process.

AR(p)

In general, the AR(p) process can be formulated equivalently by means of a lag polynomial:

(3.12)

Under the condition

the autoregressive polynomial A(L) is causally invertible with absolutely summable coefficients according to Proposition 3.3:

Therefore, under the condition of invertibility it holds that

and {x t } is a stationary MA(∞) process, see Proposition 3.2. Hence, the autocovariance sequence is absolutely summable. Furthermore, in this case it holds for h > 0 (w.l.o.g.⁴ we set ν = 0 for simplification),

Dividing by γ(0) yields the recursive relation from the subsequent proposition: The autocorrelations are given by a deterministic difference equation of order p. The still missing necessary condition of stationarity from Proposition 3.4 (A(1) > 0) will be derived in Problem 3.6.

Proposition 3.4 (AR(p))

Let {x t } be an AR(p) process from (3.12) with index set⁵ $$\mathbb{T} =\{ -\infty,\ldots,T\}$$.

(a)

The process has an absolutely summable MA(∞) representation according to (3.2) if and only if it holds that

Then the process is stationary with expectation $$\mu =\nu /A(1)$$ . The condition $$A(1) = 1 -\sum _{j=1}^{p}a_{j}> 0$$ is necessary for this.

(b)

For stationary processes the autocorrelation sequence is absolutely summable where it holds that ρ(h) ≠ 0 for all integer numbers h and

Again note that the absolute summability of ρ(h) implies: ρ(h) → 0 for h → ∞. The farther x t and x t+h are apart from each other the weaker tends to be their correlation.

Certain properties of the AR(1) process are lost for p > 1. In Problem 3.8 we show for a special case (p = 2) that the AR(p) process, p > 1, is not a Markov process in general.

AR(2)

Let p = 2,

with the autoregressive polynomial

From Corollary 3.1, we know the conditions under which $$\left (A(L)\right )^{-1}$$ can be expanded as an absolutely summable filter:

Consequently, under these three parameter restrictions the AR(2) process is stationary. The restrictions become even more obvious if they are solved for a ₂ and depicted in a coordinate system:

So, three lines are given, above or below which a ₂ has to be, respectively. This is why one also talks about the stability or stationarity triangle, see Fig. 3.3. Within the triangle lies the stationarity region.

Fig. 3.3

Stationarity triangle for AR(2) processes

The autocorrelation series of the AR(2) case is determined from Proposition 3.4. For h = 0, it naturally holds that ρ(0) = 1. For h = 1, one obtains

Because of the symmetry, $$\rho (-h) =\rho (h)$$, it follows that

Similarly, it follows that

By repeated insertion into the second order difference equation,

the entire autocorrelation sequence is determined. Next, four numerical examples will be considered.

Example 3.5 (AR(2))

We now consider four numerical examples for AR(2) processes in order to characterize typical patterns of autocorrelation. In all cases, the stability conditions can be proven to be fulfilled. The corresponding autocorrelograms can be found in Fig. 3.4.

(i)

a₁ = 0.7, a₂ = 0.1: In this case, all the autocorrelations are positive and they converge to zero with h; their behavior is similar to the autocorrelogram of a AR(1) process with a₁ > 0, see Fig. 3.2.

(ii)

$$a_{1} = -0.4$$, a₂ = 0.4: Starting with ρ(1) < 0, the autocorrelations alternate similarly to an AR(1) process with a₁ < 0, cf. Fig. 3.2.

(iii)

$$a_{1} = -1.0$$, $$a_{2} = -0.8$$: In this case, we find a dynamic which cannot be generated by an AR(1) model; two negative autocorrelations of the first and second order are followed by a seemingly irregularly alternating pattern.

(iv)

a₁ = 1.0, $$a_{2} = -0.5$$: In the last case, the autocorrelations swing from the positive area to the negative area, then to the positive one and again to the negative one whereas the last-mentioned can hardly be perceived because of the small absolute values.

Fig. 3.4

Autocorrelograms for AR(2) processes

Therefore, the cases (iii) and (iv) show that the AR(2) process allows for richer dynamics and dependence structures than the simpler AR(1) process. ■

Autoregressive Moving Average Processes

Now, we consider a combination of AR and MA processes. Again, {x t } is given by a stochastic difference equation of order p, only that $$\varepsilon _{t}$$ from (3.12) is replaced by an MA(q) process:

Abbreviating, we talk about ARMA(p,q) processes assuming a p ≠ 0 and b q ≠ 0. Again, a more compact representation follows by using lag polynomials,

(3.13)

where it is assumed that both polynomials

do not have common roots.

A stationary MA(∞) representation hinges on the autoregressive polynomial such that the stationarity condition can be adopted from the pure AR(p) case in Proposition 3.4. It amounts to an absolutely summable expansion of (A(L))⁻¹ such that the process possesses an absolutely summable representation as MA(∞) process, see (3.2). If stationarity is given, the absolutely summable autocorrelation sequence can again be determined from a stable difference equation (see e.g. Brockwell & Davis, 1991, p. 93)

Proposition 3.5 (ARMA(p, q))

Let {x t } be an ARMA(p,q) process from (3.13) with $$\mathbb{T} =\{ -\infty,\ldots,T\}$$.

(a)

The process has an absolutely summable MA(∞) representation according to (3.2) if and only if it holds that

Then the process is stationary with expectation $$\mu =\nu /A(1)$$ . The condition $$A(1) = 1 -\sum _{j=1}^{p}a_{j}> 0$$ is necessary for this.

(b)

For stationary processes the autocorrelation sequence is absolutely summable where it holds that ρ(h) ≠ 0 for all integer numbers h and

For h ≥ max(p, q + 1) the autocorrelations satisfy the same difference equation as in the pure AR(p) case. It is known that the solution to such a difference equation is bounded exponentially. Consequently, we have the following result, see e.g. Brockwell and Davis (1991, Prob. 3.11): For a stationary ARMA process there exist positive constants c and g with 0 < g < 1, such that

(3.14)

for all h = 1, 2, …. Hence, the decay rate of the autocorrelations is bounded exponentially. This shows again that stationary ARMA process are characterized be absolutely summable autocorrelations, see Problem 3.2. Proposition 3.5 will be discussed below for $$p = q = 1$$.

Before turning to the ARMA(1,1) case, we note that an analogous result to Proposition 3.5 (a) is available for an AR(∞) representation according to Proposition 3.3: The ARMA process has an absolutely summable AR(∞) representation (see Example 3.3) if and only if

In this case the ARMA process is called invertible.

ARMA(1,1)

We now wish to obtain the autocorrelation structure of the ARMA(1,1) process,

where | a | < 1 for stationarity, and | b | < 1 to ensure invertibility (see Example 3.3). The condition of no common roots of 1 − aL and 1 + bL is given for a ≠ − b. In the case of common roots, the lag polynomials could be reduced and one would obtain

Due to the invertibility of 1 − aL, the process can be formulated as an infinite MA process,

where a shift of subscripts was carried out. In the notation of (3.2) the MA(∞) coefficients result as

In this way Proposition 3.2 yields for the variance:

The autocovariance at lag one follows in the same way,

such that it holds:

Furthermore we learn from Proposition 3.5:

Hence, the MA(1) component (that is b) influences directly only ρ(1) having only an indirect effect beyond the autocorrelation of the first order: For h ≥ 2 a recursive relation between ρ(h) and ρ(h − 1) holds true just as it applies to the pure AR(1) process. Thus, this yields four typical patterns. In order to identify these, it suffices entirely to concentrate on the numerator of ρ(1) as well as on the sign of a as the denominator of ρ(1) is always positive because it is a multiple of the variance. As 1 + ab is positive due to the stationarity and the invertibility of the MA polynomial, the behavior of the autocorrelogram depends on the signs of a + b and a only. The exponential bound for the autocorrelations is easily verified:

Therefore, (3.14) applies with g = | a | and $$c = \vert \rho (1)\vert /\vert a\vert$$.

Example 3.6 (ARMA(1,1))

The four possible patterns of the autocorrelogram of a stationary and invertible ARMA(1,1) model will be discussed and illustrated by numerical examples, cf. Fig. 3.5.

Case 1:

Obviously, for a + b > 0 it holds that ρ(1) > 0. If, furthermore, a > 0 then the entire autocorrelogram proceeds above the zero line.

Case 2:

For a + b < 0 and a > 0 one obtains exclusively negative autocorrelations.

Case 3:

An alternating pattern, starting with ρ(1) > 0, is obtained for a < 0 and a > −b.

Case 4:

For a < 0 and a < −b the autocorrelation series is as well alternating but starts with a negative value.

Fig. 3.5

Autocorrelograms for ARMA(1,1) processes

Note that cases 1 and 4 can be generated qualitatively by a pure AR(1) process as well. For cases 2 and 3, however, there occur patterns which cannot be produced by an AR(1) process. Therefore, the ARMA(1,1) process allows for richer dynamic modeling than the AR(1) model does. Also when comparing with the AR(2) case, we find that the ARMA(1,1) model allows for additional dynamics. ■ 

3.5 Problems and Solutions

Problems

3.1 Where does the first order autocorrelation of an MA(1) process ($$\rho (1) = \frac{b} {1+b^{2}}$$) have its maximum and minimum?

3.2 Show for $$g \in \mathbb{R}\setminus \{1\}$$ (geometric series):

Conclusion: For | g | < 1 and n → ∞ it holds that:

3.3 Prove part (b) from Proposition 3.2.

3.4 Derive the series expansion

for real a with | a | < 1.

3.5 Prove Proposition 3.3 (b).

3.6 Prove the necessary condition of causal invertibility from Proposition 3.4, that is:

where $$A(z) = 1 - a_{1}\,z -\ldots -a_{p}\,z^{\,p}$$.

3.7 Let $$\{\varepsilon _{t}\} \sim \mbox{ WN}(0,\sigma ^{2})$$ be a Gaussian process. Show that {x t } with $$x_{t} = a_{1}\,x_{t-1} +\varepsilon _{t}$$ is a Markov process.

3.8 Let $$\{\varepsilon _{t}\} \sim \mbox{ WN}(0,\sigma ^{2})$$. Show that {x t } with $$x_{t} = a_{2}\,x_{t-2} +\varepsilon _{t}$$ is not a Markov process (a ₂ ≠ 0).

Solutions

3.1 The traditional way to solve the problem is curve sketching. We consider the first order autocorrelation to be a function of b:

Then the quotient rule for the first order derivative yields

The roots of the derivative are given by | b | = 1. In $$b = -1$$ there is a change of sign of f′(b), namely from a negative to a positive slope. Hence, in $$b = -1$$ there is a relative (and also an absolute) minimum. Because of f(b) being an odd function (symmetric about the origin), there is a maximum in b = 1. Therefore, the maximum possible correlation in absolute value is

One may also tackle the problem by more elementary means. Note that for b ≠ 0

which is equivalent to

with f(b) = ρ(1) defined above. Since $$\vert \,f(-1)\vert = f(1) = \frac{1} {2}$$, this solves the problem.

3.2 By S n we denote the following sum for finite n:

Multiplication by g yields

Therefore, it holds that

By ordinary factorization the formula

and therefore the claim is verified.

3.3 The absolute summability of γ(h) follows from the absolute summability of the linear coefficients {c j } allowing for a change of the order of summation. In order to do so, we first apply the triangle inequality:

where at the end round brackets were placed for reasons of clarity. The final term is further bounded by enlarging the expression in brackets:

Therefore, the claim follows indeed from the absolute summability of {c j }.

3.4 For the proof we denote $$(1 - aL)^{-1}$$ as $$\sum _{j=0}^{\infty }\alpha _{j}L^{\,j}$$,

and determine the coefficients α j . By multiplying this equation with 1 − aL, we obtain

Now, we compare the coefficients associated with L  j on the left- and on the right-hand side:

As claimed, the solution of the difference equation obtained in this way, ($$\alpha _{j} = a\,\alpha _{j-1}$$), is obviously α j  = a  j .

3.5 We factorize $$P(z) = 1 + b_{1}z +\,\ldots \, +b_{p}z^{\,p}$$ with roots $$z_{1},\ldots,z_{p}$$ of this polynomial (fundamental theorem of algebra):

From each bracket we factorize − z j out such that

Because of P(0) = 1, we obtain $$b_{p}\,(-1)^{p}\,z_{1}\,\ldots z_{p} = 1$$. Therefore the factorization simplifies to

with

where $$\zeta _{k} = 1/z_{k}$$. From part a) we know that

if and only if

Now, consider the convolution (sometimes called Cauchy product) for k ≠ ℓ:

with

We have $$\sum _{j=0}^{\infty }\vert c_{j}\vert <\infty$$ if and only if both $$P_{k}^{-1}(L)$$ and $$P_{\ell}^{-1}(L)$$ are absolutely summable, which holds true if and only if

Repeating this argument we obtain that

if and only if (3.10) holds. Quod erat demonstrandum.

3.6 At first we reformulate the autoregressive polynomial $$A(z) = 1 - a_{1}z -\,\ldots \,-a_{p}z^{\,p}$$ in its factorized form with roots $$z_{1},\ldots,z_{p}$$ (again by the fundamental theorem of algebra):

For z = 1 this amounts to

(3.15)

Because of A(0) = 1 we obtain as well:

(3.16)

Now we proceed in two steps, treating the cases of complex and real roots separately.

(A)

Complex roots: Note that for a root $$z_{1} \in \mathbb{C}$$ it holds that the complex conjugate, $$z_{2} = \overline{z_{1}},$$ is a root as well. Then calculating with complex numbers yields for the product

Hence, for p > 2, complex roots contribute positively to A(1) in (3.15). If p = 2, the roots are only complex if a ₂ < 0, since the discriminant is $$a_{1}^{2} + 4a_{2}$$; hence, A(1) > 0 by (3.15).

(B)

Since the effect of complex roots is positive, we now concentrate on real roots z i , for which it holds that | z i  | > 1 by assumption. So, we assume without loss of generality that the polynomial has no complex roots, or that all complex roots have been factored out. Two sub-cases have to be distinguished. (1) Even degree: For an even p we again distinguish between two cases. Case 1, a p  > 0: Because of (3.16) there has to be an odd number of negative roots and therefore there has to be an odd number of positive roots as well. For the latter it holds that (1 − z i ) < 0 while the first naturally fulfill (1 − z i ) > 0. Hence, as claimed, it follows from (3.15) that A(1) is positive. Case 2, a p  < 0: In this case one argues quite analogously. Because of (3.16) there is an even number of positive and negative roots such that the requested claim follows from (3.15) as well. (2) Odd degree: For an odd p one obtains the requested result as well by distinction of the two cases for the sign of a p . We omit details.

Hence, the proof is complete.

3.7 The normality of $$\{\varepsilon _{t}\}$$ implies a multivariate Gaussian distribution of

with the identity matrix I s of dimension s. The s-fold substitution yields

The sum over the white noise process has the moments

and, furthermore, it is normally distributed:

Hence, x t+s given x t follows a Gaussian distribution with the corresponding moments:

As x t+s can be expressed as a function of x t and $$\varepsilon _{t+1},\ldots,\varepsilon _{t+s}$$ alone, the further past of the process does not matter for the conditional distribution of x t+s . Therefore, for the entire information $$\mathcal{I}_{t}$$ up to time t it holds that:

Hence, the Markov property (2.​9) has been shown. It holds independently of the concrete value of a ₁.

3.8 For

we obtain for s = 1 the conditional expectations $$\mbox{ E}(x_{t+1}\vert \,\mathcal{I}_{t}) = a_{2}\,x_{t-1},$$ and

with $$\mathcal{I}_{t} =\sigma (x_{t},x_{t-1},\ldots,x_{1}).$$ As the conditional expectations are not equivalent, the conditional distributions are not the same. Hence, it generally holds that

which proves that {x t } is not a Markov process.

References

Andrews, D. W. K., & Chen, H.-Y. (1994). Approximately median-unbiased estimation of autoregressive models. Journal of Business & Economic Statistics, 12, 187–204.

Brockwell, P. J., & Davis, R. A. (1991). Time series: Theory and methods (2nd ed.). New York: Springer.CrossRef

Campbell, J. Y., & Mankiw, N. G. (1987). Are output fluctuations transitory? Quarterly Journal of Economics, 102, 857–880.CrossRef

Fuller, W. A. (1996). Introduction to statistical time series (2nd ed.). New York: Wiley.

Sydsæter, K., Strøm, A., & Berck, P. (1999). Economists’ mathematical manual (3rd ed.). Berlin/New York: Springer.CrossRef

Wold, H. O. A. (1938). A study in the analysis of stationary time series. Stockholm: Almquist & Wiksell.

Footnotes

1

For S = 1 we obtain the MA(1) case, however, without a seasonal interpretation.

2

Typically, one assumes that δ t is identically equal to zero for all t, since “perfectly predictable” only allows for trivial processes like e.g. $$\delta _{t} = (-1)^{t}A$$ or δ t  = A, where A is some random variable, such that $$\delta _{t} = -\delta _{t-1}$$ or $$\delta _{t} =\delta _{t-1}$$, respectively. Of course, this does not rule out the case of a constant mean μ different from zero as assumed in (3.2).

3

Many authors also speak of the backshift operator and write B.

4

The abbreviation stands for “without loss of generality”. It is frequently used for assumptions that are substantially not necessary and that are only made to simplify the argument or the notation. In the example at hand, generally it would have to be written $$(x_{t-j}-\mu )$$ for all j; just as well, one can set $$\mu =\nu /A(1)$$ equal to zero and simply write x t−j .

5

Also in the following, the notation { −∞,…,T} is always to denote the set of all integers without $$\{T + 1,T + 2,\ldots \}$$.

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_4

4. Spectra of Stationary Processes

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

4.1 Summary

Spectral analysis (or analysis in the frequency domain) aims at detecting cyclical movements in a time series. These may originate from seasonality, a trend component or from a business cycle. The theoretical spectrum of a stationary process is the quantity measuring how strongly cycles with a certain period, or frequency, account for total variance. Typically, elaborations on spectral analysis are formally demanding requiring e.g. knowledge of complex numbers and Fourier transformations. In this textbook we have tried for a way of presenting and deriving the relevant results being less elegant but in return managing with less mathematical burden. The next section provides the definitions and intuition behind spectral analysis. Section 4.3 is analytically more demanding containing some general theory. This theory is exemplified with the discussion of spectra from particular ARMA processes, hence building on the previous chapter.

4.2 Definition and Interpretation

In this chapter we assume the most general case considered previously, i.e. the infinite MA process that is only square summable, $$\{x_{t}\}_{t\in \mathbb{T}}$$, $$\mathbb{T} \subseteq \mathbb{Z}$$,

(4.1)

with $$\{\varepsilon _{t}\} \sim \mbox{ WN}(0,\sigma ^{2})$$. The autocovariances,

are given in Proposition 3.​2 (a). We do not assume that {c j } and hence {γ(h)} are absolutely summable,¹ simply because this will not hold under long memory treated in the next chapter. We wish to construct a function f that allows to express the autocovariances as weighted cosine waves of different periodicity,²

The basic ingredient of an analysis of periodicity is the cosine cycle whose properties we want to recall as an introduction.

Periodic Cycles

By c λ (t) we denote the cycle based on the cosine,³

where λ with λ > 0 is called frequency . The frequency is inversely related to the period P,

For λ = 1 one obtains the cosine function which is 2π−periodic and even (symmetric about the ordinate):

More generally, it holds with $$P = 2\pi /\lambda$$ that:

Therefore the cosine cycle $$c_{\lambda }\left (t\right )$$ with frequency λ has the period $$P = 2\pi /\lambda.$$ Of course, the symmetry of c ₁(t) carries over:

For λ = 1, λ = 2 and λ = 0. 5 these properties are graphically illustrated in Fig. 4.1.

Fig. 4.1

Cosine cycle with different frequencies

Finally, remember the derivative of the cosine,

which we will use repeatedly.

Definition

For convenience, we now rephrase the MA(∞) process in terms of the lag polynomial C(L) of infinite order,

Next, we define the so-called power transfer function $$T_{C}\left (\lambda \right )$$ of this polynomial:⁴

(4.2)

Note that T C may not exist everywhere, there may be singularities at some frequency λ ∗ such that $$T_{C}\left (\lambda \right )$$ goes off to infinity as λ → λ ∗; but at least the power transfer function is integrable. The key result in Proposition 4.1 (e) is from from Brockwell and Davis (1991, Coro. 4.3.1, Thm. 5.7.2); it will be proved explicitly in Problem 4.1 under the simplifying assumption of absolute summability. The first four statements in the following proposition are rather straightforward and will be justified below.

Proposition 4.1 (Spectrum)

Define for {x t } from (4.1) the spectrum

It has the following properties:

(a)

$$f(-\lambda ) = f(\lambda )$$ ,

(b)

$$f(\lambda ) = f(\lambda +2\pi )$$ ,

(c)

f(λ) ≥ 0,

(d)

f(λ) is continuous in λ under absolute summability, $$\sum _{j}\vert c_{j}\vert <\infty$$ .

(e)

For all $$h \in \mathbb{Z}$$ :

Substituting the autocovariance expression from Proposition 3.​2 into (4.2), the following representation of the spectrum exists:

(4.3)

The symmetry of the spectrum in Proposition 4.1 (a) immediately follows from the symmetry of the cosine function. From the periodicity of the cosine, (b) follows as well. Both results jointly explain why the spectrum is normally considered on the restricted domain [0, π] only. Property (c) follows from the definition of the power transfer function, see Footnote 6 below. Finally, the continuity of f(λ) claimed in (d) under absolute summability results from uniform convergence, see Fuller (1996, Thm. 3.1.9).

We call the function f (or f x , if we want to emphasize that {x t } is the underlying process) the spectrum of {x t }. Frequently, one also talks about spectral density or spectral density function as f is a non-negative function which could be standardized in such a way that the area beneath it would be equal to one.

Interpretation

The usual interpretation of the spectrum is based on Proposition 4.1. Result (e) and (4.3) jointly show the spectrum and the autocovariance series to result from each other. In a sense, spectrum and autocovariances are two sides of the same coin. The spectrum can be determined from the autocovariances by definition and having the spectrum, Proposition 4.1 provides the autocovariances. The case h = 0 with

is particularly interesting. This equation implies: The spectrum at λ ₀ measures how strongly the cycle with frequency λ ₀ and therefore of period $$P_{0} = 2\pi /\lambda _{0}$$ adds to the variance of the process. If f has a maximum at λ ₀, then the dynamics of {x t } is dominated by the corresponding cycle or period; inversely, if the spectrum has a minimum at λ ₀, then the corresponding cycle is of less relevance for the behavior of {x t } than all other cycles. For λ → 0, period P converges to infinity. A cycle with an infinitely long period is interpreted as a trend or a long-run component. Hence, f(0) indicates how strongly the process is dominated by a trend component.

Frequently, the analysis of the autocovariance structure or the autocorrelation structure of a process is called “analysis in the time domain” as γ(h) measures the direct temporary dependence between x t and x t+h . Correspondingly, the spectral analysis is often referred to as “analysis in the frequency domain”. Proposition 4.1 and the definition in (4.3) show how to move back and forth between time and frequency domain.

Examples

Example 4.1 (White Noise)

Let us consider the white noise process $$x_{t} =\varepsilon _{t}$$ being free from serial correlation. By definition it immediately follows that the spectrum is constant:

According to Proposition 4.1 all frequencies account equally strongly for the variance of the process. Analogously to the perspective in optics that the “color” white results if all frequencies are present equally strongly, serially uncorrelated processes are also often called “white noise”. ■

Example 4.2 (Season)

Let us consider the ordinary seasonal MA process from Example 3.​1,

with

and $$\gamma \left (h\right ) = 0$$ else. By definition we obtain for the spectrum from (4.3)

or

In Problem 4.2 we determine that there are extrema at

The corresponding values are

Depending on the sign of b, maxima and minima are followed by each other, respectively. In Fig. 4.2 we find two typical shapes of the spectrum of the seasonal MA process for⁵ S = 4 (quarterly data) with b = 0.7 and $$b = -0.5$$. First, let us interpret the case b > 0. There are maxima at the frequencies 0,π∕2 and π. Corresponding cycles are of the period

The trend is the first infinitely long “period”. The second cycle has the period P₁ = 4, i.e. four quarters which is why this is the annual cycle . The third cycle with P₂ = 2 is the semi-annual cycle with only two quarters. These three cycles dominate the process for b > 0. Inversely, for b < 0 it holds that these very cycles add particularly little to the variance of the process. ■

Fig. 4.2

Spectra (2π f(λ)) of the MA(S) process from Example 4.2

Example 4.3 (MA(1))

Specifically for S = 1 the seasonal MA process passes into the MA(1) process. Accordingly, one obtains two extrema at zero and π:

In between the spectrum reads

For b = 0.7 and $$b = -0.5$$, respectively, the spectra were calculated, see Fig. 4.2. For b < 0 one spots the relative absence of a trend (frequency zero matters least) while for b > 0 precisely the long-run component as a trend dominates the process. ■

Example 4.4 (Business Cycle)

The spectrum is not only used for modelling seasonal patterns but as well for determining the length of a typical business cycle. Let us assume a process with annual observations having the spectrum depicted in Fig. 4.3. The maximum is at λ = 0.73. How do we interpret this fact with regard to contents? The dominating frequency λ = 0.73 corresponds to a period of about 8.6 (years). A frequency of this magnitude is often called “business cycle frequency” being interpreted as the frequency which corresponds to the business cycle. In fact, Fig. 4.3 does not comprise an empirical spectrum. Rather, one detects the theoretical spectrum of the AR(2) model whose autocorrelogram is depicted in Fig. 3.​4 down to the right. The cycle, which can be seen in the autocorrelogram there, translates into the spectral maximum from Fig. 4.3. ■

Fig. 4.3

Spectrum (2π f(λ)) of business cycle with a period of 8.6 years

4.3 Filtered Processes

The ARMA process or more generally the infinite MA process have been defined as filtered white noise. In order to systematically derive a formal expression for their spectra, we start quite generally with the relation between input and output of a filter in the frequency domain.

Filtered Processes

As in the previous chapter, we consider the causal, time-invariant, linear filter F(L),

where L again denotes the lag operator and p = ∞ is allowed for. The filter is assumed to be absolutely summable, which trivially holds true for finite order p. Let the process {x t } be generated by filtering of the stationary process {e t },

Then, how does the corresponding spectrum of {x t } for a given spectrum f e of {e t } read? The answer is based on the power transfer function $$T_{F}\left (\lambda \right )$$ that we briefly touched upon in the previous section⁶:

(4.4)

At a first glance, this expression appears cumbersome. However, in the next section we will see that for concrete ARMA processes it simplifies radically. If $$F\left (L\right )$$ is a finite filter (i.e. with finite p), then the sums of $$T_{F}\left (\lambda \right )$$ are truncated accordingly, see (4.8) in the next section. With $$T_{F}\left (\lambda \right )$$ the following proposition considerably simplifies the calculation of theoretical spectra (for a proof of an even more general result see Brockwell and Davis (1991, Thm. 4.4.1), while Fuller (1996, Thm. 4.3.1) covers our case where {e t } has absolutely summable autocovariances).

Proposition 4.2 (Spectra of Filtered Processes)

Let {e t } be a stationary process with spectrum $$f_{e}\left (\lambda \right )$$ . The filter

be absolutely summable, $$\sum \limits _{j=0}^{\infty }\mid w_{j}\mid <\infty$$ , and {x t } be

Then, {x t } is stationary with spectrum

where $$T_{F}\left (\lambda \right )$$ is defined in (4.4) .

Example 4.5 (Infinite MA)

Let $$e_{t} =\varepsilon _{t}$$ from Proposition 4.2 be white noise with

and consider an absolutely summable MA(∞) process,

Then Proposition 4.2 kicks in:

(4.5)

This special case of Proposition 4.2 will be verified in Problem 4.3. Note that the spectrum given in (4.5) equals of course the result from Proposition 4.1 with (4.2), which continues to hold without absolute summability. ■

Persistence

We now return more systematically to the issue of persistence that we have touched upon in the example of the AR(1) process in the previous chapter. Loosely speaking, we understand by persistence the degree of (positive) autocorrelation such that subsequent observations form clusters: positive observations tend to be followed by positive ones, while negative observations tend to induce negative ones. With persistence we try to capture the strength of such a tendency, which depends not only on the autocorrelation coefficient at lag one but also on higher order lags. In the previous chapter we mentioned that it has been suggested to measure persistence by means of the cumulated impulse responses CIR defined in (3.​3). This quantity shows up in the spectrum at frequency zero by Proposition 4.2. Assume that {x t } is an MA(∞) process with absolutely summable impulse response sequence {c j }. We then have:

or

Hence, the larger CIR, the stronger is the contribution of the trend component at frequency zero to the variance of the process, which formalizes our concept of persistence. Cogley and Sargent (2005) applied as relative spectral measure for persistence the ratio of $$2\pi f_{x}(0)/\gamma _{x}(0)$$ pioneered previously by Cochrane (1988); it can be interpreted as a variance ratio and is hence abbreviated as VR:

(4.6)

In the case of a stationary AR(1) process, $$x_{t} = a_{1}\,x_{t-1} +\varepsilon _{t}$$, it holds that (see Problem 4.6)

(4.7)

In the case of a ₁ = 0 (white noise) we have no persistence, and VR = 1. For a ₁ > 0 the process is all the more persistent the larger a ₁ is. Following Hassler (2014), one may say that a process has negative persistence if VR < 1. The plot of a series under negative persistence will typically display a zigzag pattern as observed in the last plot in Fig. 3.​2 The limiting cases of VR = 0 (also called antipersistent) and VR = ∞ (also called strongly persistent) will be dealt with in Chap. 5

ARMA Spectra

As a consequence of the previous proposition, we can derive what the spectrum of a stationary ARMA process {x t } looks like. Remember the definition from (3.​13),

Now, define

By Proposition 4.2 one obtains for the spectra

The assumption of a stationary MA(∞) representation⁷ implies $$T_{A}\left (\lambda \right )> 0$$. Consequently, one may solve for f x rendering the following corollary.

Corollary 4.1 (ARMA Spectra)

Let {x t } be a stationary ARMA(p,q) process

Its spectrum is given by

where $$T_{B}\left (\lambda \right )$$ and $$T_{A}\left (\lambda \right )$$ are the power transfer functions of $$B\left (L\right )$$ and $$A\left (L\right )$$ .

Often, we restrict the class of stationary ARMA processes to the invertible ones, meaning we assume that the moving average polynomial B(L) satisfies the invertibility condition of Proposition 3.​3: All solutions of B(z) = 0 are larger than 1 in absolute value. This implies as in Footnote 7 that T B (λ) > 0, such that the invertible ARMA spectrum is strictly positive for all λ.

In the next section we will learn that the calculation of the functions $$T_{A}\left (\lambda \right )$$ and $$T_{B}\left (\lambda \right )$$ and thereby the calculation of the spectra do not pose any problem, cf. Eq. (4.8).

4.4 Examples of ARMA Spectra

The ARMA filters A(L) and B(L) are assumed to be of finite order. In order to calculate the spectrum, the power transfer function is needed due to Corollary 4.1. Thus, next we will get to know a simple trick allowing for quickly calculating the power transfer function of a finite filter.

Summation over the Diagonal

We consider for finite p the filter F(L) with the coefficients $$w_{0},w_{1},\,$$ … , w p being collected in a vector:

The outer product yields a matrix where w ′ stands for the transposition of the column w:

Obviously, the matrix is symmetric. Now, we add the cosine as function of | j − i | , $$\cos \left (\lambda \mid j - i\mid \right )$$, to the entries $$w_{i}w_{j}$$. Let the resulting matrix be called $$M_{F}\left (\lambda \right )$$. It becomes:

The rule for calculating $$T_{F}\left (\lambda \right )$$ reads in words: “Add up the sums over all diagonals of $$M_{F}\left (\lambda \right )$$”:

This corresponds exactly to (4.4) for finite p:

(4.8)

AR(1) Spectra

The autoregressive polynomial of order one reads

i.e. the filter coefficients are

Hence, for the power transfer function, (4.8) provides us with

and Corollary 4.1 yields for the spectrum

In Problem 4.4 we will show that there are extrema at λ = 0 and λ = π, where the slope of the spectrum is zero. For a ₁ > 0 the spectrum decreases on $$\left [0,\pi \right ]$$, i.e. the most significant frequency is λ = 0: The process is dominated by trending behavior. Figure 4.4 shows that this is the more true the greater a ₁ is: The greater a ₁, the steeper and higher grows the spectrum in the area around zero. Mirror-inversely, for a ₁ < 0 it holds that the trend component matters least, see Fig. 4.5. The direct comparison to the time domain in Fig. 3.​2 is also interesting. The case in which a ₁ > 0n with the spectral maximum at λ = 0 translates in persistence of the process: Observations temporarily lying close together have similar numerical values, i.e. the autocorrelation function is positive. For a ₁ < 0, however, observations following each other have the tendency to change their sign as, in this case, there is just no trending behavior.

Fig. 4.4

AR(1) spectra (2π f(λ)) with positive autocorrelation

Fig. 4.5

AR(1) spectra (2π f(λ)), cf. Fig. 3.​2

AR(2) Spectra

The AR(2) process is given by

In Problem 4.5 we recapitulate the principle of the “summation over the diagonal” and thus we show

Therefore, due to Corollary 4.1, the corresponding spectrum reads

For a ₂ = 0 one obtains the AR(1) case.

In Fig. 4.6 spectra for four parameter constellations are depicted; these are exactly the four cases for which autocorrelograms are given in Fig. 3.​4. The top left case could be well approximated by an AR(1) process. This is also roughly true for the top right case; however, closer inspection reveals that the AR(2) spectrum is not minimal at frequency zero. Both the lower spectra entirely burst the AR(1) scheme. On the bottom right we have the example of the business cycle, see Fig. 4.3. The spectrum on the bottom left is even more extreme: Except for a rather small area around λ = 2, it is zero almost everywhere which is why there is no trend component. The process is determined by almost only one cycle which can be seen in the autocorrelogram as well.

Fig. 4.6

AR(2) spectra (2π f(λ)), cf. Fig. 3.​4

ARMA(1,1) Spectra

Now, let us consider the two filters

We know the filter transfer function of B(L) from Example 4.3:

The transformation of A(L) was determined at the beginning of this section. Due to Corollary 4.1, we put the spectrum together as follows:

In order to have this illustrated, consider the examples from Fig. 4.7. The cases correspond in their graphical arrangement to the autocorrelograms from Fig. 3.​5. The cases top right and bottom left are interesting. At the top on the right, the entire absence of a trend is reflected in a negative autocorrelogram close to zero. At the bottom on the left, beside the trend, cycles of higher frequencies add to the process as well, the process consequently being positively autocorrelated of the first order and then exhibiting an alternating pattern of autocorrelation.

Fig. 4.7

ARMA(1,1) spectra (2π f(λ)), cf. Fig. 3.​5

Multiplicative Seasonal AR Process

If one wants to have a decaying autocorrelation function not dropping to zero, then one does not choose a pure MA model as in Example 4.2. The most basic seasonal autoregressive model is based on the filter $$(1 - a_{S}L^{\,S})$$. Frequently, the trend component is to have an additional weight which is why one adds the AR(1) factor (1 − a ₁ L):

Therefore, we have an AR(S + 1) model with parameter restrictions. The spectrum is adopted from Problem 4.6 in which T A (λ) is given:

In Fig. 4.8, we show two examples for the quarterly case (S = 4). With the frequencies λ = π and $$\lambda =\pi /2$$ the semi-annual cycles with P = 2 quarters period and the annual cycles with P = 4 quarters length are modelled (one also talks about seasonal cycles). As a ₁ = 0. 5 is positive in both the spectra, the trend (at frequency zero) dominates the seasonal cycles. The annual and semi-annual cycles add both equally strongly to the variance of the process. However, in the case a ₄ = 0. 8, the seasonal component is more pronounced than in the case a ₄ = 0. 5 as in the upper spectrum both the seasonal peaks are not only higher than in the lower one (note the scale on the ordinate) but most of all steeper: In the upper graph, the area beneath the spectrum substantially concentrates on the three frequencies 0, π∕2 and π, whereas it is more spread over all frequencies in the lower one.

Fig. 4.8

Spectra (2π f(λ)) of multiplicative seasonal AR processes (S = 4)

4.5 Problems and Solutions

Problems

4.1 Prove Proposition 4.1 (e) under the additional assumption of absolute summability.

4.2 Determine the extrema in the spectrum of the seasonal MA process from Example 4.2.

4.3 Prove the structure of the spectrum (4.5) for absolutely summable MA(∞) processes.

4.4 Determine the extrema of the AR(1) spectrum.

4.5 Determine the power transfer function T A (λ) of the filter $$A(L) = 1 - a_{1}L - a_{2}L^{2}$$.

4.6 Determine the power transfer function T A (λ) of the multiplicative quarterly AR filter $$A(L) = (1 - a_{1}\,L)(1 - a_{4}\,L^{4}) = 1 - a_{1}\,L - a_{4}\,L^{4} + a_{1}\,a_{4}\,L^{5}$$.

4.7 Determine the persistence measure VR from (4.6) for a stationary and invertible ARMA(1,1) process. Discuss its behavior in particular for the MA(1) model (in comparison with the AR(1) case given in (4.7)).

Solutions

4.1 We define the entity A h and will show that it equals γ(h). Due to the symmetry of the cosine function and of the even spectrum it holds by definition that:

Because of the absolute summability, the order of summation and integration is interchangeable:

For h = 0 it holds that⁸

Accordingly, for h ≠ 0 it holds that

where the trigonometric formula

was used. By this we obtain

as one can see that for $$k \in \mathbb{Z}\setminus \{0\}$$ the integral is

So, we finally obtain

for h ≠ 0 as well. Hence, A h  = γ(h) for all h, and the proof is complete.

4.2 The spectrum

is given. In order to determine the extrema, we consider the derivative,

with (S + 1) zeros

on the interval $$\left [0,\pi \right ]$$. The sign of the second derivative depends on b:

One obtains

Accordingly, maxima and minima follow each other. For b > 0, the sequence of extrema begins with a maximum at zero; for b < 0, one obtains a minimum, inversely.

4.3 The autocovariances of

are known from Proposition 3.​2:

For the spectrum $$f_{x}\left (\lambda \right )$$ it follows:

Hence, the claim is verified.

4.4 For

one obtains by differentiation

Obviously, candidates for extrema are λ = 0 and λ = π:

The sign of the derivative depends on a ₁ only:

Accordingly,

are maxima and minima, depending on the sign of a ₁.

4.5 With the vector of coefficients

we obtain as outer product

Adding the cosine, it follows that

By summation over the diagonal we obtain due to symmetry

which results from (4.8) as well. This is in accordance with the result in the text.

4.6 Using (4.8) with p = 5 yields the following expression:

This is simply an exercise in concentration and is simplified by the following equalities:

4.7 In the previous section we discussed the ARMA(1,1) process with the polynomials

Evaluating the spectrum given there we have:

The variance we copy from Chap. 3:

By (4.6) we obtain

If b ₁ = 0, the AR(1) case from (4.7) is of course reproduced. If a ₁ = 0, the MA(1) case results as

We hence have negative persistence for b ₁ < 0, which reflects the negative autocorrelation. For b ₁ > 0, it is straightforward to verify that VR is growing with b ₁, reaching a maximum value of VR = 2 for b ₁ = 1. This corresponds to the persistence of an AR(1) process with $$a_{1} = 1/3$$. Hence, the invertible MA(1) process with | b ₁ | < 1 can only capture very moderate persistence in comparison with the AR(1) case where VR grows with a ₁ beyond any limit.

References

Brockwell, P. J., & Davis, R. A. (1991). Time series: Theory and methods (2nd ed.). New York: Springer.CrossRef

Cochrane, J. H. (1988). How big is the random walk in GNP? Journal of Political Economy, 96, 893–920.CrossRef

Cogley, T., & Sargent T. S. (2005). Drifts and volatilities: Monetary policies and outcomes in the post WWII US. Review of Economic Dynamics, 8, 262–302.CrossRef

Fuller, W. A. (1996). Introduction to statistical time series (2nd ed.). New York: Wiley.

Hamilton, J. (1994). Time series analysis. Princeton: Princeton University Press.

Hassler, U. (2014). Persistence under temporal aggregation and differencing. Economics Letters, 124, 318–322.CrossRef

Footnotes

1

The assumption of absolute summability underlies most textbooks when it comes to spectral analysis, see e.g. Hamilton (1994) or Fuller (1996).

2

From Brockwell and Davis (1991, Coro. 4.3.1) in connection with Brockwell and Davis (Thm. 5.7.2) one knows that such an expression exists.

3

Here, the so-called amplitude is equal to one ($$\mid c_{\lambda }(t)\mid \,\leq \,1)$$, and the phase shift is zero $$\left (c_{\lambda }\left (0\right ) = 1\right )$$.

4

A more detailed and technical exposition is reserved for the next section. Our expression in (4.2) can be derived from the expression in Brockwell and Davis (1991, eq. 5.7.9), which is given in terms of complex numbers.

5

The variance of the white noise is set to one, σ ² = 1. This is also true for all spectra of this chapter depicted in the following.

6

The mathematically experienced reader will find the expression in (4.4) to be unnecessarily complicated as the transformation $$T_{F}\left (\lambda \right )$$ can be written considerably more compactly by using the exponential function in the complex space. It holds that

where Euler’s formula allows for expressing the complex-valued exponential function by sine and cosine,

with the conjugate complex number $$e^{-i\lambda } =\cos \lambda -i\sin \lambda \,,$$ where i denotes the imaginary unit. Instead of burdening the reader with complex numbers and functions, we rather expect him or her to handle the more cumbersome definition from (4.4). By the way, the term “power transfer function” stems from calling F(e −i λ ) alone transfer function of the filter F(L), and $$T_{F}(\lambda ) = \left \vert F(e^{-i\lambda })\right \vert ^{2}$$ being the power thereof.

7

According to Proposition 3.​5 we rule out autoregressive roots on the unit circle, such that A(e −i λ ) ≠ 0, and $$\left \vert A(e^{-i\lambda })\right \vert ^{2}> 0$$. By assumption, | z | = 1 implies A(z) ≠ 0, and here, $$z = e^{-i\lambda }$$ with

8

We use

and sin(π k) = 0 for $$k \in \mathbb{Z}$$.

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_5

5. Long Memory and Fractional Integration

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

5.1 Summary

Below Proposition 3.​5 we saw that the autocorrelation sequence of any stationary ARMA process dies out at exponential rate: | ρ(h) | ≤ c g h , see (3.​14). This is too restrictive for many time series of stronger persistence, which display long memory in that the autocovariance sequence vanishes at a slower rate. In some fields of economics and finance long memory is treated as an empirical stylized fact.¹ Fractional integration as a model for long memory will be presented in this chapter. In the same paper where Granger (1981) introduced the Nobel prize winning concept of cointegration (see Chap. 16) he addressed the idea of fractional integration, too. For an early survey on fractional integration and applications see Baillie (1996).

5.2 Persistence and Long Memory

We have already briefly touched upon the so-called random walk, see Eq. (1.​9). In terms of the difference operator this can be written as $$\varDelta x_{t} =\varepsilon _{t}$$, i.e. the process has to be differenced once to obtain stationarity. Alternatively, the process is given by a cumulation or summation over the shocks, $$x_{t} =\sum _{ j=1}^{t}\varepsilon _{j}$$, see (1.​8), which is the reason to call the process {x t } integrated of order 1, see also Chap. 14 In this section, differencing or integration of order 1 will be extended by introducing non-integer orders of differencing and integration.

Persistence

By persistence we understand how strongly a past shock affects the presence of a stochastic process. We stick to the MA(∞) representation behind the Wold decomposition of a stationary process that we briefly touched upon in Chap. 3,

with $$\{\varepsilon _{t}\}$$ forming a white noise sequence. The impulse responses coefficients {c j } measures the response of x t on a shock j periods ago. With stationary processes, the shocks are transitory in that $$\lim _{j\rightarrow \infty }c_{j} = 0$$. In particular, for stationary ARMA processes we know that the impulse responses die out so fast that they are summable in absolute value, see (3.​2). To model a stronger degree of persistence and long memory , we require a slower convergence to zero. The model of fractional integration of order d will impose the so-called hyperbolic decay rate,

Under d < 1, the sequence {c j } converges to zero. Clearly, the larger d, the stronger is the persistence in that j d−1 dies out more slowly. Hence, the parameter d measures the strength of persistence. Contrary to the exponential case characteristic of ARMA processes, hyperbolic decay is so slow for positive d > 0, that the impulse responses are not summable. In Problem 5.1 we will establish the following convergence result concerning the so-called (generalized) harmonic series, often also called p-series:

(5.1)

Moreover, we will show in Problem 5.2 that exponential decay to zero is faster than hyperbolic one:

In order to illustrate the different decay rates, we display in Figs. 5.1 and 5.2 sequences j d−1 and g j−1, respectively; by construction they all have the value 1 at j = 1.

Fig. 5.1

j d−1 for d = 0. 85, 0. 65, 0. 45, 0. 25

Fig. 5.2

g  j−1 for g = 0. 85, 0. 65, 0. 45, 0. 25

For a process to be stationary, we require the impulse response sequence to be square summable, see Proposition 3.​2. From (5.1) we learn that $$\sum _{j=1}^{\infty }j^{2d-2}$$ is finite if and only if d < 0. 5, which hence turns out to be the stationarity condition for processes with impulse responses {j d−1}.The model of fractional integration, however, does not directly assume $$c_{j} = c\,j^{d-1}$$; rather this power law will hold only true for large j,

where “ ∼ for j → ∞” is to be understood as a proper limit in the following way:

(5.2)

Fractional Differencing and Integration

With the usual difference operator $$\varDelta = \left (1 - L\right )$$, see Example 3.​2, we define fractional differences by binomial expansion²:

Readers not be familiar with binomial series for $$d\notin \mathbb{N}$$ may wish to consult e.g. Trench (2013, Sect. 4.​5). The binomial series results from a Taylor expansion of (1 − z) d about z = 0, hence also called Maclaurin series. The coefficients {π j } are given in terms of binomial coefficients

yielding the recursion

(5.3)

For natural numbers d one has the more familiar finite expansions,

while the expansion in (5.3) holds for non-integer (or fractional) values of d, too. In Problem 5.3 we derive the behavior for large j,

(5.4)

where Γ(⋅ ) is the so-called Gamma function introduced at greater detail below its definition in (5.18) in the Problem section.

Similarly to fractional differences, we may define the fractional integration operator upon inversion,

where the coefficients are given by simply replacing d by − d in (5.3):

(5.5)

The same arguments establishing (5.4) hence show

(5.6)

Fractional integration thus imposes the hyperbolic decay rate discussed above, where the speed of convergence varies with d. From (5.1) we observe with (5.6) that {ψ j } is summable if and only if d < 0, in which case

(5.7)

Further, {ψ j } is square summable if and only if d < 0. 5. These are the ingredients to define a fractionally integrated process.

5.3 Fractionally Integrated Noise

We now apply the above findings to the simplest case of fractional noise (which is short for: fractionally integrated noise), define long memory in the time domain in (5.9), and translate it into the frequency domain.

Fractional Noise and Long Memory

In case of fractionally integrated noise the fractional differencing filter Δ d has to be applied to {x t } in order to obtain white noise $$\{\varepsilon _{t}\}$$ with variance σ ²: $$\varDelta ^{d}x_{t} =\varepsilon _{t}$$. Equivalently, we write after inverting the differences

(5.8)

with {ψ j } from (5.5) being the sequence of impulse response coefficients measuring the effect of a past shock on the presence. As we have discussed above the impulse responses die out the more slowly the larger d is. In that sense we interpret d as measure of persistence or memory. The impulse responses die out so slowly that they are not absolutely summable for positive memory parameter d. Consequently for d > 0, {x t } from (5.8) does not belong to the class of processes with absolutely summable autocovariances characterized in (3.​2), while all stationary ARMA processes belong to this class. Hence, fractional integration models for d > 0 a feature that is not captured by traditional ARMA processes, which we call strong persistence ; it is defined in the time domain by

Consequently, CIR or VR defined in (3.​3) and (4.​6), respectively, do not exist. Even though the MA coefficients are not summable for positive d, the process is still stationary as long as d < 0. 5 because $$\sum _{j}\psi _{j}^{2} <\infty$$ due to (5.6) and (5.1). Further, the process is often called invertible for d > −0. 5 since an autoregressive representation exists that is square summable³:

Note that the existence of an autoregressive representation in the mean square sense does not require square summability; in fact, Bondon and Palma (2007) extend the range of invertibility in the mean square to d > −1. Given the existence of the MA(∞) representation the following properties of fractionally integrated noise are proven in the Problem section.

Proposition 5.1 (Fractional noise, time domain)

For fractionally integrated noise from (5.8) it holds with − 1 < d < 0.5 that

(a)

the variance equals

with Γ(⋅) being defined in (5.18) , and γ(0;d) achieves its minimum for d = 0;

(b)

the autocovariances equal

with

where γ d < 0 if and only if d < 0;

(c)

the autocorrelations ρ(h) = ρ(h;d) grow with d for d > 0.

Let us briefly comment those results. First, since Γ(1) = 1, the minimum variance obtained in the white noise case (d = 0) is of course γ(0; 0) = σ ². Second, from the hyperbolic decay of the autocovariance sequence we observe that γ(h) converges to zero with h as long as d < 0. 5, but for d > 0 so slowly, that we have long memory defined as

(5.9)

In particular, the autocovariances die out the more slowly the larger the memory parameter d is. Obviously, the same feature can be rephrased in terms of autocorrelations. The recursion carries over to the autocorrelations, and Proposition 5.1 (b) yields

For a numerical and graphical illustration see Fig. 5.3. The asymptotic constant γ d has the same sign as d, meaning that in case of long memory the autocovariances converge to zero from above, and vice versa from below zero for d < 0, see again Fig. 5.3. Note, however, that γ d collapses to zero as d → 0, simply meaning that the hyperbolic decay rate does not hold for d = 0. Third, a similar effect that d has on ρ(h) at long lags, holds true for finite h. More precisely, Proposition 5.1 (c) says for each finite h that the autocorrelation grows with d (for d > 0), which reinforces the interpretation of d as measure of persistence or long memory.⁴

Fig. 5.3

ρ(h) from Proposition 5.1 for $$d = 0.45,0.25,-0.25,-0.45$$

The case of negative d results in short memory in that the autocovariances are absolutely summable, which is clear again from the p-series in (5.1). This case is sometimes called antipersistent , the reason for that being

This property translates into a special case of short memory, namely

as we will become obvious from the spectrum at frequency zero.

Long Memory in the Frequency Domain

It is obvious from the definition of the spectrum in (4.​3) that it does not exist at the origin under long memory (d > 0), because the autocovariances are not summable. Still, the previous chapter has been set up sufficiently general to cover long memory, see (4.​1). Given a singularity at frequency λ = 0, one still may determine the rate at which f(λ) goes off to infinity as λ approaches 0. To determine f, we have to evaluate the power transfer function of $$(1 - L)^{-d}$$ from Proposition 4.​1 and obtain⁵

where the trigonometric half-angle formula was used for the second equality:

(5.10)

We hence have the following result.

Proposition 5.2 (Fractional noise, frequency domain)

Under the assumptions of Proposition 5.1 it holds for the spectrum of fractional noise $$x_{t} = (1 - L)^{-d}\varepsilon _{t}$$ that

(5.11)

and

(5.12)

The second statement in Proposition 5.2 is again understood to be asymptotic: Similarly to (5.2) we denote for two function a(x) and b(x) ≠ 0:

(5.13)

Since $$\lim _{x\rightarrow 0}\sin (x)/x = 1$$ we write sin(x) ∼ x for x → 0. Consequently, (5.12) arises from (5.11).

From Proposition 5.2 we learn that long memory (d > 0) translates into a spectral singularity at frequency zero, and the negative slope is the steeper the larger d is. In other words: the longer the memory, the stronger is the contribution of the long-run trend to the variance of the process. The antipersistent case in contrast is characterized by the opposite extreme: f(0) = 0. For an illustration, have a look at Fig. 5.4.

Fig. 5.4

2π f(λ) from Proposition 5.2 for $$d = 0.45,0.25,-0.25,-0.45$$

Example 5.1 (Fractionally Integrated Noise)

Although the fractional noise is dominated by the trend component at frequency zero (strongly persistent) for d > 0, the process is stationary as long as d < 0.5. Consequently, a typical trajectory can not drift off but displays somehow reversing trends. In Fig. 5.5 we see from simulated data that the deviations from the zero line are stronger for d = 0.45 than for d = 0.25. The antipersistent series ($$d = -0.45$$), in contrast, displays an oscillating behavior due to the negative autocorrelation. ■

Fig. 5.5

Simulated fractional noise for $$d = 0.45,0.25,-0.45$$

5.4 Generalizations

On top of long memory as implied by fractional integration for 0 < d < 0. 5, we now want to allow for additional short memory. We assume that Δ d has to be applied to {x t } in order to obtain a short memory process {e t }: $$\varDelta ^{d}x_{t} = e_{t}$$. At the end of this section, the order of integration d will be extended beyond d = 0. 5 to cover nonstationary processes, too. Thus we define general fractionally integrated processes of order d, in short x t  ∼ I(d).⁶

Fractionally Integrated ARMA Processes (ARFIMA)

Since the papers by Granger and Joyeux (1980) and Hosking (1981), it is often assumed that {e t } is a stationary and invertible ARMA(p, q) process, $$A(L)e_{t} = B(L)\varepsilon _{t}$$, with spectrum

see Corollary 4.​1. An ARFIMA(p,d,q) process is defined by replacing $$\varepsilon _{t}$$ in (5.8) by e t , such that

With the expansion from (5.5) one obtains

(5.14)

Under stationarity and invertibility of the ARMA process, the spectrum f e of {e t } is bounded and bounded away from zero everywhere:

(5.15)

The results from Propositions 5.1 and 5.2 carry over to the ARFIMA(p,d,q) process, see also Brockwell and Davis (1991, Sect. 13.​2).

Proposition 5.3 (ARFIMA)

Let the ARMA process with $$A(L)e_{t} = B(L)\varepsilon _{t}$$ be stationary and invertible. Then the ARFIMA process with $$A(L)\varDelta ^{d}x_{t} = B(L)\varepsilon _{t}$$ and − 1 < d < 0.5 is stationary, and it holds that

(a)

the spectral density f(λ) is given as

(b)

the autocovariances satisfy

with γ d from Proposition 5.1 .

Hosking (1981, Thm. 2) and Brockwell and Davis (1991, Thm. 13.2.2) cover only | d | < 0. 5, but their proof carries over to $$-1 <d \leq -0.5$$. Further, they state only $$\gamma (h) \sim C\,h^{2d-1}$$ for some constant C ≠ 0; looking at the details of the proof of Brockwell and Davis (Thm. 13.2.2), however, it turns out that $$C =\gamma _{d}\,f_{e}(0)\,2\pi$$, which of course covers the case of Proposition 5.1, too. In particular, we find again that long memory defined by a non-summable autocovariance sequence translates into a spectral peak at λ = 0. This feature occurs for d > 0.

Semiparametric Models

The parametric assumption that {e t } is an ARMA process is by no means essential for Proposition 5.3 to hold. More generally, we now define a stationary process {e t } to be integrated of order 0, e t  ∼ I(0), if

(5.16)

where b ₀ = 1 and $$\{\varepsilon _{t}\} \sim \mbox{ WN}(0,\sigma ^{2})$$. The absolute summability of {b k } rules out long memory (or d > 0) in that the autocovariances of {e t } are absolutely summable, see Proposition 3.​2; the second condition that the sequence {b k } does not sum up to zero rules out that {e t } is integrated of order d with d < 0, see (5.7). This motivates to call processes from (5.16) integrated of order 0. Consequently, {x t } from (5.14) with {e t } from (5.16) is called integrated of order d, x t  ∼ I(d). The spectrum of {e t } is of course given by Proposition 4.​2, see (4.​5). With f e being the spectrum of the I(0) process, Proposition 5.3 continues to hold without changes, provided 0 < d (see Giraitis et al., 2012, Prop. 3.1.1).

A further question is the behavior of the impulse responses of a general I(d) process without parametric model: Does the decay rate j d−1 of {ψ j } from Δ −d carry over? The answer is almost yes, but mild additional assumptions have to be imposed on {b k } from (5.16). Denote

where the MA coefficients are given by convolution:

Hassler and Kokoszka (2010) prove that a necessary and sufficient condition for

is under long memory

(5.17)

This is a very weak condition satisfied by all stationary ARMA models and most other processes of practical interest. Hassler (2012) proves that this condition remains necessary in the case of antipersistence, d < 0, and establishes a mildly stronger sufficient condition.

The statistical literature often refrains from a fractionally integrated model of the type $$x_{t} = (1 - L)^{-d}e_{t}$$, and directly assumes for the corresponding spectral behavior in a vicinity of the origin:

When it comes to estimation of d, technical smoothness restrictions are imposed on g, including of course the minimum assumption

which is required to identify d.

Nonstationary Processes

The simplest way to define a process that is integrated of a higher order than d < 0. 5 is as follows. Consider a sequence {y t } that has to be differenced once in order to obtain an I(δ) process, $$\varDelta y_{t} = x_{t}$$, x t  ∼ I(δ) or $$y_{t} = y_{t-1} + x_{t}$$. Given a starting value y ₀ = 0, the solution of this difference equation for t ∈ { 1, 2, …, n} is

Since {y t } is given by integration over an I(δ) process, we say that {y t } is integrated of order d, y t  ∼ I(d), with $$d =\delta +1$$. For d ≥ 0. 5, i.e. δ ≥ −0. 5, the process {y t } is necessarily nonstationary. We illustrate this type of nonstationarity or drift by means of an example.

Example 5.2 (Nonstationary Fractional Noise)

The middle graph in Fig. 5.6 displays a realization of a random walk (d = 1). It drifts off from the zero line for very long time spans and crosses only a few times. The I(1.45) process drifts even more pronouncedly displaying a much smoother trajectory than the random walk. The I(0.55) process does not drift as strongly, hitting the zero line much more often. In fact, comparing the I(0.55) series with the I(0.45) case from Fig. 5.5, one can imagine that it may be hard to tell apart stationarity and nonstationarity in finite samples. ■

Fig. 5.6

Nonstationary fractional noise for d = 1. 45, 1. 0, 0. 55

The case of δ = 0 or y t  ∼ I(1) is of particular interest in many financial and economic applications. Hence, one may wish to test whether a process is I(1) or not,

One method to discriminate more specifically between d = 1 and d = 0 is the celebrated test by Dickey and Fuller (1979), see Chap. 15 In a fractionally integrated framework it is more generally possible to decide e.g. whether a process is nonstationary or not,

or whether a process has short memory or not,

Demetrescu, Kuzin, and Hassler (2008) suggested a simple procedure similar to the Dickey-Fuller test, to test for arbitrary values d ₀.

5.5 Problems and Solutions

Problems

5.1 Consider the p-series $$\sum _{j=1}^{J}j^{-p}$$ for J → ∞. Show that the limit is finite if and only if p > 1, see (5.1).

5.2 Show that exponential decay is faster than hyperbolic decay, i.e.

5.3 Show (5.4).

Hint: Use properties of the Gamma function

(5.18)

5.4 Establish the following expression for the autocovariance γ(h) of fractionally integrated noise:

(5.19)

Hint: Use Proposition 4.​1 with (5.11), and apply the following identity from Gradshteyn and Ryzhik (2000, 3.631.8):

5.5 Show Proposition 5.1 (a).

Hint: Use (5.19).

5.6 Show Proposition 5.1 (b).

Hint: Use (5.19).

5.7 Show Proposition 5.1 (c).

5.8 Consider the ARFIMA(0,d,1) model $$\varDelta ^{d}x_{t} = B(L)\,\varepsilon _{t}$$, $$B(L) = 1 + b\,L$$. Show for this special case that the proportionality constant from Proposition 5.3 (b) is indeed $$\gamma _{d}\,f_{e}(0)\,2\pi$$.

Solutions

5.1 We define the p-series of the first J terms,

First, we discuss the case separating the convergence region from the divergent one (p = 1). For convenience choose J = 2 n , such that

Since the lower bound of $$S_{2^{n}}(1)$$ diverges to infinity with n, S J (1) must diverge, too.

Second, consider the case where p < 1, such that j p  < j, or $$j^{-p}> j^{-1}$$. Hence, $$S_{J}(\,p)> S_{J}(1)$$, and divergence of S J (1) implies divergence for p < 1.

Third, for p > 1, we group the terms for $$J = 2^{n} - 1$$ as follows.

We now abbreviate $$g = \frac{1} {2^{p-1}}$$ with 0 < g < 1 since p > 1. Consequently,

where we use the geometric series, see Problem 3.2. Hence, $$S_{2^{n}-1}(\,p)$$ is bounded for every n, while growing monotonically at the same time, which establishes convergence for p > 1. Hence, the proof of (5.1) is complete.

We want to add a final remark. While convergence is ensured for p > 1, an explicit expression for the limit is by no means obvious, and indeed only known for selected values. For example, for p = 2 one has the famous result by Leonhard Euler:

5.2 Since the limit of the ratio of interest is indeterminate, we apply L’Hospital’s rule:

If − 1 < d < 0, the expression $$g^{\,j}/j^{d}$$ is still indeterminate. Application of L’Hospital’s rule twice, however, yields:

This establishes the claim.

5.3 Prior to solving the problem, we review some useful properties of the Gamma function that is often employed to simplify manipulations with binomial expressions, see e.g. Sydsæter, Strøm, and Berck (1999, p.52), and in much greater detail Gradshteyn and Ryzhik (2000, Sect. 8.31), or Rudin (1976, Ch. 8) containing proofs. For integer numbers, Γ coincides with the factorial,

which implies a recursive relation holding in fact in general:

(5.20)

Hence, obviously $$\varGamma (1) =\varGamma (2) = 1$$, and a further value often encountered is $$\varGamma (0.5) = \sqrt{\pi }$$. The recursive relation further yields the rate of divergence at the origin,

which justifies the convention $$\varGamma (0)/\varGamma (0) = 1$$. Finally, we want to approximate the Gamma function for large arguments. Remember Stirling’s formula for factorials,

It generalizes to

for x → ∞, which again has to be read as

Consequently, we have for finite x and y and large integer n that⁷

(5.21)

Now, we turn to establishing (5.4). Repeated application of (5.20) gives

By definition of the binomial coefficients we conclude from (5.3) with $$\varGamma (j + 1) = j!$$ that

where the approximation relies on (5.21). This is the required result.

5.4 With Proposition 4.​1 and (5.11) we compute

where we substituted $$\frac{\lambda }{2} = x$$. Both sin²(x) and cos(2hx) are even symmetric around $$\frac{\pi }{2}$$, such that

Hence, the integration formula 3.631.8. from Gradshteyn and Ryzhik (2000) can be applied with $$\nu = 1 - 2d> 0$$ as long as d < 0. 5 and a = 2h:

Using (5.20) once more, one can show that

Therefore, we finally have

which is (5.19).

5.5 With (5.19) we obtain

where we assumed σ ² = 1 without loss of generality. Instead of the variance, we will equivalently minimize the natural logarithm thereof. We determine as derivative

where the so-called psi function is defined as logarithmic derivative of the Gamma function,

The psi function is strictly increasing for x > 0, which can be shown in different ways, see e.g. Gradshteyn and Ryzhik (2000, 8.362.1). Consequently,

which proves that $$\log \left (\gamma (0;d)\right )$$, and hence γ(0; d), takes on its minimum for d = 0. This solves the problem.

5.6 The recursive relative for γ(h) is obvious with (5.19) and (5.20) at hand.

For h → ∞, the approximation in (5.21) yields

which defines the constant from Proposition 5.1 (b):

The Gamma function is positive for positive arguments and negative on the interval (−1, 0). Hence, the sign of γ d equals the sign of Γ(d) since d < 0. 5, which completes the proof of Proposition 5.1 (b).

5.7 In terms of autocorrelations the recursion from Proposition 5.1 (b) becomes

where the factor f(h; d) is positive for d > 0,

such that ρ(h; d) > 0 since ρ(0; d) = 1. Hence, we have

which is positive by induction since

Hence, ρ(h; d) is growing with d as required.

5.8 The spectral density of the MA(1) component $$e_{t} =\varepsilon _{t} + b\varepsilon _{t-1}$$ is known from Example 4.​3:

We now express x t in terms of a fractional noise called $$y_{t} =\varDelta ^{-d}\varepsilon _{t}$$:

Let γ x (h) and γ y (h) denote the autocovariances of {x t } and {y t }, respectively. It holds that

With the behavior of γ y (h) from Proposition 5.1 it follows

as required.

References

Baillie, R. T. (1996). Long memory processes and fractional integration in econometrics. Journal of Econometrics, 73, 5–59.CrossRef

Bondon, P., & Palma, W. (2007). A class of antipersistent processes. Journal of Time Series Analysis, 28, 261–273.CrossRef

Brockwell, P. J., & Davis, R. A. (1991). Time series: Theory and methods (2nd ed.). New York: Springer.CrossRef

Demetrescu, M., Kuzin, V., & Hassler, U. (2008). Long memory testing in the time domain. Econometric Theory, 24, 176–215.CrossRef

Dickey, D. A., & Fuller, W. A. (1979). Distribution of the estimators for autoregressive time series with a unit root. Journal of the American Statistical Association, 74, 427–431.

Giraitis, L., Koul, H. L., & Surgailis, D. (2012). Large sample inference for long memory processes. London: Imperial College Press.CrossRef

Gradshteyn, I. S., & Ryzhik, I. M. (2000). Table of integrals, series, and products (6th ed.). London/San Diego: Academic Press.

Granger, C. W. J. (1981). Some properties of time series data and their use in econometric model specification. Journal of Econometrics, 16, 121–130.CrossRef

Granger, C. W. J., & Joyeux, R. (1980). An Introduction to long-memory time series models and fractional differencing. Journal of Time Series Analysis, 1, 15–29.CrossRef

Hassler, U. (2012). Impulse responses of antipersistent processes. Economics Letters, 116, 454–456.CrossRef

Hassler, U. (2014). Persistence under temporal aggregation and differencing. Economics Letters, 124, 318–322.CrossRef

Hassler, U., & Kokoszka (2010). Impulse responses of fractionally integrated processes with long memory. Econometric Theory, 26, 1855–1861.

Hosking, J. R. M. (1981). Fractional differencing. Biometrika, 68, 165–176.CrossRef

Maasoumi, E., & McAleer, M. (2008). Realized volatility and long memory: An overview. Econometric Reviews, 27, 1–9.CrossRef

Rudin, W. (1976). Principles of mathematical analyis (3rd ed.). New York: McGraw-Hill.

Sydsæter, K., Strøm, A., & Berck, P. (1999). Economists’ mathematical manual (3rd ed.). Berlin/New York: Springer.CrossRef

Trench, W. F. (2013). Introduction to real analysis. Free Hyperlinked Edition 2.04 December 2013. Downloaded on 10th May 2014 from http://​digitalcommons.​trinity.​edu/​mono/​7.

Footnotes

1

See e.g. the special issue edited by Maasoumi and McAleer (2008) in Econometric Reviews on “Realized Volatility and Long Memory”.

2

For the rest of this chapter we maintain d > −1, which guarantees that {π j } converges to 0 with growing j, making the infinite expansion meaningful.

3

A more technical exposition can be found in Brockwell and Davis (1991, Thm. 13.2.1) or Giraitis, Koul, and Surgailis (2012, Thm. 7.2.1), although they consider only the range | d | < 0. 5.

4

More complicated is the effect of changes in d if d < 0, see Hassler (2014).

5

Readers not familiar with complex numbers, $$i^{2} = -1$$, may skip the following equation, see also Footnote 6 in Chap. 4:

6

The use of ‘ ∼ ’ with a differing meaning from that one in (5.2) should not be a source for confusion.

7

Use $$(1 + x/n)^{n} \rightarrow e^{x}$$.

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_6

6. Processes with Autoregressive Conditional Heteroskedasticity (ARCH)

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

6.1 Summary

In particular in the case of financial time series one often observes a highly fluctuating volatility (or variance) of a series: Agitated periods with extreme amplitudes alternate with rather quiet periods being characterized by moderate observations. After some short preliminary considerations concerning models with time-dependent heteroskedasticity, we will discuss the model of autoregressive conditional heteroskedasticity (ARCH), for which Robert F. Engle was awarded the Nobel prize in the year 2003. After a generalization (GARCH), there will be a discussion on extensions relevant for practice. Throughout this chapter, the innovations or shocks $$\{\varepsilon _{t}\}$$ stand for a pure random process as defined in Example 2.​7.

6.2 Time-Dependent Heteroskedasticity

The heteroskedasticity allowed for here is modeled as time-dependent volatility by¹

(6.1)

where $$\{\sigma _{t}\}$$ is the volatility process being stochastically independent of $$\{\varepsilon _{t}\}$$ by assumption, and $$\{\varepsilon _{t}\}$$ is a pure random process with unit variance. There exist two routes to model the volatility process. The first one, often labeled as processes with stochastic volatility, assumes an unobserved, or latent, process {h t } behind the volatility: $$\sigma _{t} = \mbox{ exp}(h_{t}/2)$$. This implies for the squared data

For an early survey on stochastic volatility processes see Taylor (1994). A second strand in the literature assumes that $$\sigma _{t}$$ depends on observed data, in particular on past observations x t−j . This class of models has been called autoregressive conditional heteroskedasticity (ARCH). ARCH processes are widely spread and successful in practice and will be the focus of attention in the present chapter.

Heteroskedasticity as a Function of the Past

In this chapter the variance function is modeled by the observed past of the process itself:

(6.2)

By plugging in x t−j from (6.1) one obtains:

We will show that the process from (6.1) is a martingale difference sequence. Remember the definition of the information set $$\mathcal{I}_{t-1}$$ generated by the past of x t up to x t−1. Then it holds that²

as $$\varepsilon _{t}$$ is independent of x t−j for j > 0 by construction. With $$\varepsilon _{t}$$ being zero on average, it follows that

which proves (integrability of {x t } assumed) that {x t } is in fact a martingale difference sequence, see below (2.​11). The variance of the martingale difference is determined in the following way (as x t is zero on average,

as $$\sigma _{t}^{2}$$ from (6.2) and $$\varepsilon _{t}$$ (with variance 1 from (6.1)) are stochastically independent. Hence, the following proposition is verified.

Proposition 6.1 (Heteroskedastic Martingale Differences )

Let {x t } be from (6.1)and $$\{\sigma _{t}^{2}\}$$ from (6.2) with $$\mbox{ E}(\sigma _{t}^{2}) < \infty $$ independent of $$\{\varepsilon _{t}\}$$ . Then {x t } is a martingale difference sequence with variance

Let us remember Proposition 2.​2. Due to the martingale difference property it holds that

Hence, the process is serially uncorrelated with expectation zero which would be supposed e.g. for returns. However, the process is generally not independent over time. The (weak) stationarity of {x t } depends on the possibly variable variance; if the variance Var(x t ) is constant, then the entire process is stationary.

Heuristics

Now, the question is how the functional dependence in (6.2) should be specified and parameterized. Heteroskedasticity as an empirical phenomenon has been known to observers on financial markets for a long time. Before ARCH models were introduced, it had been measured by moving a window of width B through the data and averaging over the squares:

For every point in time t one averages over the past preceding B values in order to determine the variance in t. In doing so, we do not center x t−i around the arithmetic mean as we think of returns with E(x t ) = 0 when applying the procedure, cf. Proposition 6.1. With daily observations (with five trading days a week) one chooses e.g. B = 20 which approximately corresponds to a time window of a month. A first improvement of the time-dependent volatility measurement is obtained by using a weighted average where the weights, g i  ≥ 0, are not negative:

(6.3)

Example 6.1 (Exponential Smoothing)

For the weighting function gi one often uses an exponential decay:

Note that the denominator is just defined such that it holds that $$\sum _{i=1}^{B}g_{i} = 1$$. With growing B one furthermore obtains

Inserting the exponentially decaying weights in (6.3), we get the following result for $$B \rightarrow \infty $$:

Now it is an easy exercise to verify the following recursive relation:

(6.4)

We will call $$s_{t}^{2}(\lambda )$$ the exponentially smoothed volatility or variance. In order to be able to calculate it for t = 2,…,n, we need a starting value. Typically, $$s_{1}^{2}(\lambda ) = x_{1}^{2}$$ is chosen which leads to $$s_{2}^{2}(\lambda ) = x_{1}^{2}$$. ■

The ARCH and GARCH processes which are subsequently introduced are models leading to volatility specifications which generalize s t ² and $$s_{t}^{2}(\lambda )$$ from (6.3) and (6.4), respectively.

6.3 ARCH Models

So-called autoregressive conditional heteroskedasticity models can be traced back to Engle (1982). We consider the case of the order q and specify $$\sigma _{t}^{2}$$ from (6.2) as follows:

(6.5)

where it is assumed that

(6.6)

in order to guarantee $$\sigma _{t}^{2} > 0$$. Note that this variance function corresponds to s t ² from (6.3). For $$\alpha _{1} = \cdots =\alpha _{q} = 0$$, however, the case of homoskedasticity is modeled.

Conditional Moments

Given $$x_{t-1},\ldots,x_{t-q}$$, one naturally obtains zero for the conditional expectation as ARCH processes are martingale differences, see Proposition 6.1. For the conditional variance it holds that

as $$\varepsilon _{t}$$ is again independent of x t−j and has a unit variance. Hence, for the variance it conditionally holds that:

which explains the name of the models: The conditional variance is modeled autoregressively (where “autoregressive” means in this case: dependent on the past of the process). Thus, extreme amplitudes in the previous period are followed by high volatility in the present period resulting in so-called volatility clusters. If the assumption of normality of the innovations is added, then the conditional distribution of x t given the past is normal as well:

But, although the original work by Engle (1982) assumed $$\varepsilon _{t} \sim \mbox{ ii}\,\mathcal{N}(0,1)$$, the assumption of normality is not crucial for ARCH effects.

Stationarity

By Proposition 6.1 it holds for the variance that

If the process is stationary, $$\mbox{ Var}(x_{t}) = \mbox{ Var}(x_{t-j})$$, j = 1, …, q, then its variance results as

For a positive variance expression, this requires necessarily (due to α ₀ > 0):

In fact, this condition is sufficient for stationarity as well. In Problem 6.1 we therefore show the following result.

Proposition 6.2 (Stationary ARCH)

Let {x t } be from (6.1) and $$\{\sigma _{t}^{2}\}$$ from (6.5) with (6.6) . The process is weakly stationary if and only if it holds that

Correlation of the Squares

We define $$e_{t} = x_{t}^{2} -\sigma _{t}^{2}$$. Due to Proposition 6.1 the expected value is zero: E(e t ) = 0. Adding x t ² to both sides of (6.5), one immediately obtains

From this we learn that an ARCH(q) process implies an autoregressive structure for the squares {x t ²}. The serial dependence of an ARCH process originates from the squares of the process. Because of α i  ≥ 0, x t ² and x t−i ² are positively correlated which again allows to capture volatility clusters.

In Figs. 6.1 and 6.2 ARCH(1) time series of the length 500 are simulated. For this purpose pseudo-random numbers $$\{\varepsilon _{t}\}$$ are generated as normally distributed and α ₀ = 1 is chosen. The effect of α ₁ is now quite obvious. The larger the value of this parameter, the more obvious are the volatility clusters. Long periods with little movement are followed by shorter periods of vehement, extreme amplitudes which can be negative as well as positive. These volatility clusters become even more obvious in the respective lower panel of the figures, in which the squared observations {x t ²} are depicted. Because of the positive autocorrelation of the squares, small amplitudes tend again to be followed by small ones while extreme observations appear to follow each other.

Fig. 6.1

ARCH(1) with α ₀ = 1 and α ₁ = 0. 5

Fig. 6.2

ARCH(1) with α ₀ = 1 and α ₁ = 0. 9

Skewness and Kurtosis

In the first section of the chapter on basic concepts from probability theory we have defined the kurtosis by means of the fourth moment of a random variable and we have denoted the corresponding coefficient by γ ₂. For γ ₂ > 3 the density function is more “peaked” than the one of the normal distribution: On the one hand the values are more concentrated around the expected value, on the other hand there occur extreme observations in the tail of the distribution with higher probability (“fat-tailed and highly peaked”). For stationary ARCH processes with Gaussian innovations ($$\varepsilon _{t} \sim \mbox{ ii}\,\mathcal{N}(0,1)$$) it holds that the kurtosis exceeds 3 (provided it exists at all):

The corresponding derivation can be found in Problem 6.2. Due to this excess kurtosis, ARCH is generally incompatible with the assumption of an unconditional Gaussian distribution.

We define the skewness coefficient γ ₁ similarly to the kurtosis by the third moment of a standardized random variable. The skewness coefficient of ARCH models depends on the symmetry of $$\varepsilon _{t}$$. If this innovation is symmetric, then it follows that $$\mbox{ E}(\varepsilon _{t}^{3}) = 0$$. Hence, γ ₁ = 0 follows for the corresponding ARCH process (due to independence of $$\sigma _{t}$$ and $$\varepsilon _{t}$$):

Thereby it was only used that $$\varepsilon _{t}$$ is symmetrically distributed.

Example 6.2 (ARCH(1))

In particular for a stationary ARCH(1) process with $$\alpha _{1}^{2} < \frac{1} {3}$$ and Gaussian innovations $$\varepsilon _{t}$$ the kurtosis is finite and it results as (see Problem 6.3):

For $$\alpha _{1}^{2} \geq \frac{1} {3}$$ there occur extreme observations with a high probability such that the kurtosis is no longer constant. Consider a stationary ARCH(1) process (α ₁ < 1) with $$\alpha _{1}^{2} = 1/3$$. Under this condition one has for μ 4,t = E(xt ⁴) with $$\mbox{ E}(x_{t-1}^{2}) =\alpha _{0}/(1 -\alpha _{1})$$ assuming Gaussianity:

where the constant c is appropriately defined. Continued substitution yields thus

Hence, we observe that the kurtosis grows linearly over time if 3 α ₁ ² = 1. ■

6.4 Generalizations

Some extensions of the ARCH model having originated from empirical features of financial data will be covered in the following.

GARCH

In practice, with many financial series it can be observed that the correlation of the squares reaches far into the past. Therefore, for an adequate modeling a large q is needed, i.e. a large number of parameters. A very economical parametrization, however, is allowed for by the GARCH model.

Generalized ARCH processes of the order p and q were introduced by Bollerslev (1986) and are defined by their volatility function

(6.7)

The result process is abbreviated as GARCH(p, q). In addition to the parameter restrictions from (6.6) it is required that

(6.8)

Jointly, these restrictions are clearly sufficient for $$\sigma _{t}^{2} > 0$$ but stricter than necessary. Substantially weaker assumptions were derived by Nelson and Cao (1992).

We adopt the stationarity conditions for GARCH models from Bollerslev (1986, Theorem 1). The resulting variance will be determined in Problem 6.4. Thus, we obtain the following results.

Proposition 6.3 (Stationary GARCH)

Let {x t } be from (6.1) and $$\{\sigma _{t}^{2}\}$$ from (6.7) with (6.6) and (6.8) . The process is weakly stationary if and only if

Then it holds for the variance that

It can be shown that the stationary GARCH process can be considered as an ARCH($$\infty $$) process. Under the conditions from Proposition 6.3 it holds that (see Problem 6.5):

(6.9)

Thus, the GARCH process allows for modeling an infinitely long dependence of the volatility on the past of the process itself with only p + q parameters although this dependence decays with time (i.e. γ i  → 0 for $$i \rightarrow \infty $$). The fact that GARCH can be considered as ARCH($$\infty $$) has the nice consequence that results for stationary ARCH processes also hold for GARCH models. In particular, GARCH models are again special cases of processes with volatility (6.2) and therefore examples of martingale differences, i.e. Proposition 6.1 holds true. If we assume a Gaussian distribution of $$\{\varepsilon _{t}\}$$, it follows, just as for the ARCH(q) process of finite order, that the skewness is zero and that the kurtosis exceeds the value 3.

Example 6.3 (GARCH(1,1))

Consider the GARCH(1,1) case more explicitly. It is by far the most frequently used GARCH specification in practice. Continued substitution shows under the assumption of stationarity that (α ₁ +β ₁ < 1):

Hence, we have an explicit ARCH($$\infty $$) representation of GARCH(1,1). Assuming that

the kurtosis is defined. In Problem 6.6 we show (with Gaussian distribution of $$\{\varepsilon _{t}$$}):

Furthermore one shows by

the equation

with

The GARCH(1,1) process {xt} therefore corresponds to an ARMA(1,1) structure of the squares {xt ²}. ■

In Figs. 6.3 and 6.4 the influence of the sum of the parameters α ₁ +β ₁ is illustrated by means of simulated GARCH(1,1) observations. We therefore fix α ₀ = 1 and α ₁ = 0. 3 and vary β ₁ in such a way that stationarity is ensured. The larger β ₁ (and therefore the sum of α ₁ +β ₁), the more pronounced is the change from quiet periods with little or, in absolute value, moderate amplitudes to excited periods in which extreme amplitudes follow each other. Again, this pattern of volatility becomes particularly apparent with the serially correlated squares in the lower panel, respectively.

Fig. 6.3

GARCH(1,1) with α ₀ = 1, α ₁ = 0. 3 and β ₁ = 0. 3

Fig. 6.4

GARCH(1,1) with α ₀ = 1, α ₁ = 0. 3 and β ₁ = 0. 5

IGARCH

Considering the volatility of GARCH(1,1),

we are reminded of s t ²(α) from (6.4). The difference being that α ₀ = 0, and it holds that $$\alpha _{1} +\beta _{1} = 1$$ (i.e. $$\alpha _{1} = 1-\lambda$$ and $$\beta _{1} =\lambda$$, respectively). Models with such a restriction violate the stationarity condition (α ₁ +β ₁ < 1). This can be shown when forming the expected value of $$\sigma _{t}^{2}$$ with $$x_{t-1}^{2} =\sigma _{ t-1}^{2}\,\varepsilon _{t-1}^{2}$$:

With $$\alpha _{1} +\beta _{1} = 1$$ one obtains

In other words: The expectations for the increments of the volatility are positive for every point in time, modeling a volatility expectation which tends to infinity with t. This idea was generalized in literature. With

one talks about integrated GARCH processes (IGARCH) since Engle and Bollerslev (1986). This is a naming which becomes more understandable in the chapter on integrated processes.

In Fig. 6.5, an IGARCH(1,1) process ($$\alpha _{1} +\beta _{1} = 1$$) was simulated according to the scheme from Figs. 6.3 and 6.4. In comparison to the previous figures, in this case we find considerably more extreme volatility clusters which, however, are not exaggerated. The kind of depicted dynamics in Fig. 6.5 can be frequently observed in financial practice.

Fig. 6.5

IGARCH(1,1) with α ₀ = 1, α ₁ = 0. 3 and β ₁ = 0. 7

GARCH-M

We talk about “GARCH in mean”³ (GARCH-M) if the volatility term influences the (mean) level of the process. In order to explain this with regard to contents, we think of risk premia: For a high volatility of an investment (high-risk), a higher return is expected, on average. In equation form we write this down as follows:

(6.10)

where {u t } is a GARCH process:

(6.11)

Therefore, in this case the mean function μ t is set to $$\theta \,\sigma _{t}$$. In some applications it has proved successful to model the risk premium as a multiple of the variance instead of modeling it by the standard deviation ($$\theta \,\sigma _{t}$$):

For both mean functions the GARCH-M process {x t } is no longer free from serial correlation for $$\theta > 0$$; it is no longer a martingale difference sequence.

In Fig. 6.6 a GARCH-M series was generated as in (6.11). The volatility cluster can be well identified in the lower panel of the squares.The effect of $$\theta = 1$$ becomes apparent in the upper panel: In the series of x t local, reversing trends can be spotted. Upward trends involve a high volatility, whereas quiet periods are marked by a decreasing or lower level.

Fig. 6.6

GARCH(1,1)-M from (6.11) with α ₀ = 1, α ₁ = 0. 3 and β ₁ = 0. 5

EGARCH

We talk about exponential GARCH when the volatility is modeled as an exponential function of the past squares x t−i ². This suggestion originates from Nelson (1991) and was made in order to capture the asymmetries in the volatility.⁴ It is observed that decreasing stock prices (negative returns) tend to involve higher volatilities than increasing ones. This so-called leverage effect is not captured by ordinary GARCH models.

In EViews the variance for EGARCH is calculated as follows:

where $$\varepsilon _{t-j}$$ is defined as

For γ j  = 0 the sign is not an issue. However, for γ j  < 0 in the negative case

has a stronger effect on $$\log \sigma _{t}^{2}$$ than in the positive case

Note that for EGARCH the expression $$\sigma _{t}^{2}$$ is without parameter restrictions always positive by construction. Applying the exponential function, it results that

Example 6.4 (EGARCH (1,1))

Again, as special case we treat the situation with $$p = q = 1$$,

or after applying the exponential function:

In this case it is again shown that for γ ₁ < 0 the leverage effect is modeled in such a way that negative observations have a larger volatility effect than positive observations of the same absolute value. ■

In Fig. 6.7 a realization of a simulated EGARCH(1,1) process is depicted. The “leverage parameter” is $$\gamma _{1} = -0.5$$. When the graphs of the squared and the original observations are compared, it can be detected that the most extreme amplitudes are in fact negative. Furthermore, it can be observed that periods with predominantly negative values are characterized by a high volatility.

Fig. 6.7

EGARCH(1,1) with ω = 1, α ₁ = 0. 3, β ₁ = 0. 5 and $$\gamma _{1} = -0.5$$

YAARCH

The works of Engle (1982) and Bollerslev (1986) have set the stage for a downright ARCH industry. A large number of generalizations and extensions has been published and applied in practice. Most of these versions were published under more or less appealing acronyms. When Engle (2002) balanced the books after 20 years of ARCH, he added with some irony another acronym: YAARCH standing for Yet Another ARCH. There is no end in sight for this literature.

6.5 Problems and Solutions

Problems

6.1 Prove Proposition 6.2.

Hint: According to Engle (1982, Theorem 2) the process is stationary if and only if it holds that

(6.12)

with $$\alpha (z):= 1 -\alpha _{1}\,z -\ldots -\alpha _{q}\,z^{q}$$.

6.2 Show that the kurtosis γ ₂ of an ARCH process exceeds the value 3. Assume a Gaussian distribution of $$\{\varepsilon _{t}\}$$ and $$\mbox{ E}(\sigma _{t}^{4}) < \infty $$.

6.3 Calculate the kurtosis of a stationary ARCH(1) process as given in Example 6.2 for the case that it exists. Assume a Gaussian distribution of $$\{\varepsilon _{t}\}$$.

6.4 Assume {x t } to be a stationary GARCH process. Determine the variance expression from Proposition 6.3.

6.5 Assume {x t } to be a stationary GARCH process with (6.6) and (6.8). Determine the ARCH($$\infty $$) representation from (6.9).

6.6 Calculate the kurtosis of a stationary GARCH(1,1) process as given in Example 6.3 for the case that it exists. Assume a Gaussian distribution of $$\{\varepsilon _{t}\}$$.

Solutions

6.1 We have to show the equivalence of (6.12) and the condition from Proposition 6.2, given (6.6). This condition can also be written as α(1) > 0. Hence, we have to prove the equivalence:

We proceed in two steps.

“$$\Rightarrow $$”: Under the condition by Engle (1982) it holds that

Due to α ₀ > 0 it immediately follows that α(1) ≥ 0. The case α(1) = 0, however, is due to (6.12) excluded, such that α(1) > 0 can be concluded.

“$$\Leftarrow $$”: For a root z of α(z) it holds that:

By the triangle inequality and applying (6.6) it follows that:

where the assumption was used for the last inequality. Therefore, we have shown for a root z of α(z) that it holds that $$\max _{j}\,\vert z^{j}\vert > 1$$ and thus | z |  > 1.Hence, the proof is completed.

6.2 From Example 2.​4 we adopt due to the Gaussian distribution

Therefore, in a first step it follows due to the independence of $$\sigma _{t}$$ and $$\varepsilon _{t}$$:

Hence, because of E(x t ) = 0 and Proposition 6.1 the kurtosis of x t results as:

The usual variance decomposition, see Eq. (2.​1),

yields

Hence, the claim is verified: γ ₂ ≥ 3.

6.3 As $$\sigma _{t}$$ and $$\varepsilon _{t}$$ are stochastically independent, it holds that

whereby the k-th central moment is given, as {x t } is a martingale difference sequence with zero expectation. On the assumption of a standard normally distributed random process one obtains

This implies for the ARCH(1) process that the skewness is zero due to E(x t ³) = 0. 

In order to determine the kurtosis, we first observe that the fourth moment is constant under the condition 3α ₁ ² < 1. To that end define μ 4, t and use $$\mu _{2} = \mbox{ Var}(x_{t}) = \frac{\alpha _{0}} {1-\alpha _{1}}$$:

where the constant c is defined appropriately. Infinite substitution yields

With a constant μ ₄ (and μ ₂) one obtains

or

From this it follows that

Of course, these transformations were only possible for 1 − 3α ₁ ² > 0. Hence, this is the condition for a finite, constant kurtosis.

6.4 We use the fact that $$\sigma _{t}$$ from (6.7) is again independent of $$\varepsilon _{t}$$. This can be shown by substitution of $$\sigma _{t-j}^{2}$$ and x t−i ² according to (6.1). Thus, as in Proposition 6.1, for stationarity and for arbitrary points in time, it holds that:

Hence, by forming the expected value we obtain from (6.7):

Therefore, we can solve

as claimed.

6.5 We define the lag polynomial β(L) with

Hence, it holds that

By assumption

In Problem 6.1 we have shown that this is equivalent to

This is in turn the condition of invertibility known from Proposition 3.​3 which guarantees a causal, absolutely summable series expansion with coefficients {c j }:

By comparison of coefficients one obtains from

as usual

Thus, the inversion of β(L) yields:

where γ i , i > 0, results by convolution of

The non-negativity and summability of {α k } and {c j } is conveyed to the series {γ i }. Hence, the proof is complete.

6.6 As for the ARCH(1) case it holds that

Applying $$\mbox{ E}\left (x_{t}^{2}\right ) = \mbox{ E}\left (\sigma _{t}^{2}\right ) = \frac{\alpha _{0}} {1-\alpha _{1}-\beta _{1}}$$ yields:

As for the ARCH(1) case one has to show that $$\mbox{ E}\left (\sigma _{t}^{4}\right )$$ turns out to be constant under stationarity and the condition $$1 - 2\,\alpha _{1}^{2} - (\alpha _{1} +\beta _{1})^{2} > 0$$. We omit this step here and take $$\mbox{ E}\left (\sigma _{t}^{4}\right ) = \mbox{ E}\left (\sigma _{t-1}^{4}\right )$$ for granted. It then holds that:

From this it follows that

This is in accordance with the claim.

References

Bollerslev, T. (1986). Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics, 31, 307–327.CrossRef

Engle, R. F. (1982). Autoregressive conditional heteroskedasticity with estimates of the variance of U.K. inflation. Econometrica, 50, 987–1008.

Engle, R. F. (2002). New frontiers for ARCH models. Journal of Applied Econometrics, 17, 425–446.CrossRef

Engle, R. F., & Bollerslev T. (1986). Modelling the persistence of conditional variances. Econometric Reviews, 5, 1–50.CrossRef

Engle, R. F., Lilien, D. M., & Robins, R. P. (1987). Estimating time-varying risk premia in the term structure: the ARCH-M model. Econometrica, 55, 391–407.CrossRef

Nelson, D. B. (1991). Conditional heteroskedasticity in asset returns: A new approach. Econometrica, 59, 347–370.CrossRef

Nelson, D. B., & Cao, Ch. Q. (1992). Inequality constraints in the univariate GARCH model. Journal of Business & Economic Statistics, 10, 229–235.

Taylor, S.J. (1994). Modeling stochastic volatlity: A review and comparative study. Mathematical Finance, 4, 183–204.CrossRef

Footnotes

1

The following equation could be extended by a mean function, e.g. of a regression-type,

or

We restrict our exposition and concentrate on modeling volatility exclusively, although in practice time-dependent heteroskedasticity is often found with regression errors.

2

When conditioning on $$\mathcal{I}_{t-1}$$, one often writes $$\mbox{ E}\left (\cdot \,\vert \,x_{t-1},x_{t-2},\cdots \,\right )$$ instead of $$\mbox{ E}\left (\cdot \,\vert \,\mathcal{I}_{t-1}\right )$$.

3

Originally, the ARCH-M model was proposed by Engle, Lilien, and Robins (1987).

4

We do not exactly present Nelson’s model but a slightly modified implementation which is used in the software package EViews.

Part II
Stochastic Integrals

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_7

7. Wiener Processes (WP)

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

7.1 Summary

The Wiener process (or the Brownian motion) is the starting point and the basis for all the following chapters.¹ This is why we will consider this process more explicitly. It is a continuous-time process having as prominent a position in stochastic calculus as the Gaussian distribution in statistics. After introducing its defining properties intuitively, we will discuss important characteristics in the third section. Examples derived from the Wiener process will conclude the exposition.

7.2 From Random Walk to Wiener Process

We consider a nonstationary special case of the AR(1) process and thereby try to arrive at the Wiener process which is the most important continuous-time process in the fields of our applications.

Random Walks

The cumulation of white noise is labeled random walk ,

Obviously, it holds that

In other words: The random walk results as an AR(1) process for the parameter value a = 1 and with the starting value zero²

As the process is nonstationary,

it cannot have an infinitely long past, i.e. the index set is finite, $$\mathbb{T} =\{ 1,2,\ldots,n\}$$.

In a way, the random walk models the way home of a drunk who at a point in time t turns to the left or to the right by chance and uncorrelated with his previous path. Put more formally: The random walk is a martingale. We briefly want to convince ourselves of this fact. By substitution the AR(1) process yields

Therefore, for s > 0 it holds that

where $$\mathcal{I}_{t}$$ again denotes the information set of the AR(1) process. Thus, the martingale condition (2.​11) for AR(1) processes is fulfilled if and only if a = 1. The second martingale condition, $$\mbox{ E}(\vert x_{t}\vert ) < \infty \,,$$ is given as $$\sigma ^{2} < \infty $$ and hence $$\mbox{ E}(x_{t}^{2}) = t\,\sigma ^{2} < \infty $$.

Example 7.1 (Discrete-Valued Random Walk)

Let the set of outcomes contain only two elements (e.g. coin toss: heads or tails),

with probabilities $$\mbox{ P}\left (\{\omega _{1}\}\right ) = \frac{1} {2} = \mbox{ P}\left (\{\omega _{0}\}\right )$$. Let $$\{\varepsilon _{t}\}$$ be a white noise process assigning the numerical values 1 and − 1 to the events,

For every point in time, this induces the probabilities

Then, for expectation and variance it is immediately obtained:

For t = 1,…,n, the corresponding random walk $$x_{t} =\sum _{ j=1}^{t}\varepsilon _{j}$$ can only take on the countably many values { − n,−n + 1,…,n − 1,n} and is therefore also called discrete-valued. ■

Example 7.2 (Continuous-Valued Random Walk)

If $$\{\varepsilon _{t}\}$$ is a Gaussian random process,

then, obviously, the random walk based thereon is also Gaussian, where the variance grows linearly with time:

In this case, {xt} is a continuous random variable by assumption and hence this random walk is also called continuous-valued. ■

Wiener Process

At this point, the continuous-time Wiener process will not yet be defined rigorously, but we will approach it intuitively step by step. In order to do so, we choose the index set $$\mathbb{T} = [0,1]$$ with the equidistant, disjoint partition

Now, the random walk is multiplied by the factor $$1/\sqrt{n}$$ and expanded into a step function X n (t). Interval by interval, we define as continuous-time process:

(7.1)

In addition, we assume $$\varepsilon _{t} \in \{-1,1\}$$ and set for t = 1

For t = 0, i.e. i = 1 in (7.1), we follow the convention that a sum equals zero if the upper summation limit is smaller than the lower one, which is why X n (0) = 0 begins in the origin. Apparently, X n (t) is a constant step function on an interval of the length 1∕n, respectively; if X n (t) was only observed at the jump discontinuities, a time-discrete random walk would be obtained. Being dependent on the choice of n (i.e. the fineness of the partition), the process X n (t) is indexed accordingly.

If $$\varepsilon _{t}$$ is from Example 7.1, i.e. $$\vert \varepsilon _{t}\vert = 1$$, this means that each individual step of the step function has the height $$1/\sqrt{n}$$ in absolute value. Hence, X n (t) only takes on values from

Therefore, X n (t) is a continuous-time but discrete-valued process.

Now, the starting point for the Wiener process is the step function X n (t) with $$\varepsilon _{t}$$ from Example 7.1. The number of the steps obviously depends on n which indicates the fineness of the partition of the unit interval. Simultaneously, the step height of the steps with n −0. 5 becomes flatter, the finer it is partitioned. Note that, due to this fact, the range becomes finer and finer and larger and larger as n grows. Hence, with n growing, X n (t) becomes “more continuous”, in the sense that the step heights n −0. 5 turn out to be smaller; simultaneously, the jump discontinuities move together more closely (the steps of the width 1∕n get narrower) such that X n (t) can take on more and more possible values. In the limit $$(n \rightarrow \infty )$$ a process named after Norbert Wiener is obtained which we will always denote by W in the following:

where “$$\Rightarrow $$” denotes a mode of convergence which will be clarified in Chap. 14 Intuitively speaking, it holds that for each of the uncountably many points in time t the function X n (t) converges in distribution to W(t) just at this point. The transition of discrete-time and discrete-valued step functions from (7.1) to the Wiener process (for n growing) is illustrated in Fig. 7.1.³

The Wiener process W(t) as a limit of X n (t) is continuous-valued with range $$\mathbb{R} = (-\infty,\,\infty )$$ and of course it is continuous-time with t ∈ [0, 1]. Furthermore, the Wiener process is a Gaussian process (normally distributed) which is not that surprising. As, due to the central limit theorem for $$n \rightarrow \infty $$, it holds for the standardized sum of uncorrelated random variables $$\{\varepsilon _{j}\}$$ (whose variance is one and whose expectation is zero) that it tends to a standard normal distribution:

(7.2)

Here, “$$\stackrel{d}{\rightarrow }$$” denotes the usual convergence in distribution ; cf. Sect. 8.​4 As X n (1) tends to W(1) at the same time, the Wiener process has to be a standard normally distributed random variable at t = 1. After giving a formal definition, we will again bridge the gap from the Wiener process to X n (t).

Formal Definition

The Wiener process (WP) W(t), t ∈ [0, T], is defined by three assumptions. Put into words, they read: It is a process with starting value zero and independent, normally distributed, stationary increments. These assumptions are to be concretized and specified. Hence, the Wiener process is defined by:

(W1)

The starting value is zero with probability one, P(W(0) = 0) = 1;

(W2)

non-overlapping increments W(t ₁) − W(t ₀), …, W(t n ) − W(t n−1), with 0 ≤ t ₀ ≤ t ₁ ≤ … ≤ t n , are independent for arbitrary n;

(W3)

the increments follow a Gaussian distribution with the variance equalling the difference of the arguments, W(t) − W(s) $$\sim $$ $$\mathcal{N}(0,\,t - s)$$ with 0 ≤ s < t.

Note that the variance of the increments does not depend on the point in time but only on the temporal difference. Furthermore, the covariance of non-overlapping increments is zero due to the independence and the joint distribution results as the product of the marginal distributions. Hence, the joint distribution of non-overlapping increments is multivariate normal. If all increments are measured over equidistant constant time intervals, t i − t i−1 = const, then the variances are identical. Therefore, such a series of increments is (strictly) stationary.

Although the WP is defined by its increments, they translate into properties of the level. Obviously, the first and the third property⁴ imply

(7.3)

i.e. the Wiener process is clearly a stochastic function being normally distributed at every point in time with linearly growing variance t. More precisely, the WP is even a Gaussian process in the sense of the definition from Chap. 2 The autocovariances being necessary for the complete characterization of the multivariate normal distribution $$\left (W(t_{1}),\,\ldots,\,W(t_{n})\right )'$$, are determined as follows (see Problem 7.3):

(7.4)

The Wiener process, which here will always be denoted by W, is for us a special case of the more general Brownian motion.⁵ So to speak, it takes over the role of the standard normal distribution, and by multiplication with a constant a general Brownian motion is obtained as

The assumptions (W1) to (W3) seem very natural if the WP is accepted as a limit of X n (t) from (7.1). For this process it holds by construction that

X n (t) = 0 for t ∈ [0, 1∕n),

e.g. the increments

and

are uncorrelated (or even independent if $$\{\varepsilon _{t}\}$$ is a pure random process),

X n (1) − X n (0) is approximately normally distributed due to (7.2).

The three properties (W1) to (W3) just reflect the properties of the step function X n (t).

7.3 Properties

We have already come to know some properties of the WP, for example its autocovariance structure and the Gaussian distribution. For the understanding and handling of Wiener processes further properties are important.

Pathwise Properties

Loosely speaking, it holds that the Brownian motion is everywhere (i.e. for all t) continuous in terms of conventional calculus⁶; however, it is nowhere differentiable. These are pathwise properties, i.e. for a given ω ₀, W(t) = W(t; ω ₀) can be regarded as a function which is continuous in t but which is nowhere differentiable. This is a matter of properties being mathematically rather deep but which can be made heuristically plausible at least. Concerning this, let us consider

For h → 0 the given Gaussian distribution degenerates to zero suggesting continuity: W(t + h) − W(t) ≈ 0 for h ≈ 0. Analogously, we observe a difference quotient whose variance tends to infinity for h → 0,

which suggests that a usual derivative does not exist. Related to contents, this means that it is not possible to add a tangent line to W(t) which would allow for approximating W(t + h) for an ever so small h. Three simulated paths of the WP in Fig. 7.2 illustrate these properties.

Fig. 7.2

Simulated paths of the WP on the interval [0,1]

Markov and Martingale Property

In the previous section, we have learned that the random walk is a martingale. For the WP as a continuous-time counterpart, a corresponding result can be obtained (where $$\mathcal{I}_{t} =\sigma \left (W(r),\,r \leq t\right )$$ contains all the information about the past up to t):

The WP satisfies the Markov property (2.​9) as well. In order to show this, we use the fact that the increment W(t + s) − W(t) for s > 0 is independent of the information set $$\mathcal{I}_{t}$$ due to (W2). Hence, for W(t) = v it holds that:

At the same time it holds again due to independence that:

which just verifies the Markov property.

Scale Invariance

The Wiener process is a function being Gaussian for every point in time t with expectation zero and variance t. However, time can be measured in minutes, hours or other units. If the time scale is blown up or squeezed by the factor $$\sigma > 0$$, then it holds that

The same distribution is obtained for the $$\sqrt{\sigma }$$-fold of the Wiener process:

That is why the Wiener process is called scale-invariant (or self-similar). Hence, $$W(\sigma \,t)$$ and $$\sqrt{\sigma }\,W(t)$$ are equal in distribution, which we formulate as

(7.5)

as well. Such an equality in distribution is to be handled with care and by no means to be confused with ordinary equality. Naturally, it does not hold that, e.g. the double of W(t) is equal to the value at the point in time 4t:

In other words: Scale invariance is a distributional and not a pathwise property.

Up to this point, it has not been emphasized that the Wiener process is nonstationary. This can already be noted in (7.4) as for s > 0 the covariance Cov(W(t), W(t + s)) = t does not result as dependent on the temporal distance s, but as dependent on the point in time t itself. As we have said, the increments of the WP from (W2), however, are multivariate normal with expectations and covariances of zero and variances which are not affected by a shift on the time axis. The trending behavior of the nonstationary Wiener process will now be clarified by two propositions.

Hitting Time

Let T b be the point in time at which the WP attains (or hits) a given value b > 0 for the first time.⁷ By variable transformation it is shown that this random variable has the distribution function (see Eq. (7.14) in Problem 7.5)

Thereby statement (a) from the following proposition is proved; statement (b) is obtained by means of the corresponding density function (see Problem 7.5).

Proposition 7.1 (Hitting Time)

For the hitting time T b , where the WP hits b > 0 for the first time, it holds that:

(a)

$$P(T_{b} > t)\ \rightarrow \ 0\ $$ for $$t \rightarrow \infty $$ ;

(b)

E(T b ) does not exist.

The result T b  > t is tantamount to the fact that W(s) has not attained the value b up to t:

Laxly formulated this proposition implies that, paradoxically, (a) sooner or later, the WP exceeds every value with certainty; (b) on average, this takes infinitely long: $$E(T_{b}) = \infty $$.

Zero Crossing

Next, let p(t ₁, t ₂) with $$0 < t_{1} < t_{2}$$ be the probability of the WP hitting the zero line at least once between t ₁ and t ₂ (even if not necessarily crossing it). We then talk about a zero crossing. The following proposition states how to calculate it. For a proof see e.g. Klebaner (2005, Theorem 3.25).

Proposition 7.2 (Arcus Law)

The probability of a zero crossing between t ₁ and t ₂ , 0 < t ₁ < t ₂ , equals

where $$\arctan$$ denotes the inverse of the tangent function $$\tan = \frac{\sin }{\cos }$$ .

It is interesting to fathom out the limiting cases of Proposition 7.2. From the shape of the inverse function of the tangent function it results that

Hence, substantially, for $$t_{2} \rightarrow \infty $$ it follows that the probability of attaining the zero line tends to one; for $$t_{2} \rightarrow t_{1}$$, however, it naturally converges to zero.

In the literature, an equivalent formulation of the Arcus Law is found:

The equivalence is based on the formula

see e.g. Gradshteyn and Ryzhik (2000, 1.624-8), where “$$\arccos$$” stands for the inverse of the cosine function.

7.4 Functions of Wiener Processes

When applying stochastic calculus, one is often concerned with processes derived from the Brownian motion. In this section, some of these will be covered and illustrated graphically. We simulate processes on the interval [0, 1]; for this purpose, the theoretically continuous processes are calculated at 1000 sampling points and plotted. The resulting graphs are based on pseudo-random variables. Details regarding the simulation of stochastic processes are treated in Chap. 12 on stochastic differential equations.

Brownian Motion B(t)

In Fig. 7.3 a path of a WP and a Brownian motion based thereon with only half the standard deviation,

are depicted. Obviously, the one graph is just half of the other.

Fig. 7.3

WP and Brownian motion with $$\sigma = 0.5$$

Brownian Motion with Drift X(t) = μ t +σ W(t)

Here it holds that both the expectation and the variance grow linearly with t:

In Fig. 7.4 the WP from Fig. 7.3 and the process based thereon with drift are depicted. The drift parameter is μ = 2 for $$\sigma = 1$$, and the expectation function 2 t is also displayed.

Fig. 7.4

WP and Brownian motion with drift, where $$\sigma = 1$$

Brownian Bridge X(t) = B(t) − t B(1)

This process is based on the Brownian motion, $$B(t) =\sigma \, W(t)$$, and fundamentally, it is only defined for t ∈ [0, 1]. The name comes from the fact that the starting and the final value are equal with probability one by construction: X(0) = X(1) = 0. One can verify easily that (see Problem 7.6):

(7.6)

Hence, for t ∈ (0, 1] it holds that Var(X(t)) < Var(B(t)). This is intuitively clear: With being forced back to zero, the Brownian bridge has to exhibit less variability than the Brownian motion. This is also shown in Fig. 7.5 for $$\sigma = 1$$.

Fig. 7.5

WP and Brownian bridge ($$\sigma = 1$$)

Reflected Wiener Process X(t) =  | W(t) | 

Due to $$W(t) \sim \mathcal{N}(0,t)$$, for the distribution function it is elementary to obtain (see Problem 7.7):

Note that one integrates over twice the density of a Gaussian random variable with expectation zero. Therefore it immediately holds that

(7.7)

Expectation and variance of the reflected Wiener process can be determined from the corresponding density function. They read (see Problem 7.7):

(7.8)

As the reflected Wiener process cannot become negative, it has a positive expected value growing with t. For the same reason its variance is smaller than the one of the unrestricted Wiener process, see Fig. 7.6.

Fig. 7.6

WP and reflected WP along with expectation

Geometric Brownian Motion X(t) = e μ t+σ W(t)

By definition, it holds in this case that the logarithm⁸ of the process is a Brownian motion with drift and therefore Gaussian,

A random variable Y whose logarithm is Gaussian is called – as would seem natural – log-normal (logarithmically normally distributed). If it holds that

then we know how the two first moments of Y look like, cf. e.g. Sydsæter, Strøm, and Berck (1999, p. 189) or Johnson, Kotz, and Balakrishnan (1994, Ch. 14):

Hence, by plugging in we obtain for the geometric Brownian motion

(7.9)

While $$\log X(t)$$ is Gaussian with a linear trend, μ t, as expectation, X(t) exhibits an exponentially growing expectation function. Particularly for μ = 0 and $$\sigma = 1$$ the results

(7.10)

are obtained. The on average exponential growth in the case of $$\mu > -\sigma ^{2}/2$$ is illustrated in Fig. 7.7. In Fig. 7.8 we find graphs of the WP and a geometric Brownian motion with expectation one, namely with μ = −0. 5 and $$\sigma = 1$$. Generally, for $$\mu = -\sigma ^{2}/2$$ an expectation function of one is obtained. Then, one also says that the process does not exhibit a trend (or drift).

Fig. 7.7

Geometric Brownian motion with μ = 1. 5 and $$\sigma = 1$$ along with expectation

Fig. 7.8

WP and geometric Brownian motion with μ = −0. 5 and $$\sigma = 1$$

In comparison to the expectation, the median of a geometric Brownian motion does not depend on $$\sigma$$. Rather, it holds that (see Problem 7.8):

such that the median results as e μ t .

Maximum of a WP $$X(t) =\max _{0\leq s\leq t}W(s)$$

At t, the maximum process is assigned the maximal value which the WP has taken on up to this point in time. Therefore, in periods of a decreasing Wiener process path, X(t) is constant on the historic maximum until a new relative maximum is attained. However, this process has a distribution function that we have already come to know. By applying the distribution function of the stopping time which is given above Proposition 7.1, one shows (see Problem 7.9) that the maximum process and the reflected WP are equal in distribution:

Therefore, expectation and variance of the maximum process of | W(t) | can naturally be copied:

(7.11)

The expected value is positive and grows with time as the WP again will replace a relative positive maximum by a new relative maximum. Due to the process being again and again constant over times, it is not surprising that its variance is smaller than the one of the underlying WP, cf. Fig. 7.9.

Fig. 7.9

WP and maximum process along with expectation

Integrated Wiener Process $$X(t) =\int _{ 0}^{t}W(s)\,ds$$

As the Brownian motion is a continuous function, the Riemann integral can be defined pathwise. Hence, e.g. the following random variable is obtained:

Behind this random variable hides a normal distribution. The latter can be proved by using the definition of the Riemann integral or as a simple conclusion of the Proposition 8.​3 below:

(7.12)

Basically, by using the integral of a WP, a new stochastic process can also be generated by making the upper limit of integration time-dependent:

This idea forms the starting point for the subsequent chapter. In Fig. 7.10 the relation is shown between the WP and the integral X(t) as the area beneath the graph.

Fig. 7.10

WP and integrated WP

7.5 Problems and Solutions

Problems

7.1 Consider with $$\sigma > 0$$

Determine the mean and variance of X(t).

7.2 Consider

Determine the covariance of X(t) and W(t), Cov(X(t), W(t)).

7.3 Derive the autocovariance function of a WP, (7.4). Find a simple expression in t and s only for the autocorrelations

7.4 Choose $$d \in \mathbb{R}$$ such that T d−0. 5 W(t) and W(T t) are equal in distribution.

7.5 Prove Proposition 7.1 using the hints given in the text.

7.6 Derive the autocovariance function of a Brownian bridge, and hence show (7.6).

7.7 Determine the distribution function, (7.7), and the moments, (7.8), of a reflected Wiener process.

7.8 Show that in the general case of a geometric Brownian motion, $$e^{\mu \,t+\sigma \,W(t)}$$, the median is given by e μ t .

7.9 Show by means of the hints in the text that the maximum process of a WP and the corresponding reflected WP are equal in distribution:

Solutions

7.1 As the Wiener process is on average zero at every point in time, this obviously holds for X(t) as well. Therefore, the variance is calculated as follows:

7.2 Due to E(W(t)) = E(X(t)) = 0 one obtains:

Because of

it follows that

7.3 We simply apply the defining properties (W1), (W2) and (W3) or put differently (7.3). Due to (7.3) the WP has an expectation of zero such that

W.l.o.g. let s ≤ t. By using (W1) and (W2) and after adding zero, we then write:

where the last equality uses Var(W(s)) = s and the independence of non-overlapping increments. As one could also assume t ≤ s w.l.o.g., (7.4) is verified.

With the autocovariance one obtains

7.4 This is a problem on self-similarity or scale invariance. Due to (7.3) it obviously holds that:

and

Therefore, the corresponding variances are equal for d = 1. The corresponding result is obtained from (7.5) as well:

7.5 Proof of Proposition 7.1(a): Our proof consists of three steps. At first, we establish the equation

(7.13)

Secondly, by using this we show:

(7.14)

Note that in (7.14) the integrand amounts to the density function of the standard normal distribution. Hence, thirdly for $$t \rightarrow \infty $$ the claim immediately follows from (7.14) due to P(T b  > t) = 1 −P(T b  ≤ t). 

In order to accept (7.13), we remember (7.3). Accordingly, for T b  < t it holds that

which is why from the symmetry of the Gaussian distribution with W(T b ) = b it follows for the conditional probability that:

Hence, it results that:

where the last equality is caused by the fact that T b  < t is implied by W(t) > b. Thus, we obtain Eq. (7.13) which will now be applied for deriving (7.14).

Due to (7.3) it holds by definition that

By variable transformation, $$y = \frac{x} {\sqrt{t}}$$, it follows that

whereby (7.14) and hence claim (a) is proved due to P(T b  < t) = P(T b  ≤ t).

Proof of Proposition 7.1(b): With the density function f b (t) = F b ′(t) the approach for the expected value reads as follows:

Note that the distribution function derived in (7.14) has the following structure with the antiderivative H, H′ = h:

Therefore, due to the chain rule it holds for the density that

Hence, the variable transformation results in t = b ² u ⁻² with dt = −2 b ² u ⁻³ du:

However, this last integral written down symbolically does not exist since the antiderivative of u ⁻² is − u ⁻¹, and

diverges as $$\varepsilon \rightarrow 0$$. This completes the proof.

7.6 First, we determine that

has zero expectation:

By multiplying out and application of (7.4) one can show that

In particular, for s = t the variance formula (7.6) is obtained.

7.7 At first we determine the distribution function (7.7) for X(t) =  | W(t) | :

where the symmetry of the Gaussian distribution was used. With $$W(t) \sim \mathcal{N}(0,t)$$ we therefore have

or for the density

In order to calculate expectation and variance, we determine the r-th moment in general:

By substitution, these moments can be reduced to the Gamma function, which was introduced in Problem 5.3, see also below Eq. (5.​20). For a > 0 and with a x ² = u and du = 2ax dx, we obtain:

The Gamma function possesses a number of nice properties and special values, remember in particular e.g.

With $$a = \frac{1} {2t}$$, for the moments it therefore follows that:

The variance formula is obtained by the usual variance decomposition:

7.8 The random variable $$\sigma \,W(t)$$ follows for fixed t a Gaussian distribution with expectation and median equal to zero. Hence, it follows that

By multiplying the inequality by e μ t , we obtain

Therefore, the median of $$X(t) = e^{\mu \,t+\sigma \,W(t)}$$ is determined independently of $$\sigma$$ as e μ t as claimed.

7.9 As $$X(t) =\max _{0\leq s\leq t}W(s)$$ is a continuous random variable for given t, it holds that

Remember the random variable T b from Proposition 7.1 specifying the point in time at which W(t) hits the value b for the first time. The event $$\max _{0\leq s\leq t}W(s) < b$$ is equivalent to the fact that the hitting time of b is larger than t. Therefore, when using the distribution function from Proposition 7.1, it holds that

Naturally, the number 1 can be written as an integral over the density of the standard normal distribution:

By substitution,

and due to (7.7) the desired result is immediately obtained:

References

Billingsley, P. (1986). Probability and measure (2nd ed.). New York: Wiley.

Gradshteyn, I. S., & Ryzhik, I. M. (2000). Table of integrals, series, and products (6th ed.). London/San Diego: Academic Press.

Johnson, N. L., Kotz, S., & Balakrishnan, N. (1994). Continuous univariate distributions, Volume 1 (2nd ed.). New York: Wiley.

Klebaner, F. C. (2005). Introduction to stochastic calculus with applications (2nd ed.). London: Imperical College Press.

Sydsæter, K., Strøm, A., & Berck, P. (1999). Economists’ mathematical manual (3rd ed.). Berlin/New York: Springer.

Footnotes

1

Norbert Wiener, 1894–1964, was a US-American mathematician. He succeeded in finding a mathematically solid definition and discussion of the so-called Brownian motion named after the nineteenth century British botanist Brown. With a microscope, Brown initially observed and described erratic paths of molecules.

2

This special assumption for the starting value is made out of convenience; it is by no means crucial for the behavior of a random walk.

3

In order not to violate the concept of functions, strictly speaking, the vertical lines would not be allowed to occur in the graphs of the figure. We ignore this subtlety to enhance clarity.

Fig. 7.1

Step function from (7.1) on the interval [0,1]

4

To be completely accurate, this needs to read: $$W(t) - W(0) \sim \mathcal{N}(0,\,t)$$. As W(0) is zero with probability one, we set W(0) equal to zero here and in the following; then, the corresponding statements only hold with probability one.

5

This convention does not hold beyond these pages. Many authors use the terms Wiener process or Brownian motion interchangeably or they apply one of them exclusively.

6

Occasionally, the pathwise continuity is claimed to be the fourth defining property. This is to be understood as follows. Billingsley (1986, Theorem 37.1) proves more or less the following: If one has a WP W with (W1) to (W3) at hand, then a process W ∗ can be constructed which is a WP in the sense of (W1) to (W3), as well, which has the same distribution as W and which is pathwise continuous. As W ∗ and W are equal in distribution, they cannot be distinguished and therefore, w.l.o.g. it can safely be assumed that one may work with the continuous W ∗.

7

The random variable T b is a so-called “stopping time”. This is a term from the theory of stochastic processes which we will not elaborate on here.

8

By “log” we denote the natural logarithm and not the common logarithm.

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_8

8. Riemann Integrals

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

8.1 Summary

In this chapter we deal with stochastic Riemann integrals, i.e. with ordinary Riemann integrals with a stochastic process as the integrand.¹ Mathematically, these constructs are relatively unsophisticated, they can be defined pathwise for continuous functions as in conventional (deterministic) calculus. However, this pathwise definition will not be possible any longer for e.g. Ito integrals in the chapter after next. Hence, at this point we propose a way of defining integrals as a limit (in mean square) which will be useful later on. If the stochastic integrand is in particular a Wiener process, then the Riemann integral follows a Gaussian distribution with zero expectation and the familiar formula for the variance. A number of examples will facilitate the understanding of this chapter.

8.2 Definition and Fubini’s Theorem

As one has done in deterministic calculus, we will define the Riemann integral by an adequate partition as the limit of a sum.

Partition

In order to define an integral of a function from 0 to t, we decompose the interval into n adjacent, non-overlapping subintervals which are allowed to intersect at the endpoints:

(8.1)

In the following, we always assume that the partition $$P_{n}\left ([0,\,t]\right )$$ becomes increasingly fine with n growing (“adequate partition ”):

(8.2)

By s i ∗ we denote an arbitrary point in the i-th interval,

Occasionally, we will sum up the lengths of the subintervals. Obviously, it holds that

In general, for a function $$\varphi$$ one obtains:

(8.3)

Sometimes, we will operate with the example of the equidistant partition . It is given by $$s_{i} = it/n$$:

Due to $$s_{i} - s_{i-1} = 1/n$$ the required refinement from (8.2) for $$n \rightarrow \infty $$ is guaranteed.

Definition and Existence

Now, the product of a deterministic function f and a stochastic process X is to be integrated. To this end, the Riemann sum is defined by means of the notation introduced:

(8.4)

Here, we have a sum of rectangular areas, each with a width of $$\left (s_{i} - s_{i-1}\right )$$ and the height f(s i ∗) X(s i ∗). With n growing, the area beneath f(s) X(s) on [0, t] is to be approximated all the better. If the limit of this sum for $$n \rightarrow \infty $$ exists uniquely and independently of the partition and of the choice of s i ∗, then it is defined as a (stochastic) Riemann integral. In this case, the convergence occurs in mean square (“$$\stackrel{2}{\rightarrow }$$”)²:

One then says that the Riemann integral exists. For this existence, there is a sufficient and a necessary condition formulated in the following proposition. The proof is carried out with part (b) from Lemma 8.2 below, see Problem 8.1. Further elaborations on mean square convergence can be found at the end of the chapter.

Proposition 8.1 (Existence of the Riemann Integral)

The Riemann sum from Eq.  (8.4) converges in mean square for $$n \rightarrow \infty $$ under (8.2) if and only if the double integral

exists.

A sufficient condition for the existence of the Riemann integral is that the function f is continuous and that furthermore $$\mbox{ E}\left (X(s)\,X(r)\right )$$ is continuous in both arguments. In order to find this, we define

Now, if the function $$\mbox{ E}\left (X(s)\,X(r)\right )$$ is continuous in both arguments, this implies continuity of $$\varphi$$ for a continuous f as the integral is a continuous functional, see e.g. Trench (2013, p. 462). Therefore, the ordinary Riemann integral of $$\varphi$$ exists,

Hence, the Riemann sum from (8.4) converges due to Proposition 8.1.

Fubini’s Theorem

Frequently, we are interested in the average behavior, i.e. the expected value of Riemann integrals. The expected value, however, is defined as an integral itself such that one is confronted with double integrals. For calculating these, there is a simple rule which is finally based on the fact that the order of integration does not matter for double integrals over continuous functions. In deterministic calculus, this fact is also known as “Fubini’s theorem” (see Footnote 6 in Sect. 2.​3). Adapted to our problem of the expected value of a Riemann integral, the corresponding circumstances are given in the following proposition, also cf. Billingsley (1986, Theorem 18.3).

Proposition 8.2 (Fubini’s Theorem)

If $$\int _{0}^{t}\mbox{ E}(\vert X(s)\vert )\,ds$$ exists, it holds for a continuous X that:

The statement is easy to comprehend if one thinks of the integral as a finite Riemann sum. As is well known, in the discrete case, summation and expectation is interchangeable:

Now, Fubini’s theorem just guarantees a continuation of this interchangeability for $$n \rightarrow \infty $$.

Example 8.1 (Expected Value of the Integrated WP)

Consider the special case of the integrated WP with X(s) = W(s) and f(s) = 1. With the WP being continuous, | W(t) | is a continuous process as well. In (7.​8) we have determined the following expression as the expected value:

Before applying Proposition 8.2, we check:

As this quantity is finite, the requirements of Fubini’s theorem are fulfilled. Hence, it follows that

 ■ 

General Rules

Note that our definition of the integral seems to be unnecessarily restrictive. However, the restriction on the interval [0, t] is by no means crucial. The usual rules apply and are here symbolically described for an integrand g (without proof):

8.3 Riemann Integration of Wiener Processes

In this section, we concentrate on Riemann integrals where the stochastic part of the integrand is a WP: X(t) = W(t).

Normal Distribution

Frequently, Gaussian random variables are hidden behind Riemann integrals. In fact, it holds that all of the integrals discussed in this section follow Gaussian distributions with zero expectation. The variances can be determined according to the following proposition (for a proof see Problem 8.3).

Proposition 8.3 (Normality of Riemann Integrals)

Let f be a continuous deterministic function on [0,t]. Then, it holds

The normality follows from the fact that the WP is a Gaussian process. Hence, the Riemann sum R n from (8.4) follows a Gaussian distribution for finite n. As R n converges in mean square, it follows from Lemma 8.1 (see below) that the limit is Gaussian as well. Note that the finiteness of the variance expression from Proposition 8.3 is just sufficient and necessary for the existence of the Riemann integral (Proposition 8.1).

Example 8.2 (Variance of the Integrated WP)

Consider an integrated WP with f(s) = 1 as in Example 8.1. We look for a closed expression for the variance of $$\int _{0}^{t}W(s)ds$$. Due to Proposition 8.3, the starting point is:

Now, we employ a useful trick for many applications. The integral with respect to r is decomposed into the sum of two integrals with s as the integration limit such that the minimum function can be specified explicitly:

Now, the integration of the power functions yields the requested variance:

 ■ 

Autocovariance Function

With the time-dependent integration limit, $$\int _{0}^{t}f(s)\,W(s)\,ds$$ itself is a stochastic process. Therefore, it suggests itself to not only determine the variance as in Proposition 8.3, but the covariance function as well. The general result is given in the following proposition, which will be verified in Problem 8.7.

Proposition 8.4 (Autocovariance Function)

For a continuous function f with integrable antiderivative F and $$Y (t) =\int _{ 0}^{t}f(s)\,W(s)ds$$ it holds that:

where h ≥ 0.

Therefore, with h = 0 an alternative expression for the variance from Proposition 8.3 is obtained. For concrete functions f, the formula can be simplified considerably. This is to be shown by the following example.

Example 8.3 (Autocovariance of the Integrated WP)

Once again, we examine the integrated WP with f(s) = 1 and F(s) = s as in Examples 8.1 and 8.2. Then, plugging in yields:

Hence, for h = 0 the variance of the integrated Wiener process reads:

Of course, we already know this from Example 8.2. ■ 

Examples

For three special Gaussian integrals, which we will be confronted with over and over, the variances are to be calculated. We put the results in front.

Corollary 8.1

It holds that

where $$c \in \mathbb{R}$$ and $$\sigma _{R}^{2} = \frac{8-25c+20c^{2}} {60} > 0$$ .

The normality in (a) and (c) is clear due to Proposition 8.3. In (b) we have the sum of two Gaussian random variables which does not necessarily have to be Gaussian again unless a multivariate Gaussian distribution is present. Thus, the normality of (b) can only be proven in connection with Stieltjes integrals (see Problem 9.2).

The result from (a) is a special case of Example 8.2 with t = 1. We show in Problem 8.4 that the variance in (b) is just 1∕3. The proof of (c) for c = 0 is given in Problem 8.5; for an arbitrary c, the proof is basically similar, however, it gets computationally more involved. Note that the variance $$\sigma _{R}^{2}$$ cannot be zero or negative for any c (Problem 8.6).

Again, there should be a word of warning concerning equality in distribution. From (b) it follows that:

Therefore, the following random variables are equal in distribution,

although, pathwise it obviously holds that:

8.4 Convergence in Mean Square

Now, we hand in some basics which are not necessary for the understanding of Riemann integrals; however, they are helpful for some technical properties. In particular, for the elaboration on the Ito integral following below, the knowledge of convergence in mean square is advantageous for a complete understanding. For a brief introduction to the basics of asymptotic theory, Pötscher and Prucha (2001) can be recommended.

Definition and Properties

Let {X n }, $$n \in \mathbb{N}$$, be a sequence of real random variables with

(8.5)

For a sequence {X n } and a random variable X, we define the mean squared error as distance or norm:

One says, {X n } converges in mean square to X for n tending to infinity if

Abbreviating, we write for this as well

This limit is unique with probability one. Of course, it can be a random variable itself or a constant. In any case, due to (8.5) it holds that: $$\mbox{ E}(X^{2}) < \infty $$. In fact, expected value and variance of X can be determined from the moments of X n . In particular, the limit of Gaussian random variables is again Gaussian. More precisely, the following lemma holds (see Problem 8.8 for a proof).

Lemma 8.1 (Properties of the Limit in Mean Square)

Let {X n } with (8.5) converge in mean square to X. Then it holds for $$n \rightarrow \infty $$:

(a)

E(X n ) →E(X);

(b)

E(X n ² ) →E(X ² );

(c)

if {X n } is Gaussian, then X follows a Gaussian distribution as well.

Naturally, the parameters of the Gaussian distribution X from (c) follow according to (a) and (b).

Convergence to a Constant

If the limit is a constant, then it is particularly easy to establish convergence in mean square. For this purpose, we consider the following derivation. By zero addition and the binomial formula, the following expression is obtained:

The expectation operator yields:

If X is a constant (a “degenerate random variable”), X = c, then the second term becomes zero and the third term is the expected value of a constant. In other words, this yields:

Hence, {X n } converges in mean square to a constant c if and only if it holds that

As is well known, this implies that {X n } converges to c in probability as well (see Lemma 8.3 below). Next, we cover criteria of convergence.

Test of Convergence

Now, we still need a convenient criterion in order to decide whether a series is convergent in mean square. In fact, we have two equivalent criteria. For the proof see Problem 8.9. The name goes back to the famous French mathematician Augustin Louis Cauchy (1789–1857).

Lemma 8.2 (Cauchy Criterion)

A series {X n } with (8.5) converges in mean square …

(a)

… if and only if it holds for arbitrary n and m that

(b)

… or put equivalently, if and only if it holds for arbitrary n and m that

where $$c \in \mathbb{R}$$ is a constant.

Note that the convergence of the Cauchy criterion holds independently of how m and n tend to infinity. As well, the constant c results independently of the choice of m and n. As the criteria are sufficient and necessary, the proof of existence for mean square convergence can be supplied without determining the limit explicitly.

Example 8.4 (Law of Large Numbers)

Let $$\{\varepsilon _{t}\}$$ be a white noise process, $$\varepsilon _{t} \sim \mbox{ WN}(0,\sigma ^{2})$$. Then, it can be shown that the arithmetic mean,

converges in mean square without specifying the limit. It namely holds that

for $$m\,,n \rightarrow \infty $$. Due to Lemma 8.2(b) we conclude that $$\overline{\varepsilon }_{n}$$ has a limit in mean square.

Let the limit of $$\overline{\varepsilon }_{n}$$ simply be called $$\varepsilon$$. Naturally, it can be determined immediately. Due to

it follows from Lemma 8.1(a) and (b) for the limit that

Hence, the limit is equal to zero (with probability one). From this, it follows for $$x_{t} =\mu +\varepsilon _{t}$$ that the arithmetic mean of x t converges in mean square to the true expected value, μ. In the literature, this fact is also known as the “law of large numbers”. ■ 

Further Modes of Convergence

Two weaker concepts of convergence can be defined via probability statements. First, we say {X n } converges in probability to X if it holds for arbitrary $$\varepsilon > 0$$ that:

Symbolically, this is denoted as³

Secondly, one talks about convergence in distribution of {X n } to X if it holds for all points $$x \in \mathbb{R}$$ at which the distribution function F n (x) of X n is continuous, that F n (x) tends to the distribution function F(x) of X:

The word “distribution” suggests the symbolic notation:

From Grimmett and Stirzaker (2001, p. 310) or Pötscher and Prucha (2001, Theorem 5 and 9) we adopt the following results.

Lemma 8.3 (Implications of Convergence)

The following implications hold, $$n \rightarrow \infty $$.

(a)

Convergence in mean square implies convergence in probability:

(b)

Convergence in probability implies convergence in distribution:

In general, the converse of Lemma 8.3(a) or (b) does not hold. However, if X = c is a constant, then convergence in probability and convergence in distribution are equivalent, see Grimmett and Stirzaker (2001, p. 310) or Pötscher and Prucha (2001, Theorem 10).

8.5 Problems and Solutions

Problems

8.1 Prove Proposition 8.1.

Hint: Use Lemma 8.2.

8.2 Determine the expected value from Proposition 8.3.

8.3 Determine the variance from Proposition 8.3.

8.4 Calculate the variance from Corollary 8.1(b).

8.5 Calculate the variance from Corollary 8.1(c) for the special case of c = 0.

8.6 Show that the variance from Corollary 8.1(c) is positive.

8.7 Prove Proposition 8.4.

8.8 Prove Lemma 8.1.

8.9 Prove Lemma 8.2.

Solutions

8.1 Analogously to the partition (8.1) and the Riemann sum R n from (8.4), we define for arbitrary m with $$m \rightarrow \infty $$:

In order to apply the existence criterion from Lemma 8.2(b), we formulate the product of the two Riemann sums as follows:

Hence, the Riemann integral as limit of R n exists if and only if E(R n  R m ) converges. Further,

converges if and only if the ordinary Riemann double integral

exists. The Cauchy criterion from Lemma 8.2 therefore amounts to a proof of Proposition 8.1.

8.2 For the WP it holds that

The minimum function is continuous in both the arguments. If f is a continuous function as well, then we know, with the considerations following Proposition 8.1, that the stochastic Riemann integral

exists. In order to calculate the expected value, Fubini’s theorem will be applied. For this purpose, we check that

is finite. The bound is based on the continuity and hence the finiteness of f. The integral

was determined in Example 8.1 for t fixed to be finite. As the WP is continuous, Proposition 8.2 can be applied. According to this, it holds that:

8.3 Let us denote the Riemann integral by Y (t):

We have already shown that E(Y (t)) = 0. Hence, it follows that

By applying Fubini’s theorem twice, we obtain:

which is the requested result.

8.4 Let us define

with E(Y (t)) = 0. Then, it holds that

where the variance from Corollary 8.1(a) and Fubini’s theorem were used. On [0, 1] it holds that:

Hence, the variance results as claimed:

8.5 For c = 0 the claim reads

According to Proposition 8.3, the variance results as a double integral for f(s) = s,

where the inner integral is appropriately decomposed to facilitate the calculation:

This corresponds to the claimed result.

8.6 We consider the numerator of $$\sigma _{R}^{2}$$,

and show that it does not have any real zeros. Setting n(c) = 0 yields:

Thus, no real zeros exist. Consequently, since n(0) = 8 > 0 the in c continuous function n(c) cannot be zero or negative; the same holds true for $$\sigma _{R}^{2}$$, which proves the claim.

Of course, an alternative proof consists in determining the real extrema of n(c). There exists an absolute minimum and this turns out to be positive.

8.7 If the Riemann integral is denoted by Y (t), then it holds, as for the derivation of the variance, that:

Partial integration yields the following relation:

By plugging in we obtain the claim.

8.8 Proof of (a): By bounding the difference of the two expected values by means of the Cauchy-Schwarz inequality (2.​5) one can immediately tell that this difference tends to zero in the case of convergence in mean square:

Proof of (b): The simple trick

yields upon expectation:

where again the Cauchy-Schwarz inequality (2.​5) was used.

Proof of (c): As is well known, convergence in mean square implies convergence in distribution, see Lemma 8.3. The latter is equivalent to the fact that the characteristic function ϕ n (u) of X n tends to the characteristic function ϕ(u) of X.⁴ Now we show: If ϕ n (u) belongs to a Gaussian distribution, then this holds for ϕ(u) as well. Hence, (a) and (b) in combination with the premise of a Gaussian sequence {X n } imply:

for $$n \rightarrow \infty $$. Thus, the characteristic function of X as $$n \rightarrow \infty $$ is that of a Gaussian distribution as well.

8.9 Proof of (a): Elementarily, it can be shown that the Cauchy criterion follows from convergence in mean square:

where the bounding is again based on the Cauchy-Schwarz inequality (2.​5). It is somewhat more involved that, inversely, the condition from (a) implies convergence in mean square as well. For the proof, we refer e.g. to the exposition on Hilbert spaces in Brockwell and Davis (1991, Ch. 2).

Proof of (b): If we take (a) for granted, the proof is simple. Due to

one can immediately tell that the condition from (a) implies:

Inversely, from the condition from (b) it naturally follows that

This completes the proof.

References

Billingsley, P. (1986). Probability and measure (2nd ed.). New York: Wiley.

Brockwell, P. J., & Davis, R. A. (1991). Time series: Theory and methods (2nd ed.). New York: Springer.

Grimmett, G. R., & Stirzaker, D. R. (2001). Probability and random processes (3rd ed.). Oxford: Oxford University Press.

Pötscher, B. M., & Prucha, I. R. (2001). Basic elements of asymptotic theory. In B. H. Baltagi (Ed.), A companion to theoretical econometrics (pp. 201–229). Malden: Blackwell.

Trench, W. F. (2013). Introduction to real analysis. Free Hyperlinked Edition 2.04 December 2013. Downloaded on 10th May 2014 from http://​digitalcommons.​trinity.​edu/​mono/​7.

Footnotes

1

Bernhard Riemann (1826–1866) studied with Gauss in Göttingen where he himself became a professor. Already before his day, integration had been used as a technique which reverses differentiation by forming an antiderivative. However, Riemann explained for the first time under which conditions a function possesses an antiderivative at all.

2

A definition and discussion of this mode of convergence can be found in the fourth section of this chapter.

3

In particular in econometrics, one often writes alternatively plimX n  = X as $$n \rightarrow \infty $$.

4

See e.g. sections 5.7 through 5.10 in Grimmett and Stirzaker (2001) for an introduction to the theory and application of characteristic functions. In particular, it holds for the characteristic function of a random variable with a Gaussian distribution, $$Y \sim \mathcal{N}(\mu,\sigma ^{2})$$, that:

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_9

9. Stieltjes Integrals

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

9.1 Summary

Below, we will encounter Riemann-Stieltjes integrals (or more briefly: Stieltjes integrals) as solutions of certain stochastic differential equations. They can be reduced to the sum of a Riemann integral and a multiple of the Wiener process. Stieltjes integrals are again Gaussian. As an example we consider the Ornstein-Uhlenbeck process which is defined by a Stieltjes integral and which will be dealt with in detail in the chapter on interest rate models.

9.2 Definition and Partial Integration

As a first step towards the Ito integral, we define Stieltjes¹ integrals which can be reduced to Riemann integrals by integration by parts.

Definition

The Riemann-Stieltjes integral (or Stieltjes integral), as it is considered here, integrates over a deterministic function f(s). Nevertheless, the Stieltjes integral is random as it is integrated with respect to the stochastic Wiener process W(s). In order to understand what is meant by this, we recall the partition (8.​1):

with s i ∗ ∈ [s i−1, s i ]. Hence, the Riemann-Stieltjes sum is defined as

(9.1)

If an expression well-defined in mean square follows from this for $$n \rightarrow \infty $$ under (8.​2), then we define it as a Stieltjes integral with the obvious notation

Note that dW(s) does not stand for the derivative of the Wiener process as it does not exist. It is just a common symbolic notation.

If f is continuously differentiable,² then the existence of the Stieltjes integral is guaranteed, see Soong (1973, Theorem 4.5.2).

Integration by Parts

If f is continuously differentiable, then the Stieltjes integral can be expressed by a Riemann integral and the WP. This relation is also known as integration by parts. In Chap. 11 we will understand that it is a special case of Ito’s lemma, which is why we do not have to concern ourselves with a proof of Proposition 9.1 at this point.

Proposition 9.1 (Stieltjes Integral; Integration by Parts)

For a continuously differentiable, deterministic function f we have that

(a)

the Stieltjes sum from (9.1) converges in mean square if it holds that $$\max (s_{i} - s_{i-1}) \rightarrow 0$$ ,

(b)

and

where the last equality holds with probability one. ³

The result from (b) corresponds to the familiar rule of partial integration. As a refresher, we write this rule for two deterministic functions f and g:

(9.2)

Hence, this is the integral form of the product rule of differentiation:

or

or

Therefore, one can make a mental note of Proposition 9.1 (b) by the well-known partial integration from (9.2).

Example 9.1 (Corollary)

As an application of Proposition 9.1 we consider Riemann-Stieltjes integrals for three particularly simple functions. We will encounter these relations repeatedly. The proof amounts to a simple exercise in substitution. It holds

(a)

for the identity function f(s) = s:

(b)

for f(s) = 1 − s:

(c)

for the constant function f(s) = 1:

In (c) we again observe a formal analogy of the WP with the random walk. Just like the latter is defined as the sum over the past of a pure random process, see (1.​8), the WP is the integral of its past independent increments. ■ 

9.3 Gaussian Distribution and Autocovariances

The reduction of Stieltjes integrals to Riemann integrals suggests that there are Gaussian processes hiding behind them. In fact, it holds that all Stieltjes integrals follow Gaussian distributions with expectation zero.

Gaussian Distribution

The Gaussian distribution itself is obvious: The Riemann-Stieltjes sum from (9.1) is, as the sum of multivariate Gaussian random variables, Gaussian as well. Then, this also holds for the limit of the sum due to Lemma 8.​1. The expected value is zero due to Propositions 9.1(b) and 8.3. The variance results as a special case of the autocovariance given in Proposition 9.3. Hence, we obtain the following proposition.

Proposition 9.2 (Normality of Stieltjes integrals)

For a continuously differentiable, deterministic function f, it holds that

The variance of the Stieltjes integral is well motivated as follows. For the variance of the Riemann-Stieltjes sum,

it follows for $$n \rightarrow \infty $$, due to the independence of the increments of the WP, that:

The convergence takes place as f ² is continuous and thus Riemann-integrable. Hence, for $$n \rightarrow \infty $$ the expression from Proposition 9.2 is obtained.

Let us consider the integrals from Example 9.1 and calculate the variances for t = 1 (see Problem 9.1).

Example 9.2 (Corollary)

For the functions from Example 9.1 it holds:

(a)

for the identity function f(s) = s:

(b)

for f(s) = 1 − s:

(c)

for the constant function f(s) = 1:

 ■ 

Autocovariance Function

As a generalization of the variance, an expression for the covariance is to be found. Hence, let us define the process $$Y (t) =\int _{ 0}^{t}f(s)dW(s)$$. The autocovariance of Y (t) and Y (t + h) with h ≥ 0 can be well justified if one takes into account that the increments dW(t) of the WP are stochastically independent provided they do not overlap. Therefore, one should expect $$\int _{0}^{t}f(s)dW(s)$$ and $$\int _{t}^{t+h}f(r)dW(r)$$ to be uncorrelated:

If this is true, then, due to

the following result is obtained:

Therefore, for an arbitrary h ≥ 0 the autocovariance coincides with the variance in t. In fact, this result can be verified more rigorously (see Problem 9.5).

Proposition 9.3 (Autocovariance of Stieltjes Integrals)

For a continuously differentiable, deterministic function f it holds that

with h ≥ 0.

Of course, for h = 0 the variance from Proposition 9.2 is obtained.

Example 9.3 (Autocovariance of the WP)

As an example, let us consider f(s) = 1 with

Then, it follows for h ≥ 0:

Trivially, this just reproduces the autocovariance structure of the Wiener process already known from (7.​4). ■ 

9.4 Standard Ornstein-Uhlenbeck Process

The so-called Ornstein-Uhlenbeck process has been introduced in a publication by the physicists Ornstein and Uhlenbeck in 1930.

Definition

We define the Ornstein-Uhlenbeck process (OUP) with starting value X c (0) = 0 for an arbitrary real c as a Stieltjes integral,

(9.3)

For c = 0 in (9.3) the Wiener process, X ₀(t) = W(t), is obtained. More precisely, X c (t) from (9.3) is a standard OUP; a generalization will be offered in the chapter on interest rate dynamics. By definition, it holds that:

where $$\varepsilon (t + 1)$$ was defined implicitly. Note that the increments dW(s) from t on in $$\varepsilon (t + 1)$$ are independent of the increments up to t as they appear in X c (t). Hence, the OUP is a continuous counterpart of the AR(1) process from Chap. 3 where the autoregressive parameter is denoted by e c . For c < 0 this parameter is less than one, such that in this case we expect a stable adjustment or, in a way, a quasi-stationary behavior. This will be reflected by the behavior of the variance and the covariance function which are given, among others, in the following proposition.

Properties

The proof of Proposition 9.4 will be given in an exercise problem. It comprises an application of Propositions 9.1, 9.2 and 9.3.

Proposition 9.4 (Ornstein-Uhlenbeck Process)

It holds for the Ornstein-Uhlenbeck process from (9.3) that:

where h ≥ 0.

Statement (a) establishes the usual relation between Stieltjes and Riemann integrals and, seen individually, it is not that thrilling. As for c = 0 the OUP coincides with the WP, it is interesting to examine the variance from (b) for c → 0. L’Hospital’s rule yields:

Hence, for c → 0 the variance of the WP is embedded in (b). The covariance from (c) allows for determining the autocorrelation:

Now, let us assume that c < 0. Then it holds for t growing that:

Accordingly, it holds for the autocorrelation that:

Thus, for c < 0 we obtain the “asymptotically stationary” case with asymptotically constant variance and an autocorrelation being asymptotically dependent on the lag h only. Thereby, the autocorrelation results as the h-th power of the “autoregressive parameter” a = e c . With h growing, the autocovariance decays gradually. This finds its counterpart in the discrete-time AR(1) process. Just as the random walk arises from the AR(1) process with the parameter value one, the WP with c = 0, i.e. a = e ⁰ = 1, is the corresponding special case of the OUP. Hence, we can definitely consider the OUP as a continuous-time analog to the AR(1) process.

Simulation

The theoretical properties of the process for c < 0 can be illustrated graphically. In Fig. 9.1 the simulated paths of two parameter constellations are shown. It can be observed that the process oscillates about the zero line where the variance or the deviation from zero for c = −0. 1 is much larger⁴ than in the case c = −0. 9. This is clear against the background of (b) from Proposition 9.4 in which the first moment and the variance are given: The expected value is zero and the variance decreases with the absolute value of c increasing. The positive autocorrelation (cf. Proposition 9.4(c)) is obvious as well: Positive values tend to be followed by positive values and the inverse holds for negative observations. The closer to zero c is, the stronger the autocorrelation gets. That is why the graph for c = −0. 1 is strongly determined by the “local” trend and does not cross the zero line for longer time spans while for c = −0. 9 the force which pulls the observations back to the zero line is more effective such that the graph looks “more stationary” for c = −0. 9.

Fig. 9.1

Standard Ornstein-Uhlenbeck processes

9.5 Problems and Solutions

Problems

9.1 Calculate the variances from Example 9.2.

9.2 Verify the Gaussian distribution from Corollary 8.​1(b).

9.3 Verify the following equality (with probability 1):

9.4 Determine the variance of the process X(t) with

9.5 Prove Proposition 9.3.

9.6 Prove (a) from Proposition 9.4.

9.7 Show (b) from Proposition 9.4.

9.8 Prove (c) from Proposition 9.4.

Solutions

9.1 From Proposition 9.2 it obviously follows for (a) that:

Equally, one shows (b):

Finally, the result from (c) is known anyway.

9.2 The result follows from the examples of this chapter. From Example 9.1(a) we obtain for t = 1:

Due to Example 9.2(a) the claim is verified.

9.3 This is a straightforward application of Proposition 9.1. With f(s) = s ² and f′(s) = 2s the claim is established.

9.4 From Proposition 9.2 with f(s) = s ² it follows for the variance

9.5 With $$Y (t) =\int _{ 0}^{t}f(s)dW(s)$$ we know from Proposition 9.1 that:

Hence, the covariance results as

where the expressions on the right-hand side are defined by multiplying Y (t) and Y (t + h). Now, we consider them one by one.

For A we obtain immediately:

By Fubini’s theorem it holds for B that:

Integration by parts in the following form,

(9.4)

applied to f′ yields:

where F(s) denotes the antiderivative of f(s). In the same way, we obtain

For the fourth expression Proposition 8.​4 provides us with f′ instead of f:

In addition to (9.4), we apply integration by parts in the form of

Then it holds that:

If we assemble the terms, then we obtain the autocovariance function in the desired form:

9.6 We use Proposition 9.1 with f(s) = e −cs :

Multiplying by e ct yields

On the left-hand side, we have the OUP X c (t) by definition which was to be verified.

9.7 Due to Proposition 9.2 the OUP is Gaussian with expectation zero and variance

This is equal to the claimed variance.

9.8 As for the derivation of the variance, we use

From Proposition 9.3 we know that this is also the expression for the autocovariance of the Stieltjes integrals (h ≥ 0):

Hence, we obtain for the OUP:

Reference

Soong, T. T. (1973). Random differential equations in science and engineering. New York: Academic Press.

Footnotes

1

Thomas J. Stieltjes lived from 1856 to 1894. The Dutch mathematician generalized the concept of integration by Riemann.

2

We call a function continuously differentiable if it has a continuous first order derivative.

3

Remember that we assumed P(W(0) = 0) = 1, which justifies the last statement. Whenever we have equalities in a stochastic setting, they are typically understood to hold with probability one for the rest of the book.

4

If the arithmetic mean of the 1000 observations of this time series is calculated, then by − 0. 72344 a notably negative number is obtained although the theoretical expected value is zero. Details on the simulation of OUP paths are to follow in Sect. 13.​2

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_10

10. Ito Integrals

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

10.1 Summary

Kiyoshi Ito (1915–2008) was awarded the inaugural Gauss Prize by the International Mathematical Union in 2006. Stochastic integration in the narrow sense can be traced back to his early work published in Japanese in the forties of the last century. We precede the general definition of the Ito integral with a special case. Concluding, we discuss the (quadratic) variation of a process without which a sound understanding of Ito’s lemma will not be possible.

10.2 A Special Case

We start with a special case of Ito integration, so to speak the mother of all stochastic integrals. Thereby we will understand that, besides the Ito integral, infinitely many related integrals of a similar structure exist.

Problems with the Definition

The starting point is again a partition

that gets finer for n growing since we continue to maintain (8.​2). Given this decomposition of [0, t], we define analogously to the Riemann-Stieltjes sum for s i ∗ ∈ [s i−1, s i ):

(10.1)

For $$n \rightarrow \infty $$ we would like to denote the limit as $$\int _{0}^{t}W(s)\,dW(s)$$, which looks like a Stieltjes integral of a WP with respect to a WP. However, we will realize that:

1.

The limit of S n (W) is not unique, but depends on the choice of s i ∗;

2.

the limit of S n (W) is not defined as a Stieltjes integral.

As the Stieltjes integral has a unique limit independently of s i ∗, the second claim follows from the first one. If one chooses in particular the lower endpoint of the interval as support, $$s_{i}^{{\ast}} = s_{i-1}$$, then this leads to the Ito integral. Hence, this special case is called the Ito sum :

(10.2)

The following proposition specifies the dependence on s i ∗. The special case γ = 0 leading to the Ito integral will be proved in Problem 10.1; the general result is established e.g. in Tanaka (1996, eq. (2.40)). The convergence is again in mean square.

Proposition 10.1 (Stochastic Integrals in Mean Square)

Let s i ∗ = (1 −γ) s i−1 + γ s i with 0 ≤γ < 1. Then it holds for the sum from (10.1) with $$n \rightarrow \infty $$ under (8.​2) :

Before we discuss two special cases of Proposition 10.1, this striking result is to be somewhat better understood. We call it striking because it is counter-intuitive at first glance that the choice of s i ∗ should matter with the intervals [s i−1, s i ) getting narrower and narrower for $$n \rightarrow \infty $$. To better understand this, we temporarily denote the limit of S n (W) as S(γ):

Then one observes immediately:

This means that the variance of all these stochastic integrals S(γ) is identical, i.e. equal to the variance of S(0). Hence, the choice of different support points s i ∗ is only reflected in the expected value:

However, this expected value can be well understood as for finite sums it can be shown that (see Problem 10.4):

This simply follows from the fact that W(s i ∗) is not independent of W(s i ) − W(s i−1) for γ > 0. Next, we turn towards the case γ = 0.

Ito Integral

For γ = 0, S n (W) from Proposition 10.1 merges into I n (W) from (10.2). The proposition guarantees two different things: First, that the limit of I n (W) converges in mean square. We call this limit the Ito integral and write instead of S(0) the following integral:

Secondly, the proposition yields an expression for this Ito integral:

(10.3)

By the way, (10.3) is just the “stochastified chain rule” for Wiener processes from (1.​14).¹ Note the analogy and the contrast to the deterministic case (with f(0) = 0):

(10.4)

In Eq. (10.3) we find, so to speak, the archetype of Ito calculus, i.e. of stochastic calculus using Ito’s lemma. The latter will be covered in the next chapter.

The moments of the Ito integral can be determined by (10.3). According to this equation it holds for the expected value :

The variance of the integral as well can be calculated elementarily²:

where the kurtosis of 3 for Gaussian random variables was used. Hence, we have the first two results of the following proposition

Proposition 10.2 (Moments of $$\int _{0}^{t}W(s)\,dW(s)$$)

For $$I(t) =\int _{ 0}^{t}W(s)\,dW(s)$$ it holds that

and

As a third result it holds in Proposition 10.2, just as for the Stieltjes integral, that the autocovariance coincides with the variance, i.e.

We will prove this in Problem 10.2.

Stratonovich Integral

For a reason that will soon become evident, sometimes a considered competitor of the Ito integral is the Stratonovich integral . It is defined as the limit of S n (W) from (10.1) with the midpoints of the intervals as s i ∗:

This corresponds to the choice of γ = 0. 5 in Proposition 10.1. Let the limit in mean square be denoted as follows:

where “∂” does not stand for the partial derivative but denotes the Stratonovich integral in contrast to the Ito integral. By the way, with γ = 0. 5 Proposition 10.1 yields:

Hence, the Stratonovich integral stands out due to the fact that the familiar integration rule known from ordinary calculus holds true. In differential notation this rule can be formulated symbolically as follows:

This just corresponds to the ordinary chain rule, cf. (10.4). Although the Ito and the Stratonovich integral are distinguished from each other only by the choice of s i ∗ with intervals getting shorter and shorter, they still have drastically different properties. Obviously, it holds for the expected value

while the Ito integral is zero on average. However, as aforementioned, the variances of $$\int W(s)dW(s)$$ and $$\int W(s)\partial W(s)$$ coincide, cf. Problem 10.3 as well.

Example 10.1 (Alternative Stratonovich Sum)

Sometimes the Stratonovich integral is defined as the limit of the following sum, see e.g. Klebaner (2005, eq. (5.65)):

The intuition behind this is, due to the continuity of the WP, that

In fact, it can be shown more explicitly that the following difference becomes negligible in mean square:

For this purpose we consider as the mean square deviation with $$s_{i}^{{\ast}} = (s_{i-1} + s_{i})/2$$:

Due to $$s_{i-1} < s_{i}^{{\ast}} < s_{i}$$ the familiar variance and covariance formulas yield:

As for $$n \rightarrow \infty $$ the partition gets finer and finer, $$s_{i} - s_{i-1} \rightarrow 0$$, the replacement of $$W(s_{i}^{{\ast}})$$ by $$\left (W(s_{i-1}) + W(s_{i})\right )/2$$ is asymptotically well justified. ■ 

10.3 General Ito Integrals

After covering general Ito integrals, we define so-called diffusions that we will be concerned with in the following chapters.

Definition and Moments

In order to define general Ito integrals, we consider for a stochastic process X as a generalization of the sum I n (W):

(10.5)

For the Ito integral two special things apply: First, the lower endpoint of the interval is chosen as $$s_{i}^{{\ast}} = s_{i-1}$$, i.e. X(s i−1); and secondly, we integrate with respect to the WP, $$\left (W(s_{i}) - W(s_{i-1})\right )$$. If X was integrated with respect to another stochastic process, then one would obtain even more general stochastic integrals, which we are not interested in here.

If X(t) is a process with finite variance where the variance varies continuously in the course of time, and if X(t) only depends on the past of the WP, W(s) with s ≤ t, but not on its future, then the Ito sum converges uniquely and independently of the partition. The limit is called Ito integral and is denoted as follows:

The assumptions about X(t) are stronger than necessary, however, they guarantee the existence of the moments of an Ito integral, too. Similar assumptions can be found in Klebaner (2005, Theorem 4.3) or Øksendal (2003, Corollary 3.1.7).

Proposition 10.3 (General Ito Integral)

Let X(s) be a stochastic process on [0,t] with two properties:

(i)

$$\mu _{2}(s) = \mbox{ E}\left (X^{2}(s)\right ) < \infty $$ is a continuous function,

(ii)

X(s) is independent of W(s j ) − W(s i ) with s ≤ s i < s j .

Then it holds that

(a)

the sum from (10.5) converges in mean square:

(b)

the moments of the Ito integral are determined as:

Naturally, for X(s) = W(s) the extensively discussed example from the previous section is obtained. In particular, in (b) the moments from Proposition 10.2 are reproduced, and it holds for the variance that:

Example 10.2 (Stieltjes Integral)

Consider the special case where X(s) is not stochastic but deterministic,

where f(s) is continuous. Then, the conditions of existence are fulfilled which can easily be verified: The square, μ ₂(s) = f ²(s), is continuous as well, and the deterministic function is independent of W(s). Hence, it holds that

In other words: For deterministic processes, X(s) = f(s), the Stieltjes and the Ito integral coincide; the former is a special case of the latter. Due to $$\mbox{ E}\left (f^{2}(s)\right ) = f^{2}(s)$$ the already familiar formulas for expectation and variance from Proposition 9.​2 are embedded in the general Proposition 10.3. ■ 

Distribution and Further Properties

As is well known, the special case of the Stieltjes integral is Gaussian. For the Ito integral this does not hold in general. This can clearly be seen in (10.3):

i.e. the support of the distribution is bounded. Then again, the integral of a WP with respect to a thereof stochastically independent WP amounts to a Gaussian distribution. The following result is by Phillips and Park (1988).

Proposition 10.4 (Ito Integral of an Independent WP)

Let W(t) and V (s) be stochastically independent Wiener processes. Then it holds that

By showing the conditional distribution of the left-hand side given V (t) to just follow a $$\mathcal{N}(0,1)$$ distribution and therefore not to depend on this condition, one proves the result claimed in Proposition 10.4; see Phillips and Park (1988, Appendix) for details.

Note that the Ito integral again defines a stochastic process in the bounds from 0 to t whose properties could be discussed, which we will not do at this point. In the literature, however, it can be looked up that the Ito integral and the Wiener process share the continuity and the martingale properties. What is more, the integration rules outlined at the end of Sect. 8.​2 hold true for Ito integrals as well, see e.g. Klebaner (2005, Thm. 4.3).

Diffusions

For economic modeling the Ito integral is an important ingredient. However, it gains its true importance only when combined with Riemann integrals. In the following chapters, the sum of both integrals constitutes so-called diffusions ³ (diffusion processes). Hence, we now define processes X(t) (with starting value X(0)) as follows:

Frequently, we will write this integral equation in differential form as follows:

The conditions set to μ(s) and $$\sigma (s)$$ that guarantee the existence of such processes can be adopted from Propositions 8.​1 and 10.3. In general, μ(s) and $$\sigma (s)$$ are stochastic; particularly, they are allowed to be dependent on X(s) itself. Therefore, we write μ(s) and $$\sigma (s)$$ as abbreviations for functions which firstly explicitly depend on time and secondly depend on X simultaneously:

Processes μ and $$\sigma$$ satisfying these conditions are used to define diffusions X(t):

(10.6)

Recall that this differential equation actually means the following:

Example 10.3 (Brownian Motion with Drift)

We consider the Brownian motion with drift and a starting value 0:

Therefore, the differential notation reads

Hence, this is a diffusion whose drift and volatility are constant:

 ■ 

10.4 (Quadratic) Variation

From (10.3) we know that

Now, we want to understand where the expression t comes from that is subtracted from W ²(t). It will be made clear that this is the so-called quadratic variation.

(Absolute) Variation

Again, the considerations are based on an adequate partition of the interval [0, t],

For a function g the variation over this partition is defined as⁴:

If the limit exists independently of the decomposition for $$n \rightarrow \infty $$ under (8.​2), then one says that g is of finite variation and writes⁵:

The finite sum V n (g, t) measures for a certain partition the absolute increments of the function g on the interval [0, t]. If the function evolves sufficiently smooth, then V (g, t) takes on a finite value for $$(n \rightarrow \infty )$$. For very jagged functions, however, it may be that for an increasing refinement $$(n \rightarrow \infty )$$ the increments of the graph of g become larger and larger even for fixed t, such that g is not of finite variation.

Example 10.4 (Monotonic Functions)

For monotonic finite functions the variation can be calculated very easily and intuitively. In this case, V (g, t) is simply the absolute value of the difference of the function at endpoints of the interval, | g(t) − g(0) | . First, let us assume that g grows monotonically on [0, t],

Obviously, it then holds by (8.​3) that

For a monotonically decreasing function, it results quite analogously:

Monotonic functions are hence of finite variation. ■ 

Without the requirement of monotonicity, an intuitive sufficient condition exists for the function to be smooth enough to be of finite variation where this variation then has a familiar form as well.

Proposition 10.5 (Variation of Continuously Differentiable Functions)

Let g be a continuously differentiable function with derivative g′ on [0,t]. Then g is of finite variation and it holds that

The proof is given in Problem 10.6.

Example 10.5 (Sine Wave)

Let us consider a sine cycle of the frequency k on the interval [0, 2π]:

The derivative reads

Accounting for the sign one obtains as the variation:

In Fig. 10.1 it can be observed, how the sum of (absolute) differences in amplitude grows with k growing. Accordingly, the absolute variation of $$g_{k}(s) =\sin (ks)$$ multiplies with k. For $$k \rightarrow \infty $$, g k ′ tends to infinity such that this derivative is not continuous anymore. Consequently, the absolute variation is not finite in the limiting case $$k \rightarrow \infty $$. ■ 

Fig. 10.1

Sine cycles of different frequencies (Example 10.5)

Quadratic Variation

In the same way as V n (g, t) a q-variation can be defined where we are only interested in the case q = 2, – the quadratic variation:

As would seem natural, g is called of finite quadratic variation if it holds that

If g is a stochastic function, i.e. a stochastic process, then Q(g, t) and V (g, t) are defined as limits in mean square. Between the absolute variation V (g, t) and the quadratic variation Q(g, t) there are connections which we want to deal with now. If a continuous function is of finite variation, then it is of finite quadratic variation as well, where the latter is in fact zero. This is the statement of the following proposition. As it seems counterintuitive at first sight that Q, as the limit of a positive sum of squares Q n , can become zero, we start with an example.

Example 10.6 (Identity Function)

Let id be the identity function on [0, t]:

As the functions increases monotonically, it is of finite variation with

For finite n it holds that:

Q n consists of n terms, where the lengths s i − s i−1 > 0 are of the magnitude $$\frac{1} {n}$$. Due to the squaring, the n terms are of the magnitude $$\frac{1} {n^{2}}$$. Hence, the sum converges to zero for $$n \rightarrow \infty $$. This intuition can be formalized as follows:

as $$\max (s_{i} - s_{i-1}) \rightarrow 0$$ for $$n \rightarrow \infty $$. ■ 

The next proposition gives a sufficient condition for the quadratic variation to vanish.

Proposition 10.6 (Absolute and Quadratic Variation)

Let g be a continuous function on [0,t]. It then holds under (8.​2) for $$n \rightarrow \infty $$:

implies

If g is a stochastic process, then “ $$\rightarrow $$ ” is to be understood as convergence in mean square.

The proof is given in Problem 10.7. From the proposition it follows by contraposition that: If we have a positive (finite) quadratic variation, then the process does not have a finite variation. Formally, we write: From

it follows that there is no finite variation:

If a function g is so smooth that it has a continuous derivative, then Q(g, t) = 0 by Propositions 10.5 and 10.6; the other way round, values of Q(g, t) > 0 characterize how little smooth or jagged the function is.

Wiener Processes

As we know, the WP is nowhere differentiable, therefore it is everywhere so jagged that there is no valid tangent line approximation. Due to this extreme jaggedness the WP is of infinite variation as well, as we will show in a moment. More explicitly, we prove that the WP is of positive quadratic variation and does not have a finite absolute variation due to Proposition 10.6. We save the proof for an exercise (Problem 10.8).

Proposition 10.7 (Quadratic Variation of the WP)

For the Wiener process with $$n \rightarrow \infty $$ it holds under (8.​2) :

The expression Q(W, t) = t characterizes the level of jaggedness or irregularity of the Wiener process on the interval [0, t]. This non-vanishing quadratic variation causes the problems and specifics of the Ito integral. Let us recapitulate: If the Wiener process was continuously differentiable, then it would be of finite variation due to Proposition 10.5 and it would have a vanishing quadratic variation due to Proposition 10.6. However, this is just not the case.

Symbolic Notation

In finance textbooks one frequently finds a notation for time that is strange at first sight:

(10.7)

How is this to be understood? Formal integration yields

As would seem natural, the “integral” on the left-hand side here stands for Q(W, t):

Therefore, the integral equation and hence (10.7) is justified by Proposition 10.7: Q(W, t) = t. We adopt the result into the following proposition. The expressions

(10.8)

are to be understood similarly, namely in the sense of Proposition 10.8.

Proposition 10.8 (Symbolic Notation)

It holds for $$n \rightarrow \infty $$ under (8.​2) :

In symbols, these facts are frequently formulated as in (10.7) and (10.8) .

Note that the expression in (c) in Proposition 10.8 is the quadratic variation of the identity function id(s) = s:

Hence, the third claim is already established by Example 10.6. The expression from (b) in Proposition 10.8 is sometimes also called covariation (of W(s) and id(s) = s). The claimed convergence to zero in mean square is shown in Problem 10.9.

10.5 Problems and Solutions

Problems

10.1 Prove Proposition 10.1 for γ = 0 (Ito integral).

Hint: Use Proposition 10.7.

10.2 Prove the autocovariance from Proposition 10.2.

10.3 Derive that the Ito integral from (10.3) and the corresponding Stratonovich integral have the same variance.

10.4 Show for S n (W) from (10.1) with s i ∗ from Proposition 10.1,

that it holds:

10.5 Show: $$\varGamma _{n}\stackrel{2}{\rightarrow }0$$ with

for γ ∈ [0, 1] for an adequate partition, i.e. for s i − s i−1 → 0.

10.6 Prove Proposition 10.5.

10.7 Prove Proposition 10.6.

10.8 Determine the quadratic variation of the Wiener process, i.e. verify Proposition 10.7.

10.9 Show (b) from Proposition 10.8.

10.10 Determine the covariance of W(s) and $$\int _{0}^{t}W(r)\,dW(r)$$ for s ≤ t.

Solutions

10.1 In order to prove Proposition 10.1 for γ = 0, it has to be shown that I n (W) from (10.2) converges in mean square, namely to the expression given in (10.3). For this purpose, we write I n (W) as follows:

where for the last equation we added zero, $$\left (W^{2}(s_{i}) - W^{2}(s_{i})\right )$$. Hence, it furthermore follows by means of the quadratic variation:

The Wiener process is of finite quadratic variation and we know from Proposition 10.7: $$Q_{n}(W,t)\stackrel{2}{\rightarrow }t$$. This verifies the claim (as it holds that W(0) = 0 with probability 1).

10.2 Based on (10.3) we consider the process

Due to the vanishing expected value, it holds for the autocovariance that:

By adding zero one obtains:

If we assume w.l.o.g. that s ≤ t, then due to the independence of non-overlapping increments of W it holds that:

and

As the kurtosis of a Gaussian random variable is 3, it follows that

Therefore, these results jointly yield the claimed outcome:

10.3 We know about the aforementioned Ito integral from Proposition 10.2 that its variance is t ²∕2. Hence, it is to be shown that:

We use Proposition 10.1,

from which it follows immediately that

The usual variance decomposition, see (2.​1), hence yields

Due to a kurtosis of 3, the fourth moment of a $$\mathcal{N}(0,t)$$-distribution just amounts to 3 t ². Thus, one obtains

which proves the claim.

10.4 The expectation of the sum S n (W) is equal to the sum of the expectations. Therefore, we consider an individual expectation,

where simply the well-known covariance formula was used. Hence, summation yields as desired

10.5 Convergence in mean square implies that the mean squared error tends to zero. The MSE with the limit zero reads MSE(Γ n , 0) = E(Γ n ²). Therefore, it remains to be shown that: E(Γ n ²) → 0. 

For this purpose one considers with s i ∗ = (1 −γ)s i−1 +γ s i :

Forming expectation yields:

Hence, for $$n \rightarrow \infty $$ the required result is obtained as s i − s i−1 tends to zero.

10.6 For a given partition we write

As the derivative is continuous, | g′(s) | is continuous as well and hence integrable. According to the mean value theorem an s i ∗ ∈ [s i−1, s i ] exists with

Thus it follows that

Quod erat demonstrandum.

10.7 The claim is based on the bound

Due to continuity it holds that

Hence, the claim immediately follows from the bound.

10.8 It is to be shown that the mean squared error,

tends to zero. For this purpose we proceed in two steps. In the first one we show that the MSE coincides with the variance of Q n (W, t). In the second step it will be shown that the variance converges to zero.

(1)

For the first step we only need to derive E(Q n (W, t)) = t. With

the required expectation can be easily determined:

(2)

Due to the independence of the increments of the WP one has

Due to $$W(s_{i}) - W(s_{i-1}) \sim \mathcal{N}(0,s_{i} - s_{i-1})$$ and with a kurtosis of 3 for Gaussian random variables, it furthermore holds that:

Hence, plugging in yields

which completes the proof.

10.9 Let us call the aforementioned covariation CV n :

The claim reads: MSE(CV n , 0) → 0. As it obviously holds that E(CV n ) = 0, we obtain

Hence, it remains to be shown that this variance tends to zero: Due to the independence of the increments of the WP, one determines

and hence

where Q n (id, t) is the quadratic variation of the identity function, see Example 10.6. Hence, the claim is established.

10.10 We want to obtain the expected value of Y (s, t) with

Due to (10.3) it again holds that:

Therefore, we study

Let us consider the last three terms one by one. Due to the independence of the increments, one obtains:

Moreover, it is obvious that the second term is also zero. For the third term the symmetry of the Gaussian distribution yields $$\mbox{ E}\left (W^{3}(s)\right ) = 0$$. Summing up, we have shown that

and hence

References

Klebaner, F. C. (2005). Introduction to stochastic calculus with applications (2nd ed.). London: Imperical College Press.CrossRef

Øksendal, B. (2003). Stochastic differential equations: An introduction with applications (6th ed.). Berlin/New York: Springer.CrossRef

Phillips, P. C. B., & Park, J. Y. (1988). Asymptotic equivalence of ordinary least squares and generalized least squares in regressions with integrated regressors. Journal of the American Statistical Association, 83, 111–115.CrossRef

Tanaka, K. (1996). Time series analysis: Nonstationary and noninvertible distribution theory. New York: Wiley.

Footnotes

1

In particular for t = 1, (10.3) accomplishes the transition from (1.​11) to (1.​12) for the Dickey-Fuller distribution.

2

An alternative, interesting method uses the fact that the variance of a chi-squared distributed random variable equals twice its degrees of freedom:

as it holds that $$W(t)/\sqrt{t} \sim \mathcal{N}(0, 1)$$ and therefore

3

The name stems from molecular physics, where diffusions are used to model the change of location of a molecule due to a deterministic component (drift) and an erratic (stochastic) component. Physically, the influence of temperature on the motion hides behind the stochastics: The higher the temperature of the matter in which the particles move, the more erratic is their behavior.

4

Sometimes we speak of absolute variation in order to avoid confusion with e.g. quadratic variation.

5

If g is a deterministic function, then “$$\rightarrow $$” means the usual convergence of analysis. If we allow for g(t) to be a stochastic process, then we mean the convergence in mean square: “$$\stackrel{2}{\rightarrow }$$”.

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_11

11. Ito’s Lemma

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

11.1 Summary

If a process is given as a stochastic Riemann and/or Ito integral, then one may wish to determine how a function of the process looks. This is achieved by Ito’s lemma as an ingredient of stochastic calculus. In particular, stochastic integrals can be determined and stochastic differential equations can be solved with it; we will get to know stochastic variants of familiar rules of differentiation (chain and product rule). For this purpose we approach Ito’s lemma step by step by first discussing it for Wiener processes, then by generalizing it for diffusion processes and finally by considering some extensions.

11.2 The Univariate Case

The WP itself is a special case of a diffusion as defined in (10.​6). With

Eq. (10.​6) becomes (with probability one)

Thus, we consider this special case first.

For Wiener Processes

As a revision, let us recall (10.​3), which can be written equivalently as

If g(W) = W ² is defined with derivatives g′(W) = 2W and g″(W) = 2, then this equation can also be formulated as follows:

Now, this is just the form of Ito’s lemma for functions g of a Wiener process. It is a corollary of the more general case (Proposition 11.1) which will be covered in the following. Throughout, we will assume that g has a continuous second derivative (“twice continuously differentiable”).

Corollary 11.1 (Ito’s Lemma for WP)

Let g​​: $$\mathbb{R} \rightarrow \mathbb{R}$$ be twice continuously differentiable. Then it holds that

In integral form this corollary to Ito’s lemma is to be read as follows:

Strictly speaking, this integral equation is the statement of the corollary, which is abbreviated by the differential notation. However, in doing so it must not be forgotten that the WP is not differentiable. Sometimes one also writes even more briefly:

Example 11.1 (Powers of the WP)

For $$g(W) = \frac{1} {2}\,W^{2}$$ this special case of Ito’s lemma just proves (10.​3). In general, one obtains for m ≥ 2 from Corollary 11.1 with $$g(W) = \frac{W^{m}} {m}$$:

or in integral notation

 ■ 

Explanation and Proof

Corollary 11.1 can be considered as a stochastic chain rule and can loosely be formulated as follows: the derivative of g(W(t)) results as the product of the outer derivative (g′(W)) and the inner derivative (dW), plus an Ito-specific extra term consisting of the second derivative of g times $$\frac{1} {2}$$. Where this term comes from (second order Taylor series expansion) and why no further terms occur (higher order derivatives), we want to clarify now. For this purpose we prove Corollary 11.1 (almost completely) although it is, as mentioned above, a corollary to Proposition 11.1.

With s n  = t and s ₀ = 0 it holds due to (8.​3) that:

Now, on the right-hand side a second order Taylor expansion of g(W(s i )) about W(s i−1) yields

with $$\theta _{i}$$ between W(s i−1) and W(s i ):

By substitution of $$g(W(s_{i})) - g(W(s_{i-1}))$$, g(W(t)) − g(W(0)) can be expressed by two sums:

with

Now, $$\varSigma _{1}$$ just coincides with the Ito sum from (10.​5) such that it holds due to Proposition 10.​3 that:

Furthermore, we know from the section on quadratic variation (Proposition 10.​8)

As the quadratic variation of the WP is not negligible (Proposition 10.​7), this suggests the following approximation:

A corresponding convergence in mean square can actually be established, which we will dispense with at this point. Hence, except for this technical detail, Corollary 11.1 is verified.

Additionally, we want to consider why higher order derivatives do not matter for Ito’s lemma. For a third order Taylor expansion e.g. it follows

Thus, due to the summation, the term

occurs. However, it is negligible:

as the quadratic variation of the WP tends to t and as it furthermore holds that

For Diffusions

Now, we turn to Ito’s lemma for diffusions. In this section, we consider the univariate case of only one diffusion that depends on one WP only. The following variant of Ito’s lemma is again a kind of stochastic chain rule and the idea for the proof is again based on a second order Taylor expansion.

Proposition 11.1 (Ito’s Lemma with One Dependent Variable)

Let g: $$\mathbb{R} \rightarrow \mathbb{R}$$ be twice continuously differentiable and X(t) a diffusion on [0,T] with ( 10.6 ), or briefly:

Then it holds that

If X(t) = W(t) is a Wiener process, i.e. μ(t) = 0 and $$\sigma (t) = 1$$, then Corollary 11.1 is obtained as a special case.

The statement in Proposition 11.1 is given somewhat succinctly It can be condensed even more by suppressing the dependence on time:

However, it needs to be clear that by substituting dX(t) one obtains for the differential $$dg\left (X(t)\right )$$ the following lengthy expression:

The corresponding statement in integral notation naturally looks yet more extensive.

Example 11.2 (Differential of the Exponential Function)

Let a diffusion X(t) be given,

Then, how does the differential of e X(t) read? This example is particularly easy to calculate as it holds for g(x) = e x that:

Hence Ito’s lemma yields:

If X(t) is deterministic, i.e. $$\sigma (t) = 0$$, then it results

which just corresponds to the traditional chain rule (outer derivative times inner derivative). ■ 

On the Proof

Just as for the proof of Corollary 11.1, one obtains with $$\theta _{i}$$, where

from the Taylor expansion:

The first sum is approximated as desired:

The second sum is approximated by

By multiplying out the square of the differential of the Ito process,

one shows due to (cf. Proposition 10.​8),

for the second sum:

This verifies Proposition 11.1 at least heuristically.

11.3 Bivariate Diffusions with One WP

A generalization of Proposition 11.1, which is sometimes needed, is presented by the following variant of Ito’s lemma. The function g be dependent on two diffusions X ₁ and X ₂, where both are driven by the very same Wiener process. Occasionally, we will call this case (referring to the literature on interest rate models) the one-factor case as it is the identical factor W(t) driving both diffusions.

One-Factor Case

Let g be a function in two arguments, whose partial derivatives are denoted by

Then, the following proposition is a special case of Proposition 11.3.

Proposition 11.2 (Ito’s Lemma with Two Dependent Variables)

Let g​​: $$\mathbb{R} \times \mathbb{R} \rightarrow \mathbb{R}$$ be twice continuously differentiable with respect to both arguments, and let X i (t) be diffusions on [0,T] with the same WP:

Then it holds that

Note that substitution of dX i (t) in Proposition 11.2 leads again to an integral equation for the process $$g\left (X_{1}(t),\,X_{2}(t)\right )$$ including Riemann and Ito integrals.

Frequently, the time dependence of the processes will be suppressed in order to obtain a more economical formulation of Proposition 11.2:

By this notation one recognizes, that again a second order Taylor expansion hides behind Proposition 11.2, but now of the two-dimensional function g,

because the mixed second derivatives coincide due to the continuity assumed. With (10.​7) and (10.​8) it can easily be shown that (we again suppress the arguments)

and for the covariance expression as well

(11.1)

Example 11.3 (One-Factor Product Rule)

Proposition 11.2 provides us with a stochastic product rule for $$X_{1}(t)\,X_{2}(t)$$:

(11.2)

Under $$\sigma _{1}(t) = 0$$ or $$\sigma _{2}(t) = 0$$ (no stochastics), the well-known product rule is just reproduced. The derivation of (11.2) follows for $$g(x_{1},x_{2}) = x_{1}\,x_{2}$$ with

and

Hence, we obtain an abbreviated form:

where the second derivatives were plugged in. If one substitutes the first derivatives, then one obtains the result from (11.2). ■ 

Time as a Dependent Variable

Frequently it is of interest to consider another special case of Proposition 11.2. Again, g is a function in two arguments; however, the first one is time t, and the second one is a diffusion X(t):

We consciously suppress the fact that the diffusion is time-dependent as well. Since, when we talk about the derivative of g with respect to time, then we refer strictly formally to the partial derivative with respect to the first argument. Sometimes this is confusing for beginners. For example for

the derivative with respect to t refers to:

Hence, for the partial derivatives we purposely do not consider that X itself is a function of t.

With X ₁(t) = t and X(t) = X ₂(t) from Proposition 11.2 we obtain for μ ₁(t) = 1 and $$\sigma _{1}(t) = 0$$ the following circumstance.

Corollary 11.2 (Ito’s Lemma with Time as a Dependent Variable)

Let g​​: $$[0,T] \times \mathbb{R} \rightarrow \mathbb{R}$$ be twice continuously differentiable with respect to both arguments and let X(t) be a diffusion on [0,T] with ( 10.6 ), or briefly

Then it holds that

Again, suppressing time-dependence, this can be condensed to

Example 11.4 (OUP as a Diffusion)

As an application we can just prove that the standard Ornstein-Uhlenbeck process X c (t) from (9.​3) is a diffusion with

For this purpose we define as an auxiliary quantity the process

or

With this variable we define the function g such,

that it holds for the OUP:

With the derivatives

it follows from Corollary 11.2:

where dX(t) was substituted. ■ 

Further examples for practicing Corollary 11.2 can be found in the problem section.

K-Variate Diffusions

Concerning the contents, there is no reason why Proposition 11.2 should be written just with two processes. Let us consider as a generalization the case where g depends on K diffusions, all of them given by the identical WP:

Then it holds with dX k (t), k = 1, …, K, due to a second order Taylor expansion that:

As in the bivariate case, one obtains $$dX_{k}\,dX_{j} =\sigma _{k}\,\sigma _{j}\,dt$$, cf. (11.1). Sometimes, as in Corollary 11.2, time as a further variable is allowed for,

and

11.4 Generalization for Independent WP

We keep to the multivariate generalization, however, allowing for several, stochastically independent Wiener processes behind the diffusions.

The General Case

Now, $$W_{1}(t),\ldots,W_{d}(t)$$ denote stochastically independent standard Wiener processes. We allow for d factors driving each of the K diffusions. According to this, let X(t) be a K-dimensional diffusion¹ $$\mathbf{X}'(t) = (X_{1}(t),\ldots,X_{K}(t))$$, defined by d factors W j (t), j = 1, …, d:

In order to have a diffusion, it holds for μ k and $$\sigma _{kj}$$ that they may only depend on X(t) and t:

For a function g, which maps X(t) to the real numbers, Ito’s lemma reads as follows, cf. Øksendal (2003, Theorem 4.2.1).

Proposition 11.3 (Ito’s Lemma (Independent WP))

Let g​: $$\mathbb{R}^{K} \rightarrow \mathbb{R}$$ be twice continuously differentiable with respect to all the arguments, and let X k (t) be diffusions on [0,T] depending on d independent Wiener processes:

Then it holds for $$\mathbf{X}'(t) = \left (X_{1}(t),\ldots,X_{K}(t)\right )$$ that

with

(11.3)

Heuristically, (11.3) can be well justified. For this purpose we consider vectors of length d:

such that

Neglecting the dependence on time, it follows

due to (see Proposition 10.​8)

and

Let us consider the matrix

As is well known, it holds due to (10.​7) that:

Furthermore, it can be shown for stochastically independent Wiener processes that

Overall, we hence obtain

with the d-dimensional identity matrix I d . All in all it follows

which is given in (11.3).

The 2-Factor Case

Let us consider the case $$K = d = 2$$. Then, Proposition 11.3 becomes more clearly

with

and

Two interesting special cases result:

1.

$$\sigma _{12} =\sigma _{22} = 0$$ (one-factor model),

2.

$$\sigma _{12} =\sigma _{21} = 0$$ (independent diffusions).

The first case naturally corresponds to the one from the previous section: Both diffusions only depend on the same WP. The second case is the opposite extreme where both diffusions depend only on one or other of the two stochastically independent processes:

We want to discuss both borderline cases based on two examples.

Example 11.5 (2-Factor Product Rule)

Proposition 11.3 with $$K = d = 2$$ yields with the derivatives from Example 11.3 as product rule:

(11.4)

In the borderline case of only one factor, naturally the result from Eq. (11.2) is reproduced. In the second borderline case of stochastically independent diffusions, however, it holds, as in the deterministic case, that

Without restrictions (11.4) reads as follows:

The example illustrates that a proper application of Ito’s lemma needs to account for the number of factors underlying the diffusions. ■ 

Example 11.6 (2-Factor Quotient Rule)

For X ₂(t) ≠ 0 and

we obtain:

Hence Proposition 11.3 yields with $$K = d = 2$$ suppressing the arguments:

(11.5)

If X ₂ is a deterministic function $$(\sigma _{21} =\sigma _{22} = 0)$$, then the conventional quotient rule is reproduced. ■ 

11.5 Problems and Solutions

Problems

11.1 Prove part (a) from Example 9.​1.

Hint: Choose $$g\left (t,W\right ) = t\,W$$ in Corollary 11.2 or in (11.2).

11.2 Prove part (b) from Example 9.​1.

Hint: Choose $$g\left (t,W\right ) = (1 - t)\,W$$ in Corollary 11.2 or in (11.2).

11.3 Prove part (b) from Proposition 9.​1 (integration by parts).

Hint: Choose $$g\left (t,W\right ) = f(t)\,W$$ in Corollary 11.2 or in (11.2).

11.4 Prove statement (a) from Proposition 9.​4 with Ito’s lemma.

Hint: Choose $$g\left (t,W\right ) = e^{-ct}W$$.

11.5 Prove for the OUP from Proposition 9.​4:

Note that for c = 0 (WP) this reproduces (10.​3).

Hint: Choose $$g(X_{c}(t)) = X_{c}^{2}(t)$$ in Ito’s lemma.

11.6 Determine the differential of W(t)∕e W(t) according to the one-factor product rule (11.2).

11.7 Determine the differential of W(t)∕e W(t) directly from Corollary 11.1.

Solutions

11.1 For the proof we use Corollary 11.2 with

The derivatives needed read:

Hence, one determines with Corollary 11.2:

Due to g(0, W(0)) = 0, we obtain as an integral equation

which was to be shown.

11.2 As in Problem 11.1 we consider

with

Therefore, substitution into Corollary 11.2 yields

As W(0) = 0 with probability one, it follows that

which was to be shown.

11.3 As an adequate function g we choose

where f(t) is deterministic. Then, Corollary 11.2 is used with

This yields for the differential:

In integral notation this reads

As W(0) = 0 with probability one, we hence obtain the desired result:

11.4 If one chooses $$g(t,W) = e^{-ct}W$$ with

then Corollary 11.2 allows for the following calculation:

i.e.

or

Rearranging terms completes the proof.

11.5 With the function $$g(X_{c}) = X_{c}^{2}$$ and its derivatives,

Proposition 11.1 can be applied. We know that X c (t) is a diffusion with (see Example 11.4):

Plugging in into Proposition 11.1 shows:

With starting value X c (0) = 0 this translates into the following integral equation:

This is equivalent to

which amounts to the claim.

11.6 We define

and we are interested in the differential of the product. In order to apply the one-factor product rule, we need the differentials of the factors. For X ₁ it obviously holds that: dX ₁ = dW. For e −W(t) Example 11.2 yields

Hence, we have

Plugging in into the product rule (11.2) yields:

11.7 As $$g(W) = \frac{W} {e^{W}}$$ is a simple function of W, Corollary 11.1 yields a direct approach to the differential. For this purpose, we only need the derivatives (quotient rule):

Thus, it follows from Ito’s lemma that

Of course, this result coincides with the one from the previous problem.

Reference

Øksendal, B. (2003). Stochastic differential equations: An introduction with applications (6th ed.). Berlin/New York: Springer.

Footnotes

1

For vectors and matrices, the superscript denotes transposition and not differentiation. Further, the dimension d of the multivariate Wiener process should not be confused with the differential operator denoted by the same symbol.

Part III
Applications

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_12

12. Stochastic Differential Equations (SDE)

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

12.1 Summary

In the following section we discuss the most general stochastic differential equation considered here, whose solution is a diffusion. Then, linear differential equations (with variable coefficients) will be studied extensively. Here we obtain analytical solutions by Ito’s lemma. We discuss special cases that are widespread in the literature on finance. In the fourth section we turn to numerical solutions allowing to simulate processes. The sample paths displayed in the figures of Chap. 13 are constructed that way.

12.2 Definition and Existence

After a definition and a discussion of conditions for existence, we will consider the deterministic case as well. Deterministic differential equations are embedded into the stochastic ones as special cases.

Diffusions

We defined the solution of

as a diffusion process, where μ(t) and $$\sigma (t)$$ are allowed to depend on t and on X(t) itself. As the most general case of this chapter we consider diffusions as in (10.​6):

(12.1)

The solutions¹ of such differential equations can also be written in integral form:

(12.2)

Under what conditions is such a definition possible? In other words: Which requirements have to be met by the functions μ(t, x) and $$\sigma (t,x)$$, such that a solution of (12.1) exists at all – and uniquely so? This mathematical aspect is not to be overly deepened at this point, however, neither is it to be completely neglected. We consider stronger but simpler sufficient conditions than necessary. For a profound discussion see e.g. Øksendal (2003). The first assumption requires that μ and $$\sigma$$ are smooth enough in the argument x:²

(E1)

The partial derivatives of μ(t,x) and $$\sigma (t,x)$$ with respect to x exist and are continuous in x.

Secondly, we maintain a linear restriction of the growth of the diffusion process:

(E2)

There exist constants K ₁ and K ₂ with

And finally we need a well defined starting value, which may be stochastic:

(E3)

X(0) is independent of W(t) with $$\mbox{ E}(X^{2}(0)) <\infty$$ .

Under these assumptions Øksendal (2003, Theorem 5.2.1) proves the following proposition.

Proposition 12.1 (Existence of a Unique Solution)

Under the assumptions (E1) to (E3) , Eq. (12.1) has a unique solution X(t) of the form (12.2) with continuous paths and $$\mbox{ E}(X^{2}(t)) <\infty$$ .

The assumption (E3) can always be met by assuming a fixed starting value. The second assumption is necessary for the existence of a (finite) solution while (E1) guarantees the uniqueness of this solution. This is to be illustrated by means of two deterministic examples.

Example 12.1 (Violation of the Assumptions)

We examine two examples known from the literature on deterministic differential equations where $$\sigma (t,x) = 0$$. Similar cases can be found e.g. in Øksendal (2003). In the first example we set $$\mu (t,X(t)) = X^{2/3}(t)$$:

We define for an arbitrary a > 0 infinitely many solutions:

By differentiating one can observe that any X a (t) indeed satisfies the given equation. The reason for the ambiguity of the solutions lies in the violation of (E1) as the partial derivative,

does not exist at x = 0.

The second example reads for μ(t, X(t)) = X ²(t):

Again, by elementary means one proves that the solution reads

and hence tends to $$\infty$$ for t → 1. The reason for this lies in a violation of (E2): The quadratic function μ(t, x) = x ² cannot be linearly bounded. ■ 

Linear Coefficients

In order to be able to state analytical solutions, we frequently restrict generality and consider linear differential equations:

(12.3)

where the variable coefficients c i (t) and $$\sigma _{i}(t)$$, i = 1, 2, are continuous deterministic functions of time. Here, X(t) enters μ and $$\sigma$$ just linearly. Obviously, the partial derivatives from (E1) are constant (in x) and thus continuous. In addition, one obtains a linear bound:

As c i (t) and $$\sigma _{i}(t)$$ are continuous in t and hence bounded for finite t, positive constants K ₁ and K ₂ can be specified such that the inequality above holds true. Therefore, (E2) is satisfied. Therefore, a unique solution exists for linear stochastic differential equations. What is more: Ito’s lemma will allow as well for the specification of an explicit form of this analytical solution from which one can determine first and second moments as functions of time. The next section is reserved for studying equation (12.3). Before, we consider the borderline case of a deterministic linear differential equation.

Deterministic Case

By setting $$\sigma _{1}(t) =\sigma _{2}(t) = 0$$ in (12.3), we obtain a deterministic linear differential equation (in small letters to distinguish from the stochastic case),

(12.4)

or as well

Frequently, one speaks of first order differential equations, as only the first derivative is involved. As is well known, the solution reads (see Problem 12.1)

(12.5)

with

(12.6)

For c ₂(t) = 0 one obtains from (12.4) the related homogeneous differential equation (with starting value 1),

which just has z(t) from (12.6) as a solution. The following example presents the special case of constant coefficients.

Example 12.2 (Constant Coefficients)

In the case of constant coefficients,

the solution from (12.5) simplifies, see Problem 12.1:

Hence, for negative values of c ₁ it holds that the equation is stable in the sense that the solution tends towards a fixed value:

Basically, one can already observe this from the equation itself:

Namely, if x(t) lies above the limit μ, then the expression in brackets is positive and hence the change is negative such that x(t) adjusts towards the limit μ. Conversely, x(t) < μ causes a positive derivative such that x(t) grows and moves towards the limit. All in all, for c ₁ < 0 a convergence to μ is modeled. ■ 

In the following, we will see that the solution of the deterministic linear equation is embedded into the stochastic one for $$\sigma _{1}(t) =\sigma _{2}(t) = 0$$.

12.3 Linear Stochastic Differential Equations

For the solution of the equation (12.3) we expect a similar structure as in the deterministic case, (12.5), i.e. a homogeneous solution as a multiplicative factor has to be expected. Hence, we start with the solution of a homogeneous stochastic equation.

Homogeneous Solution

For $$c_{2}(t) =\sigma _{2}(t) = 0$$ one obtains from (12.3) the corresponding homogeneous linear equation . In doing so, we rename X and choose 1 as the starting value³:

(12.7)

Now, Ito’s lemma (Proposition 11.​1) is applied to $$g(Z(t)) =\log (Z(t))$$. Thus, we obtain as the solution of (12.7),

(12.8)

see Problem 12.2. Hence, for $$\sigma _{1}(t) = 0$$ the deterministic solution from (12.6) is reproduced. The solution with an arbitrary starting value different from zero therefore reads

General Solution

Let us return to the solution of equation (12.3). Now, analogously to the deterministic case (12.5), let us define Z(t) from (12.8) as a homogeneous solution. At the end of the section we will establish the following proposition whilst applying two versions of Ito’s lemma. Two interesting, alternative proofs will be given in exercise problems.

Proposition 12.2 (Solution of Linear SDE with Variable Coefficients)

The solution of (12.3) with in t continuous deterministic coefficients is

(12.9)

with the homogeneous solution

For $$\sigma _{1}(t) =\sigma _{2}(t) = 0$$ we again obtain the known result of a deterministic differential equation, cf. (12.5).

Expected Value and Variance

The process defined by (12.3) reads in integral notation

Let us define the expectation function as

then it holds due to Propositions 8.2 (Fubini) and 10.3 that:

This corresponds exactly with the deterministic equation (12.4). Hence, the solution is known from (12.5) and one obtains the form given in Proposition 12.3. The derivation of an expression for the second moment is somewhat more complex,

see Problem 12.3.

Proposition 12.3 (Moments of the Solution of a Linear SDE)

Under the assumptions of Proposition  12.2 it holds that

(12.10)

and

(12.11)

where

Example 12.3 (Homogeneous Linear SDE (Constant Coefficients))

 Since the works by Black and Scholes (1973) and Merton (1973) one assumes for the stock price X(t) the model of a homogeneous linear SDE with constant coefficients (and starting value X(0), cf. (1.​3)):

The solution resulting from (12.9) or rather from Proposition 12.2 is a geometric Brownian motion ,

This process has already been discussed in Chap. 7 With the generally derived formulas we can now recheck the moment functions from (7.​9). Proposition 12.3 yields (see Problem 12.4)

Now, assume a fixed starting value X(0). Then, it holds that

and hence

With X(0) = 1, $$\mu = c_{1} -\frac{1} {2}\sigma _{1}^{2}$$ and $$\sigma =\sigma _{1}$$ this corresponds to the notation from Chap. 7 The moments from (7.​9) are indeed reproduced. ■ 

Inhomogeneous Linear SDE with Additive Noise

For c ₂(t) ≠ 0 the linear SDE is inhomogeneous. However, at the same time the increments of the Wiener process (“noise”) enter into (12.3) additively, i.e. $$\sigma _{1}(t) = 0$$:

(12.12)

The solution results from (12.9) in Proposition 12.2 as

(12.13)

where z(t) is a deterministic function:

Note that X(t), as a Stieltjes integral, is a Gaussian process due to Proposition 9.​2. Its moments result correspondingly (for a fixed starting value X(0)). We collect these results in a corollary.

Corollary 12.1 (Additive Noise)

The solution of ( 12.12 ) with in t continuous deterministic coefficients is given by ( 12.13 ). The starting value X(0) be deterministic. Then, the process is Gaussian with:

(12.14)

(12.15)

We illustrate the corollary with the following example.

Example 12.4 (Convergence to Zero)

As a concrete example, let us consider the process given by the following equation with starting value 0:

This equation is a special case of additive noise as it holds that $$\sigma _{1}(t) = 0$$. The remaining coefficient restrictions read:

What behavior is to be expected intuitively for X(t)? The volatility term, $$\sigma _{2}(t)$$, tends to zero with t growing; does this also hold true for the variance of the process? And $$c_{1}(t) = -1$$ implies that positive values influence the change negatively and vice versa; does the process hence fluctuate around the expectation of zero? In fact, we can show that the process with vanishing variance varies around zero and therefore converges to zero.⁴ For this we need the first two moments. These can be obtained from (12.14) and (12.15):

What can be learned from this about the variance for t increasing? In Problem 12.7 we show

Then, this proves Var(X(t)) → 0 for $$t \rightarrow \infty$$. Hence, it is obvious that X(t) indeed tends to zero in mean square. ■ 

Proof of Proposition 12.2

With the homogeneous solution

of

we define the two auxiliary quantities

Note that X(t) is the process defined by (12.3) such that the differential of X ₂(t) is shown in (12.3). The proof suggested here uses the product rule for $$d\left (X_{1}(t)\,X_{2}(t)\right )$$. However, for a valid application the derivation of dX ₁(t) is necessary as well.

As a first step we use Ito’s lemma in the form of Proposition 11.​1 in order to determine the differential for X ₁(t) with

The differential becomes

In a second step, we can now apply the stochastic product rule (see Eq. (11.​2)) as an implication of Proposition 11.​2 to the auxiliary quantities⁵:

If the differentials dX ₁(t) and dX ₂(t) are plugged in, then some terms cancel each other such that it just remains:

Due to

it follows by integrating in a third step:

As Z(0) = 1, we have established (12.9) and hence completed the proof. Two alternative proofs, which are again based on Ito’s lemma (or implications thereof), are covered as exercise problems.

12.4 Numerical Solutions

Even if an analytical expression for the solution of a SDE is known, numerical solutions in the sense of simulated approximations to paths of a process are of interest. Such a simulation of a solution is, on the one hand, desired for reasons of a graphic illustration; on the other hand, in practice a whole family of numerical solutions is simulated in order to obtain a whole scenario of possible trajectories.

Euler Approximation

The interval [0, T] from (12.1) is divided w.l.o.g. in n equidistant intervals of the length $$\frac{T} {n}$$. The corresponding partition reads:

The theoretical solution from (12.2) of an arbitrary diffusion is now considered on the subinterval $$[t_{i-1},t_{i}]$$, i = 1, …, n:

This allows for the following approximation⁶ as it is discussed e.g. in Mikosch (1998):

which can also be written as:

For this purpose

was used. Hence, we have a recursive scheme. Given X ₀ = X(0) one calculates for i = 1:

and in general, for i = 1, …, n:

(12.16)

Thus we obtain n observations X i (i.e. n + 1 observations including the starting value), with which a path of the continuous-time process X(t) on [0, T] is simulated. However, this simulation requires Gaussian pseudo random numbers in (12.16),

For this purpose, a series of stochastically independent $$\mathcal{N}\left (0, \frac{T} {n}\right )$$-distributed random variables $$\varepsilon _{i}$$ need to be simulated instead of $$W(t_{i}) - W(t_{i-1})$$, in order to obtain a numerical solution X i , i = 1, …, n for the diffusion X(t) from (12.2) according to (12.16). Naturally, with n growing the approximation of a numerical solution improves.

12.5 Problems and Solutions

Problems

12.1 Show that the function given in (12.5) solves the deterministic differential equation (12.4). How does it look like in the case of constant coefficients?

12.2 Show that Z(t) from (12.8) solves the homogeneous SDE (12.7) with Z(0) = 1.

Hint: See the text.

12.3 Prove (12.11) from Proposition 12.3.

Hint: Determine for $$g\left (X(t)\right ) = X^{2}(t)$$ an expression with Ito’s lemma.

12.4 Derive the expectation and the variance of the geometric Brownian motion with Proposition 12.3,

12.5 Determine the process X(t) for which it holds that:

Hint: Proposition 12.2.

12.6 Find the solution of

for X(0) = 0. Show that it tends to zero in mean square.

Hint: Proposition 12.2.

12.7 Show for the Example 12.4:

12.8 Determine the solution of

with X(0) = 0. Show that $$\mbox{ Var}(X(t)) = (1 - t)\,t$$ and hence that X(t) tends to zero in mean square for t → 1. (This reminds us of the Brownian bridge, see (7.​6). In fact, the above SDE defines a Brownian bridge, cf. Grimmett & Stirzaker, 2001, p. 535.)

12.9 Prove Proposition 12.2 by directly applying Proposition 11.​2. Hint: Choose $$g(X,Z) = X/Z$$.

12.10 Prove Proposition 12.2 with the quotient rule from (11.​5). Hint: First derive the quotient rule for the one-factor case (d = 1) as a special case of (11.​5).

Solutions

12.1 The solution from (12.5) reads

with

Let us define the square bracket as b(t):

and

The derivative of z(t) is

Hence, the product rule yields:

which just corresponds to the claim.

In the case of constant coefficients, x(t) from (12.5) with $$z(t) = e^{c_{1}t}$$ becomes:

12.2 For Z(t) from (12.7) it holds that

with

Therefore, Proposition 11.​1 yields:

With

we hence obtain

Integration yields

Because of Z(0) = 1, the exponential function yields as desired:

12.3 Proposition 11.​1 with

is applied to X ² where the differential dX(t) is given by Eq. (12.3). This leads to

As an integral equation this reads as follows:

The expectation of the second integral is zero due to Proposition 10.​3. The expectation of the first integral results due to Fubini’s theorem as

With the definition of μ ₁(s) and μ ₂(s) it thus follows that

In differential notation this equation reads

where the functions γ ₁(t) and γ ₂(t) in the proposition following (12.11) were adequately defined. Therefore, the second moment results as the solution of a deterministic differential equation of the form (12.4). Its solution can be found in (12.5). Hence, the proposition is verified.

12.4 The geometric Brownian motion solves the homogeneous linear equation with constant coefficients:

For a stochastic starting value it holds that:

By plugging in, Proposition 12.3 yields

With the definitions from Proposition 12.3 one determines

Hence, substitution yields

Thus, the variance is calculated as

12.5 The equation at hand is linear with

Furthermore, it holds that

Therefore, the solution deduced from Proposition 12.2 reads:

with

In particular for X(0) = 1 (analogously to e ⁰ = 1) it hence holds that:

Due to the analogy to $$de^{t} = e^{t}\,dt$$ with e ⁰ = 1 this process X(t) is sometimes called “Ito exponential”. It is noteworthy that the Ito exponential is not given by $$\exp \{W(t)\}$$.

12.6 The equation is linear, see Eq. (12.3), and corresponds to the special case of additive noise, cf. (12.12), i.e. $$\sigma _{1}(t) = 0$$. The remaining coefficients read:

Hence, the expression for the solution from (12.13) yields with X(0) = 0:

where

Since $$\sigma _{2}(t)/z(t) = 1$$, the solution simplifies radically:

For this solution it obviously holds that:

Thus, for $$t \rightarrow \infty$$ we have established

which just corresponds to the required convergence in mean square.

12.7 In order to prove the inequality claimed, we define the function

with the derivative (quotient rule)

Let us call the integral of interest I,

Then it follows

where the bound follows from $$(1 + s) \leq (1 + s)^{2}$$. By rearranging terms it results that

With the definition of g it follows

which was to be shown.

12.8 This is again an inhomogeneous linear equation with additive noise:

With X(0) = 0, X(t) from (12.13) turns out to be:

with

i.e.

Due to $$c_{2}(t) = X(0) = 0$$, (12.14) yields:

Due to (12.15), the variance is:

For t → 1 the variance shrinks to zero such that it holds that X(t) tends to 0 in mean square:

12.9 The key problem with this exercise is not to confuse the different meanings of $$\sigma _{i}(t),\,i = 1,2\,$$, in Proposition 11.​2 and Eq. (12.3). Hence, firstly we adapt Proposition 11.​2 for the processes X(t) from (12.3) and Z(t) from (12.7):

Following the hint, we consider

with

Hence, Ito’s lemma (Proposition 11.​2) yields:

Integration yields:

If this equation is multiplied by Z(t), then, due to Z(0) = 1, one obtains the desired result.

12.10 We apply (11.​5) with

where X and Z are driven by the same Wiener process, say W ₁ = W. Then the one-factor quotient rule is obtained by the following restrictions:

For this purpose, $$\sigma _{x}$$ and $$\sigma _{z}$$ were defined in the previous problem. Then, the one-factor quotient rule yields:

As before, we obtain the desired result by integration and multiplication by Z(t).

References

Black, F., & Scholes, M. (1973). The pricing of options and corporate liabilities. The Journal of Political Economy, 81, 637–654.CrossRef

Grimmett, G. R., & Stirzaker, D. R. (2001). Probability and random processes (3rd ed.). Oxford: Oxford University Press.

Merton, R. C. (1973). Theory of rational option pricing. The Bell Journal of Economics and Management Science, 4, 141–183.CrossRef

Mikosch, Th. (1998). Elementary stochastic calculus with finance in view. Singapore: World Scientific Publishing.

Øksendal, B. (2003). Stochastic differential equations: An introduction with applications (6th ed.). Berlin/New York: Springer.CrossRef

Footnotes

1

Strictly speaking, this is a so-called “strong solution” in contrast to a “weak solution”. For a weak solution the behavior of X(t) is only characterized in distribution. We will not concern ourselves with weak solutions.

2

Normally, one demands that they satisfy a Lipschitz condition. A function f is called Lipschitz continuous if it holds for all x and y that there exists a constant K with

We can conceal this condition by requiring the stronger sufficient continuous differentiability.

3

The renaming justifies the assumption regarding the starting value. Consider

with a starting value different from zero, then by division one can normalize $$Z(t) = X(t)/X(0)$$.

4

For this purpose we do not need an explicit expression for the process which, however, can be easily obtained from (12.13) with X(0) = 0:

5

There is the risk of confusing the symbols $$\sigma _{i},\,i = 1, 2$$, from Eq. (12.3) with the ones from Eq. (11.​1). Note that the volatility of X ₁ (i.e. “$$\sigma _{1}$$”) is given by $$-\sigma _{1}\,X_{1}$$ while the volatility term “$$\sigma _{2}$$” of X ₂ just reads $$\sigma _{1}\,X_{2} +\sigma _{2}$$!

6

In the literature, one speaks of an Euler approximation. An improvement is known under the keyword Milstein approximation. In order to explain what is meant by “improve” in this case, one would have to become more involved in numerics.

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_13

13. Interest Rate Models

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

13.1 Summary

The results from the previous chapter will be applied to stochastic differential equations that were suggested in the literature for modeling interest rate dynamics. However, we do not model yield curves with various maturities, but consider the model for one interest rate only driven by one Wiener process (one-factor model). The next section starts with the general Ornstein-Uhlenbeck process which has the drawback of allowing for negative values. Subsequently, we discuss linear models for which negativity is ruled out. Finally, a class of nonlinear models will be considered.

13.2 Ornstein-Uhlenbeck Process (OUP)

We have already encountered the standard OUP in the chapter on Stieltjes integrals. Now, we discuss the general case, which has served as an interest rate model in the literature on finance.

Vasicek

We now assume constant coefficients for the inhomogeneous linear SDE with additive noise in (12.​12):

This defines the general Ornstein-Uhlenbeck process,

(13.1)

According to Corollary 12.​1 the solution is

(13.2)

In particular for c ₂ = 0 and $$\sigma _{2} = 1$$ we obtain the standard OUP with X(0) = 0 from (9.​3) in Sect. 9.​4 The following equation sheds additional light on the OUP:

(13.3)

In this manner, Vasicek (1977) modeled the interest rate dynamics, cf. (1.​7). Due to (13.2), the solution of this interest rate equation reads:

Setting the starting value to μ, X(0) = μ, then one obtains the form immediately corresponding to the standard OUP (9.​3),

with expectation μ. From (12.​14) and (12.​15) we obtain for an arbitrary fixed starting value X(0):

(13.4)

(13.5)

Particularly the mean value function μ ₁(t) results as a convex combination of the long-term mean value μ and the starting value X(0). For c ₁ < 0, these moments tend to a fixed value and the process can be understood as asymptotically stationary:

where the limits are taken as $$t \rightarrow \infty$$. Processes with this property are also called “mean-reverting”. The adjustment parameter c ₁ < 0 measures the “speed of mean-reversion”, i.e. the strength of adjustment: The smaller (more negative) c ₁, the more strongly dX(t) reacts as a function of the deviation of X(t) from μ. If X(t) > μ, then c ₁ < 0 causes c ₁(X(t) −μ) to have a negative impact on X(t) such that the process tends to decrease and therefore approaches μ; conversely, it holds that for X(t) < μ the process experiences a positive impulse. Furthermore, one observes the distinct influence of the parameter c ₁ on the (asymptotic) variance: The smaller the negative c ₁, the smaller is the asymptotic expression; however, for a negative c ₁ near zero the variance becomes large and the OUP loses the property of “mean reversion” for c ₁ = 0.

Determining the autocovariance function for c ₁ < 0 is also useful. We denote it by γ(t, t + h) at lag h:

where $$X_{c_{1}}(t)$$ is the standard OUP with zero expectation. From Proposition 9.​4 we hence adopt

Thus, for a large t there results an autocovariance function only depending on the temporal distance h. All in all, this is why the OUP with c ₁ < 0 can be labeled as asymptotically ($$t \rightarrow \infty$$) weakly stationary.

Simulations

In the following, processes with T = 20 and n = 1000 are simulated. For reasons of graphical comparability, the same WP is always assumed, i.e. the 1000 random variables filtered by the recursion (12.​16) are always the same ones.

First, we examine the impact of the adjustment parameter c ₁ on the behavior of the OUP. In Fig. 13.1, $$\sigma _{2} = 0.01$$. As we want to think about interest rates when looking at this figure, expectation and starting value are chosen to be μ = 5 (%). Here, it is obvious that the solid line deviates less strongly from the expected value for $$c_{1} = -0.9$$ and is thus “more stationary” than the dashed graph for $$c_{1} = -0.1$$. This is evident as the smaller (i.e. the more negative) c ₁, the smaller is the variance $$\sigma _{2}^{2}/(-2c_{1})$$ for t growing.

Fig. 13.1

OUP for $$c_{1} = -0.9$$ (X ₁) and $$c_{1} = -0.1$$ (X ₂) ($$X(0) =\mu = 5$$, $$\sigma _{2} = 0.01$$)

In the second figure, we have an OUP with the same parameter set-up for $$c_{1} = -0.9$$, however, with a starting value different from μ = 5, X(0) = 5. 1. Furthermore, the expected value function μ ₁(t) is given and one can observe how rather rapidly it approaches the value μ = 5 (Fig. 13.2).

Fig. 13.2

OUP for $$c_{1} = -0.9$$ and Starting Value X(0) = 5. 1 including Expected Value Function (μ = 5, $$\sigma _{2} = 0.01$$)

Despite the convenient property of mean reversion, the OUP is only partly suitable for interest rate modeling: Note that the process takes on negative values with a positive probability. This is due to the fact that the OUP, as a Stieltjes integral, is Gaussian:

Subsequent to Vasicek (1977), interest rate models without this drawback have been discussed.

13.3 Positive Linear Interest Rate Models

For now we stay in the class of linear SDEs, however, we restrict the discussion to the case in which positivity (more precisely: nonnegativity) is guaranteed.

Sufficient Condition

A sufficient condition for a positive evolution of the solution of a linear SDE is easy to be specified. For this purpose we naturally consider the general solution from Proposition 12.​2. Note that Z(t), as an exponential function, is always positive. With the restriction $$\sigma _{2}(t) = 0$$ the following diffusion is obtained:

(13.6)

With a positive starting value and c ₂(t) ≥ 0, a positive evolution of X(t) is ensured. The models in this section are of the form (13.6).

Dothan

Let us consider a special case more extensively. Dothan (1978) suggested for the interest rate dynamics a special case of the geometric Brownian motion:

With $$c_{2}(t) =\sigma _{2}(t) = 0$$ it holds in this case that

and hence, the interest rate X(t) can in fact not get negative. Not X(t) follows a Gaussian distribution but $$\log (X(t))$$ does. Furthermore, in Example 12.​3 we have determined the moments (for a fixed starting value):

Thus, the variance of the process increases exponentially which is why the model may not be satisfactory for interest rates.

Brennan-Schwartz

Brennan and Schwartz (1980) suggested another attractive variant. It consists of a combination of Vasicek (1977) and Dothan (1978); we choose the drift component just as for the Ornstein-Uhlenbeck process and the volatility just as for the geometric Brownian motion:

(13.7)

where, for simplicity, the starting value is set equal to μ. For c ₁ < 0 it holds that $$c_{2} = -c_{1}\mu> 0$$ such that we have indeed a positive interest rate dynamics. For this model one can show (see Problem 13.4) that the expected value results just as for Dothan (1978),

while it holds for the variance:

If $$c_{1} <-\sigma _{1}^{2}/2$$ (i.e. $$2\,c_{1} +\sigma _{ 1}^{2} <0$$), then it holds that the variance tends to a fixed positive value ($$t \rightarrow \infty$$):

If the volatility parameter $$\sigma _{1}$$ is relatively small compared to the absolute value of the negative adjustment parameter c ₁, then the model (13.7) provides a process with a fixed expected value and an asymptotically constant variance. Again, one speaks of “mean reversion”. Interestingly, the variance is not only influenced by $$\sigma _{1}$$ and c ₁ in an obvious manner: The greater $$\sigma _{1}$$, the greater Var(X(t)), and the greater c ₁ in absolute value, the more strongly or the faster the adjustment happens (and the smaller is the variance). The parameter μ > 0 from the drift function as well has a positive effect on the variance. Intuitively, this is obvious: The smaller μ (i.e. the closer to zero), the lesser X(t) can spread as the process does not get negative; conversely, it holds that the scope for the variance between the zero line and μ increases with μ growing.

Simulations

Again, processes with T = 20 and T = 1000 were simulated. For reasons of graphical comparability, the same WP as in the previous section is assumed, i.e. the 1000 random variables being filtered by the recursion (12.​16) are identical.

In Fig. 13.3 it is obvious how the variance of the geometric Brownian motion suggested by Dothan (1978) increases with the parameter $$\sigma _{1}$$. Although the expected value is constant and equal to the starting value, long periods are possible and probable in which the process does not cross the expected value. A more plausible interest rate dynamics can be observed in Fig. 13.4 for two values of the adjustment parameter c ₁. The values are chosen small enough (relative to $$\sigma _{1}$$), such that the variance remains bounded and converges to a fixed value. It is obvious: The greater c ₁ in absolute value, the smaller the variance.

Fig. 13.3

Dothan for $$\sigma _{1} = 0.01$$ (X ₁) and $$\sigma _{1} = 0.02$$ (X ₂) ($$X(0) =\mu = 5$$)

Fig. 13.4

Brennan-Schwartz for $$c_{1} = -0.9$$ (X ₁) and $$c_{1} = -0.1$$ (X ₂) ($$X(0) =\mu = 5$$, $$\sigma _{1} = 0.01$$)

13.4 Nonlinear Models

Chan, Karolyi, Longstaff, and Sanders (1992) [in short: CKLS] considered the following class of nonlinear equations for modeling short-term interest rates which is covered in this section:

(13.8)

Thus, the modeling of the drift component always corresponds to the one by Vasicek (1977). The OUP from (13.1) just results for γ = 0 while γ = 1 leads to the just discussed process from (13.7). Noninteger values of γ in between provide a nonlinear interest rate dynamics. The process from (13.8) is sometimes also called model with constant elasticity as it holds for the elasticity with the derivative of the volatility $$\sigma \,X^{\gamma }(t)$$ with respect to X that:

In order to show the interpretation of γ as an elasticity, we consider a discretization of the CKLS process as for the computer simulation. For this purpose, we define for discrete steps of the length 1, t = 1, 2, …, T:

The discrete-time version of (13.8) hence reads

or

For the conditional variance it holds:

Correspondingly, it holds for the conditional standard deviation that e.g. a doubling of x t−1 leads to a multiplication by the factor 2 γ :

Two simulated paths of the CKLS model are depicted in Fig. 13.5. They only differ in the elasticity γ. It is not surprising that the deviations from μ get greater with a greater γ.

Fig. 13.5

CKLS with γ = 0. 25 (X ₁) and γ = 0. 75 (X ₂) for $$c_{1} = -0.9$$ ($$X(0) =\mu = 5$$, $$\sigma = 0.01$$)

Cox, Ingersoll & Ross [CIR]

A particularly prominent representative of (13.8) is obtained for γ = 0. 5. This model is often used following Cox, Ingersoll, and Ross (1985):

(13.9)

The conditional standard deviation is modeled as a square root which is why one also speaks of (13.9) as a “square root process”. Consequently, the conditional variance of the increments is proportional to the level of the process.

For this nonlinear SDE it can be formally shown, which is also intuitive: If X(t) (starting from a positive starting value X(0) > 0) takes on the value zero, then the variance is zero as well, but the change dX(t) gets a positive impulse of the strength − c ₁ μ such that the process is reflected on the zero line for μ > 0. Insofar the square root process overcomes the deficiency of the OUP as an interest rate model. However, an analytical representation of the solution of (13.9) is not known.

Already for the ordinary square root process from (13.9) with $$\sigma (t,x) =\sigma \, \sqrt{x}$$ the condition of existence (E1) from Proposition 12.​1 is not fulfilled anymore as the derivative at zero does not exist. Fortunately, there are weaker conditions ensuring the existence of a solution of (13.9) – however, they do not guarantee the finiteness of the first two moments anymore. In order to show that finite moments exist up to the second order, we would need more fundamental arguments. Instead, we start with calculating the moments (finiteness assumed).

For reasons of simplicity, assume a fixed starting value equal to μ in the following: X(0) = μ. Then we obtain (see Problem 13.5), as for the OUP, on average

Under the assumption on the starting value X(0) = μ, for the second moment we obtain (cf. Problem 13.6)

from which it immediately follows for the variance

The asymptotic variance for $$t \rightarrow \infty$$ hence coincides with the one of the OUP if μ = 1; for μ < 1 it turns out to be smaller (as the process is reflected on the zero line and therefore varies in a narrow band) while it is obviously greater for μ > 1. The border case μ = 0 makes sense as well: Here, the asymptotic variance is zero as, sooner or later, the process is absorbed by the zero line.

For Fig. 13.6 an OUP with $$c_{1} = -0.9$$ and $$\sigma _{2} = 0.01$$ was simulated but the expected value of 5 % is now written as 0.05. In the example it becomes clear that the OUP can definitely become negative. In comparison, we observe a numerical solution of the corresponding square root process from (13.9) with the same volatility parameter and the same drift component. The picture confirms the theoretical considerations: The process exhibits a smaller variance and does not get negative.

Fig. 13.6

OUP and CIR for $$c_{1} = -0.9$$ ($$X(0) =\mu = 0.05$$, $$\sigma =\sigma _{2} = 0.01$$)

Further Models and Parameter Estimation

Marsh and Rosenfeld (1983) mention the variant with μ = 0 as a borderline case of (13.8). Cox, Ingersoll, and Ross (1980) consider a version with γ > 1 for a special investigation:

Finally, some models are applied which leave the framework of CKLS from (13.8) entirely, e.g. Constantinides and Ingersoll (1984) with

where both drift and volatility are nonlinear.

Given the copious possibilities for specifying a diffusion process, it is not surprising that it has been tried to, first, estimate unknown parameters and second to statistically discriminate between the different model classes. Beside the work by CKLS, the papers by Broze, Scaillet, and Zakoïan (1995) and Tse (1995) should be mentioned. As a first introduction to the topic of estimation of diffusion parameters, the corresponding chapter by Gourieroux and Jasiak (2001) is recommended.

13.5 Problems and Solutions

Problems

13.1 Derive the solution (13.2) of Eq. (13.1).

13.2 Derive the moments, (13.4) and (13.5), of the OUP (a fixed starting value X(0) assumed).

13.3 Discuss Eq. (13.1) for c ₁ = 0 as a special case of the OUP (a proposal by Merton, 1973). For this purpose, consider the solution, the expected value and the variance for c ₁ → 0, if necessary with L’Hospital’s rule. (You should be familiar with the results. By which name do you know the process as well?)

13.4 Consider now, as a combination of the interest models by Vasicek (1977) and Dothan (1978), the process from (13.7) by Brennan and Schwartz (1980),

particularly with the starting value X(0) = μ. Determine expectation and variance. How do these behave for $$t \rightarrow \infty$$ if it holds that $$2\,c_{1} <-\sigma _{1}^{2}$$?

13.5 Consider the square root process (13.9) by Cox et al. (1985). Under the assumption X(0) = μ for the starting value, derive an expression for the expected value.

13.6 Again, consider the square root process (13.9) by Cox et al. (1985). Under the assumption X(0) = μ for the starting value, derive an expression for the variance.

Solutions

13.1 Equation (13.1) is a special case of (12.​12) with constant coefficients. Hence, the solution results from (12.​13) with

From this it follows

Thus, from (12.​13) we obtain the desired result:

13.2 The expected value function is determined from (12.​14):

With $$\mu = -\frac{c_{2}} {c_{1}}$$ the formula from (13.4) results.

The variance expression follows from (12.​15):

This expression coincides with (13.5).

13.3 L’Hospital’s rule provides for c ₁ → 0:

Hence, X(t) from (13.2) merges for c ₁ → 0 into

Equation (13.4) is not suitable for determining the expected value as $$\mu = \frac{-c_{2}} {c_{1}}$$ is not defined for c ₁ → 0. Instead, one directly obtains (for X(0) fixed):

This linear growth is distinctive for the Brownian motion with drift, cf. Chap. 7

In order to determine the variance from (13.5), it is again argued with L’Hospital’s rule:

Therefore it holds that

This is the familiar variance of a Brownian motion with drift. Indeed, a Brownian motion with drift is the same as the process resulting from the OUP for c ₁ = 0 or c ₁ → 0.

13.4 This equation is linear and does not belong to the category “additive noise”. It is rather a special case of (12.​3) with

With

due to Proposition 12.​3 the expected value is given by

Hence, for c ₁ < 0 it holds:

For the second moment, we determine from (12.​11):

In particular for X(0) = μ, this simplifies, due to E(X(t)) = μ, to (with μ ₂(0) = μ ²):

Thus, the variance for X(0) = μ reads:

If $$2c_{1} <-\sigma _{1}^{2}$$, then it hence holds

as $$t \rightarrow \infty$$.

13.5 In order to determine the expected value function, we write Eq. (13.9) in integral form:

By assumption, $$\mu _{1}(0) = \mbox{ E}(X(0)) = \mbox{ E}(\mu ) =\mu.$$ Thus, Propositions 8.​2 and 10.​3(b) yield:

or rather

Due to (12.​5), the solution of this deterministic differential equation reads:

13.6 In order to determine the function of the second moment analogously to the expected value in Problem 13.5, we search for an integral equation for X ²(t). This is provided by Ito’s lemma for g(X) = X ² with g′(X) = 2X and $$g''(X) = 2$$:

where $$\sigma (t) =\sigma \sqrt{X(t)}$$ from (13.9) was substituted. Plugging in the definition of dX(t) further provides:

or rather

With X(0) = μ forming expectation yields from Propositions 8.​2 and 10.​3(b):

as μ ₁(t) = μ is constant. Thus, for μ ₂(t) a deterministic differential equation results:

With

the solution reads

Thus, in a last step the required variance is calculated as

References

Brennan, M. J, & Schwartz, E. S. (1980). Analyzing convertible bonds. The Journal of Financial and Quantitative Analysis, 15, 907–929.CrossRef

Broze, L., Scaillet, O., & Zakoïan, J.-M. (1995). Testing for continuous-time models of the short-term interest rate. Journal of Empirical Finance, 2, 199–223.CrossRef

Chan, K. C., Karolyi, G. A., Longstaff, F. A., & Sanders, A. B. (1992). An empirical comparision of alternative models of the short-term interest rate. The Journal of Finance, XLVII, 1209–1227.

Constantinides, G. M., & Ingersoll, J. E., Jr. (1984). Optimal bond trading with personal taxes. Journal of Financial Economics, 13, 299–335.CrossRef

Cox, J. C., Ingersoll, J. E., Jr., & Ross S. A. (1980). An analysis of variable rate loan contracts. The Journal of Finance, 35, 389–403.CrossRef

Cox, J. C., Ingersoll, J. E., Jr., & Ross S. A. (1985). A theory of the term structure of interest rates. Econometrica, 53, 385–407.CrossRef

Dothan, L. U. (1978). On the term structure of interest rates. Journal of Financial Economics, 6, 59–69.CrossRef

Gourieroux, Chr., & Jasiak, J. (2001). Financial econometrics: Problems, models, and methods. Princeton: Princeton University Press.

Marsh, T. A., & Rosenfeld, E. R. (1983). Stochastic processes for interest rates and equilibrium bond prices. The Journal of Finance, XXXVIII, 635–646.

Merton, R. C. (1973). Theory of rational option pricing. The Bell Journal of Economics and Management Science, 4, 141–183.CrossRef

Tse, Y. K. (1995). Some international evidence on the stochastic behavior of interest rates. Journal of International Money and Finance, 14, 721–738.CrossRef

Vasicek, O. (1977). An equilibrium characterization of the term structure. Journal of Financial Economics, 5, 177–188.CrossRef

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_14

14. Asymptotics of Integrated Processes

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

14.1 Summary

This chapter aims at providing the basics in order to understand the asymptotic distributions of modern time series econometrics. In the first section, we treat the mathematical problems of a functional limit theory as solved and get to know the basic ingredients of a functional limit theory. Then, we proceed somewhat more abstractly by presenting the mathematical hurdles to be overcome in order to arrive at a functional limit theory. Finally, we consider multivariate generalizations.

14.2 Limiting Distributions of Integrated Processes

Under classical assumptions it holds that the arithmetic mean of a sample converges to the expected value of the sample variables for the growing sample size.¹ However, if the sample is generated by a random walk, then this does not hold any longer. Hence, limits and limiting distributions for so-called integrated processes will now be discussed.

Long-Run Variance

In order to technically formulate the concept of an integrated process, we need the so-called long-run variance . However, this involves an old acquaintance from Chap. 4

Let {e t } denote a stationary discrete-time process with zero expectation and the auotcovariances

As long-run variance we define

(14.1)

Here, we rule out the case of a fractionally integrated process with long memory, I(d) with d > 0, as introduced in Chap. 5, since we require the autocovariances to be summable: $$\omega _{e}^{2} <\infty$$. In the case of a pure random process or white noise, $$e_{t} =\varepsilon _{t}$$, variance and long-run variance naturally coincide:

In general, it holds that the long-run variance is a multiple of the spectrum at the frequency zero, see (4.​3):

Now, we further assume an MA($$\infty$$) process for {e t }, see (3.​2):

(14.2)

with absolutely summable coefficients:

(14.3)

In order to determine the long-run variance, we establish an alternative expression for MA($$\infty$$) processes in Problem 14.1:

(14.4)

Due to 2π f e (0) = ω e ², one can directly read this relation from Eq. (4.​5), too.

Example 14.1 (Long-run Variance of MA(1))

We consider a moving average process of order 1,

From Example 4.​3 we adopt the spectrum,

Thus, at the origin the long-run variance results,

At the minimum, this expression takes on the value zero which happens for b = −1:

In this case {e t } is “overdifferenced”. What is meant by this, will be explained in the following. ■ 

Integrated Processes

We revisit Example 14.1 and consider the differences of a stationary MA($$\infty$$) process {e t }:

Therefore, {Δ e t } is also a stationary process where the coefficients are now called {d j }:

By definition, it hence holds

Thus, we obtain for the long-run variance:

The process {Δ e t } is overdifferenced: It is differenced more often than necessary for stationarity as {e t } itself is already stationary.² Overdifferencing is reflected in the long-run variance being zero.

If the stationary, absolutely summable process {e t } has a positive long-run variance, then we call it integrated of order zero (in symbols: $$e_{t} \sim I(0)$$):

Verbally, this means: We have to difference zero times in order to attain stationarity, and it is not differenced once more than needed. Technically, in terms of Chap. 5, this means that the process is fractionally integrated of order d = 0.

Finally, the process {x t } with

is called integrated of order one , I(1), as it is defined as sum (“integral”) of an I(0) process. The random walk from (1.​8) e.g. is integrated of order one and obviously nonstationary. It holds for I(1) random walks that differencing once,

is required by definition to obtain stationarity. Hence, I(1) processes are sometimes called difference-stationary. Also, I(1) processes are often labelled as unit root processes, or are said to have an autoregressive unit root. We briefly want to elaborate on this terminology. Assume that {e t } is a stationary autoregressive process of order p,

Consequently, the I(1) process is autoregressive of order p + 1 since

where

Hence, A x (z) has a unit root, meaning that A x (z) = 0 has a solution on the unit circle, namely the real root z = 1: A x (1) = 0.

The Functional Central Limit Theorem (FCLT)

Now, we make statements on the distribution of the stochastic step function (the partial sum process)

(14.5)

Here, ⌊y⌋ denotes the integer part of a real number y. In order to be able to divide by ω e , the process {e t } has to be I(0). In (7.​1), we have already considered a precursor of this step function as it holds:

For a graphical illustration, recall Fig. 7.​1 The following proposition for MA($$\infty$$) processes from (14.2) holds under some additional assumptions.³

Proposition 14.1 (FCLT)

Let {e t } from (14.2) be integrated of order zero and satisfy some additional assumptions. Then it holds for X n (s) from ( 14.5 ) that,

where ω e ² > 0 is from ( 14.1 ).

Note that the FCLT, so to speak, consists of infinitely many central limit theorems. For a fixed $$\overline{s}$$ it namely holds that

However, as this holds for each $$\overline{s} \in \left [0,1\right ]$$, we have quasi uncountably many central limit theorems collected in Proposition 14.1. The mathematically precise collection is the so-called weak convergence in function spaces which is symbolized by “$$\Rightarrow$$”. For the following, an intuitive notion thereof suffices, somewhat more rigorous remarks will be given in the next section.

As the I(1) process {x t } was defined as the sum of the past of {e t }, the circumstance from the proposition can also be expressed as follows: It holds for $$x_{t} =\sum _{ j=1}^{t}e_{j}$$ that

The first FCLT was proved by Donsker for pure random processes (Donsker, 1951). Particularly for an iid sequence $$e_{t} =\varepsilon _{t}$$, one hence speaks of Donsker’s theorem . Frequently, a FCTL also operates under the name “invariance principle” as it is invariant with respect to the distribution of {e t }.

First Implications

The following proposition assembles some implications being of immediate relevance in application. As an exercise, we encourage the reader to come up with the proof in order to understand why which powers of the sample size n appear in the normalization of the sums; see Problems 14.3 through 14.5. Note that the weak convergence, “$$\Rightarrow$$”, will be discussed in the next section, while “$$\stackrel{d}{\rightarrow }$$” stands for the usual convergence in distribution.

Proposition 14.2 (Some Limiting Distributions)

Let x t = x t−1 + e t with x ₀ = 0, t = 1,…,n, i.e.

where {e t } is I(0) as in Proposition 14.1 . Then it holds for $$n \rightarrow \infty:$$

with γ e (0) = V ar(e t ) and ω e ² from (14.1).

Two remarks on the functional form of the statements shall be given.

Remark 1

Note the elegant and evocative functional analogy of the sums on the left-hand side, respectively, and the integrals on the right-hand side in (a)–(c) and (e): Here, the sums are substituted by integrals, the Wiener process corresponds to the I(1) process {x t }, and the increments of the WP dW correspond to the I(0) increments Δ x t  = e t .

Remark 2

If $$e_{t} =\varepsilon _{t}$$ is white noise, then Ito’s lemma in form of (10.​3) also yields an accordance of the functional form of sample variables and limiting distributions in (f):

As well, the limiting process appearing in (d) is intuitively well justified. It is a Brownian bridge with W(1) − 1 W(1) = 0 which just reflects

for s = 1.

Example 14.2 (Demeaned WP)

From Proposition 14.2 (a) results due to x ₀ = 0:

Hence, it holds for {x t } after demeaning the following FCTL:

where

is also called a demeaned Wiener process. ■ 

In the example it was argued that it is negligible for the asymptotics whether we sum over x t−1 or x t . This holds in Proposition 14.2 (a), (c), (e) but not in (f), where on the right-hand side the sign in front of γ e (0) changes. The following corollary summarizes the corresponding results, cf. as well Problem 14.2.

Corollary 14.1 (Some Limiting Distributions)

Let x t = x t−1 + e t with x ₀ = 0, t = 1,…,n, i.e.

where {e t } is I(0) as in Proposition 14.1 . Then it holds for $$n \rightarrow \infty:$$

with γ e (0) = V ar(e t ) and ω e ² from ( 14.1 ).

14.3 Weak Convergence of Functions

In this subsection we want to briefly occupy ourselves with the mathematical concepts hiding behind Proposition 14.1. More rigorous expositions addressing an econometric audience can be found in Davidson (1994) or White (2001), see also the classical mathematical reference by Billingsley (1968).

Metric Function Spaces

Recall the stochastic step function that has led us to the WP, see (7.​1),

which can also be written more compactly as

Furthermore, we define $$\tilde{X} _{n}(t)$$ as the function that coincides with X n (t) at the lower endpoint of the interval. However, it is not constant on the intervals, but varies linearly:

By construction, $$\tilde{X}_{n}(t)$$ is a continuous function on $$\left [0,1\right ]$$, for which we also abbreviate

In contrast, X n (t) is only right-continuous and exhibits (removable) discontinuities of the first type (i.e. jump discontinuities). It belongs to the set of so-called cadlag ⁴ functions that is denoted by $$D\left [0,1\right ]$$ due to the discontinuities:

Obviously, the set of continuous functions is a subset of the cadlag functions, i.e. $$C\left [0,1\right ] \subseteq D\left [0,1\right ]$$. Now, we want X n (t) as well as $$\tilde{X}_{n}(t)$$ to converge to a WP W(t). For this purpose we need a distance measure in function spaces, a metric d. A precise mathematical definition follows.

Metric space: Let M be an arbitrary set and d a metric, i.e. a mapping,

which assigns to x and y from M a non-negative number such that the following three conditions are satisfied:

Then, M endowed with d is called a metric space, (M, d).

Example 14.3 (Supremum Metric)

Particularly $$C\left [0,1\right ]$$ or $$D\left [0,1\right ]$$ are readily endowed with the supremum metric:

In Problem 14.6 it is shown that the above-mentioned three defining properties are indeed fulfilled. ■ 

However, as X n (t) and W(t) are stochastic functions, a convergence of {X n } to W cannot simply be based on d s (X n , W). The convergence of {X n } to W has to be formulated rather as a statement on probabilities or expected values. In order to specify this, we need the concept of continuous functionals.

Continuous Functionals

Let the mapping h assign a real number to the function $$f \in D\left [0,1\right ]$$,

As the argument of h is a function, one often speaks of a functional .

Now, let the set of cadlag functions be equipped with a metric d, i.e. let $$(D\left [0,1\right ],d)$$ be a metric space. Then the functional h with h: $$D\left [0,1\right ] \rightarrow \mathbb{R}$$ is called continuous with respect to d if it holds for all $$f,g \in D\left [0,1\right ]$$ that

for

An alternative definition of continuity reads: h is called continuous with respect to d if there exists a δ > 0 with

for each $$\varepsilon> 0$$.

Strictly speaking, continuity is a “pointwise” property; however, if a functional is continuous for every considered function, then one generally speaks of continuity of the functional. The integral over a function is a typical example for a continuous functional.

Example 14.4 (Three Functionals)

Frequently, we encounter the following functionals in econometrics:

It can be shown that they are continuous on $$D\left [0,1\right ]$$ with respect to the supremum metric (cf. Problem 14.7). ■ 

Weak Convergence

We consider a set of stochastic elements, let them be random variables or stochastic functions. Let M be a set of stochastic elements and d a metric. We define somewhat loosely, see Billingsley (1968, Thm. 2.1): A sequence S n  ∈ M, $$n \in \mathbb{N}$$, converges weakly to S ∈ M for $$n \rightarrow \infty$$ if

for all real-valued mappings h that are bounded and uniformly continuous with respect to d. Symbolically, we write

This definition in terms of expected values is not very illustrative as it is hard to imagine all mappings which are bounded and continuous. In order to translate weak convergence into a probability statement, we consider the indicator function I a for an arbitrary real a and $$x \in \mathbb{R}$$:

By linearization on $$\left [a,a+\varepsilon \right ]$$ for an arbitrarily small $$\varepsilon> 0$$, the indicator function can be continuously approximated by

The approximation can become arbitrarily close to I a for small $$\varepsilon$$. Let us now choose $$M = D\left [0,1\right ].$$ Then it holds for the stochastic cadlag processes X n (t) and X(t) that

Hence, it holds for the continuous bounded functional $$h = \tilde{I} _{a}$$ for an arbitrary $$a \in \mathbb{R}$$ in case of weak convergence of {X n (t)} to X(t), i.e. for $$\mbox{ E}\left [\tilde{I} _{a}(X_{n}(t))\right ] \rightarrow \mbox{ E}\left [\tilde{I} _{a}(X(t))\right ]$$, that

Hence, we have the following illustration of {X n (t)} converging weakly to X(t): For every point in time t it holds that the sequence of distribution functions, P(X n (t) ≤ a), tends to the distribution function of X(t).

If M particularly denotes the set of real random variables and if $$X_{n} \Rightarrow X$$ holds, then the same argument shows for the distribution function that:

With the definition from the end of Chap. 8, weak convergence of random variables hence implies their convergence in distribution , $$X_{n}\stackrel{d}{\rightarrow }X$$. The converse holds as well: For random variables {X n } and X, weak convergence is synonymous with convergence in distribution.

Continuous Mapping Theorem

A further ingredient of the proof of statements as in Proposition 14.2 is presented by the continuous mapping theorem (actually: about mappings which are discontinuous only on “infinitesimal sets”); see Billingsley (1968, Thm. 5.1), Davidson (1994), and White (2001). We consider two versions of the proposition which are both special cases of a more general formulation.

Proposition 14.3 (Continuous Mapping Theorem (CMT))

For continuous mappings of convergent series it holds:

(a)

Let {X n } be a sequence of real random variables and h, h: $$\mathbb{R} \rightarrow \mathbb{R},$$ a continuous function. From $$X_{n}\stackrel{d}{\rightarrow }X$$ for $$n \rightarrow \infty$$ it follows

(b)

Let {X n (s)} and X(s) belong to $$D\left [0,1\right ]$$ and be h, h: $$D\left [0,1\right ] \rightarrow \mathbb{R},$$ a continuous functional. From $$X_{n}(s) \Rightarrow X(s)$$ for $$n \rightarrow \infty$$ it follows

Verbally, the continuous mapping theorem means that mapping and limits can be interchanged without altering the result: It does not matter whether h is applied first and then n is let to infinity, or whether $$n \rightarrow \infty$$ first is followed by the mapping. At first sight, this may seem trivial which it is definitely not, see Example 14.5.

Remember that for X = c = const convergence in distribution is equivalent to convergence in probability, see Sect. 8.​4 Hence, the CMT holds as well for convergence in probability to a constant: From

for $$n \rightarrow \infty$$ it follows that h(X n ) tends in probability to the corresponding constant:

In the literature, this fact is also known as Slutsky’s theorem . For this, we consider an example.

Example 14.5 (Consistency of Moment Estimators)

Let {y t } with $$y_{t} =\mu +\varepsilon _{t}$$ be a white noise process with expected value μ. For the arithmetic mean of a sample of the size n we know from Example 8.​4 (law of large numbers):

i.e. the empirical mean is a consistent estimator for the theoretical mean. Frequently, however, one is interested in a parameter which is a function of μ:

An estimator for $$\theta$$ constructed according to the method of moments is simply based on the substitution of the unknown expected value by its consistent estimator:

Slutsky’s theorem as a special case of (a) from Proposition 14.3 then guarantees the consistency of the moment estimator, provided h is continuous:

Such an interchangeability of some operation and a mapping is by no means trivial. It does e.g. not hold for the expectation in general: For non-linear functions h one has:

If e.g. {y t } is exponentially distributed with the parameter $$\lambda$$, i.e.

then it holds that

The function h is continuous in μ > 0, which is why the moment estimator for $$\lambda$$ is consistent:

However, one can show that it holds for an iid sample that

which is why the estimator is not unbiased for $$\lambda$$ in finite samples. ■ 

In order to justify the limit theory from the first section (i.e. in order to prove something like Proposition 14.1), mathematicians have followed two paths. First, the treatment of $$\tilde{X} _{n}(t) \in C[0,1]$$ with the ordinary supremum metric. For the proof of econometric propositions as e.g. Proposition 14.2 this has the disadvantage that the impractical “continuity appendage”,

needs to be dragged along, cf. e.g. Tanaka (1996). Second, the treatment of the more compact cadlag functions X n (t) which, however, requires a more complicated metric (Skorohod metric) and additional considerations. These mathematical difficulties are indeed solved and do not have to bother us, see e.g. Billingsley (1968) or Davidson (1994). Hence, we always work with {X n (t)} in this book.

14.4 Multivariate Limit Theory

The multivariate limit theory is based, among others, on a vector variant of Proposition 14.1 and yields generalizations of Proposition 14.2 or Corollary 14.1. For the sake of simplicity, we narrow the exposition down to the bivariate case. The following elements of a functional limit theory are kept sufficiently general to cover the case of cointegration as well as the case of no cointegration in the following two chapters. Here, we deviate from the convention of Sect. 11.​4 and do not use bold letters to denote vectors or matrices.

Integrated Vectors

The transposition of the vector z t is denoted by $$z_{t}^{{\prime}}$$. Let $$z_{t}^{{\prime}} = (z_{1,t},\,z_{2,t})$$ be a bivariate I(1) vector with starting value zero (we assume this for convenience), this means both components are I(1). Then it holds for the differences by definition,

that they are stationary with expectation zero, more precisely: Integrated of order zero. In generalization of the univariate autocovariance function we define

Note that these matrices are not symmetric in h. Rather it holds that

The long-run variance matrix is defined as a generalization of (14.1),

(14.6)

This matrix is symmetric ($$\varOmega =\varOmega ^{{\prime}}$$) and positive semi-definite (Ω ≥ 0) by construction; sometimes we omit the subscript and write Ω instead of Ω w . Note that Ω cannot be equal to the zero matrix as {w t } is not “overdifferenced” but I(0). Nevertheless, the matrix does not have to be invertible. In the following chapter we will learn that the presence of so-called cointegration depends on the rank of the matrix.

Now, let W(t) denote a vector of the length 2, namely the bivariate standard Wiener process. Its components are stochastically independent such that this vector is bivariate Gaussian with the identity matrix I ₂:

The corresponding Brownian motion is defined as a vector as follows:

with

For the existence and construction of a matrix Ω 0. 5 with the given properties, which is to some extent a “square root of a matrix”, we refer to the literature, e.g. Dhrymes (2000, Def. 2.35). However, concrete numerical examples are provided here.

Example 14.6 ($$\varOmega =\varOmega ^{0.5}(\varOmega ^{0.5})^{{\prime}}$$)

First consider a matrix of rank 1,

Now, let us define

Multiplied by itself, P ₁ just yields the starting matrix Ω ₁. In the second example of a diagonal matrix with full rank,

the construction of the square root becomes even more obvious:

As Ω ₂ is a diagonal matrix, P ₂ = Ω ₂ 0. 5 is just generated by taking the square root of the diagonal. Let us consider a third example with full rank:

Here it is not obvious which form Ω ₃ 0. 5 may have. However, one can check that

In the case where Ω has full rank it is actually easy to come up with one specific factorization. Under full rank, Ω has a strictly positive determinant such that

One may hence define the following triangular matrix factorizing Ω from (14.6),

(14.7)

which is sometimes called Cholesky decomposition of Ω. For Ω ₃ one obtains that way

We hence reinforce the postulate that Ω 0. 5 of a matrix Ω is not unique. In particular, T ₃ is not symmetric, while P ₃ is. Still, it holds $$T_{3}\,T_{3}^{{\prime}} = P_{3}P_{3} =\varOmega _{3}$$. ■ 

Functional Limit Theory

Phillips (1986) and Phillips and Durlauf (1986) introduced under appropriate assumptions multivariate generalizations of Proposition 14.1 into econometrics:

(14.8)

For the individual components this means:

where the Brownian motions are generally not independent of each other. Independence is only present if Ω w is diagonal (ω ₁₂ = 0) as it then holds:

Under adequate technical conditions, which need not to be specified here, the following proposition holds, cf. as well Johansen (1995, Theorem B.13).

Proposition 14.4 (I(1) Asymptotics)

Let {z t } be a 2-dimensional integrated process and Δz t = z t − z t−1 = w t with $$\mbox{ E}(w_{t}) = (0,\,0)^{{\prime}}$$ and Ω w from ( 14.6 ). Then it holds under some additional assumptions that

as $$n \rightarrow \infty$$ .

Naturally, these results can be expressed in terms of B = Ω w 0. 5 W as well:

The limit from Proposition 14.4 (a) is to be read as a vector of Riemann integrals,

In (b), we have a square matrix:

Concluding, both these outcomes are results from (14.8) and from a multivariate version of the continuous mapping theorem, cf. Proposition 14.3. The third result from Proposition 14.4, the matrix of Ito integrals, corresponds to result (f) from Proposition 14.2, also cf. Corollary 14.1:

In the multivariate setting, such a convergence cannot be elementarily derived any longer. For a proof see e.g. Phillips (1988) or Hansen (1992a).

14.5 Problems and Solutions

Problems

14.1 Derive with elementary means the expression (14.4) for the long-run variance of the MA($$\infty$$) process {e t } from (14.2).

14.2 Derive the limiting distribution of $$n^{-1}\sum _{t=1}^{n}x_{t}e_{t}$$ from Corollary 14.1.

14.3 Prove Proposition 14.2(a), (c), (d) and (e). When doing this, you may assume the functionals to be continuous with respect to an appropriate metric.

14.4 Prove Proposition 14.2(b).

14.5 Prove Proposition 14.2(f).

14.6 Check that d s (f, g) with

is a metric (supremum metric).

14.7 Show that the integral functionals h ₁, h ₂ and h ₃ from Example 14.4 are continuous on $$D\left [0,1\right ]$$ with respect to the supremum metric.

Solutions

14.1 First we adopt the autocovariance function from Proposition 3.​2:

In (14.4), the long-run variance is formulated as follows:

As the coefficients {c j } are absolutely summable, the infinite sums can be multiplied out:

Hence, the equivalence of the representations of the long-run variance from (14.1) and (14.4) is derived.

14.2 Due to x t  = x t−1 + e t we write

The limiting behavior of the first sum on the right-hand side is known from Proposition 14.2, and the second sum on the right-hand side tends to Var(e t ) = γ e (0). Thus, elementary transformations yield

The application of Ito’s lemma completes the proof.

14.3

(a)

The interval [0, 1) is split up into n subintervals of the same length,

On each of these subintervals, we define the step function X n (s) as the appropriately normalized I(1) process,

and on the endpoint, it holds for s = 1:

Due to Proposition 14.1 we have ($$n \rightarrow \infty$$)

Furthermore, we take a trick into account which allows for expressing a sum over x t−1 as an integral:

Equipped with this, one explicitly obtains

As we may assume that the functional

is continuous with respect to an appropriate metric, Proposition 14.3 yields for $$n \rightarrow \infty$$:

Hence, the proof is complete.

(c)

Just as for the proof of (a) we define the stochastic step function

and in addition the analogously constructed deterministic step function

t = 1, …, n, and T n (1) = 1. Then it holds that

For $$n \rightarrow \infty$$ it holds

and hence, due to the continuity of the integral function, as claimed

(d)

Again, with the definition of X n (s) it is shown:

(e)

The proof is entirely analogous to (a),

14.4 The result can be shown in three steps.

(i)

By definition it holds:

(ii)

Thus, the sum of interest is reduced to known quantities:

or

where Proposition 14.1 for s = 1 and Proposition 14.2 (a) were applied.

(iii)

From Example 9.​1 (a) with t = 1 we know:

Hence, the claim is proved.

14.5 The first result is based on the binomial formula applied to x t  = x t−1 + e t :

Solving this for the mixed term, we obtain:

Thus, we come to the second claim. Obviously, it holds that

The special case (10.​3) of Ito’s lemma then establishes the equality claimed.

14.6 In order to have a metric, three conditions from the text need to be fulfilled.

(i)

The condition f = g means

which is equivalent to

From this it immediately follows d s (f, g) = 0. Conversely,

immediately implies | f(t) − g(t) | = 0. Therefore, two functions f and g in are indeed equal if and only if they have zero distance according to the supremum metric.

(ii)

The symmetry condition is obviously given as it holds for the absolute value:

(iii)

Finally, the triangle inequality can be established by adding zero:

The second, rather plausible inequality follows from the properties of the supremum metric, see e.g. Sydsæter, Strøm, and Berck (1999, p.77).

14.7 We consider three functionals h i (f),  i = 1, 2, 3, which assign a real number to the function f(t).

(i)

The first functional is just the integral from zero to one. Here it holds:

Hence, the smaller the deviation of f and g, the nearer are h ₁(f) and h ₁(g). This exactly corresponds to the definition of continuity.

(ii)

The second functional is the integral over a quadratic function. Here we obtain with the binomial formula and the triangle inequality:

For the last inequality, it was approximated by the supremum as in (i) and then integrated from 0 to 1. Hence, it holds by definition

As g(t) belongs to D[0, 1] and is thus absolutely integrable, h ₂(f) tends to h ₂(g) if the distance between f and g gets smaller, which amounts to continuity.

(iii)

The third functional is $$\frac{1} {h_{2}}$$. Hence, we reduce the continuity of h ₃ to the one of h ₂:

Hence, from the (quadratic) integrability of f and g and the continuity of h ₂ follows, as required, the continuity of h ₃.

References

Billingsley, P. (1968). Convergence of probability measures. New York: Wiley.

Davidson, J. (1994). Stochastic limit theory: An introduction for econometricians. Oxford/New York: Oxford University Press.CrossRef

Dhrymes, Ph. J. (2000). Mathematics for econometrics (3rd ed.). New York: Springer.CrossRef

Donsker, M. D. (1951). An invariance principle for certain probability limit theorems. Memoirs of the American Mathematical Society, 6, 1–12.

Hansen, B. E. (1992a). Convergence to stochastic integrals for dependent heterogeneous processes. Econometric Theory, 8, 489–500.CrossRef

Johansen, S. (1995). Likelihood-based inference in cointegrated vector autoregressive models. Oxford/New York: Oxford University Press.CrossRef

Phillips, P. C. B. (1986). Understanding spurious regressions in econometrics. Journal of Econometrics, 33, 311–340.CrossRef

Phillips, P. C. B. (1987). Time series regression with a unit root. Econometrica, 55, 277–301.CrossRef

Phillips, P. C. B. (1988). Weak convergence of sample covariance matrices to stochastic integrals via martingale approximations. Econometric Theory, 4, 528–533.CrossRef

Phillips, P. C. B., & Durlauf, S. N. (1986). Multiple time series regression with integrated processes. Review of Economic Studies, LIII, 473–495.

Phillips, P. C. B, & Perron, P. (1988). Testing for a unit root in time series regression. Biometrika, 75, 335–346.CrossRef

Phillips, P. C. B., & Solo, V. (1992). Asymptotics for linear processes. The Annals of Statistics, 20, 971–1001.CrossRef

Pötscher, B. M., & Prucha, I. R. (2001). Basic elements of asymptotic theory. In B. H. Baltagi (Ed.), A companion to theoretical econometrics (pp. 201–229). Malden: Blackwell.

Sydsæter, K., Strøm, A., & Berck, P. (1999). Economists’ mathematical manual (3rd ed.). Berlin/New York: Springer.CrossRef

Tanaka, K. (1996). Time series analysis: Nonstationary and noninvertible distribution theory. New York: Wiley.

White, H. (2001). Asymptotic theory for econometricians (2nd ed.). London/San Diego: Academic Press.

Footnotes

1

For a review on the convergence of random sequences, we recommend Pötscher and Prucha (2001).

2

In econometrics, this overdifferencing is also described by the fact that {Δ e t } is integrated of order − 1 , $$\varDelta e_{t} \sim$$ I( − 1): {e t } is differenced one more time although the process is already stationary.

3

Phillips and Solo (1992) assume that the innovations $$\{\varepsilon _{t}\}$$ form an iid sequence and that $$\sum _{j=0}^{\infty }j\vert c_{j}\vert <\infty$$, which is more restrictive than (14.3). Phillips (1987) or Phillips and Perron (1988) do without the iid assumption, but require more technical restrictions. For a discussion of further sets of assumptions ensuring Proposition 14.1 see also Davidson (1994).

4

This French acronym (sometimes also “càdlàg”) stands for “continue à droite, (avec une) limite à gauche”: right-continuous and bounded on the left.

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_15

15. Trends, Integration Tests and Nonsense Regressions

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

15.1 Summary

Now we consider some applications of the propositions from the previous chapter. In particular, {e t } and {x t } are integrated of order 0 and integrated of order 1, respectively, cf. the definitions above Proposition 14.​2. It turns out that the regression of a time series on a linear trend leads to asymptotically Gaussian estimators. However, test statistics constructed to distinguish between integration of order 1 and $$0$$ are not Gaussian. Finally, we cover the problem of nonsense regressions, which occur particularly in the case of independent integrated variables.

15.2 Trend Regressions

Let {y t } be trending in the sense that the expectation follows a linear time trend , E(y t ) = β t. The slope parameter is estimated following the least squares (LS) method. The residuals, $$\widehat{res}_{t} = y_{t} -\hat{\beta } t$$, are then the detrended series. Estimation relies on a sample of size n.

Detrending

The time series {y t } is regressed on a linear time trend according to the least squares method. For the sake of simplicity, we neglect a constant intercept that would have to be included in practice. The LS estimator $$\hat{\beta }$$ of the regression

(15.1)

with the empirical residuals $$\{\widehat{res}_{t}\}$$ is

For the denominator the following formula holds

(15.2)

such that one obtains asymptotically ($$n \rightarrow \infty$$)

This limit is a special case of a more general result dealt with in Problem 15.1.

In this chapter we consider two models with a linear time trend in the mean. First, if the deviations from the linear trend are stationary, i.e. if the true model reads

(15.3)

then we say that {y t } is trend stationary . More precisely, {e t } satisfies the assumptions of an I(0) process discussed in the previous chapter with long-run variance $$\omega _{e}^{2}$$ defined in (14.​1). Second, the stochastic component may be I(1). Then one says that {y t } is integrated with drift (β ≠ 0): $$\varDelta y_{t} =\beta +e_{t}$$. By integrating (summing up) with a starting value of zero, this translates into

(15.4)

An example will illustrate the difference between these two trend models.

Example 15.1 (Linear Time Trend)

In Fig. 15.1 we see two time series, following the slope 0. 1 on average, t = 1, 2, …, 250. The upper graph shows a trend stationary series,

where $$\varepsilon _{t} \sim \text{ii}\,\mathcal{N}(0,1)$$. The lower diagram shows with identical $$\{\varepsilon _{t}\}$$

i.e. $$y_{t}^{(1)}$$ is I(1) with drift:

Fig. 15.1

Linear Time Trend

The deviations from the linear time trend are in the lower case I(1) and are hence much stronger than in the upper case. ■ 

Trend Stationarity

The following proposition contains the properties of LS detrending under trend stationarity.

Proposition 15.1 (Trend Stationary)

Let {y t } from ( 15.3 ) be trend stationary, and let $$\hat{\omega }_{e}$$ denote a consistent estimator for ω e . It then holds for $$\hat{\beta }$$ from ( 15.1 ) that

(15.5)

as $$n \rightarrow \infty$$ .

A proof is provided in Problem 15.2 relying on Proposition 14.​2 (b). In practice, a consistent estimator $$\hat{\omega }_{e}$$ for ω e will be built from LS residuals , $$\widehat{res}_{t} = y_{t} -\hat{\beta }\, t$$, see below for details.

Notice the fast convergence of the estimator $$\hat{\beta }$$ to its true value (with rate n 1. 5). In real applications we would typically calculate a trend regression with intercept,

Hassler (2000) showed that limiting normality and the fast convergence rate of the LS estimator with intercept, $$\bar{\beta }$$, pertains, although the variance is affected:

again, this requires that $$\bar{\omega }_{e}$$ constructed from the residuals is consistent.

I(1) with Drift

Now we assume {y t } to be integrated of order 1, possibly with drift. Note, however, that the following proposition does not require β ≠ 0, as we can learn from the proof given in Problem 15.3.

Proposition 15.2 (I(1) with Drift)

Let {y t } from ( 15.4 ) be I(1), possibly with drift: $$\varDelta y_{t} =\beta +e_{t}$$ . Let $$\hat{\omega }_{e}$$ denote a consistent estimator for ω e . It then holds for $$\hat{\beta }$$ from ( 15.1 ) that

(15.6)

as $$n \rightarrow \infty$$ .

Consistent estimation of the long-term variance will rely on the differences of the residuals ($$\{\varDelta \widehat{res}_{t}\}$$). Here, the LS estimator obviously converges much more slowly, namely with the more usual rate n 0. 5 instead of n 1. 5 as in the trend stationary case. This does not come as a surprise given the impression from Fig. 15.1: Since the trend stationary series follows the straight line more closely, the estimation of the slope is more precise than in the I(1) case with drift.

A final word on Proposition 15.2: The same variance as in (15.6) is obtained in the case of a regression with intercept, see Durlauf and Phillips (1988).

Consistent Estimation of the Long-Run Variance

As we have just seen, a consistent estimation of $$\omega _{e}^{2}$$ is frequently needed in practice in order to apply the functional limit theory. For this purpose, {e t } is to be approximated by residuals or differences thereof: In the trend stationary case we have $$\widehat{res}_{t} =\hat{ e}_{t}$$, while for I(1) processes it holds $$\widehat{res}_{t} =\hat{ x}_{t}$$, such that $$\varDelta \widehat{res}_{t} =\hat{ e}_{t}$$. The intuition behind an estimator $$\hat{\omega }_{e}^{2}$$ is readily available from (14.​1), $$\omega _{e}^{2} =\gamma _{e}(0) + 2\,\sum _{h=1}^{\infty }\gamma _{e}(h)$$, although three modifications are required. First, the theoretical autocovariances need to be replaced by the sample analogues,

Second, the infinite sum has to be cut off as sample autocovariances can be computed only up to the lag n − 1. Third, in order to really have a consistent (and positive) estimator, a weight function w B (⋅ ) depending on a tuning parameter B is needed (whose required properties will not be discussed at this point, but see Example 15.2). In the statistics literature, w B (⋅ ) is often called a kernel. Put together, we obtain as sample counterpart to (14.​1):

(15.7)

For most kernels w B (⋅ ), the parameter B (the so-called bandwidth) takes over the role of a truncation, i.e. the weights are zero for arguments greater than B:

In fact, the choice of B is decisive for the quality of an estimation of the long-run variance. On the one hand, B needs to tend to infinity with the sample size, on the other it needs to diverge more slowly than n. Further issues on bandwidth selection and choice of kernels have been pioneered by Andrews (1991); see also the exposition in Hamilton (1994, Sect. 10.​5).

Example 15.2 (Bartlett Weights)

According to Maurice S. Bartlett (English statistician, 1910–2002), a very simple weight function has a triangular form:

Hence, the sequence of weights reads for $$h = 1,2,\ldots B + 1$$:

Plugging this in (15.7), the sum is indeed truncated at B:

Although the simple Bartlett weights are by no means optimal, they are widespread in econometrics up to the present time, often also named after Newey and West (1987) who popularized them in their paper. ■ 

15.3 Integration Tests

We consider one test for the null hypothesis that there is integration of order 1 and one for the null hypothesis that there is integration of order 0.

Dickey-Fuller [DF] Test for Nonstationarity

The oldest and the most frequently applied test on the null hypothesis of integration of order 1 stems from Dickey and Fuller (1979).¹ In the simplest case (without deterministics) the regression model reads

with the null hypothesis

against the alternative of stationarity, H ₁: | a | < 1. For the LS estimator $$\hat{a}$$ from

(15.8)

we obtain under H ₀ the limiting distribution of the normalized LS estimator $$n\,(\hat{a} - 1)$$. Often, one does not work with $$n\,(\hat{a} - 1)$$, but the associated t-statistic:

The limiting distributions from Proposition 15.3 are established in Problem 15.4.

Proposition 15.3 (Dickey-Fuller Test)

Let the I(1) process {x t } (Δx t = e t ) satisfy the assumptions from Proposition 14.2 . It then holds

(a)

for $$\hat{a}$$ from regression ( 15.8 ) without intercept that

(b)

and for the t-statistic that

as $$n \rightarrow \infty$$ .

In this elegant form these limiting distributions were first given by Phillips (1987).² Note that the distributions depend on two parameters γ e (0) and $$\omega _{e}^{2}$$ called “nuisance parameters” in this context. They are a nuisance because we have to somehow deal with them (remove their effect) without being economically interested in their values. Particularly, if $$e_{t} =\varepsilon _{t}$$ is a white noise process, then the first limit simplifies to the so-called Dickey-Fuller distribution, cf. (1.​11) and (1.​12):

This expression does not depend on unknown nuisance parameters anymore; hence, quantiles can be simulated and approximated. One rejects for small (too strongly negative) values as the test is one-sided against the alternative of stationarity ( | a | <  1). Similarly, for $$\omega _{e}^{2} =\gamma _{e}(0)$$ the limit of the t-statistic simplifies to a ratio free of nuisance parameters:

(15.9)

The numerator equals that one of $$\mathcal{D}\mathcal{F}_{a}$$ from (1.​11).

In the relevant case that {e t } is serially correlated, one can run two paths in practice. Firstly, the test statistics can be appropriately modified by estimators for $$\omega _{e}^{2}$$ and γ e (0). Phillips (1987) and Phillips and Perron (1988) paved this way. Secondly, one frequently calculates the regression augmented by K lags (ADF test):

or with $$\phi = a - 1$$

If K is so large that the error term is free of serial correlation, then the t-statistic belonging to the test on a = 1, i.e. ϕ = 0, converges to the Dickey-Fuller distribution, and available tabulated percentiles serve as critical values; for further details see Said and Dickey (1984) and Chang and Park (2002). In practice, one would run a regression with intercept. This leaves the functional shape of the limiting distributions unaffected; only replace the WP W by a so-called demeaned WP, see also Problem 15.5.

KPSS Test for Stationarity

Now, the null and the alternative hypotheses are interchanged. The null hypothesis of the test suggested by Kwiatkowski, Phillips, Schmidt, and Shin (1992) claims that the time series {y t } is integrated of order 0 while it exhibits a random walk component under the alternative (hence, it is I(1)). Actually, this is a test for parameter constancy. The model reads

with the hypotheses

Under the null hypothesis, the intercept is again estimated by LS:

From this, the partial sum process {S t } is obtained:

Again, we assume that the long-run variance is consistently estimated from the residuals $$\hat{e}_{t}$$ under H ₀. Then, the test statistic is formulated as

The limiting distribution under the null hypothesis of stationarity is provided in Problem 15.6.

Proposition 15.4 (KPSS Test)

Let the I(0) process {e t } satisfy the assumptions from Proposition 14.2 , and $$y_{t} = c + e_{t}$$ . It then holds that

as $$n \rightarrow \infty$$ .

This expression does not depend on unknown nuisance parameters, and critical values are tabulated. In econometrics one often speaks about the KPSS distribution although this distribution has a long tradition in statistics where it also trades under the name of the Cramér-von-Mises ($$\mathcal{C}\mathcal{M}$$) distribution. Quantiles were first tabulated by Anderson and Darling (1952). The limit $$\mathcal{C}\mathcal{M}$$ is constructed from a Brownian bridge with $$W(1) - 1\,W(1) = 0$$, which reflects of course that S n  = 0 by construction.

Linear Time Trends

Many economic and financial time series are driven by a linear time trend in the mean. Consider a trend stationary process $$x_{t} =\beta t + e_{t}$$, which is not integrated of order 1. Still, it holds that

Hence, it is not surprising that a regression of x t on x t−1 results in a Dickey-Fuller statistic not rejecting the false null hypothesis of integration of order 1. In order to avoid the confusion of a stochastic trend (unit root process integrated of order 1) and a linear time trend, one has to include time as explanatory variable in the lag-augmented regression estimated by LS (we also add now a constant intercept)³:

Under the null hypothesis that {x t } is integrated of order 1 (possibly with drift), the t-statistic associated with $$\tilde{a} - 1$$ obeys the following limiting distribution:

(15.10)

The functional form is identical to that from (15.9), only that the WP is replaced by a so-called detrended WP $$\tilde{W}$$ defined for instance in Park and Phillips (1988, p. 474):

We call $$\widetilde{\mathcal{D}\mathcal{F}}$$ also the detrended Dickey-Fuller distribution; critical values are tabulated in the literature.

Similarly, the KPSS test may be modified to account for a linear time trend. Simply replace the demeaned series $$\hat{e}_{t} = y_{t} -\overline{y}$$ by the detrended one:

Computing the KPSS statistic $$\tilde{\eta }$$ from $$\tilde{e}_{t}$$ results asymptotically in a detrended Cramér-von-Mises distribution ($$\widetilde{\mathcal{C}\mathcal{M}}$$ say) as long as the null hypothesis of (trend) stationarity holds true. Details on $$\widetilde{\mathcal{C}\mathcal{M}}$$ and critical values thereof are given in Kwiatkowski et al. (1992): $$\tilde{\eta }\stackrel{d}{\rightarrow }\widetilde{\mathcal{C}\mathcal{M}}$$ with

(15.11)

15.4 Nonsense Regression

Nonsense or spurious regressions occur if two integrated processes are regressed on each other without being cointegrated. Hence, we need to start by defining cointegration and by briefly recapping the standard statistics from the regression model.

Cointegration

The starting point for the econometric analysis of integrated time series is the concept of cointegration , which is also rooted in the equilibrium paradigm of economic theory. The idea of cointegration was introduced by Granger (1981) and has firmly been embedded in econometrics by the work of Engle and Granger (1987).

Let us consider two integrated processes {x t } and {y t } integrated of order 1. Sometimes we assume that there is a linear combination with b ≠ 0 such that

(15.12)

is integrated of order 0. Here, y = b x is interpreted as a long-run equilibrium relation, as postulated by economic theory, from which, however, the empirical observations deviate at a given point in time t by v t . If there is no linear combination of two I(1) processes that is stationary, then {x t } and {y t } are called not cointegrated.

Estimators and Statistics in the Regression Model

We consider the regression model without intercept (for the sake of simplicity) that is estimated by means of the least squares (LS) method:

(15.13)

In this section we work under the assumption that {x t } and {y t } are integrated of order 1 but not cointegrated. Hence, each linear combination, $$u_{t} = y_{t} -\beta \, x_{t}$$, is necessarily I(1) as well. Let {y t } and {x t } be both components of {z t }:

Hence, it holds with (14.​8):

If the regressand y t and the regressor x t are stochastically independent, then this property is transferred to the limiting processes B ₁ and B ₂. Nevertheless, we will show that then $$\hat{\beta }$$ from (15.13) does not tend to the true value that is zero. Instead, a (significant) relation between the independent variables is spuriously obtained. Since Granger and Newbold (1974), this circumstance is called spurious or nonsense regression.

The LS estimator from the regression without intercept reads

The t-statistic⁴ belonging to the test on the parameter value 0 is based on the difference of estimator and hypothetical value divided by the estimated standard error of the estimator:

As a measure of fit, the (uncentered) coefficient of determination is frequently calculated,⁵

Finally, the Durbin-Watson statistic is a well-established measure for the first order residual autocorrelation,

Let us briefly recall the behavior of these measures if we worked with I(0) variables x and y not being correlated (β = 0). Then, $$\hat{\beta }$$ would tend to 0, the t-statistic would converge to a normal distribution and the coefficient of determination would tend to zero. Finally, the Durbin-Watson statistic would converge to 2 (1 −ρ ₁) > 0, where ρ ₁ denotes the first order autocorrelation coefficient of the regression errors. In the case of nonsense regressions, we obtain qualitatively entirely different asymptotic results.

Asymptotics

Due to Proposition 14.​4 (b) it holds for the denominator of the LS estimator

and for the numerator we obtain

Both results put together yield

In particular, if y t and x t are stochastically independent, then $$\hat{\beta }$$ does not tend to the true value 0 but to the random variable $$\beta _{\infty }$$. And as if that was not enough, the t-statistic belonging to the test on the true parameter value β = 0 tends to infinity in absolute value! Hence, in this situation t-statistics highly significantly reject the true null hypothesis of no correlation and therefore report absurd relations as being significant. This phenomenon was experimentally discovered for small samples by Granger and Newbold (1974) and asymptotically proved by Phillips (1986). For $$n \rightarrow \infty$$ it namely holds that n −0. 5 t β has a well-defined limiting distribution. In the problem section we prove in addition the further properties of the following proposition.

Proposition 15.5 (Nonsense Regression)

For I(1) processes {x t } and {y t } it holds in case of no cointegration with the notation introduced that

as $$n \rightarrow \infty$$ .

As already has been emphasized: The results (a), (b) and (d) justify to speak of a nonsense regression. A first hint at the lack of cointegration is obtained from the first order residual autocorrelation: For nonsense or spurious regressions, the Durbin-Watson statistic tends to zero.

Example 15.3 (Hendry, 1980)

Hendry (1980) shows the real danger of nonsense regressions. In his polemic example, the price development (measured by the consumer price index P) is to be explained. For this purpose, first a money supply variable M is used. Then a second variable C is considered (for which we could think of consumption), and it appears that this time series explains the price development better than M does. However, there can in fact be no talk of explanation; this is a nonsense correlation as behind C the cumulated rainfalls are hidden (hence, the precipitation amount)! By the way, note that P and C are not (only) integrated of order 1 but in addition exhibit a deterministic time trend, which only aggravates the problem of nonsense regression. ■ 

If integrated variables are not cointegrated, they have to be analyzed in differences, i.e. Δ y t is regressed on Δ x t , resulting in the familiar stationary regression model. Naturally, not all integrated economic variables lead to nonsense regressions. This does not happen under cointegration, which brings us to the final chapter.

15.5 Problems and Solutions

Problems

15.1 Show

as $$n \rightarrow \infty$$.

15.2 Prove Proposition 15.1.

15.3 Prove Proposition 15.2.

15.4 Prove Proposition 15.3.

15.5 Derive an expression for the limiting distribution of $$n(\overline{a} - 1)$$ under a = 1 if a model with intercept is estimated in the Dickey-Fuller test:

Assume $$\varDelta x_{t} =\varepsilon _{t}$$ to be white noise.

15.6 Prove Proposition 15.4.

15.7 Prove the statements (b), (c) and (d) from Proposition 15.5.

15.8 Prove statement (e) from Proposition 15.5.

Solutions

15.1 Define the continuous and hence Riemann-integrable function f on [0, 1] with antiderivative F:

Further, we work with the equidistant partition

Hence,

which proves the claim.

15.2 According to the hints in the text, it holds

or

The denominator tends to $$\frac{1} {3}$$ whereas Proposition 14.​2 (b) guarantees the following limiting distribution:

From Example 9.​2 it follows that

If ω e is replaced by a consistent estimator,

then the result from (15.5) is established.

15.3 The I(1) case is treated in an analogous way as the trend stationary case, see Proposition 14.​2 (c):

From Corollary 8.​1 (c) it follows for c = 0

Hence, (15.6) is proved.

15.4 Under H ₀, the LS estimator is given as:

Insofar it holds

Thus, numerator as well as denominator are in a form accessible for Proposition 14.​2,

which is the first distribution to be established.

To find the second limit, one has to handle the standard error s a of $$\hat{a}$$. It is based on the residual variance estimation

where Proposition 14.​2 (e), (f) and $$n^{-1}\sum _{t=1}^{n}e_{t}^{2} \rightarrow \gamma _{e}(0)$$ as well as the circumstance that $$n(1 -\hat{ a})$$ converges in distribution were used. All in all, it hence holds for the t-statistic:

This completes the proof.

15.5 With the means

it holds for the LS estimator when including an intercept under a = 1:

Thus, we obtain

For $$\varepsilon _{t} \sim \text{WN}(0,\sigma ^{2})$$ with $$\omega _{\varepsilon }^{2} =\sigma ^{2} =\gamma _{\varepsilon }(0)$$ it therefore holds due to Proposition 14.​2 (a), (e) and (f) and according to Proposition 14.​1 for s = 1:

If one defines the demeaned WP,

then one observes two interesting identities:

and

Hence, the limiting distribution in the case of a regression with intercept (which is equivalent to running a regression of demeaned variables!) can also be written as:

Thus, one obtains the same functional form as in the case without intercept, save that W(s) is replaced by $$\underline{W}(s)$$.

15.6 For the partial sum S t we obtain under the null hypothesis:

The adequately normalized squared sum yields

where S ₀ = 0 was used and $$S_{\lfloor sn\rfloor }$$ is the step function from Proposition 14.​2:

Note that S n is zero by construction:

Hence, we obtain

As there is a continuous functional on the right-hand side, Proposition 14.​3 with Proposition 14.​2 yields:

If $$\omega _{e}^{2}$$ is replaced by a consistent estimator, then we just obtain the given limiting distribution $$\mathcal{C}\mathcal{M}$$ as a functional of a Brownian bridge. This completes the proof.

15.7 The limit $$\beta _{\infty }$$ of $$\hat{\beta }$$ given in Proposition 15.5 is adopted from the text.

The s ² is based on the LS residuals $$\hat{u}_{t} = y_{t} -\hat{\beta } x_{t}$$. Hence, the sum of squared residuals becomes

Thus, again with Proposition 14.​4 (b), one immediately obtains for the uncentered coefficient of determination:

The relation used above between the sum of squared residuals and the coefficient of determination further yields

Moreover, the asymptotics of s ² enables us to state the behavior of the t-statistic as well. The required normalization is obvious:

15.8 Due to $$\hat{u}_{t} = y_{t} -\hat{\beta }\, x_{t}$$, the numerator of the Durbin-Watson statistic yields:

where γ i (0) denote the variances of the processes {w i, t }, i = 1, 2. By definition one obtains

Consequently, dw tends to zero in probability.

References

Anderson, T. W., & Darling, D. A. (1952). Asymptotic theory of certain “Goodness of Fit” criteria based on stochastic processes. Annals of Mathematical Statistics, 23, 193–212.CrossRef

Andrews, D. W. K. (1991). Heteroskedasticity and autocorrelation consistent covariance matrix estimation. Econometrica, 59, 817–858.CrossRef

Chang, Y., & Park, J. (2002). On the asymptotics of ADF tests for unit roots. Econometric Reviews, 21, 431–447.CrossRef

Dickey, D. A., & Fuller, W. A. (1979). Distribution of the estimators for autoregressive time series with a unit root. Journal of the American Statistical Association, 74, 427–431.

Durlauf, S. N., & Phillips, P. C. B. (1988). Trends versus random walks in time series analysis. Econometrica, 56, 1333–1354.CrossRef

Elliott, G., Rothenberg, T. J., & Stock, J. H. (1996). Efficient tests for an autoregressive unit root. Econometrica, 64, 813–836.CrossRef

Engle, R. F., & Granger, C. W. J. (1987). Co-integration and error correction: Representation, estimation, and testing. Econometrica, 55, 251–276.CrossRef

Granger, C. W. J. (1981). Some properties of time series data and their use in econometric model specification. Journal of Econometrics, 16, 121–130.CrossRef

Granger, C. W. J., & Newbold P. (1974). Spurious regressions in econometrics. Journal of Econometrics, 2, 111–120.CrossRef

Hamilton, J. (1994). Time series analysis. Princeton: Princeton University Press.

Hassler, U. (2000). Simple regressions with linear time trends. Journal of Time Series Analysis, 21, 27–32.CrossRef

Hendry, D. F. (1980). Econometrics – alchemy or science? Economica, 47, 387–406.CrossRef

Kwiatkowski, D., Phillips, P. C. B., Schmidt, P., & Shin Y. (1992). Testing the null hypothesis of stationarity against the alternative of a unit root. Journal of Econometrics, 54, 159–178.CrossRef

Newey, W. K., & West, K. D. (1987). A Simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica, 55, 703–708.CrossRef

Park, J. Y., & Phillips, P. C. B. (1988). Statistical inference in regressions with integrated processes: Part I. Econometric Theory, 4, 468–497.CrossRef

Phillips, P. C. B. (1986). Understanding spurious regressions in econometrics. Journal of Econometrics, 33, 311–340.CrossRef

Phillips, P. C. B. (1987). Time series regression with a unit root. Econometrica, 55, 277–301.CrossRef

Phillips, P. C. B, & Perron, P. (1988). Testing for a unit root in time series regression. Biometrika, 75, 335–346.CrossRef

Said, S. E., & Dickey, D. A. (1984). Testing for unit roots in autoregressive-moving average models of unknown order. Biometrika, 71, 599–607.CrossRef

Footnotes

1

Many more procedures have been developed over the last decades, notably the test by Elliott et al. (1996) with certain optimality properties.

2

Through numerous works by Peter Phillips the functional central limit theory has found its way into econometrics. This kind of limiting distributions was then celebrated as “non-standard asymptotics”; meanwhile it has of course become standard.

3

Equivalently, one might feed detrended data into the ADF regression above.

4

For the following calculation of s ² we divide by n without correcting for degrees of freedom, which does not matter asymptotically ($$n \rightarrow \infty )$$.

5

“Uncentered”, as the regression is calculated without intercept.

© Springer International Publishing Switzerland 2016

Uwe HasslerStochastic Processes and CalculusSpringer Texts in Business and Economics10.1007/978-3-319-23428-1_16

16. Cointegration Analysis

Uwe Hassler1 

(1)

Faculty of Economics and Business Administration, Goethe University Frankfurt, Frankfurt, Germany

16.1 Summary

This chapter is addressed to the analysis of cointegrated variables. Properties like superconsistency of the LS estimator and conditions for asymptotic normality are extensively discussed. Error-correction is the reverse of cointegration, which is why we provide an introduction to the analysis of error-correction models as well. In particular, we discuss cointegration testing. In 2003, Clive W.J. Granger was awarded the Nobel prize for introducing the concept of cointegration. Finally, we stress once more the effect of linear time trends underlying the series.

16.2 Error-Correction and Cointegration

Before cointegration had been launched, so-called error-correction models impressed due to their empirical performance, cf. e.g. Davidson, Hendry, Srba, and Yeo (1978). Today we know that these models are just the other side of the cointegration coin. By way of example, the key statement of Granger’s representation theorem from Engle and Granger (1987) is illustrated, which demonstrates the fact that error-correction and cointegration are equivalent.

Autoregressive Distributed Lag Model

Let us consider a dynamic regression model in which {y t } has an autoregressive structure on the one hand and which is explained by (lagged) exogenous variables x t−j on the other:

Hence, this is an extension of the AR(p) process. Because of the additional exogenous explanatory variables, we sometimes speak of ARX(p, ℓ) models, although such processes are more often called autoregressive distributed lag models, ARDL(p, ℓ). We assume that {x t } is integrated of order one. In order to have cointegration with {y t }, the ARDL model has to be stable. We adopt the stability condition from Proposition 3.​4:

Example 16.1 (ARDL(2,2))

With $$p =\ell= 2$$ we consider

Due to the assumed stability, the parameter b can be defined:

In fact, the denominator is not only different from zero but positive if stability is given (which we again know from Proposition 3.​4): $$1 - a_{1} - a_{2}> 0$$. Thus, the following parameter γ is negative:

Elementary manipulations lead to the reparameterization (cf. Problem 16.1)

(16.1)

In this equation differences of y are related to their own lags and differences of x. In addition, they depend on a linear combination of lagged levels (in square brackets). This last aspect is the one constituting error-correction models. The cointegration relation y = b x is understood as a long-run equilibrium relation and $$v_{t-1} = y_{t-1} - b\,x_{t-1}$$ as a deviation from it in t − 1. This deviation from the equilibrium again influences the increments of y t . The involved linear combination of y t−1 and x t−1 needs to be stationary because {Δ y t } is stationary by assumption. Hence, in the example it is obvious that such a relation between differences and levels implies cointegration. Indeed, it is the lagged deviation from the equilibrium v t−1 influencing the increments of Δ y t with a negative sign. If y t−1 is greater than the equilibrium value, v t−1 > 0, then this affects the change from y t−1 to y t negatively, i.e. y is corrected towards the equilibrium, and vice versa for values below the equilibrium value. What economists know as deviation from the equilibrium is called “error” (in the sense of a deviation from a set target) in engineering, which explains the name error-correction model. ■ 

The example can be generalized. Cointegrated ARDL models of arbitrary order can always be formulated as error-correction models. This is not surprising against the background of Granger’s representation theorem.

Granger’s Representation Theorem

Error-correction adjustment is the downside to cointegration. The relation between cointegration and error-correction is explained by the following proposition where we, however, will not spell out all the technical details. The result goes back to Granger, cf. Engle and Granger (1987) or Johansen (1995, Theorem 4.2).

Proposition 16.1 (Representation Theorem)

Let {y t } and {x t } be integrated of order one. They are cointegrated if and only if they have an error-correction representation,

(16.2)

(16.3)

(16.4)

where at least one of the so-called adjustment coefficients γ or γ x is different from zero.

Of course, not all a j ($$a_{j}^{(x)}$$) and α j ($$\alpha _{j}^{(x)}$$) need to be different from zero. The error sequences $$\{\varepsilon _{t}\}$$ and $$\{\varepsilon _{x,t}\}$$ are white noise and may be contemporaneously correlated. Frequently in practice, additional contemporaneous differences of the respective other variable are hence incorporated on the right-hand side. For Eq. (16.2) this means e.g. the inclusion of $$\alpha _{0}\,\varDelta x_{t}$$. Then, one sometimes also speaks of the conditional or structural error-correction equation.

Cointegration and the Long-Run Variance Matrix

In (14.​6) the symmetric long-run variance matrix Ω of a stationary vector has been defined. We now consider an I(1) vector $$z_{t}^{{\prime}} = (z_{1,t},z_{2,t})$$ such that $$\varDelta z_{t} = (w_{1,t},w_{2,t})^{{\prime}}$$ is integrated of order zero, which is why Ω cannot be equal to the zero matrix. Nevertheless, the matrix does not have to be invertible. It rather holds: If the vector {z t } is cointegrated, then Ω has the reduced rank one and is not invertible. Equivalently, this means: If Ω is invertible, then {z 1, t } and {z 2, t } are not cointegrated. To this, we consider the following example.

Example 16.2 (Ω under Cointegration)

In this example, let {z 2, t } be a random walk, the cointegration parameter be one, and the deviation from the equilibrium $$v_{t} =\varepsilon _{1,t}$$ be iid and independent of {Δ z 2, t }:

Hence, if $$\{\varepsilon _{1,t}\}$$ and $$\{\varepsilon _{2,t}\}$$ are independent with variances $$\sigma _{1}^{2}$$ and $$\sigma _{2}^{2}$$, then one shows with $$w_{1,t} =\varDelta z_{1,t} =\varepsilon _{2,t} +\varepsilon _{1,t} -\varepsilon _{1,t-1}$$ and $$w_{2,t} =\varDelta z_{2,t} =\varepsilon _{2,t}$$:

For h > 1, Γ w (h) = 0. Thus, it holds:

i.e. the matrix is of rank one and not invertible. ■ 

Conversely, it holds as well that full rank of Ω follows from the absence of cointegration. This is to be illustrated by the following example.

Example 16.3 (Ω without Cointegration)

Now, let {z 1, t } and {z 2, t } be two random walks independent of each other:

Since $$\{\varepsilon _{1,t}\}$$ and $$\{\varepsilon _{2,t}\}$$ are independent with variances $$\sigma _{1}^{2}$$ and $$\sigma _{2}^{2}$$, one shows with $$w_{1,t} =\varepsilon _{1,t}$$ and $$w_{2,t} =\varepsilon _{2,t}$$:

For h > 0, Γ w (h) = 0. Thus, it holds:

where this matrix has full rank 2 in the case of positive variances and is thus invertible. ■ 

Hence, the presence of cointegration of the I(1) vector {z t } depends on the matrix Ω. The examples show that no cointegration of {z t } is equivalent to the full rank of Ω, cf. Phillips (1986).

Linearly Independent Cointegration Vectors

For more than two I(1) variables linearly independent cointegration vectors can exist. In such a situation there can be no talk of “the true” cointegration vector. Each and every linear combination of independent cointegration vectors is itself again a cointegration vector. Although this cannot occur under our assumption of a bivariate vector, we want to become aware of this problem by means of a three-dimensional example.

Example 16.4 (Three Interest Rates)

Assume z ₁, z ₂ and z ₃ to be interest rates for one-month, two-month and three-month loans integrated of order one. Then one expects (due to the expectations hypothesis of the term structure) the interest rate differentials $$z_{1} - z_{2}$$ and $$z_{2} - z_{3}$$ to provide stable relations:

Here, $$b_{1}^{{\prime}} = (1,-1,0)$$ and $$b_{2}^{{\prime}} = (0,1,-1)$$ are linearly independent, and both are cointegrating vectors for $$z_{t}^{{\prime}} = (z_{1,t},z_{2,t},z_{3,t})$$ as v 1, t and v 2, t are both assumed to be I(0):

Hence, the uniqueness of the cointegration vector is lost. It is rather that b ₁ and b ₂ form a basis for the cointegration space. Each vector contained in the plane they span, i.e. each linear combination of b ₁ and b ₂, is itself again a cointegration vector. For example for

one obtains a stationary relation comprising all three variables from z t ,

where $$v_{1,t} + (1-\alpha )v_{2,t}$$ is I(0). In particular for α = 0 one sees that z 1, t and z 3, t alone are also cointegrated with the cointegration vector $$\beta _{0}^{{\prime}} = b_{1}^{{\prime}} + b_{2}^{{\prime}} = (1,0,-1)$$. The cointegration vectors b ₁, b ₂ as well as β ₀ provide economically reasonable and theoretically secured statements on the interest rate differentials. However, β α for an arbitrary α is just as good a cointegrating vector. Hence,

for α ≠ 1 or α ≠ 0 is as well a “true” long-run equilibrium relation, even if, in contrast to the interest rate differentials, it is not readily amenable to an economic interpretation. These problems with the interpretation of more than one linearly independent cointegration vectors are of a fundamental nature and cannot be solved unless one makes a priori (economically plausible) assumptions on the form of the cointegrating vectors. The cointegration analysis is not the life belt saving us from the lack of identification: With purely statistical methods one generally cannot make economic statements. ■ 

For only two I(1) variables {y t } and {x t }, linearly independent cointegration vectors cannot exist. We show this by contradiction. Hence, let us assume with $$b_{1}\neq b_{2}$$ from $$\mathbb{R}$$ that two linearly independent relations exist. We collect them row-wise in the matrix B:

Then, it holds by assumption that

is a vector of I(0) variables {v 1, t } and {v 2, t }. Due to the independence of the cointegration vectors, B is invertible:

This yields {y t } and {x t } as linear combinations of {v i, t }, i = 1, 2, from which it follows that {y t } and {x t } themselves have to be I(0), which contradicts the assumption.

16.3 Cointegration Regressions

In the case of bivariate cointegration, the LS estimator of a static regression of y t on x t tends to the true value with the sample size (i.e. with rate n). For this fast rate of convergence the term superconsistency has been coined in the literature. At the same time limiting normality only arises under additional assumptions.

Superconsistent Estimation

We consider the LS estimator regressing y t on x t under the assumption (16.4) that cointegration is present. For the sake of simplicity, we again do not allow for an intercept (which is why we assume that the I(1) processes have the starting value zero):

(16.5)

Then we write for the LS estimator:

or

To be able to apply the functional limit theory from Chap. 14, we now define

(16.6)

instead of $$z_{t}^{{\prime}} = (y_{t},x_{t})$$ as in Sect. 15.​4 without cointegration. Then it holds with the results from Proposition 14.​4(b) and (c):

As the LS estimator tends to the true value with the sample size n instead of with only n 0. 5 as for the stationary regression model, since Stock (1987) and Engle and Granger (1987) it has become common usage to speak of superconsistency of the static cointegration estimator from (16.5); however, this result has been known from Phillips and Durlauf (1986) already.

Note that the estimation of b is consistent despite possible correlation between error term v t and regressor x t (or Δ x t ). Insofar the cointegration regression knocks out the simultaneity bias (or “Haavelmo bias”): Superconsistency is a strong asymptotic argument for single equation regressions despite possibly existing dependencies through simultaneous relations between the individual equations, i.e. despite correlation between regressors and error term. According to this, the cointegration approach can be understood as a reaction to the simultaneous equation methodology of former decades. At the same time, it is not clear anymore which variable constitutes the endogenous left-hand side and which quantity identifies the exogenous regressor. Beside (16.4), it also holds as a “true relation” that

Hence, if x t is regressed on y t , then one would obtain analogously a superconsistent estimator for b ⁻¹. This vehemently contrasts the results of the stationary standard econometrics where the (asymptotic) validity of LS crucially depends on the correct specification of the single equation and exogeneity assumptions.

Further Asymptotic Properties

In Problems 16.2 and 16.3 we prove the further properties of the following proposition. Here, the standard errors of the t-statistic, the uncentered coefficient of determination and the Durbin-Watson statistic are defined as in Sect. 15.​4.

Proposition 16.2 (Cointegration Regression)

For cointegrated I(1) processes {x t } and {y t } it holds with ( 16.4 ) and the notation introduced that

where

as $$n \rightarrow \infty$$ .

Three remarks are to help with the interpretation. (i) Note again that superconsistency holds even if the regressors x t correlate with the error terms v t . This nice property, however, comes at a price: Unfortunately, the limiting distributions from Proposition 16.2(a) and (d) are (without further assumptions) generally not Gaussian. (ii) As a rule, for regressions involving trending (integrated) variables one empirically observes values of the coefficient of determination near one, which is explained by Proposition 16.2(b). Due to that, for trending (integrated) time series the coefficient of determination cannot be interpreted as usual: Since {y t } does not have a constant variance, the coefficient of determination does not give the percentage of the variance explained by the regression. (iii) At no point it was assumed that the error terms, {v t }, are white noise. For the first order residual autocorrelation, it holds

which is just reflected by the behavior of the Durbin-Watson statistic.

Asymptotic Normality

The price for the superconsistency without exogeneity assumption is that the limiting distribution of the t-statistic is generally not Gaussian anymore. However, if v t and Δ x s are stochastically independent for all t and s, then Krämer (1986) shows that asymptotic normality arises. This assumption, however, is stronger than necessary. It suffices to require, first, Δ xv  = 0 and, second, that the Brownian motions B ₁ and B ₂ are independent. For independence of B ₁ and B ₂ we only need

(16.7)

as under this condition it holds that $$B_{i} =\omega _{i}W_{i}$$. Due to Proposition 10.​4 the following corollary is obtained (also cf. Problem 16.4).

Corollary 16.1 (Asymptotic Normality)

If Δ xv from Proposition 16.2 is zero, then it holds under ( 16.7 ) that

as $$n \rightarrow \infty$$ .

If one has consistent estimators for the variance and the long-run variance, then the t-statistic can be modified as follows and can be applied with standard normal distribution asymptotics under Corollary 16.1:

For the estimation of the long-run variance, we refer to the remarks in Sect. 15.​2, in particular Example 15.​2. As {v t } itself is not observable, $$\hat{\gamma }_{1}(0)$$ and $$\hat{\omega }_{1}^{2}$$ have to be calculated from $$\hat{v}_{t} = y_{t} -\hat{b}x_{t}$$.

Efficient Estimation

The assumptions Δ xv  = 0 and ω ₁₂ = 0 required for Corollary 16.1 are often not met in practice. We now consider modifications of LS resulting in limiting normality that get around such restrictions. To that end we have a closer look at the LS limit from Proposition 16.2 called now $$\mathcal{L}(\hat{b})$$:

We define the process B 1⋅ 2,

(16.8)

in such a way that it does not correlate with B ₂:

Because of normality the two processes are thus independent. The variance of the new process is

With

we may now decompose the LS limit as follows:

The first component, $$\mathcal{L}_{1}$$, is conditionally normal with mean zero, i.e.

which is true by Proposition 10.​4. The second component defined as a multiple of $$\int _{0}^{1}B_{2}(s)dB_{2}(s) = (B_{2}^{2}(1) - 1)/2$$ is stochastic and introduces skewness into $$\mathcal{L}(\hat{b})$$, while finally this distribution is shifted deterministically by Δ xv . Consequently, for arbitrary $$\varepsilon> 0$$ one has

see Saikkonen (1991, Theorem 3.1) Saikkonen. In that sense, $$\mathcal{L}_{1}$$ is more concentrated around zero than $$\mathcal{L}(\hat{b})$$ in general, such that intuitively the LS estimator is closest to zero for $$\omega _{12} =\varDelta _{xv} = 0$$. This is the intuition for the following definition: Let $$\hat{b}^{+}$$ denote a cointegration estimator with

(16.9)

for some positive constant ω; then $$\hat{b}^{+}$$ is said to be efficient (see Saikkonen 1991 for a more general discussion). To further justify this notion of efficiency we note that full information maximum likelihood estimation of a cointegrated system results in exactly this distribution, see Phillips (1991).

Efficient cointegration regressions are not only interesting because they achieve the lower bound for the standard error; more importantly, related t-type statistics are asymptotically normal, which allows for standard inference. Suppose we have an efficient estimator satisfying (16.9), and that $$\hat{\omega }$$ (typically computed from cointegration residuals) is consistent for ω. Then we define the t-type statistic

From the previous discussion it follows, see Phillips and Park (1988):

Consequently, the caveat of superconsistent LS cointegration regression, lacking normality in general, is overcome by efficient estimators.

Let us repeat once more: LS cointegration estimation is efficient under the not very realistic assumption that $$\omega _{12} =\varDelta _{xv} = 0$$. Several modifications of LS achieving efficiency without this assumption have been proposed. First, the so-called dynamic LS estimator suggested independently by Saikkonen (1991) and Stock and Watson (1993) is settled in the time domain, see also Phillips and Loretan (1991); second, so-called frequency domain based modifications of LS have been suggested by Phillips and Hansen (1990) (“fully modified LS”) or Park (1992) (“canonical cointegrating regression”); they all meet (16.9) and $$t^{+} \sim \mathcal{N}(0,1)$$, asymptotically.

Linear Time Trends

In Sect. 15.​2 we considered linear time trends and I(1) processes, i.e. so-called integrated processes with drift:

(16.10)

By repeated substitution one obtains

i.e. {x t } consists of a linear trend of the slope μ and an I(1) component; and of a starting value whose influence can be neglected such that we set x ₀ = 0 w.l.o.g. In addition, let the cointegration relation (16.4) hold true. Consequently, {y t } as well exhibits a linear trend of the slope b μ. The cointegration relation (16.4) simultaneously eliminates the deterministic linear time trend and the stochastic I(1) trend from both series. In this case the static LS regression from (16.5) yields an even faster rate of convergence (n 1. 5 instead of n) and simultaneously, the limiting distribution is Gaussian. The following proposition is a special case of the more general results from West (1988). We prove it in Problem 16.5.

Proposition 16.3 (West)

We assume cointegrated I(1) processes {x t } and {y t } with drift (i.e. ( 16.10 ) with ( 16.4 )). Then it holds for the regression ( 16.5 ) without intercept that

as $$n \rightarrow \infty$$ , where $$\omega _{1}^{2}$$ is the long-run variance of {v t } from ( 16.4 ).

For the sake of completeness, note that the asymptotics from Proposition 16.3 is qualitatively retained if the regression is run with intercept. However, the variance of the Gaussian distribution is changed. It becomes $$12\omega _{1}^{2}/\mu ^{2}$$. For practical inference, μ ² has to be estimated from {x t } or {Δ x t }, while $$\omega _{1}^{2}$$ can be estimated consistently from the cointegration residuals $$\hat{v}_{t}$$. For μ = 1 the limiting distribution from Proposition 16.3 equals that from (15.​5), which is not coincidental, see also the proof in Problem 16.5: The scalar I(1) regressor with drift is dominated by the linear time trend; hence, the cointegration regression amounts to a trend stationary regression.

Note, however, that Proposition 16.3 holds only in our special case that x t is a scalar I(1) variable. If one has a vector of I(1) regressors with drift instead of a scalar variable, then the asymptotic normality in general does not hold anymore, and the very fast convergence with rate n 1. 5 is lost as well. Instead, Hansen (1992) proved results in line with Proposition 16.2 if there are several I(1) regressors of which at least one has a drift.

16.4 Cointegration Testing

If one regresses nonstationary (integrated) time series on each other, the interpretation of the regression outcome largely depends on whether the series are cointegrated or not. Hence, one has to test for the absence or presence of cointegration.

Residual-Based Dickey-Fuller Test

The idea of the following test for the null hypothesis of no cointegration dates back to Engle and Granger (1987), although a rigorous asymptotic treatment was provided later by Phillips and Ouliaris (1990). The idea is very simple. Without cointegration any linear combination of I(1) variables results in a series that too has a unit root. Hence, the Dickey-Fuller test is applied to LS residuals, which are computed from a regression with intercept:

Due to the included intercept, $$\{\hat{u}_{t}\}$$ are zero mean by construction. Hence, the DF regression in the second step may be run w.l.o.g. without intercept:

The LS estimator $$\overline{a}$$ converges to 1 under the null hypothesis of a residual unit root with the rate known from Sect. 15.​3 However, $$\hat{\beta }$$ does not converge to 0, but a limit characterized in Proposition 15.​5. Consequently, the asymptotic distribution of $$n(\overline{a} - 1)$$ does not only depend on one WP but rather on two. Let $$\overline{t}_{a}(2)$$, involving 2 I(1) processes, denote the t-statistic related to $$\overline{a} - 1$$. Then the limit depends on two standard Wiener processes W ₁ and W ₂:

(16.11)

Interestingly, this limit is free of nuisance parameters as long as {Δ y t } and {Δ x t } are white noise; in particular, it does not depend on the eventual correlation between {Δ y t } and {Δ x t }; see also Problem 16.7. If {Δ y t } and {Δ x t } are not white noise processes, then $$\overline{a}$$ may be computed from a lag-augmented regression, or a modification to the test statistic in line with Phillips (1987) has to be applied. Critical values or p-values are most often taken from MacKinnon (1991, 1996). Here, we do not present a definition of the functional shape of the limit $$\overline{\mathcal{D}\mathcal{F}}\left (W_{1},W_{2}\right )$$; rather, to give at least an idea thereof, we consider now explicitly the less complicated case without constant. So, $$\hat{u}_{t}$$ and $$\hat{\beta }$$ are now from a regression without intercept,

We use a new notation to denote the subsequent DF regression

In Problem 16.7 the following result is given.

Proposition 16.4 (Phillips & Ouliaris)

Let {z t } with $$z_{t}^{{\prime}} = (y_{t},x_{t})$$ be a random walk such that {Δy t } and {Δx s } are white noise processes not correlated for t ≠ s, although we do allow for contemporaneous correlation. In case of no cointegration it holds with the notation introduced above that

with

where W ₁ and W ₂ are two independent Wiener processes.

The functional shape of this limit corresponds exactly to the one in (1.​11); only that the WP W is replaced by U, which is, however, no longer a WP. Not surprisingly, the new process U is defined as residual from a projection of W ₁ (corresponding to y) on W ₂ (corresponding to x). Once more this shows the power and elegance of the functional limit theory approach introduced in Chap. 14.

Residual-Based KPSS Test

It comes in natural to apply also the KPSS test to regression residuals. As in Sect. 15.​3 the hypotheses are now exchanged: We test for the null hypothesis of cointegration against the alternative of no cointegration. Working with LS cointegration residuals, $$\hat{v}_{t} = y_{t} -\hat{b}x_{t}$$, we have under the null hypothesis

where

with the limiting distribution $$b_{\infty }$$ characterized in Proposition 16.2. Interestingly, we observe by a FCLT (Proposition 14.​1) that

with B ₂ being the Brownian motion behind {x t } from $$z_{t}^{{\prime}} = \left (\sum _{j=1}^{t}v_{j},\,x_{t}\right )$$. Therefore, it holds that $$(\hat{b} - b)\,x_{t}$$ converges to zero with growing sample size, and the empirical residuals are proxies of the unobserved cointegration deviation: $$\hat{v}_{t} \approx v_{t}$$. Hence, it is tempting to believe that a KPSS test applied to the sequence $$\{\hat{v}_{t}\}$$ behaves as if applied to {v t }. This, however, is not correct, as we will demonstrate next, since the limit characterized in Proposition 15.​4 is not recovered when working with cointegration residuals.

The KPSS test builds on the partial sum process $$S_{t} =\sum _{ j=1}^{t}\hat{v}_{j}$$. Mimicking the proof of Proposition 14.​2(a) in Problem 14.3, we obtain the following FCLT for the partial sum process:

Notwithstanding that $$\hat{v}_{t} \approx v_{t}$$ we must thus not jump at the conclusion that the residual effect is negligible: The more careful analysis showed that the limit of the partial sum process depends on the distribution $$b_{\infty }$$ arising from the cointegration regression.

What is more, we know that the LS limit $$b_{\infty }$$ from Proposition 16.2 is plagued by the nuisance parameters Δ xv and ω ₁₂, except for the special case of Corollary 16.1. Therefore, Shin (1994) suggested to apply the KPSS test not with LS residuals but with residuals from an efficient regression (now with intercept),

where $$\hat{b}^{+}$$ is efficient in the sense of (16.9). Efficient cointegration regressions rely on removing Δ xv and ω ₁₂ consistently. Hence, Shin (1994) showed that the limiting distribution of the KPSS test applied to efficient residuals is free of nuisance parameters, and he provided critical values. Let $$\overline{\eta }^{+}(2)$$ denote the residual-based KPSS statistic building on $$v_{j}^{+} = y_{t} -\hat{a}^{+} -\hat{b}^{+}x_{t}$$, thus involving two I(1) variables. Under the null hypothesis of cointegration it holds asymptotically that (Shin, 1994, Thm. 2)

where

(16.12)

and $$\underline{W}$$ is again short for a demeaned Wiener process. For related work see Harris and Inder (1994) or Leybourne and McCabe (1994), although the latter paper considered only the case of LS residuals.

Error-Correction Test

The third test we look into is not residual-based. The analysis rather relies on the error-correction equation (16.2):

The fact that we restrict the analysis to the error-correction equation of {y t } and that we ignore Eq. (16.3) has to be justified by the assumption

(16.13)

This assumptions implies that, when cointegration is present, only Δ y t reacts to the deviation from the equilibrium of the previous period. Because of the assumption (16.13), absence of cointegration means γ = 0, which is the null hypothesis. In order that there is an adjustment to the equilibrium in the case of cointegration, γ < 0 under the alternative:

To provide further intuition for the test statistic, we rewrite the error-correction equation (16.2) by inserting the definition of v t−1:

(16.14)

Here, we defined

where the null hypothesis of course implies $$\theta = 0$$. Hence, one may test the null hypothesis by means of an F-type test statistic for $$\gamma =\theta = 0$$, which has been investigated by Boswijk (1994). Alternatively, one may employ a t-type test specifically for γ = 0 only as proposed by Banerjee, Dolado, and Mestre (1998). The following proposition characterizes the asymptotic behavior of the LS estimator $$\hat{\gamma }$$, cf. Banerjee, Dolado, and Mestre (1998, Proposition 1). The Wiener processes W ₁ and W ₂ are adopted from Proposition 14.​4 with $$z_{t}^{{\prime}} = (y_{t},x_{t})$$. In order to get a limiting distribution free of nuisance parameters, we assume that Δ x t and $$\varepsilon _{s}$$ are uncorrelated at arbitrary points in time:

(16.15)

see also the proof in Problem 16.6.

Proposition 16.5 (BDM)

Let the I(1) processes {x t } and {y t } be not cointegrated, and let the exogeneity assumption ( 16.15 ) be fulfilled. Then, it holds for the LS estimator from the regression ( 16.14 ) that

as $$n \rightarrow \infty$$ .

Obviously, this limiting distribution can be reshaped into the following form (which lends itself for a multivariate generalization with vectors {x t }):

In fact, Banerjee et al. (1998) suggest the computation of the t-statistic relating to $$\hat{\gamma }$$: t γ (2). As limiting distribution under H ₀ one obtains:

(16.16)

Again, in most practical situations an intercept will be included in (16.14). If the corresponding test statistic is called $$\overline{t}_{\gamma }(2)$$, then it holds under the null hypothesis that

(16.17)

This limit has the same functional shape as $$\mathcal{B}\mathcal{D}\mathcal{M}(W_{1},W_{2})$$, only that W i have to be replaced by the demeaned analogs $$\underline{W}_{i}$$, i = 1, 2.

Simulated critical values for conducting the tests can be found in Banerjee et al. (1998). One rejects for small values. From Ericsson and MacKinnon (2002) p-values are available, too.

Linear Time Trends

It has been mentioned in this and the previous chapter that in practice one would run regressions with an intercept to account for non-zero means of the series. But how should one proceed if the mean function follows a linear time trend, i.e. if the series are I(1) with drift? This is a quite realistic assumption for many economic and financial time series where positive growth rates are plausible. One might consider the analysis of detrended data, see Sect. 15.​2. Note that the regression of detrended series is equivalent to including a linear time trend in the regression (see Frisch & Waugh, 1933):

This is why we call such regressions also detrended regressions . Similarly, one may augment the error-correction regression (16.14) by a linear time trend (and a constant). Many economists, however, do not run detrended regression or detrend the series even if the data display a linear time trend by eyeball inspection. Economically, it is often more meaningful to “explain” one trend by another instead of regressing deviations from linear trends on each other. Also statistically the regression of detrended data may not seem advisable since power losses are to be expected (see Hamilton, 1994, Sect. 19.2).

Running regressions with intercept only, i.e. without detrending, in the presence of linear time trends in the regressors has some subtle implications, however. Generally, the presence of a linear time trend in the data not accounted for in the regression will affect the limiting distributions. Just remember that a simple (or bivariate) cointegration regression in the presence of a linear time trend (Proposition 16.3) resembles more the detrending of a trend stationary process (Proposition 15.​1) than a cointegration regression without linear trend (Proposition 16.2). More precisely, a linear time trend in {x t } will dominate the stochastic unit root in the following sense: If {x t } is I(1) with drift, $$\mbox{ E}(\varDelta x_{t}) =\mu +e_{t}$$, μ ≠ 0, or

then this process grows with rate n (and not n 0. 5, see Proposition 14.​1):

This provides an intuition for the following finding by Hansen (1992, Theorem 7): If {x t } is I(1) with drift and {y t } and {x t } are not cointegrated, and if a regression-based DF test for no cointegration is computed from a regression with intercept but without detrending, then the limiting distribution of the t-type DF statistic is not given by $$\overline{\mathcal{D}\mathcal{F}}(W_{1},W_{2})$$ from (16.11), but rather by the detrended univariate distribution $$\widetilde{\mathcal{D}\mathcal{F}}$$ given in (15.​10). A corresponding result was established for the error-correction test by Hassler (2000a): If {x t } and {y t } are I(1) but not cointegrated, and if {x t } is integrated with drift and the error-correction test for no cointegration is computed from a regression with intercept but without detrending, then the limiting distribution of the t-statistic is not given by $$\overline{\mathcal{B}\mathcal{D}\mathcal{M}}(W_{1},W_{2})$$ from (16.17), but by the detrended Dickey-Fuller distribution $$\widetilde{\mathcal{D}\mathcal{F}}$$. And similarly: If {x t } is I(1) with drift and cointegrated with {y t }, and if a regression-based KPSS test for cointegration is computed from a regression with intercept only, then the limiting distribution of the KPSS statistic is not given by $$\overline{\mathcal{C}\mathcal{M}}(W_{1},W_{2})$$ from (16.12), but rather by the detrended univariate distribution $$\widetilde{\mathcal{C}\mathcal{M}}$$ given in (15.​11); see Hassler (2000b). Hence, we have the following proposition.

Proposition 16.6 (Hansen & Hassler)

Consider the test statistics $$\overline{t}_{a}(2)$$ , $$\overline{t}_{\gamma }(2)$$ or $$\overline{\eta }^{+}(2)$$ computed without detrending to test for the null hypothesis of (no) cointegration of the I(1) processes {x t } and {y t }. With $$\widetilde{\mathcal{D}\mathcal{F}}$$ from ( 15.10 ) and $$\widetilde{\mathcal{C}\mathcal{M}}$$ from ( 15.11 ) it holds under the respective null hypotheses that

as $$n \rightarrow \infty$$ .

Proposition 16.6 is not restricted to bivariate regressions, but carries over to the general multiple regression case as follows:

Consider single-equation regressions estimated by LS (or efficient variants thereof); regressions with intercept only on k I(1) regressors, of which at least one has a drift, result under the null hypothesis (of cointegration or no cointegration, respectively) in a limit as if one runs a detrended regression on k − 1 I(1) regressors.

For k = 1 this reproduces Proposition 16.6. For a proof for the residual-based DF test with k > 1 see again Hansen (1992), and also the lucid discussion by Hamilton (1994, p. 596, 597); for a proof for the error-correction test see Hassler (2000a), and for the residual-based KPSS test see Hassler (2001) for k > 1.

In view of Proposition 16.6 one may identify two strategies when testing cointegration from regressions with intercept only; we restrict the discussion to the case of a scalar regressor x t . First, one might ignore the possibility of linear trends and always work with critical values from $$\overline{\mathcal{D}\mathcal{F}}(W_{1},W_{2})$$, $$\overline{\mathcal{B}\mathcal{D}\mathcal{M}}(W_{1},W_{2})$$ or $$\overline{\mathcal{C}\mathcal{M}}(W_{1},W_{2})$$ provided for the case of regressions with intercept only under no drift; we call this strategy S I (I for “ignoring”), and of course it is not correct if {x t } displays a linear trend in mean. Second, one may always account for the possibility of linear time trends and work with critical values from $$\widetilde{\mathcal{D}\mathcal{F}}$$ or $$\widetilde{\mathcal{C}\mathcal{M}}$$; let us call this strategy S A (A for “account”), and note that it is only appropriate if {x t } is indeed dominated by a linear time trend. In a numerical example we discuss the consequences of S I and S A .

Example 16.5 (Testing under the Suspicion of Time Trends)

We consider tests at a nominal significance level of 5 %. Let $$\tilde{c}_{1}$$ and $$\overline{c}_{2}$$ denote critical values from $$\widetilde{\mathcal{D}\mathcal{F}}$$ or $$\widetilde{\mathcal{C}\mathcal{M}}$$ and $$\overline{\mathcal{D}\mathcal{F}}(W_{1},W_{2})$$, $$\overline{\mathcal{B}\mathcal{D}\mathcal{M}}(W_{1},W_{2})$$ or $$\overline{\mathcal{C}\mathcal{M}}(W_{1},W_{2})$$, respectively. For the residual-based DF test by Phillips and Ouliaris (1990) we take asymptotic critical values from MacKinnon (1991): $$\tilde{c}_{1} = -3.41$$ and $$\overline{c}_{2} = -3.34$$. Coincidently, these critical values are not very distant. Strategy S I results in a slightly too liberal test (rejecting more often than with 5 % probability) in the presence of a drift, while S A is mildly conservative (rejecting less often than in 5 % of all cases) in the absence of a linear trend in the regressor. So, for the residual-based DF test, Proposition 16.6 is not so relevant, since the distributions happen to differ not that much, and $$\tilde{c}_{1} \approx \overline{c}_{2}$$. For the error-correction test by Banerjee et al. (1998), however, matters are not quite so harmless, since stronger size distortions are caused by a larger difference of the asymptotic critical values: $$\tilde{c}_{1} = -3.41$$ and $$\overline{c}_{2} = -3.19$$. With the residual-based KPSS test, things change qualitatively and quantitatively. Critical values from Kwiatkowski, Phillips, Schmidt, and Shin (1992) and Shin (1994) are $$\tilde{c}_{1} = 0.146$$ and $$\overline{c}_{2} = 0.314$$, and thus differ dramatically. Since one rejects for too large values, strategy S I implies in the presence of a linear time trend a very conservative test; it will hardly reject the null hypothesis, which comes at a price of power of course. The other way round, without linear time trends strategy S A will reject the true null hypothesis much too often resulting in an intolerably liberal test. ■ 

Clearly, none of the strategies S I or S A is generally acceptable when testing under the possibility of linear time trends. Fortunately, one often has strong a priori beliefs regarding the absence or presence of a linear time trend in the regressor. If one is convinced that a linear time trend is present in {x t }, then one would apply e.g. $$\overline{t}_{a}(2)$$ or $$\overline{t}_{\gamma }(2)$$ with critical values from $$\widetilde{\mathcal{D}\mathcal{F}}$$; if one believes that there is no linear time trend behind {x t }, then critical values from $$\overline{\mathcal{D}\mathcal{F}}(W_{1},W_{2})$$ or $$\overline{\mathcal{B}\mathcal{D}\mathcal{M}}(W_{1},W_{2})$$ must be recommended.

If one is not sure about the absence or presence of a linear time trend in the data, then there are (at least) two more strategies beyond S I or S A one may employ. As a third strategy, one may always test from detrended data. This clearly circumvents size distortions, but comes at a price of power losses as has been acknowledged for instance by Hansen (1992) or Hamilton (1994, p. 598). Fourth, one may rely on a pretest whether the regressor follows a linear trend or not. If a linear time trend in {x t } is significant, then one would apply e.g. $$\overline{\eta }^{+}(2)$$ with critical values from $$\widetilde{\mathcal{C}\mathcal{M}}$$; if not, then critical values from $$\overline{\mathcal{C}\mathcal{M}}(W_{1},W_{2})$$ should be applied. Such a pretesting strategy, however, will be troubled in small samples by the problem of controlling the significance level when carrying out a sequence of conditional tests (multiple testing). A recommendation whether the strategy of generally detrending or the strategy of pretesting is to be preferred, when the presence or absence of a linear time trend is debatable, will require future research.

16.5 Problems and Solutions

Problems

16.1 Show the equivalence of (16.1) and the ARDL(2,2) parameterization from Example 16.1.

16.2 Prove statements (b), (c) and (d) from Proposition 16.2.

16.3 Prove statement (e) from Proposition 16.2.

16.4 Prove Corollary 16.1.

16.5 Prove Proposition 16.3.

16.6 Prove Proposition 16.5 for the special case that no lagged differences are required to obtain white noise errors $$\{\varepsilon _{t}\}$$:

16.7 Prove Proposition 16.4.

Solutions

16.1 Using γ and b, one obtains with $$\varDelta = 1 - L$$ from (16.1):

Hence, the claim is already proved.

16.2 We proceed in the same way as in the proof of Proposition 15.​5, only that we work under (16.6) when appealing to Proposition 14.​4. We start with

The numerator on the right-hand side is just

The first of the three remaining terms tends to zero as $$\sum x_{t}^{2}$$ is of order n ² and $$(b -\hat{b})$$ is of order n ⁻¹; correspondingly, the second expression tends to zero as $$\sum x_{t}\,v_{t}$$ grows with n; finally, the third term converges to Var(v t ) as a law of large numbers holds for $$\{v_{t}^{2}\}$$. This proves Proposition 16.2(c). By the same arguments, one establishes:

Hence, Proposition 16.2(b) is proved as well.

Finally, the behavior of the t-statistic with $$s_{b}^{2} = s^{2}/\sum x_{t}^{2}$$ is clear again by Proposition 14.​4:

16.3 In order to analyze the behavior of the Durbin-Watson statistic, we only need to study the numerator,

As $$(b -\hat{b})$$ tends to zero, there remains asymptotically

Hence, it holds as claimed:

as s ² approaches Var(v t ) with n growing.

16.4 The result will follow from Proposition 16.2. By (16.7), we have ω ₁₂ = 0. Due to the resulting diagonality of Ω, Ω 0. 5 is diagonal as well, cf. Ω ₂ from Example 14.​6. Hence it holds:

and ω ₂ cancels from the limiting distribution of Proposition 16.2(d) (using the second assumption Δ xv  = 0):

According to Proposition 10.​4, the stochastic quotient on the right-hand side follows a standard normal distribution. Hence, it holds that

which proves the corollary.

16.5 By assumption, it holds:

We know from Proposition 14.​2(c) and (e) that the second and the third expression in square brackets have to be divided by n 2. 5 and n ², respectively, such that they converge. However, in front of the square bracket there is n ⁻³, such that it holds with (15.​2):

Hence we have the denominator of the LS estimator under control:

In order to crack the numerator, we consider

and due to Proposition 14.​1 it holds

and x t is dominated by a linear trend, i.e. x t behaves just as μ t. Thus, as for the detrending in the trend stationary case, we obtain with a standard Wiener process V that (see Sect. 15.​2):

where ω ₁ V (s) is the Brownian motion corresponding to $$\sum _{j=1}^{t}v_{j}$$, and $$\omega _{1}^{2}$$ is the long-run variance of {v t }. The normality of the integral follows from Example 9.​2. Hence, the claim is proved.

16.6 Because of the simplifying assumption we consider the regression of (16.14) without differences. As LS estimator for the vector

one hence obtains for a sample t = 1, … , n:

(16.18)

where D has the form

Plugging in $$\varDelta y_{t} =\varepsilon _{t}$$ under H ₀, we obtain

In the case of no cointegration, for using Proposition 14.​4 we choose

Then it holds for the matrix D:

(16.19)

For the inverse, this implies

where “det” stands for the determinant of the limiting matrix from (16.19). Here, the known inversion formula for $$\left (2 \times 2\right )$$-matrices was applied:

(16.20)

Note that Proposition 14.​4(b) was applied with z t−1 instead of z t . This is unproblematic and can be justified by similar arguments like those leading to Corollary 14.​1 in the univariate case.

Next, we analyze with Proposition 14.​4(c)

where we used that $$w_{1,t} =\varDelta y_{t} =\varepsilon _{t}$$ is free from serial correlation and is uncorrelated with Δ x s at each point in time, see (16.15):

Thus, under the null hypothesis of no cointegration, we obtain that $$\hat{\gamma }$$ tends to zero. For this purpose we consider the first row of the limit of $$n^{2}D^{-1}$$ multiplied by $$\int B\left (s\right )dB_{1}\left (s\right )$$:

This is almost the claim as “det” is defined as the determinant of the limit of n ⁻² D. Finally, note that $$B_{i} =\omega _{i}W_{i}$$ holds as $$\varDelta x_{t} = w_{2,t}$$ and $$\varDelta y_{s} =\varepsilon _{s} = w_{1,s}$$ are uncorrelated. Thus, the long-run variances cancel from the limiting distribution and one obtains the required result.

16.7 Under the null hypothesis of no cointegration we define $$z_{t}^{{\prime}} = (y_{t},\,x_{t})$$ with long-run variance matrix of full rank. The corresponding vector Brownian motion $$B^{{\prime}} = (B_{1},\,B_{2})$$ can be written in terms of independent WPs $$W^{{\prime}} = (W_{1},\,W_{2})$$ as

Here, T is the triangular decomposition given in (14.​7) with $$TT^{{\prime}} =\varOmega$$. The limit of $$\hat{\beta }$$ from Proposition 15.​5(a) hence becomes

(16.21)

With this result one obtains a FCLT for the residuals $$\hat{u}_{t} = y_{t} -\hat{\beta }x_{t}$$:

Unfortunately, however, Proposition 14.​2 does not apply directly since U(r) is not a WP. Still, with the techniques we used to prove Proposition 14.​2(e) and (f) in Problems 14.3 and 14.5, we can establish (omitting details)

where the last limit arises because {Δ z t } is white noise such that $$\omega _{i}^{2}$$ and ω ₁₂ coincide with the (co)variances. Further, note by B = TW that

(16.22)

Remember from (14.​7) that

Consequently, for

we get by (16.21) and (16.22) that

The numerator of this limit may be condensed. Use the product rule from Example 11.​5 to obtain

It follows that

Once more by Ito’s lemma $$\frac{W_{i}^{2}(1)} {2} =\int _{ 0}^{1}W_{ i}(s)dW_{i}(s) + \frac{1} {2}$$, such that

This provides the expression for the limiting distribution given in Proposition 16.4 as required.

References

Banerjee, A., Dolado, J. J., & Mestre R. (1998). Error-correction mechanism tests for cointegration in a single-equation framework. Journal of Time Series Analysis, 19, 267–283.CrossRef

Boswijk, H. P. (1994). Testing for an unstable root in conditional and structural error correction models. Journal of Econometrics, 63, 37–60.CrossRef

Davidson, J., Hendry, D. F., Srba, F., & Yeo S. (1978). Econometric modelling of the aggregate time-series relationship between consumers’ expenditure and income in the United Kingdom. Economic Journal, 88, 661–692.CrossRef

Engle, R. F., & Granger, C. W. J. (1987). Co-integration and error correction: Representation, estimation, and testing. Econometrica, 55, 251–276.CrossRef

Ericsson, N. R., & MacKinnon, J. G. (2002). Distributions of error correction tests for cointegration. Econometrics Journal, 5, 285–318.CrossRef

Frisch, R., & Waugh, F. V. (1933). Partial time regressions as compared with individual trends. Econometrica, 1, 387–401.CrossRef

Hamilton, J. (1994). Time series analysis. Princeton: Princeton University Press.

Hansen, B. E. (1992). Efficient estimation and testing of cointegrating vectors in the presence of deterministic trends. Journal of Econometrics, 53, 87–121.CrossRef

Harris, D., & Inder, B. (1994). A test of the null hypothesis of cointegration. In C. P. Hargreaves (Ed.), Nonstationary time series analysis and cointegration (pp. 133–152). Oxford/New York: Oxford University Press.

Hassler, U. (2000a). Cointegration testing in single error-correction equations in the presence of linear time trends. Oxford Bulletin of Economics and Statistics, 62, 621–632.CrossRef

Hassler, U. (2000b). The KPSS test for cointegration in case of bivariate regressions with linear trends. Econometric Theory, 16, 451–453.CrossRef

Hassler, U. (2001). The effect of linear time trends on the KPSS test for cointegration. Journal of Time Series Analysis, 22, 283–292.CrossRef

Johansen, S. (1995). Likelihood-based inference in cointegrated vector autoregressive models. Oxford/New York: Oxford University Press.CrossRef

Krämer, W. (1986). Least squares regression when the independent variable follows an ARIMA process. Journal of the American Statistical Association, 81, 150–154.

Kwiatkowski, D., Phillips, P. C. B., Schmidt, P., & Shin Y. (1992). Testing the null hypothesis of stationarity against the alternative of a unit root. Journal of Econometrics, 54, 159–178.CrossRef

Leybourne, S. J., & McCabe, B. P. M. (1994). A simple test for cointegration. Oxford Bulletin of Economics and Statistics, 56, 97–103.CrossRef

MacKinnon, J. G. (1991). Critical values for co-integration tests. In R. F. Engle, & C. W. J. Granger (Eds.), Long-run economic relationships (pp. 267–276). Oxford/New York: Oxford University Press.

MacKinnon, J. G. (1996). Numerical distribution functions for unit root and cointegration tests. Journal of Applied Econometrics, 11, 601–618.CrossRef

Park, J. Y. (1992). Canonical cointegrating regressions. Econometrica, 60, 119–143.CrossRef

Phillips, P. C. B. (1986). Understanding spurious regressions in econometrics. Journal of Econometrics, 33, 311–340.CrossRef

Phillips, P. C. B. (1987). Time series regression with a unit root. Econometrica, 55, 277–301.CrossRef

Phillips, P. C. B. (1991). Optimal inference in cointegrated systems. Econometrica, 59, 283–306.CrossRef

Phillips, P. C. B., & Durlauf, S. N. (1986). Multiple time series regression with integrated processes. Review of Economic Studies, LIII, 473–495.

Phillips, P. C. B., & Hansen, B. E. (1990). Statistical inference in instrumental variables regression with I(1) processes. Review of Economic Studies, 57, 99–125.CrossRef

Phillips, P. C. B., & Loretan, M. (1991). Estimating long-run economic equilibria. Review of Economic Studies, 58, 407–436.CrossRef

Phillips, P. C. B., & Ouliaris, S. (1990). Asymptotic properties of residual based tests for cointegration. Econometrica, 58, 165–193.CrossRef

Phillips, P. C. B., & Park, J. Y. (1988). Asymptotic equivalence of ordinary least squares and generalized least squares in regressions with integrated regressors. Journal of the American Statistical Association, 83, 111–115.CrossRef

Saikkonen, P. (1991). Asymptotically efficient estimation of cointegration regressions. Econometric Theory, 7, 1–21.CrossRef

Shin, Y. (1994). A residual-based test of the Null of cointegration against the alternative of no cointegration. Econometric Theory, 10, 91–115.CrossRef

Stock, J. H. (1987). Asymptotic properties of least squares estimators of cointegrating vectors. Econometrica, 55, 1035–1056.CrossRef

Stock, J. H., & Watson, M. W. (1993). A simple estimator of cointegrating vectors in higher order integrated systems. Econometrica, 61, 783–820.CrossRef

West, K. D. (1988). Asymptotic normality, when regressors have a unit root. Econometrica, 56, 1397–1418.CrossRef

References

Anderson, T. W., & Darling, D. A. (1952). Asymptotic theory of certain “Goodness of Fit” criteria based on stochastic processes. Annals of Mathematical Statistics, 23 , 193–212.

Andrews, D. W. K. (1991). Heteroskedasticity and autocorrelation consistent covariance matrix estimation. Econometrica, 59 , 817–858.

Andrews, D. W. K., & Chen, H.-Y. (1994). Approximately median-unbiased estimation of autoregressive models. Journal of Business & Economic Statistics, 12 , 187–204.

Baillie, R. T. (1996). Long memory processes and fractional integration in econometrics. Journal of Econometrics, 73 , 5–59.

Banerjee, A., Dolado, J. J., Galbraith, J. W., & Hendry, D. F. (1993). Co-integration, error correction, and the econometric analysis of non-stationary data . Oxford/New York: Oxford University Press.

Banerjee, A., Dolado, J. J., & Mestre R. (1998). Error-correction mechanism tests for cointegration in a single-equation framework. Journal of Time Series Analysis, 19 , 267–283.

Bickel, P. J., & Doksum, K. A. (2001). Mathematical statistics: Basic ideas and selected topics, volume 1 (2nd ed.). Upper Saddle River: Prentice-Hall.

Billingsley, P. (1968). Convergence of probability measures . New York: Wiley.

Billingsley, P. (1986). Probability and measure (2nd ed.). New York: Wiley.

Black, F., & Scholes, M. (1973). The pricing of options and corporate liabilities. The Journal of Political Economy, 81 , 637–654.

Bollerslev, T. (1986). Generalized autoregressive conditional heteroskedasticity. Journal of Econometrics, 31 , 307–327.

Bondon, P., & Palma, W. (2007). A class of antipersistent processes. Journal of Time Series Analysis, 28 , 261–273.

Boswijk, H. P. (1994). Testing for an unstable root in conditional and structural error correction models. Journal of Econometrics, 63 , 37–60.

Breiman, L. (1992). Probability (2nd ed.). Philadelphia: Society for Industrial and Applied Mathematics.

Brennan, M. J, & Schwartz, E. S. (1980). Analyzing convertible bonds. The Journal of Financial and Quantitative Analysis, 15 , 907–929.

Brockwell, P. J., & Davis, R. A. (1991). Time series: Theory and methods (2nd ed.). New York: Springer.

Broze, L., Scaillet, O., & Zakoïan, J.-M. (1995). Testing for continuous-time models of the short-term interest rate. Journal of Empirical Finance, 2 , 199–223.

Campbell, J. Y., & Mankiw, N. G. (1987). Are output fluctuations transitory? Quarterly Journal of Economics, 102 , 857–880.

Chan, K. C., Karolyi, G. A., Longstaff, F. A., & Sanders, A. B. (1992). An empirical comparision of alternative models of the short-term interest rate. The Journal of Finance, XLVII , 1209–1227.

Chang, Y., & Park, J. (2002). On the asymptotics of ADF tests for unit roots. Econometric Reviews, 21 , 431–447.

Cochrane, J. H. (1988). How big is the random walk in GNP? Journal of Political Economy, 96 , 893–920.

Cogley, T., & Sargent T. S. (2005). Drifts and volatilities: Monetary policies and outcomes in the post WWII US. Review of Economic Dynamics, 8 , 262–302.

Constantinides, G. M., & Ingersoll, J. E., Jr. (1984). Optimal bond trading with personal taxes. Journal of Financial Economics, 13 , 299–335.

Cox, J. C., Ingersoll, J. E., Jr., & Ross S. A. (1980). An analysis of variable rate loan contracts. The Journal of Finance, 35 , 389–403.

Cox, J. C., Ingersoll, J. E., Jr., & Ross S. A. (1985). A theory of the term structure of interest rates. Econometrica, 53 , 385–407.

Davidson, J. (1994). Stochastic limit theory: An introduction for econometricians . Oxford/New York: Oxford University Press.

Davidson, J., Hendry, D. F., Srba, F., & Yeo S. (1978). Econometric modelling of the aggregate time-series relationship between consumers’ expenditure and income in the United Kingdom. Economic Journal, 88 , 661–692.

Demetrescu, M., Kuzin, V., & Hassler, U. (2008). Long memory testing in the time domain. Econometric Theory, 24 , 176–215.

Dhrymes, Ph. J. (2000). Mathematics for econometrics (3rd ed.). New York: Springer.

Dickey, D. A., & Fuller, W. A. (1979). Distribution of the estimators for autoregressive time series with a unit root. Journal of the American Statistical Association, 74 , 427–431.

Donsker, M. D. (1951). An invariance principle for certain probability limit theorems. Memoirs of the American Mathematical Society, 6 , 1–12.

Dothan, L. U. (1978). On the term structure of interest rates. Journal of Financial Economics, 6 , 59–69.

Durlauf, S. N., & Phillips, P. C. B. (1988). Trends versus random walks in time series analysis. Econometrica, 56 , 1333–1354.

Elliott, G., Rothenberg, T. J., & Stock, J. H. (1996). Efficient tests for an autoregressive unit root. Econometrica, 64 , 813–836.

Engle, R. F. (1982). Autoregressive conditional heteroskedasticity with estimates of the variance of U.K. inflation. Econometrica, 50 , 987–1008.

Engle, R. F. (2002). New frontiers for ARCH models. Journal of Applied Econometrics, 17 , 425–446.

Engle, R. F., & Bollerslev T. (1986). Modelling the persistence of conditional variances. Econometric Reviews, 5 , 1–50.

Engle, R. F., & Granger, C. W. J. (1987). Co-integration and error correction: Representation, estimation, and testing. Econometrica, 55 , 251–276.

Engle, R. F., Lilien, D. M., & Robins, R. P. (1987). Estimating time-varying risk premia in the term structure: the ARCH-M model. Econometrica, 55 , 391–407.

Ericsson, N. R., & MacKinnon, J. G. (2002). Distributions of error correction tests for cointegration. Econometrics Journal, 5 , 285–318.

Frisch, R., & Waugh, F. V. (1933). Partial time regressions as compared with individual trends. Econometrica, 1 , 387–401.

Fuller, W. A. (1996). Introduction to statistical time series (2nd ed.). New York: Wiley.

Giraitis, L., Koul, H. L., & Surgailis, D. (2012). Large sample inference for long memory processes . London: Imperial College Press.

Gourieroux, Chr., & Jasiak, J. (2001). Financial econometrics: Problems, models, and methods . Princeton: Princeton University Press.

Gradshteyn, I. S., & Ryzhik, I. M. (2000). Table of integrals, series, and products (6th ed.). London/San Diego: Academic Press.

Granger, C. W. J. (1981). Some properties of time series data and their use in econometric model specification. Journal of Econometrics, 16 , 121–130.

Granger, C. W. J., & Joyeux, R. (1980). An Introduction to long-memory time series models and fractional differencing. Journal of Time Series Analysis, 1 , 15–29.

Granger, C. W. J., & Newbold P. (1974). Spurious regressions in econometrics. Journal of Econometrics, 2 , 111–120.

Grimmett, G. R., & Stirzaker, D. R. (2001). Probability and random processes (3rd ed.). Oxford: Oxford University Press.

Hamilton, J. (1994). Time series analysis . Princeton: Princeton University Press.

Hansen, B. E. (1992). Efficient estimation and testing of cointegrating vectors in the presence of deterministic trends. Journal of Econometrics, 53 , 87–121.

Hansen, B. E. (1992a). Convergence to stochastic integrals for dependent heterogeneous processes. Econometric Theory, 8 , 489–500.

Harris, D., & Inder, B. (1994). A test of the null hypothesis of cointegration. In C. P. Hargreaves (Ed.), Nonstationary time series analysis and cointegration (pp. 133–152). Oxford/New York: Oxford University Press.

Hassler, U. (2000). Simple regressions with linear time trends. Journal of Time Series Analysis, 21 , 27–32.

Hassler, U. (2000a). Cointegration testing in single error-correction equations in the presence of linear time trends. Oxford Bulletin of Economics and Statistics, 62 , 621–632.

Hassler, U. (2000b). The KPSS test for cointegration in case of bivariate regressions with linear trends. Econometric Theory, 16 , 451–453.

Hassler, U. (2001). The effect of linear time trends on the KPSS test for cointegration. Journal of Time Series Analysis, 22 , 283–292.

Hassler, U. (2012). Impulse responses of antipersistent processes. Economics Letters, 116 , 454–456.

Hassler, U. (2014). Persistence under temporal aggregation and differencing. Economics Letters, 124 , 318–322.

Hassler, U., & Hosseinkouchack, M. (2014). Effect of the order of fractional integration on impulse responses. Economics Letters, 125 , 311–314.

Hassler, U., & Kokoszka (2010). Impulse responses of fractionally integrated processes with long memory. Econometric Theory, 26 , 1855–1861.

Hendry, D. F. (1980). Econometrics – alchemy or science? Economica, 47 , 387–406.

Hosking, J. R. M. (1981). Fractional differencing. Biometrika, 68 , 165–176.

Johansen, S. (1995). Likelihood-based inference in cointegrated vector autoregressive models . Oxford/New York: Oxford University Press.

Johnson, N. L., Kotz, S., & Balakrishnan, N. (1994). Continuous univariate distributions, Volume 1 (2nd ed.). New York: Wiley.

Kirchgässner, G., Wolters, J., & Hassler, U. (2013). Introduction to modern time series analysis (2nd ed.). Berlin/New York: Springer.

Klebaner, F. C. (2005). Introduction to stochastic calculus with applications (2nd ed.). London: Imperical College Press.

Krämer, W. (1986). Least squares regression when the independent variable follows an ARIMA process. Journal of the American Statistical Association, 81 , 150–154.

Kwiatkowski, D., Phillips, P. C. B., Schmidt, P., & Shin Y. (1992). Testing the null hypothesis of stationarity against the alternative of a unit root. Journal of Econometrics, 54 , 159–178.

Leybourne, S. J., & McCabe, B. P. M. (1994). A simple test for cointegration. Oxford Bulletin of Economics and Statistics, 56 , 97–103.

Maasoumi, E., & McAleer, M. (2008). Realized volatility and long memory: An overview. Econometric Reviews, 27 , 1–9.

MacKinnon, J. G. (1991). Critical values for co-integration tests. In R. F. Engle, & C. W. J. Granger (Eds.), Long-run economic relationships (pp. 267–276). Oxford/New York: Oxford University Press.

MacKinnon, J. G. (1996). Numerical distribution functions for unit root and cointegration tests. Journal of Applied Econometrics, 11 , 601–618.

Marsh, T. A., & Rosenfeld, E. R. (1983). Stochastic processes for interest rates and equilibrium bond prices. The Journal of Finance, XXXVIII , 635–646.

Merton, R. C. (1973). Theory of rational option pricing. The Bell Journal of Economics and Management Science, 4 , 141–183.

Mikosch, Th. (1998). Elementary stochastic calculus with finance in view . Singapore: World Scientific Publishing.

Mills, T. C., & Markellos, R. N. (2008). The econometric modelling of financial time series (3rd ed.). Cambridge/New York: Cambridge University Press.

Nelson, D. B. (1991). Conditional heteroskedasticity in asset returns: A new approach. Econometrica, 59 , 347–370.

Nelson, D. B., & Cao, Ch. Q. (1992). Inequality constraints in the univariate GARCH model. Journal of Business & Economic Statistics, 10 , 229–235.

Newey, W. K., & West, K. D. (1987). A Simple, positive semi-definite, heteroskedasticity and autocorrelation consistent covariance matrix. Econometrica, 55 , 703–708.

Øksendal, B. (2003). Stochastic differential equations: An introduction with applications (6th ed.). Berlin/New York: Springer.

Park, J. Y. (1992). Canonical cointegrating regressions. Econometrica, 60 , 119–143.

Park, J. Y., & Phillips, P. C. B. (1988). Statistical inference in regressions with integrated processes: Part I. Econometric Theory, 4 , 468–497.

Phillips, P. C. B. (1986). Understanding spurious regressions in econometrics. Journal of Econometrics, 33 , 311–340.

Phillips, P. C. B. (1987). Time series regression with a unit root. Econometrica, 55 , 277–301.

Phillips, P. C. B. (1988). Weak convergence of sample covariance matrices to stochastic integrals via martingale approximations. Econometric Theory, 4 , 528–533.

Phillips, P. C. B. (1991). Optimal inference in cointegrated systems. Econometrica, 59 , 283–306.

Phillips, P. C. B., & Durlauf, S. N. (1986). Multiple time series regression with integrated processes. Review of Economic Studies, LIII , 473–495.

Phillips, P. C. B., & Hansen, B. E. (1990). Statistical inference in instrumental variables regression with I(1) processes. Review of Economic Studies, 57 , 99–125.

Phillips, P. C. B., & Loretan, M. (1991). Estimating long-run economic equilibria. Review of Economic Studies, 58 , 407–436.

Phillips, P. C. B., & Ouliaris, S. (1990). Asymptotic properties of residual based tests for cointegration. Econometrica, 58 , 165–193.

Phillips, P. C. B., & Park, J. Y. (1988). Asymptotic equivalence of ordinary least squares and generalized least squares in regressions with integrated regressors. Journal of the American Statistical Association, 83 , 111–115.

Phillips, P. C. B, & Perron, P. (1988). Testing for a unit root in time series regression. Biometrika, 75 , 335–346.

Phillips, P. C. B., & Solo, V. (1992). Asymptotics for linear processes. The Annals of Statistics, 20 , 971–1001.

Pötscher, B. M., & Prucha, I. R. (2001). Basic elements of asymptotic theory. In B. H. Baltagi (Ed.), A companion to theoretical econometrics (pp. 201–229). Malden: Blackwell.

Ross, S. (2010). A first course in probability (8th ed.). Upper Saddle River: Prentice-Hall.

Rudin, W. (1976). Principles of mathematical analyis (3rd ed.). New York: McGraw-Hill.

Said, S. E., & Dickey, D. A. (1984). Testing for unit roots in autoregressive-moving average models of unknown order. Biometrika, 71 , 599–607.

Saikkonen, P. (1991). Asymptotically efficient estimation of cointegration regressions. Econometric Theory, 7 , 1–21.

Shin, Y. (1994). A residual-based test of the Null of cointegration against the alternative of no cointegration. Econometric Theory, 10 , 91–115.

Soong, T. T. (1973). Random differential equations in science and engineering . New York: Academic Press.

Stock, J. H. (1987). Asymptotic properties of least squares estimators of cointegrating vectors. Econometrica, 55 , 1035–1056.

Stock, J. H., & Watson, M. W. (1993). A simple estimator of cointegrating vectors in higher order integrated systems. Econometrica, 61 , 783–820.

Sydsæter, K., Strøm, A., & Berck, P. (1999). Economists’ mathematical manual (3rd ed.). Berlin/New York: Springer.

Tanaka, K. (1996). Time series analysis: Nonstationary and noninvertible distribution theory . New York: Wiley.

Taylor, S.J. (1994). Modeling stochastic volatlity: A review and comparative study. Mathematical Finance, 4 , 183–204.

Trench, W. F. (2013). Introduction to real analysis . Free Hyperlinked Edition 2.04 December 2013. Downloaded on 10th May 2014 from http://​digitalcommons.​trinity.​edu/​mono/​7 .

Tsay, R. S. (2005). Analysis of financial time series (2nd ed.). New York: Wiley.

Tse, Y. K. (1995). Some international evidence on the stochastic behavior of interest rates. Journal of International Money and Finance, 14 , 721–738.

Vasicek, O. (1977). An equilibrium characterization of the term structure. Journal of Financial Economics, 5 , 177–188.

West, K. D. (1988). Asymptotic normality, when regressors have a unit root. Econometrica, 56 , 1397–1418.

White, H. (2001). Asymptotic theory for econometricians (2nd ed.). London/San Diego: Academic Press.

Wold, H. O. A. (1938). A study in the analysis of stationary time series . Stockholm: Almquist & Wiksell.

Index

Algebra

σ -

Borel-

ARCH model

EGARCH

GARCH

GARCH-M

IGARCH

Autocorrelation

Autocovariance

Autoregressive distributed lag model

Brownian bridge

Brownian motion

with drift

geometric

Cadlag

Causally invertible

Cholesky decomposition

Coefficient of determination

Cointegration

Comparison of coefficients

Continuous differentiability

Convergence

Cauchy criterion

in distribution

in mean square

in probability

weak

Correlation coefficient

Covariance

Cycle

annual

cosine

semi-annual

sine

Density function

Detrended regression

Detrending

Dickey-Fuller test

Difference equation

deterministic

stochastic

Differential equation

with constant coefficients

deterministic

homogeneous

stochastic

See(Stochastic differential equation)

Diffusion

Distribution

conditional

joint

marginal

multivariate

Distribution function

Drift

SeeIntegrated process

Durbin-Watson statistic

Error-correction model

Event

Expectation

conditional

Expected value

Filter

causal

difference

Fractional

differences

integration

noise

Frequency

Functional

Functional central limit theorem

Gamma function

Gaussian distribution

asymptotic

Gaussian process

Impulse response

Index set

Inequality

Cauchy-Schwarz

Chebyshev’s

Jensen’s

Markov’s

triangle

Information set

Integrated process

with drift

of order -1, I(-1)

of order 0, I(0)

of order 1, I(1)

Invariance principle

Ito integral

autocovariance

expected value

general

variance

Ito’s lemma

bivariate with one factor

for diffusions

for Wiener processes

multivariate

with time as a dependent variable

Ito sum

KPSS test

Kurtosis

Lag operator

Lag polynomial

causally invertible

invertible

Least squares estimator

Leverage effect

L’Hospital’s rule

Linear time trend

Long memory

Long-run variance

consistent estimation

matrix

Markov property

Martingale

difference

Mean squared error

Measurability

Metric

supremum

Moments

centered

Normal distribution

bivariate

log-

Ornstein-Uhlenbeck process

properties

Partial integration

Partition

adequate

disjoint

equidistant

Period

Persistence

anti-

strong

Power transfer function

Probability

space

Process

ARCH

See(ARCH model)

ARMA

autoregressive

continuous-time

discrete-time

integrated

See(Integrated process)

invertible

linear

Markov

moving average

normal

pure random

stationary

stochastic

strictly stationary

weakly stationary

Random variable

continuous

integrable

Random walk

continuous-valued

discrete-valued

Residuals

Riemann integral

autocovariance

expected value

Gaussian distribution

variance

Riemann-Stieltjes sum

Riemann sum

Schur criterion

Set of outcomes

Skewness

Spectral density function

Spectrum

Stieltjes integral

autocovariance

definition

expected value

Gaussian distribution

variance

Stochastically independent

Stochastic differential equation

with constant coefficients

with linear coefficients

inhomogeneous linear with additive noise

moments of the solution

uniqueness of solution

Stratonovich integral

Superconsistency

Theorem

Donsker’s

Fubini’s

Slutsky’s

Time series

Trend component

Trend stationary

Unit root

Variance

Variation

absolute

quadratic

Volatility

White noise

Wiener process

demeaned

hitting time

integrated

maximum

reflected

scale invariance

zero crossing

W.l.o.g.

Wold decomposition