Siegmund Brandt

Data AnalysisStatistical and Computational Methods for Scientists and Engineers4th ed. 2014

Siegmund Brandt

Department of Physics, University of Siegen, Siegen, Germany

ISBN 978-3-319-03761-5e-ISBN 978-3-319-03762-2

https://doi.org/10.1007/978-3-319-03762-2

Springer Cham Heidelberg New York Dordrecht London

Library of Congress Control Number: 2013957143

© Springer International Publishing Switzerland 2014

Additional material to this book can be downloaded fromhttp://​extras.​springer.​com

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection with reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publisher’s location, in its current version, and permission for use must always be obtained from Springer. Permissions for use may be obtained through RightsLink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Law.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein.

Printed on acid-free paper

Springer is part of Springer Science+Business Media (www.springer.com)

Preface to the Fourth English Edition

For the present edition, the book has undergone two major changes: Its appearance was tightened significantly and the programs are now written in the modern programming language Java.

Tightening was possible without giving up essential contents by expedient use of the Internet. Since practically all users can connect to the net, it is no longer necessary to reproduce program listings in the printed text. In this way, the physical size of the book was reduced considerably.

The Java language offers a number of advantages over the older programming languages used in earlier editions. It is object-oriented and hence also more readable. It includes access to libraries of user-friendly auxiliary routines, allowing for instance the easy creation of windows for input, output, or graphics. For most popular computers, Java is either preinstalled or can be downloaded from the Internet free of charge. (See Sect.  1.​3 for details.) Since by now Java is often taught at school, many students are already somewhat familiar with the language.

Our Java programs for data analysis and for the production of graphics, including many example programs and solutions to programming problems, can be downloaded from the page under www.​extras.​springer.​com .

I am grateful to Dr. Tilo Stroh for numerous stimulating discussions and technical help. The graphics programs are based on previous common work.

Siegmund Brandt

Siegen, Germany

Frequently Used Symbols and Notation

x , y , ξ , η , …

(ordinary) variable

$$\mathbf{x}$$ , $$\mathbf{y}$$ , $$\boldsymbol{\upxi }$$ , $$\boldsymbol{\upeta }$$ , …

vector variable

x , y , …

random variable

$$\mathsf{\mathbf{x}}$$ , $$\mathsf{\mathbf{y}}$$ , …

vector of random variables

A , B , C , …

matrices

B

bias

covariance

F

variance ratio

f ( x )

probability density

F ( x )

distribution function

mean value, expectation value

H

hypothesis

H ₀

null hypothesis

L , ℓ

likelihood functions

L ( S c ,  λ )

operating characteristic function

M ( S c ,  λ )

power (of a test)

M

minimum function, target function

P ( A )

probability of the event A

Q

sum of squares

sample variance

s

estimator

S c

critical region

t

variable of Student’s distribution

T

Testfunktion

x m

most probable value (mode)

x 0. 5

median

x q

quantile

sample mean

estimator from maximum likelihood or least squares

α

level of significance

1 − α

level of confidence

λ

parameter of a distribution

characteristic function

ϕ ( x ), ψ ( x )

probability density and distribution function of the normal distribution

ϕ ₀ ( x ), ψ ₀ ( x )

probability density and distribution function of the standard normal distribution

σ ( x ) =  Δ ( x )

standard deviation

σ ² ( x )

variance

χ ²

variable of the χ ² distribution

Ω ( P )

inverse function of the normal distribution

Contents

1 Introduction 1

1.​1 Typical Problems of Data Analysis 1

1.​2 On the Structure of this Book 2

1.​3 About the Computer Programs 5

2 Probabilities 7

2.​1 Experiments, Events, Sample Space 7

2.​2 The Concept of Probability 8

2.​3 Rules of Probability Calculus:​ Conditional Probability 10

2.​4 Examples 11

2.4.1 Probability for n Dots in the Throwing of Two Dice 11

2.​4.​2 Lottery 6 Out of 49 12

2.​4.​3 Three-Door Game 13

3 Random Variables:​ Distributions 15

3.​1 Random Variables 15

3.​2 Distributions of a Single Random Variable 15

3.​3 Functions of a Single Random Variable, Expectation Value, Variance, Moments 17

3.​4 Distribution Function and Probability Density of Two Variables:​ Conditional Probability 25

3.​5 Expectation Values, Variance, Covariance, and Correlation 27

3.​6 More than Two Variables:​ Vector and Matrix Notation 30

3.​7 Transformation of Variables 33

3.​8 Linear and Orthogonal Transformations:​ Error Propagation 36

4 Computer Generated Random Numbers:​ The Monte Carlo Method 41

4.​1 Random Numbers 41

4.​2 Representation of Numbers in a Computer 42

4.​3 Linear Congruential Generators 44

4.​4 Multiplicative Linear Congruential Generators 45

4.​5 Quality of an MLCG:​ Spectral Test 47

4.​6 Implementation and Portability of an MLCG 50

4.​7 Combination of Several MLCGs 52

4.​8 Generation of Arbitrarily Distributed Random Numbers 55

4.​8.​1 Generation by Transformation of the Uniform Distribution 55

4.​8.​2 Generation with the von Neumann Acceptance–Rejection Technique 58

4.​9 Generation of Normally Distributed Random Numbers 62

4.​10 Generation of Random Numbers According to a Multivariate Normal Distribution 63

4.​11 The Monte Carlo Method for Integration 64

4.​12 The Monte Carlo Method for Simulation 66

4.​13 Java Classes and Example Programs 67

5 Some Important Distributions and Theorems 69

5.​1 The Binomial and Multinomial Distributions 69

5.​2 Frequency:​ The Law of Large Numbers 72

5.​3 The Hypergeometric Distribution 74

5.​4 The Poisson Distribution 78

5.​5 The Characteristic Function of a Distribution 81

5.​6 The Standard Normal Distribution 84

5.​7 The Normal or Gaussian Distribution 86

5.​8 Quantitative Properties of the Normal Distribution 88

5.​9 The Central Limit Theorem 90

5.​10 The Multivariate Normal Distribution 94

5.​11 Convolutions of Distributions 100

5.​11.​1 Folding Integrals 100

5.​11.​2 Convolutions with the Normal Distribution 103

5.​12 Example Programs 106

6 Samples 109

6.​1 Random Samples.​ Distribution of a Sample.​ Estimators 109

6.​2 Samples from Continuous Populations:​ Mean and Variance of a Sample 111

6.​3 Graphical Representation of Samples:​ Histograms and Scatter Plots 115

6.​4 Samples from Partitioned Populations 122

6.​5 Samples Without Replacement from Finite Discrete Populations.​ Mean Square Deviation.​ Degrees of Freedom 127

6.6 Samples from Gaussian Distributions: χ ² -Distribution 130

6.7 χ ² and Empirical Variance 135

6.​8 Sampling by Counting:​ Small Samples 136

6.​9 Small Samples with Background 142

6.​10 Determining a Ratio of Small Numbers of Events 144

6.​11 Ratio of Small Numbers of Events with Background 147

6.​12 Java Classes and Example Programs 149

7 The Method of Maximum Likelihood 153

7.​1 Likelihood Ratio:​ Likelihood Function 153

7.​2 The Method of Maximum Likelihood 155

7.​3 Information Inequality.​ Minimum Variance Estimators.​ Sufficient Estimators 157

7.​4 Asymptotic Properties of the Likelihood Function and Maximum-Likelihood Estimators 164

7.​5 Simultaneous Estimation of Several Parameters:​ Confidence Intervals 167

7.​6 Example Programs 173

8 Testing Statistical Hypotheses 175

8.​1 Introduction 175

8.2 F -Test on Equality of Variances 177

8.​3 Student’s Test:​ Comparison of Means 180

8.​4 Concepts of the General Theory of Tests 185

8.​5 The Neyman–Pearson Lemma and Applications 191

8.​6 The Likelihood-Ratio Method 194

8.7 The χ ² -Test for Goodness-of-Fit 199

8.7.1 χ ² -Test with Maximal Number of Degrees of Freedom 199

8.7.2 χ ² -Test with Reduced Number of Degrees of Freedom 200

8.7.3 χ ² -Test and Empirical Frequency Distribution 200

8.​8 Contingency Tables 203

8.​9 2 × 2 Table Test 204

8.​10 Example Programs 205

9 The Method of Least Squares 209

9.​1 Direct Measurements of Equal or Unequal Accuracy 209

9.​2 Indirect Measurements:​ Linear Case 214

9.​3 Fitting a Straight Line 218

9.​4 Algorithms for Fitting Linear Functions of the Unknowns 222

9.​4.​1 Fitting a Polynomial 222

9.​4.​2 Fit of an Arbitrary Linear Function 224

9.​5 Indirect Measurements:​ Nonlinear Case 226

9.​6 Algorithms for Fitting Nonlinear Functions 228

9.​6.​1 Iteration with Step-Size Reduction 229

9.​6.​2 Marquardt Iteration 234

9.7 Properties of the Least-Squares Solution: χ ² -Test 236

9.​8 Confidence Regions and Asymmetric Errors in the Nonlinear Case 240

9.​9 Constrained Measurements 243

9.​9.​1 The Method of Elements 244

9.​9.​2 The Method of Lagrange Multipliers 247

9.​10 The General Case of Least-Squares Fitting 251

9.​11 Algorithm for the General Case of Least Squares 255

9.​12 Applying the Algorithm for the General Case to Constrained Measurements 258

9.​13 Confidence Region and Asymmetric Errors in the General Case 260

9.​14 Java Classes and Example Programs 261

10 Function Minimization 267

10.​1 Overview:​ Numerical Accuracy 267

10.​2 Parabola Through Three Points 273

10.3 Function of n Variables on a Line in an n -Dimensional Space 275

10.​4 Bracketing the Minimum 275

10.​5 Minimum Search with the Golden Section 277

10.​6 Minimum Search with Quadratic Interpolation 280

10.7 Minimization Along a Direction in n Dimensions 280

10.8 Simplex Minimization in n Dimensions 281

10.​9 Minimization Along the Coordinate Directions 284

10.​10 Conjugate Directions 285

10.​11 Minimization Along Chosen Directions 287

10.​12 Minimization in the Direction of Steepest Descent 288

10.​13 Minimization Along Conjugate Gradient Directions 288

10.​14 Minimization with the Quadratic Form 292

10.​15 Marquardt Minimization 292

10.​16 On Choosing a MinimizationMeth​od 295

10.​17 Consideration of Errors 296

10.​18 Examples 298

10.​19 Java Classes and Example Programs 303

11 Analysis of Variance 307

11.​1 One-Way Analysis of Variance 307

11.​2 Two-Way Analysis of Variance 311

11.​3 Java Class and Example Programs 319

12 Linear and Polynomial Regression 321

12.​1 Orthogonal Polynomials 321

12.​2 Regression Curve:​ Confidence Interval 325

12.​3 Regression with Unknown Errors 326

12.​4 Java Class and Example Programs 329

13 Time Series Analysis 331

13.​1 Time Series:​ Trend 331

13.​2 Moving Averages 332

13.​3 Edge Effects 336

13.​4 Confidence Intervals 336

13.​5 Java Class and Example Programs 340

Literature341

A Matrix Calculations347

A.1 Definitions: Simple Operations348

A.2 Vector Space, Subspace, Rank of a Matrix351

A.3 Orthogonal Transformations353

A.3.1 Givens Transformation354

A.3.2 Householder Transformation356

A.3.3 Sign Inversion359

A.3.4 Permutation Transformation359

A.4 Determinants360

A.5 Matrix Equations: Least Squares362

A.6 Inverse Matrix365

A.7 Gaussian Elimination367

A.8 LR-Decomposition369

A.9 Cholesky Decomposition372

A.10 Pseudo-inverse Matrix375

A.11 Eigenvalues and Eigenvectors376

A.12 Singular Value Decomposition379

A.13 Singular Value Analysis380

A.14 Algorithm for Singular Value Decomposition385

A.14.1 Strategy385

A.14.2 Bidiagonalization386

A.14.3 Diagonalization388

A.14.4 Ordering of the Singular Values and Permutation392

A.14.5 Singular Value Analysis392

A.15 Least Squares with Weights392

A.16 Least Squares with Change of Scale393

A.17 Modification of Least Squares According to Marquardt394

A.18 Least Squares with Constraints396

A.19 Java Classes and Example Programs399

B Combinatorics405

C Formulas and Methods for the Computation of Statistical Functions409

C.1 Binomial Distribution409

C.2 Hypergeometric Distribution409

C.3 Poisson Distribution410

C.4 Normal Distribution410

C.5 χ 2 -Distribution 412

C.6 F -Distribution 413

C.7 t -Distribution 413

C.8 Java Class and Example Program414

D The Gamma Function and Related Functions: Methods and Programs for Their Computation415

D.1 The Euler Gamma Function415

D.2 Factorial and Binomial Coefficients418

D.3 Beta Function418

D.4 Computing Continued Fractions418

D.5 Incomplete Gamma Function420

D.6 Incomplete Beta Function420

D.7 Java Class and Example Program422

E Utility Programs425

E.1 Numerical Differentiation425

E.2 Numerical Determination of Zeros427

E.3 Interactive Input and Output Under Java427

E.4 Java Classes428

F The Graphics Class DatanGraphics431

F.1 Introductory Remarks431

F.2 Graphical Workstations: Control Routines431

F.3 Coordinate Systems, Transformations and Transformation Methods432

F.3.1 Coordinate Systems432

F.3.2 Linear Transformations:Window – Viewport433

F.4 Transformation Methods435

F.5 Drawing Methods436

F.6 Utility Methods439

F.7 Text Within the Plot441

F.8 Java Classes and Example Programs441

G Problems, Hints and Solutions, and Programming Problems447

G.1 Problems447

G.2 Hints and Solutions456

G.3 Programming Problems470

H Collection of Formulas487

I Statistical Tables503

List of Computer Programs515

Index517

List of Examples

2.1 Sample space for continuous variables7

2.2 Sample space for discrete variables8

3.1 Discrete random variable15

3.2 Continuous random variable15

3.3 Uniform distribution22

3.4 Cauchy distribution23

3.5 Lorentz (Breit–Wigner) distribution25

3.6 Error propagation and covariance38

4.1 Exponentially distributed random numbers57

4.2 Generation of random numbers following a Breit–Wigner distribution57

4.3 Generation of random numbers with a triangular distribution58

4.4 Semicircle distribution with the simple acceptance–rejection method59

4.5 Semicircle distribution with the general acceptance–rejection method61

4.6 Computation of π 65

4.7 Simulation of measurement errors of points on a line66

4.8 Generation of decay times for a mixture of two different radioactive substances66

5.1 Statistical error74

5.2 Application of the hypergeometric distribution for determination of zoological populations77

5.3 Poisson distribution and independence of radioactive decays80

5.4 Poisson distribution and the independence of scientific discoveries81

5.5 Addition of two Poisson distributed variables with use of the characteristic function84

5.6 Normal distribution as the limiting case of the binomial distribution92

5.7 Error model of Laplace92

5.8 Convolution of uniform distributions102

5.9 Convolution of uniform and normal distributions104

5.10 Convolution of two normal distributions. “Quadratic addition of errors”104

5.11 Convolution of exponential and normal distributions105

6.1 Computation of the sample mean and variance from data114

6.2 Histograms of the same sample with various choices of bin width117

6.3 Full width at half maximum (FWHM)119

6.4 Investigation of characteristic quantities of samples from a Gaussian distribution with the Monte Carlo method119

6.5 Two-dimensional scatter plot: Dividend versus price for industrial stocks120

6.6 Optimal choice of the sample size for subpopulations125

6.7 Determination of a lower limit for the lifetime of the proton from the observation of no decays142

7.1 Likelihood ratio154

7.2 Repeated measurements of differing accuracy156

7.3 Estimation of the parameter N of the hypergeometric distribution 157

7.4 Estimator for the parameter of the Poisson distribution162

7.5 Estimator for the parameter of the binomial distribution163

7.6 Law of error combination (“Quadratic averaging of individual errors”)163

7.7 Determination of the mean lifetime from a small number of decays166

7.8 Estimation of the mean and variance of a normal distribution171

7.9 Estimators for the parameters of a two-dimensional normal distribution172

8.1 F -test of the hypothesis of equal variance of two series of measurements 180

8.2 Student’s test of the hypothesis of equal means of two series of measurements185

8.3 Test of the hypothesis that a normal distribution with given variance σ 2 has the mean λ = λ 0 189

8.4 Most powerful test for the problem of Example 8.3193

8.5 Power function for the test from Example 8.3195

8.6 Test of the hypothesis that a normal distribution of unknown variance has the mean value λ = λ 0 197

8.7 χ 2-test for the fit of a Poisson distribution to an empirical frequency distribution 202

9.1 Weighted mean of measurements of different accuracy212

9.2 Fitting of various polynomials223

9.3 Fitting a proportional relation224

9.4 Fitting a Gaussian curve231

9.5 Fit of an exponential function232

9.6 Fitting a sum of exponential functions233

9.7 Fitting a sum of two Gaussian functions and a polynomial235

9.8 The influence of large measurement errors on the confidence region of the parameters for fitting an exponential function241

9.9 Constraint between the angles of a triangle245

9.10 Application of the method of Lagrange multipliers to Example 9.9249

9.11 Fitting a line to points with measurement errors in both the abscissa and ordinate257

9.12 Fixing parameters257

9.13 χ 2-test of the description of measured points with errors in abscissa and ordinate by a given line 259

9.14 Asymmetric errors and confidence region for fitting a straight line to measured points with errors in the abscissa and ordinate260

10.1 Determining the parameters of a distribution from the elements of a sample with the method of maximum likelihood298

10.2 Determination of the parameters of a distribution from the histogram of a sample by maximizing the likelihood299

10.3 Determination of the parameters of a distribution from the histogram of a sample by minimization of a sum of squares302

11.1 One-way analysis of variance of the influence of various drugs310

11.2 Two-way analysis of variance in cancer research318

12.1 Treatment of Example 9.2 with Orthogonal Polynomials325

12.2 Confidence limits for linear regression327

13.1 Moving average with linear trend335

13.2 Time series analysis of the same set of measurements using different averaging intervals and polynomials of different orders338

A.1 Inversion of a 3×3 matrix381

A.2 Almost vanishing singular values393

A.3 Point of intersection of two almost parallel lines393

A.4 Numerical superiority of the singular value decomposition compared to the solution of normal equations396

A.5 Least squares with constraints410

© Springer International Publishing Switzerland 2014

Siegmund BrandtData Analysishttps://doi.org/10.1007/978-3-319-03762-2_1

1. Introduction

Siegmund Brandt1 

(1)

Department of Physics, University of Siegen, Siegen, Germany

1.1 Typical Problems of Data Analysis

Every branch of experimental science, after passing through an early stage of qualitative description, concerns itself with quantitative studies of the phenomena of interest, i.e., measurements. In addition to designing and carrying out the experiment, an important task is the accurate evaluation and complete exploitation of the data obtained. Let us list a few typical problems.

1.

A study is made of the weight of laboratory animals under the influence of various drugs. After the application of drug A to 25 animals, an average increase of 5 % is observed. Drug B, used on 10 animals, yields a 3 % increase. Is drug A more effective? The averages 5 and 3 % give practically no answer to this question, since the lower value may have been caused by a single animal that lost weight for some unrelated reason. One must therefore study the distribution of individual weights and their spread around the average value. Moreover, one has to decide whether the number of test animals used will enable one to differentiate with a certain accuracy between the effects of the two drugs.

2.

In experiments on crystal growth it is essential to maintain exactly the ratios of the different components. From a total of 500 crystals, a sample of 20 is selected and analyzed. What conclusions can be drawn about the composition of the remaining 480? This problem of sampling comes up, for example, in quality control, reliability tests of automatic measuring devices, and opinion polls.

3.

A certain experimental result has been obtained. It must be decided whether it is in contradiction with some predicted theoretical value or with previous experiments. The experiment is used for hypothesis testing.

4.

A general law is known to describe the dependence of measured variables, but parameters of this law must be obtained from experiment. In radioactive decay, for example, the number N of atoms that decay per second decreases exponentially with time: $$N(t) = \mbox{ const} \cdot \exp (-\lambda t)$$. One wishes to determine the decay constant λ and its measurement error by making maximal use of a series of measured values N ₁(t ₁), N ₂(t ₂), …. One is concerned here with the problem of fitting a function containing unknown parameters to the data and the determination of the numerical values of the parameters and their errors.

From these examples some of the aspects of data analysis become apparent. We see in particular that the outcome of an experiment is not uniquely determined by the experimental procedure but is also subject to chance: it is a random variable. This stochastic tendency is either rooted in the nature of the experiment (test animals are necessarily different, radioactivity is a stochastic phenomenon), or it is a consequence of the inevitable uncertainties of the experimental equipment, i.e., measurement errors. It is often useful to simulate with a computer the variable or stochastic characteristics of the experiment in order to get an idea of the expected uncertainties of the results before carrying out the experiment itself. This simulation of random quantities on a computer is called the Monte Carlo method, so named in reference to games of chance.

1.2 On the Structure of this Book

The basis for using random quantities is the calculus of probabilities. The most important concepts and rules for this are collected in Chap. 2.​ Random variables are introduced in Chap. 3.​ Here one considers distributions of random variables, and parameters are defined to characterize the distributions, such as the expectation value and variance. Special attention is given to the interdependence of several random variables. In addition, transformations between different sets of variables are considered; this forms the basis of error propagation.

Generating random numbers on a computer and the Monte Carlo method are the topics of Chap. 4.​ In addition to methods for generating random numbers, a well-tested program and also examples for generating arbitrarily distributed random numbers are given. Use of the Monte Carlo method for problems of integration and simulation is introduced by means of examples. The method is also used to generate simulated data with measurement errors, with which the data analysis routines of later chapters can be demonstrated.

In Chap. 5 we introduce a number of distributions which are of particular interest in applications. This applies especially to the Gaussian or normal distribution, whose properties are studied in detail.

In practice a distribution must be determined from a finite number of observations, i.e., from a sample. Various cases of sampling are considered in Chap. 6.​ Computer programs are presented for a first rough numerical treatment and graphical display of empirical data. Functions of the sample, i.e., of the individual observations, can be used to estimate the parameters characterizing the distribution. The requirements that a good estimate should satisfy are derived. At this stage the quantity χ ² is introduced. This is the sum of the squares of the deviations between observed and expected values and is therefore a suitable indicator of the goodness-of-fit.

The maximum-likelihood method, discussed in Chap. 7, forms the core of modern statistical analysis. It allows one to construct estimators with optimum properties. The method is discussed for the single and multiparameter cases and illustrated in a number of examples. Chapter 8 is devoted to hypothesis testing. It contains the most commonly used F, t, and χ ² tests and in addition outlines the general points of test theory.

The method of least squares, which is perhaps the most widely used statistical procedure, is the subject of Chap. 9.​ The special cases of direct, indirect, and constrained measurements, often encountered in applications, are developed in detail before the general case is discussed. Programs and examples are given for all cases. Every least-squares problem, and in general every problem of maximum likelihood, involves determining the minimum of a function of several variables. In Chap. 10 various methods are discussed in detail, by which such a minimization can be carried out. The relative efficiency of the procedures is shown by means of programs and examples.

The analysis of variance (Chap. 11) can be considered as an extension of the F-test. It is widely used in biological and medical research to study the dependence, or rather to test the independence, of a measured quantity from various experimental conditions expressed by other variables. For several variables rather complex situations can arise. Some simple numerical examples are calculated using a computer program.

Linear and polynomial regression, the subject of Chap. 12, is a special case of the least-squares method and has therefore already been treated in Chap. 9.​ Before the advent of computers, usually only linear least-squares problems were tractable. A special terminology, still used, was developed for this case. It seemed therefore justified to devote a special chapter to this subject. At the same time it extends the treatment of Chap. 9.​ For example the determination of confidence intervals for a solution and the relation between regression and analysis of variance are studied. A general program for polynomial regression is given and its use is shown in examples.

In the last chapter the elements of time series analysis are introduced. This method is used if data are given as a function of a controlled variable (usually time) and no theoretical prediction for the behavior of the data as a function of the controlled variable is known. It is used to try to reduce the statistical fluctuation of the data without destroying the genuine dependence on the controlled variable. Since the computational work in time series analysis is rather involved, a computer program is also given.

The field of data analysis, which forms the main part of this book, can be called applied mathematical statistics. In addition, wide use is made of other branches of mathematics and of specialized computer techniques. This material is contained in the appendices.

In Appendix A, titled “Matrix Calculations”, the most important concepts and methods from linear algebra are summarized. Of central importance are procedures for solving systems of linear equations, in particular the singular value decomposition, which provides the best numerical properties.

Necessary concepts and relations of combinatorics are compiled in Appendix B. The numerical value of functions of mathematical statistics must often be computed. The necessary formulas and algorithms are contained in Appendix C. Many of these functions are related to the Euler gamma function and like it can only be computed with approximation techniques. In Appendix D formulas and methods for gamma and related functions are given. Appendix E describes further methods for numerical differentiation, for the determination of zeros, and for interactive input and output under Java.

The graphical representation of measured data and their errors and in many cases also of a fitted function is of special importance in data analysis. In Appendix F a Java class with a comprehensive set of graphical methods is presented. The most important concepts of computer graphics are introduced and all of the necessary explanations for using this class are given.

Appendix G.1 contains problems to most chapters. These problems can be solved with paper and pencil. They should help the reader to understand the basic concepts and theorems. In some cases also simple numerical calculations must be carried out. In Appendix G.2 either the solution of problems is sketched or the result is simply given. In Appendix G.3 a number of programming problems is presented. For each one an example solution is given.

The set of appendices is concluded with a collection of formulas in Appendix H, which should facilitate reference to the most important equations, and with a short collection of statistical tables in Appendix I. Although all of the tabulated values can be computed (and in fact were computed) with the programs of Appendix C, it is easier to look up one or two values from the tables than to use a computer.

1.3 About the Computer Programs

For the present edition all programs were newly written in the programming language Java. Since some time Java is taught in many schools so that young readers often are already familiar with that language. Java classes are directly executable on all popular computers – independently of the operating system. The compilation of Java source programs takes place using the Java Development Kit, which for many operating systems, in particular Windows, Linux, and Mac OSX, can be downloaded free of cost from the Internet, http://​www.​oracle.​com/​technetwork/​java/​index.​html.

There are four groups of computer programs discussed in this book. These are

The data analysis library in the form of the package datan,

The graphics library in the form of the package datangraphics,

A collection of example programs in the package examples,

Solutions to the programming problems in the package solutions.

The programs of all groups are available both as compiled classes and (except for datangraphics.DatanGraphics) also as source files. In addition there is the extensive Java-typical documentation in html format.

Every class and method of the package datan deals with a particular, well defined problem, which is extensively described in the text. That also holds for the graphics library, which allows to produce practically any type of line graphics in two dimensions. For many purposes it suffices, however, to use one of 5 classes each yielding a complete graphics.

In order to solve a specific problem the user has to write a short class in Java, which essentially consists of calling classes from the data analysis library, and which in certain cases organizes the input of the user’s data and output of the results. The example programs are a collection of such classes. The application of each method from the data analysis and graphics libraries is demonstrated in at least one example program. Such example programs are described in a special section near the end of most chapters.

Near the end of the book there is a List of Computer Programs in alphabetic order. For each program from the data analysis library and from the graphics library page numbers are given, for an explanation of the program itself, and for one or several example programs demonstrating its use.

The programming problems like the example programs are designed to help the reader in using computer methods. Working through these problems should enable readers to formulate their own specific tasks in data analysis to be solved on a computer. For all programming problems, programs exist which represent a possible solution.

In data analysis, of course, data play a special role. The type of data and the format in which they are presented to the computer cannot be defined in a general textbook since it depends very much on the particular problem at hand. In order to have somewhat realistic data for our examples and problems we have decided to produce them in most cases within the program using the Monte Carlo method. It is particularly instructive to simulate data with known properties and a given error distribution and to subsequently analyze these data. In the analysis one must in general make an assumption about the distribution of the errors. If this assumption is not correct, then the results of the analysis are not optimal. Effects that are often decisively important in practice can be “experienced” with exercises combining simulation and analysis.

Here are some short hints concerning the installation of our programs. As material accompanying this book, available from the page www.​extras.​springer.​com, there is a zip file named DatanJ. Download this file, unzip it while keeping the internal tree structure of subdirectories and store it on your computer in a new directory. (It is convenient to also give that directory the name DatanJ.) Further action is described in the file ReadMe in that directory.

© Springer International Publishing Switzerland 2014

Siegmund BrandtData Analysishttps://doi.org/10.1007/978-3-319-03762-2_2

2. Probabilities

Siegmund Brandt1 

(1)

Department of Physics, University of Siegen, Siegen, Germany

2.1 Experiments, Events, Sample Space

Since in this book we are concerned with the analysis of data originating from experiments , we will have to state first what we mean by an experiment and its result. Just as in the laboratory, we define an experiment to be a strictly followed procedure, as a consequence of which a quantity or a set of quantities is obtained that constitutes the result. These quantities are continuous (temperature, length, current) or discrete (number of particles, birthday of a person, one of three possible colors). No matter how accurately all conditions of the procedure are maintained, the results of repetitions of an experiment will in general differ. This is caused either by the intrinsic statistical nature of the phenomenon under investigation or by the finite accuracy of the measurement. The possible results will therefore always be spread over a finite region for each quantity. All of these regions for all quantities that make up the result of an experiment constitute the sample space of that experiment. Since it is difficult and often impossible to determine exactly the accessible regions for the quantities measured in a particular experiment, the sample space actually used may be larger and may contain the true sample space as a subspace. We shall use this somewhat looser concept of a sample space.

Example 2.1:

Sample space for continuous variables

In the manufacture of resistors it is important to maintain the values R (electrical resistance measured in ohms) and N (maximum heat dissipation measured in watts) at given values. The sample space for R and N is a plane spanned by axes labeled R and N. Since both quantities are always positive, the first quadrant of this plane is itself a sample space. ■ 

Example 2.2:

Sample space for discrete variables

In practice the exact values of R and N are unimportant as long as they are contained within a certain interval about the nominal value (e.g., 99 k$$\varOmega < R < 101\,$$ k$$\varOmega$$, 0.49 W < N < 0. 60 W). If this is the case, we shall say that the resistor has the properties R n , N n . If the value falls below (above) the lower (upper) limit, then we shall substitute the index n by −(+). The possible values of resistance and heat dissipation are therefore $$R_{-},\:R_{n},\:R_{+},\:N_{-},\:N_{n},\:N_{+}$$. The sample space now consists of nine points:

 ■ 

Often one or more particular subspaces of the sample space are of special interest. In Example 2.2, for instance, the point R n , N n represents the case where the resistors meet the production specifications. We can give such subspaces names, e.g., A, B, … and say that if the result of an experiment falls into one such subspace, then the event A (or B, C, …) has occurred. If A has not occurred, we speak of the complementary event $$\bar{A}$$ (i.e., not A). The whole sample space corresponds to an event that will occur in every experiment, which we call E. In the rest of this chapter we shall define what we mean by the probability of the occurrence of an event and present rules for computations with probabilities.

2.2 The Concept of Probability

Let us consider the simplest experiment, namely, the tossing of a coin. Like the throwing of dice or certain problems with playing cards it is of no practical interest but is useful for didactic purposes. What is the probability that a “fair” coin shows “heads” when tossed once? Our intuition suggests that this probability is equal to 1∕2. It is based on the assumption that all points in sample space (there are only two points: “heads” and “tails”) are equally probable and on the convention that we give the event E (here: “heads” or “tails”) a probability of unity. This way of determining probabilities can be applied only to symmetric experiments and is therefore of little practical use. (It is, however, of great importance in statistical physics and quantum statistics, where the equal probabilities of all allowed states is an essential postulate of very successful theories.) If no such perfect symmetry exists—which will even be the case with normal “physical” coins—the following procedure seems reasonable. In a large number N of experiments the event A is observed to occur n times. We define

(2.2.1)

as the probability of the occurrence of the event A. This somewhat loose frequency definition of probability is sufficient for practical purposes, although it is mathematically unsatisfactory. One of the difficulties with this definition is the need for an infinity of experiments, which are of course impossible to perform and even difficult to imagine. Although we shall in fact use the frequency definition in this book, we will indicate the basic concepts of an axiomatic theory of probability due to KOLMOGOROV [1]. The minimal set of axioms generally used is the following:

(a)

To each event A there corresponds a non-negative number, its probability,

(2.2.2)

(b)

The event E has unit probability,

(2.2.3)

(c)

If A and B are mutually exclusive events, then the probability of A or B (written A + B) is

(2.2.4)

From these axioms¹ one obtains immediately the following useful results. From (b) and (c):

(2.2.5)

and furthermore with (a):

(2.2.6)

From (c) one can easily obtain the more general theorem for mutually exclusive events A, B, C, …,

(2.2.7)

It should be noted that summing the probabilities of events combined with “or” here refers only to mutually exclusive events. If one must deal with events that are not of this type, then they must first be decomposed into mutually exclusive ones. In throwing a die, A may signify even, B odd, C less than 4 dots, D 4 or more dots. Suppose one is interested in the probability for the event A or C, which are obviously not exclusive. One forms A and C (written AC) as well as AD, BC, and BD, which are mutually exclusive, and finds for A or C (sometimes written $$A \dot{+} C$$) the expression $$AC + AD + BC$$. Note that the axioms do not prescribe a method for assigning the value of a particular probability P(A).

Finally it should be pointed out that the word probability is often used in common language in a sense that is different or even opposed to that considered by us. This is subjective probability, where the probability of an event is given by the measure of our belief in its occurrence. An example of this is: “The probability that the party A will win the next election is 1∕3.” As another example consider the case of a certain track in nuclear emulsion which could have been left by a proton or pion. One often says: “The track was caused by a pion with probability 1∕2.” But since the event had already taken place and only one of the two kinds of particle could have caused that particular track, the probability in question is either 0 or 1, but we do not know which.

2.3 Rules of Probability Calculus: Conditional Probability

Suppose the result of an experiment has the property A. We now ask for the probability that it also has the property B, i.e., the probability of B under the condition A. We define this conditional probability as

(2.3.1)

It follows that

(2.3.2)

One can also use (2.3.2) directly for the definition, since here the requirement P(A) ≠ 0 is not necessary. From Fig. 2.1 it can be seen that this definition is reasonable. Consider the event A to occur if a point is in the region labeled A, and correspondingly for the event (and region) B. For the overlap region both A and B occur, i.e., the event (AB) occurs. Let the area of the different regions be proportional to the probabilities of the corresponding events. Then the probability of B under the condition A is the ratio of the area AB to that of A. In particular this is equal to unity if A is contained in B and zero if the overlapping area vanishes.

Using conditional probability we can now formulate the rule of total probability . Consider an experiment that can lead to one of n possible mutually exclusive events,

(2.3.3)

The probability for the occurrence of any event with the property B is

(2.3.4)

as can be seen easily from (2.3.2) and (2.2.7).

Fig. 2.1:

Illustration of conditional probability.

We can now also define the independence of events. Two events A and B are said to be independent if the knowledge that A has occurred does not change the probability for B and vice versa, i.e., if

(2.3.5)

or, by use of (2.3.2),

(2.3.6)

In general several decompositions of the type (2.3.3),

(2.3.7)

are said to be independent, if for all possible combinations α, β, …, ω the condition

(2.3.8)

is fulfilled.

2.4 Examples

2.4.1 Probability for n Dots in the Throwing of Two Dice

If n ₁ and n ₂ are the number of dots on the individual dice and if $$n = n_{1} + n_{2}$$, then one has $$P(n_{i}) = 1/6$$; i = 1, 2; n i  = 1, 2, …, 6. Because the two dice are independent of each other one has $$P(n_{1},n_{2}) = P(n_{1})P(n_{2}) = 1/36$$. By considering in how many different ways the sum $$n = n_{i} + n_{j}$$ can be formed one obtains

Of course, the normalization condition $$\sum _{k=2}^{12}P_{2}(k) = 1$$ is fulfilled.

2.4.2 Lottery 6 Out of 49

A container holds 49 balls numbered 1 through 49. During the drawing 6 balls are taken out of the container consecutively and none are put back in. We compute the probabilities P(1), P(2), …, P(6) that a player, who before the drawing has chosen six of the numbers 1, 2, …, 49, has predicted exactly 1, 2, …, or 6 of the drawn numbers.

First we compute P(6). The probability to choose as the first number the one which will also be drawn first is obviously 1∕49. If that step was successful, then the probability to choose as the second number the one which is also drawn second is 1∕48. We conclude that the probability for choosing six numbers correctly in the order in which they are drawn is

The order, however, is irrelevant. Since there are 6! possible ways to arrange six numbers in different orders we have

That is exactly the inverse of the number of combinations $$C_{6}^{49}$$ of 6 elements out of 49 (see Appendix B), since all of these combinations are equally probable but only one of them contains only the drawn numbers.

We may now argue that the container holds two kinds of balls, namely 6 balls in which the player is interested since they carry the numbers which he selected, and 43 balls whose numbers the player did not select. The result of the drawing is a sample from a set of 49 elements of which 6 are of one kind and 43 are of the other. The sample itself contains 6 elements which are drawn without putting elements back into the container. This method of sampling is described by the hypergeometric distribution (see Sect. 5.​3). The probability for predicting correctly ℓ out of the 6 drawn numbers is

2.4.3 Three-Door Game

In a TV game show a candidate is given the following problem. Three rooms are closed by three identical doors. One room contains a luxury car, the other two each contain a goat. The candidate is asked to guess behind which of the doors the car is. He chooses a door which we will call A. The door A, however, remains closed for the moment. Of course, behind at least one of the other doors there is a goat. The quiz master now opens one door which we will call B to reveal a goat. He now gives the candidate the chance to either stay with the original choice A or to choose remaining closed door C. Can the candidate increase his or her chances by choosing C instead of A?

The answer (astonishing for many) is yes. The probability to find the car behind the door A obviously is $$P(A) = 1/3$$. Then the probability that the car is behind one of the other doors is $$P(\bar{A}) = 2/3$$. The candidate exhausts this probability fully if he chooses the door C since through the opening of B it is shown to be a door without the car, so that $$P(C) = P(\bar{A})$$.

Literature

Literature Cited in the Text

[1]

A. Kolmogorov, Ergebn. Math. 2 (1933) 3

Footnotes

1

Sometimes the definition (2.3.1) is included as a fourth axiom.

© Springer International Publishing Switzerland 2014

Siegmund BrandtData Analysishttps://doi.org/10.1007/978-3-319-03762-2_3

3. Random Variables: Distributions

Siegmund Brandt1 

(1)

Department of Physics, University of Siegen, Siegen, Germany

Keywords

Covariance

3.1 Random Variables

We will now consider not the probability of observing particular events but rather the events themselves and try to find a particularly simple way of classifying them. We can, for instance, associate the event “heads” with the number 0 and the event “tails” with the number 1. Generally we can classify the events of the decomposition (2.​3.​3) by associating each event A i with the real number i. In this way each event can be characterized by one of the possible values of a random variable . Random variables can be discrete or continuous . We denote them by symbols like x, y, ….

Example 3.1:

Discrete random variable

It may be of interest to study the number of coins still in circulation as a function of their age. It is obviously most convenient to use the year of issue stamped on each coin directly as the (discrete) random variable, e.g., x = …, 1949, 1950, 1951, …. ■ 

Example 3.2:

Continuous random variable

All processes of measurement or production are subject to smaller or larger imperfections or fluctuations that lead to variations in the result, which is therefore described by one or several random variables. Thus the values of electrical resistance and maximum heat dissipation characterizing a resistor in Example 2.1 are continuous random variables. ■ 

3.2 Distributions of a Single Random Variable

From the classification of events we return to probability considerations. We consider the random variable x and a real number x, which can assume any value between −∞ and + ∞, and study the probability for the event x < x. This probability is a function of x and is called the (cumulative) distribution function of x:

(3.2.1)

If x can assume only a finite number of discrete values, e.g., the number of dots on the faces of a die, then the distribution function is a step function. It is shown in Fig. 3.1 for the example mentioned above. Obviously distribution functions are always monotonic and non-decreasing.

Fig. 3.1:

Distribution function for throwing of a symmetric die.

Because of (2.​2.​3) one has the limiting case

(3.2.2)

Applying Eqs. (2.​2.​5)–(3.2.1) we obtain

(3.2.3)

and therefore

(3.2.4)

Of special interest are distribution functions F(x) that are continuous and differentiable. The first derivative

(3.2.5)

is called the probability density (function) of x. It is a measure of the probability of the event (x ≤ x < x + dx). From (3.2.1) and (3.2.5) it immediately follows that

(3.2.6)

(3.2.7)

and in particular

(3.2.8)

A trivial example of a continuous distribution is given by the angular position of the hand of a watch read at random intervals. We obtain a constant probability density (Fig. 3.2).

Fig. 3.2:

Distribution function and probability density for the angular position of a watch hand.

3.3 Functions of a Single Random Variable, Expectation Value, Variance, Moments

In addition to the distribution of a random variable x, we are often interested in the distribution of a function of x. Such a function of a random variable is also a random variable:

(3.3.1)

The variable y then possesses a distribution function and probability density in the same way as x.

In the two simple examples of the last section we were able to give the distribution function immediately because of the symmetric nature of the problems. Usually this is not possible. Instead, we have to obtain it from experiment. Often we are limited to determining a few characteristic parameters instead of the complete distribution.

The mean or expectation value of a random variable is the sum of all possible values x i of x multiplied by their corresponding probabilities

(3.3.2)

Note that $$\widehat{x}$$ is not a random variable but rather has a fixed value. Correspondingly the expectation value of a function (3.3.1) is defined to be

(3.3.3)

In the case of a continuous random variable (with a differentiable distribution function), we define by analogy

(3.3.4)

and

(3.3.5)

If we choose in particular

(3.3.6)

we obtain the expectation values

(3.3.7)

which are called the ℓ−th moments of the variable about the point c. Of special interest are the moments about the mean ,

(3.3.8)

The lowest moments are obviously

(3.3.9)

The quantity

(3.3.10)

is the lowest moment containing information about the average deviation of the variable x from its mean. It is called the variance of x.

We will now try to visualize the practical meaning of the expectation value and variance of a random variable x. Let us consider the measurement of some quantity, for example, the length x ₀ of a small crystal using a microscope. Because of the influence of different factors, such as the imperfections of the different components of the microscope and observational errors, repetitions of the measurement will yield slightly different results for x. The individual measurements will, however, tend to group themselves in the neighborhood of the true value of the length to be measured, i.e., it will be more probable to find a value of x near to x ₀ than far from it, providing no systematic biases exist. The probability density of x will therefore have a bell-shaped form as sketched in Fig. 3.3, although it need not be symmetric. It seems reasonable – especially in the case of a symmetric probability density – to interpret the expectation value (3.3.4) as the best estimate of the true value. It is interesting to note that (3.3.4) has the mathematical form of a center of gravity, i.e., $$\widehat{x}$$ can be visualized as the x-coordinate of the center of gravity of the surface under the curve describing the probability density.

The variance (3.3.10),

(3.3.11)

Fig. 3.3:

Distribution with small variance (a) and large variance (b).

which has the form of a moment of inertia, is a measure of the width or dispersion of the probability density about the mean. If it is small, the individual measurements lie close to $$\widehat{x}$$ (Fig. 3.3a); if it is large, they will in general be further from the mean (Fig. 3.3b). The positive square root of the variance

(3.3.12)

is called the standard deviation (or sometimes the dispersion ) of x. Like the variance itself it is a measure of the average deviation of the measurements x from the expectation value.

Since the standard deviation has the same dimension as x (in our example both have the dimension of length), it is identified with the error of the measurement,

This definition of measurement error is discussed in more detail in Sects. 5.​6 – 5.10. It should be noted that the definitions (3.3.4) and (3.3.10) do not provide completely a way of calculating the mean or the measurement error, since the probability density describing a measurement is in general unknown.

The third moment about the mean is sometimes called skewness . We prefer to define the dimensionless quantity

(3.3.13)

to be the skewness of x. It is positive (negative) if the distribution is skew to the right (left) of the mean. For symmetric distributions the skewness vanishes. It contains information about a possible difference between positive and negative deviation from the mean.

We will now obtain a few important rules about means and variances. In the case where

(3.3.14)

it follows immediately that

(3.3.15)

and therefore

(3.3.16)

We now consider the function

(3.3.17)

It has the expectation value

(3.3.18)

and variance

(3.3.19)

The function u – which is also a random variable – has particularly simple properties, which makes its use in more involved calculations preferable. We will call such a variable (having zero mean and unit variance) a reduced variable . It is also called a standardized, normalized, or dimensionless variable.

Although a distribution is mathematically most easily described by its expectation value, variance, and higher moments (in fact any distribution can be completely specified by these quantities, cf. Sect. 5.​5), it is often convenient to introduce further definitions so as to better visualize the form of a distribution.

The mode x m (or most probable value) of a distribution is defined as that value of the random variable that corresponds to the highest probability:

(3.3.20)

If the distribution has a differentiable probability density, the mode, which corresponds to its maximum, is easily determined by the conditions

(3.3.21)

In many cases only one maximum exists; the distribution is said to be unimodal . The median x 0. 5 of a distribution is defined as that value of the random variable for which the distribution function equals 1∕2:

(3.3.22)

In the case of a continuous probability density Eq. (3.3.22) takes the form

(3.3.23)

i.e., the median divides the total range of the random variable into two regions each containing equal probability.

It is clear from these definitions that in the case of a unimodal distribution with continuous probability density that is symmetric about its maximum, the values of mean, mode, and median coincide. This is not, however, the case for asymmetric distributions (Fig. 3.4).

Fig. 3.4:

Most probable value (mode) x m, mean $$\widehat{x}$$, and median x 0. 5 of an asymmetric distribution.

The definition (3.3.22) can easily be generalized. The quantities x 0. 25 and x 0. 75 defined by

(3.3.24)

are called lower and upper quartiles . Similarly we can define deciles x 0. 1, x 0. 2, …, x 0. 9, or in general quantiles x q , by

(3.3.25)

with 0 ≤ q ≤ 1.

The definition of quantiles is most easily visualized from Fig. 3.5. In a plot of the distribution function F(x), the quantile x q can be read off as the abscissa corresponding to the value q on the ordinate. The quantile x q (q), regarded as a function of the probability q, is simply the inverse of the distribution function.

Fig. 3.5:

Median and quantiles of a continuous distribution.

Example 3.3:

Uniform distribution

We will now discuss the simplest case of a distribution function of a continuous variable. Suppose that in the interval a ≤ x < b the probability density of x is constant, and it is zero outside of this interval:

(3.3.26)

Because of (3.2.8) one has

or

(3.3.27)

The distribution function is

(3.3.28)

By symmetry arguments the expectation value of x must be the arithmetic mean of the boundaries a and b. In fact, (3.3.4) immediately gives

(3.3.29)

Correspondingly, one obtains from (3.3.10)

(3.3.30)

The uniform distribution is not of great practical interest. It is, however, particularly easy to handle, being the simplest distribution of a continuous variable. It is often advantageous to transform a distribution function by means of a transformation of variables into a uniform distribution or the reverse, to express the given distribution in terms of a uniform distribution. This method is used particularly in the “Monte Carlo method” discussed in Chap. 4.​ ■ 

Example 3.4:

Cauchy distribution

In the (x, y) plane a gun is mounted at the point $$(x,y) = (0,-1)$$ such that its barrel lies in the (x, y) plane and can rotate around an axis parallel to the z axis (Fig. 3.6).

Fig. 3.6:

Model for producing a Cauchy distribution (below) and probability density of the Cauchy distribution (above).

The gun is fired such that the angle θ between the barrel and the y axis is chosen at random from uniform distribution in the range $$-\pi /2 \leq \theta <\pi /2$$, i.e., the probability density of θ is

Since

we find by the transformation (cf. Sect. 3.7) θ → x of the variable for the probability density in x

(3.3.31)

A distribution with this probability density (in our example of the position of hits on the x axis) is called the Cauchy distribution.

The expectation value of x is (taking the principal value for the integral)

The expression for the variance,

yields an infinite result. One says that the variance of the Cauchy distribution does not exist.

One can, however, construct another measure for the width of the distribution, the full width at half maximum ‘FWHM’ (cf. Example 6.3). The function g(x) has its maximum at $$x =\widehat{ x} = 0$$ and reaches half its maximum value at the points $$x_{a} = -1$$ and x ℓ  = 1. Therefore,

is the full width at half maximum of the Cauchy distribution. ■ 

Example 3.5:

Lorentz (Breit–Wigner) distribution

With $$\widehat{x} = a = 0$$ and $$\Gamma = 2$$ we can write the probability density (3.3.31) of the Cauchy distribution in the form

(3.3.32)

This function is a normalized probability density for all values of a and full width at half maximum $$\Gamma > 0$$. It is called the probability density of the Lorentz or also Breit–Wigner distribution and plays an important role in the physics of resonance phenomena. ■ 

3.4 Distribution Function and Probability Density of Two Variables: Conditional Probability

We now consider two random variables x and y and ask for the probability that both x < x and y < y. As in the case of a single variable we expect there to exist of a distribution function (see Fig. 3.7)

(3.4.1)

Fig. 3.7:

Distribution function of two variables.

We will not enter here into axiomatic details and into the conditions for the existence of F, since these are always fulfilled in cases of practical interest. If F is a differentiable function of x and y, then the joint probability density of x and y is

(3.4.2)

One then has

(3.4.3)

Often we are faced with the following experimental problem. One determines approximately with many measurements the joint distribution function F(x, y). One wishes to find the probability for x without consideration of y. (For example, the probability density for the appearance of a certain infectious disease might be given as a function of date and geographic location. For some investigations the dependence on the time of year might be of no interest.)

We integrate Eq. (3.4.3) over the whole range of y and obtain

where

(3.4.4)

is the probability density for x. It is called the marginal probability density of x. The corresponding distribution for y is

(3.4.5)

In analogy to the independence of events [Eq. (2.​3.​6)] we can now define the independence of random variables . The variables x and y are said to be independent if

(3.4.6)

Using the marginal distributions we can also define conditional probability for y under the condition that x is known,

(3.4.7)

We define the conditional probability density as

(3.4.8)

so that the probability of Eq. (3.4.7) is given by

The rule of total probability can now also be expressed for distributions:

(3.4.9)

In the case of independent variables as defined by Eq. (3.4.6) one obtains directly from Eq. (3.4.8)

(3.4.10)

This was expected since, in the case of independent variables, any constraint on one variable cannot contribute information about the probability distribution of the other.

3.5 Expectation Values, Variance, Covariance, and Correlation

In analogy to Eq. (3.3.5) we define the expectation value of a function H(x,y) to be

(3.5.1)

Similarly, the variance of H(x,y) is defined to be

(3.5.2)

For the simple case $$H(\mathsf{x,y}) = a\mathsf{x} + b\mathsf{y}$$, Eq. (3.5.1) clearly gives

(3.5.3)

We now choose

(3.5.4)

The expectation values of such functions are the ℓ mth moments of x,y about the origin,

(3.5.5)

If we choose more generally

(3.5.6)

the expectation values

(3.5.7)

are the ℓ m-th moments about the point a, b. Of special interest are the moments about the point λ ₁₀, λ ₀₁,

(3.5.8)

As in the case of a single variable, the lower moments have a special significance, in particular,

(3.5.9)

We can now express the variance of a x + b y in terms of these quantities:

(3.5.10)

In deriving (3.5.10) we have made use of (3.3.14). As another example we consider

(3.5.11)

In this case we have to assume the independence of x and y in the sense of (3.4.6) in order to obtain the expectation value. Then according to (3.5.1) one has

(3.5.12)

or

(3.5.13)

While the quantities E(x), E(y), σ ²(x), σ ²(y) are very similar to those obtained in the case of a single variable, we still have to explain the meaning of $$\mathop{\mathrm{cov}}\nolimits (\mathsf{x,y})$$. The concept of covariance is of considerable importance for the understanding of many of our subsequent problems. From its definition we see that $$\mathop{\mathrm{cov}}\nolimits (\mathsf{x,y})$$ is positive if values $$\mathsf{x} >\widehat{ x}$$ appear preferentially together with values $$\mathsf{y} >\widehat{ y}$$. On the other hand, $$\mathop{\mathrm{cov}}\nolimits (\mathsf{x,y})$$ is negative if in general $$\mathsf{x} >\widehat{ x}$$ implies $$\mathsf{y} <\widehat{ y}$$. If, finally, the knowledge of the value of x does not give us additional information about the probable position of y, the covariance vanishes. These cases are illustrated in Fig. 3.8.

It is often convenient to use the correlation coefficient

(3.5.14)

rather than the covariance.

Both the covariance and the correlation coefficient offer a (necessarily crude) measure of the mutual dependence of x and y. To investigate this further we now consider two reduced variables u and v in the sense of Eq. (3.3.17) and determine the variance of their sum by using (3.5.9),

(3.5.15)

Fig. 3.8:

Illustration of the covariance between the variables x and y. (a) $$\mathop{\mathrm{cov}}\nolimits (\mathsf{x,y}) > 0$$; (b) $$\mathop{\mathrm{cov}}\nolimits (\mathsf{x,y}) \approx 0$$; (c) $$\mathop{\mathrm{cov}}\nolimits (\mathsf{x,y}) < 0$$.

From Eq. (3.3.19) we know that $${\sigma }^{2}(\mathsf{u}) {=\sigma }^{2}(\mathsf{v}) = 1$$. Therefore we have

(3.5.16)

and correspondingly

(3.5.17)

Since the variance always fulfills

(3.5.18)

it follows that

(3.5.19)

If one now returns to the original variables x,y, then it is easy to show that

(3.5.20)

Thus we have finally shown that

(3.5.21)

We now investigate the limiting cases ± 1. For ρ(u,v) = 1 the variance is $$\sigma (\mathsf{u - v}) = 0$$, i.e., the random variable (u − v) is a constant. Expressed in terms of x,y one has therefore

(3.5.22)

The equation is always fulfilled if

(3.5.23)

where b is positive. Therefore in the case of a linear dependence (b positive) between x and y the correlation coefficient takes the value $$\rho (\mathsf{x,y}) = +1$$. Correspondingly one finds $$\rho (\mathsf{x,y}) = -1$$ for a negative linear dependence (b negative). We would expect the covariance to vanish for two independent variables x and y, i.e., for which the probability density obeys Eq. (3.4.6). Indeed with (3.5.9) and (3.5.1) we find

3.6 More than Two Variables: Vector and Matrix Notation

In analogy to (3.4.1) we now define a distribution function of n variables x ₁, x ₂, …, x n :

(3.6.1)

If the function F is differentiable with respect to the x i , then the joint probability density is given by

(3.6.2)

The probability density of one of the variables x r , the marginal probability density , is given by

(3.6.3)

If H(x ₁, x ₂, …, x n ) is a function of n variables, then the expectation value of H is

(3.6.4)

With H(x) = x r one obtains

(3.6.5)

The variables are independent if

(3.6.6)

In analogy to Eq. (3.6.3) one can define the joint marginal probability density of ℓ out of the n variables,¹by integrating (3.6.3) over only the n − ℓ remaining variables,

(3.6.7)

These ℓ variables are independent if

(3.6.8)

The moments of order ℓ ₁,ℓ ₂,…,ℓ nabout the origin are the expectation values of the functions

that is,

In particular one has

(3.6.9)

The moments about $$(\widehat{x}_{1},\widehat{x}_{2},\ldots,\widehat{x}_{n})$$ are correspondingly

(3.6.10)

The variances of the x i are then

(3.6.11)

The moment with $$\ell_{i} =\ell _{j} = 1,\:\ell_{k} = 0\:(i\neq k\neq j)$$ is called the covariance between the variables x i and x j ,

(3.6.12)

It proves useful to represent the n variables x ₁, x ₂, …, x n as components of a vector $$\mathsf{\mathbf{x}}$$ in an n-dimensional space. We can then write the distribution function (3.6.1) as

(3.6.13)

Correspondingly, the probability density (3.6.2) is then

(3.6.14)

The expectation value of a function $$H(\mathsf{\mathbf{x}})$$ is then simply

(3.6.15)

We would now like to express the variances and covariances by means of a matrix.² This is called the covariance matrix

(3.6.16)

The elements c ij are given by (3.6.12); the diagonal elements are the variances c ii  = σ ²(x i ). The covariance matrix is clearly symmetric, since

(3.6.17)

If we now also write the expectation values of the x i as a vector,

(3.6.18)

we see that each element of the covariance matrix

is given by the expectation value of the product of the row vector $${(\mathsf{\mathbf{x}} -\widehat{\mathbf{x}})}^{\mathrm{T}}$$ and the column vector $$(\mathsf{\mathbf{x}} -\widehat{\mathbf{x}})$$, where

The covariance matrix can therefore be written simply as

(3.6.19)

3.7 Transformation of Variables

As already mentioned in Sect. 3.3, a function of a random variable is itself a random variable, e.g.,

We now ask for the probability density g(y) for the case where the probability density f(x) is known.

Clearly the probability

that y falls into a small interval dy must be equal to the probability f(x) dx that x falls into the “corresponding interval” dx, f(x) dx = g(y) dy. This is illustrated in Fig. 3.9. The intervals dx and dy are related by

Fig. 3.9:

Transformation of variables for a probability density of x to y.

The absolute value ensures that we consider the values dx, dy as intervals without a given direction. Only in this way are the probabilities f(x) dx and g(x) dy always positive. The probability density is then given by

(3.7.1)

We see immediately that g(y) is defined only in the case of a single-valued function y(x) since only then is the derivative in (3.7.1) uniquely defined. For functions where this is not the case, e.g., $$y = \sqrt{x}$$, one must consider the individual single-valued parts separately, i.e., $$y = +\sqrt{x}$$. Equation (3.7.1) also guarantees that the probability distribution of y is normalized to unity:

In the case of two independent variables x, y the transformation to the new variables

(3.7.2)

can be illustrated in a similar way. One must find the quantity J that relates the probabilities f(x, y) and g(u, v):

(3.7.3)

Figure 3.10 shows in the (x, y) plane two lines each for u = const and v = const. They bound the surface element dA of the transformed variables u, v corresponding to the element dx dy of the original variables.

Fig. 3.10:

Variable transformation from x, y to u, v.

These curves of course need not be straight lines. Since, however, dA is an “infinitesimal” surface element, it can be treated as a parallelogram, whose area we will now compute. The coordinates of the corner points a, b, c are

We can expand the last two lines in a series and obtain

The area of the parallelogram is equal to the absolute value of the determinant

(3.7.4)

where the sign is of no consequence because of the absolute value in Eq. (3.7.3). The expression

(3.7.5)

is called the Jacobian (determinant) of the transformation (3.7.2).

For the general case of n variables $$\mathsf{\mathbf{x}} = (\mathsf{x}_{1},\mathsf{x}_{2},\ldots,\mathsf{x}_{n})$$ and the transformation

(3.7.6)

the probability density of the transformed variables is given by

(3.7.7)

where the Jacobian is

(3.7.8)

A requirement for the existence of $$g(\mathbf{y})$$ is of course again the uniqueness of all derivatives occurring in J.

3.8 Linear and Orthogonal Transformations: Error Propagation

In practice we deal frequently with linear transformations of variables. The main reason is that they are particularly easy to handle, and we try therefore to approximate other transformations by linear ones using Taylor series techniques.

Consider r linear functions of the n variables $$\mathsf{\mathbf{x}} = (\mathsf{x}_{1},\mathsf{x}_{2},\ldots,\mathsf{x}_{n})$$:

(3.8.1)

or in matrix notation,

(3.8.2)

The expectation value of $$\mathsf{\mathbf{y}}$$ follows from the generalization of (3.5.3)

(3.8.3)

Together with (3.6.19) one obtains the covariance matrix for $$\mathsf{\mathbf{y}}$$,

(3.8.4)

Equation (3.8.4) expresses the well-known law of error propagation . Suppose the expectation values $$\widehat{x}_{i}$$ have been measured. Suppose as well that the errors (i.e., the standard deviations or variances) and the covariances of $$\mathsf{\mathbf{x}}$$ are known. One would like to know the errors of an arbitrary function $$\mathsf{\mathbf{y}}(\mathsf{\mathbf{x}})$$. If the errors are relatively small, then the probability density is only significantly large in a small region (on the order of the standard deviation) around the point $$\widehat{\mathbf{x}}$$. One then performs a Taylor expansion of the functions,

or in matrix notation,

(3.8.5)

with

(3.8.6)

If one neglects the higher-order terms and substitutes the first partial derivatives of the matrix T into Eq. (3.8.4), then one obtains the law of error propagation. We see in particular that not only the errors (i.e., the variances) of $$\mathsf{\mathbf{x}}$$ but also the covariances make a significant contribution to the errors of $$\mathsf{\mathbf{y}}$$, that is, to the diagonal elements of C y . If these are not taken into account in the error propagation, then the result cannot be trusted.

The covariances can only be neglected when they vanish anyway, i.e., in the case of independent original variables $$\mathsf{\mathbf{x}}$$. In this case C x simplifies to a diagonal matrix. The diagonal elements of C y then have the simple form

(3.8.7)

If we now call the standard deviation, i.e., the positive square root of the variance, the error of the corresponding quantity and use for this the symbol Δ, Eq. (3.8.7) leads immediately to the formula

(3.8.8)

known commonly as the law of the propagation of errors. It must be emphasized that this expression is incorrect in cases of non-vanishing covariances. This is illustrated in the following example.

Example 3.6:

Error propagation and covariance

In a Cartesian coordinate system a point (x, y) is measured. The measurement is performed with a coordinate measuring device whose error in y is three times larger than that in x. The measurements of x and y are independent. We therefore can take the covariance matrix to be (up to a factor common to all elements)

We now evaluate the errors (i.e., the covariance matrix) in polar coordinates

The transformation matrix (3.8.6) is

To simplify the numerical calculations we consider only the point (1, 1). Then

and therefore

We can now return to the original Cartesian coordinate system

by use of the transformation

As expected we obtain

If instead we had used the formula (3.8.8), i.e., if we had neglected the covariances in the transformation of $$r,\varphi$$ to x, y, then we would have obtained

which is different from the original covariance matrix. This example stresses the importance of covariances, since it is obviously not possible to change errors of measurements by simply transforming back and forth between coordinate systems. ■ 

Finally we discuss a special type of linear transformation. We consider the case of exactly n functions y of the n variables x. In particular we take $$\mathbf{a} = 0$$ in (3.8.2). One then has

(3.8.9)

where R is a square matrix. We now require that the transformation (3.8.9) leaves the modulus of a vector invariant

(3.8.10)

Using Eq. (A.1.9) we can write

This means

or written in terms of components,

(3.8.11)

A transformation of the type (3.8.9) that fulfills condition (3.8.11) is said to be orthogonal . We now consider the determinant of the transformation matrix

and form its square. According to rules for computing determinants we obtain from Eq. (3.8.11)

i.e., D = ±1. The determinant D, however, is the Jacobian determinant of the transformation (3.8.9),

(3.8.12)

We multiply the system of equations (3.8.9) on the left with R T and obtain

Because of Eq. (3.8.11) this expression reduces to

(3.8.13)

The inverse transformation of an orthogonal transformation is described simply by the transposed transformation matrix. It is itself orthogonal.

An important property of any linear transformation of the type

is the following. By constructing additional functions y ₂, y ₃, …, y n of equivalent form it can be extended to yield an orthogonal transformation as long as the condition

is fulfilled.

Footnotes

1

Without loss of generality we can take these to be the variables x ₁, x ₂, …, x ℓ .

2

For details on matrix notation see Appendix A.

© Springer International Publishing Switzerland 2014

Siegmund BrandtData Analysishttps://doi.org/10.1007/978-3-319-03762-2_4

4. Computer Generated Random Numbers: The Monte Carlo Method

Siegmund Brandt1 

(1)

Department of Physics, University of Siegen, Siegen, Germany

Keywords

Covariance

4.1 Random Numbers

Up to now in this book we have considered the observation of random variables, but not prescriptions for producing them. It is in many applications useful, however, to have a sequence of values of a randomly distributed variable x. Since operations must often be carried out with a large number of such random numbers , it is particularly convenient to have them directly available on a computer. The correct procedure to create such random numbers would be to use a statistical process, e.g., the measurement of the time between two decays from a radioactive source, and to transfer the measured results into the computer. In practical applications, however, the random numbers are almost always calculated directly by the computer. Since this works in a strictly deterministic way, the resulting values are not really random, but rather can be exactly predicted. They are therefore called pseudorandom.

Computations with random numbers currently make up a large part of all computer calculations in the planning and evaluation of experiments. The statistical behavior which stems either from the nature of the experiment or from the presence of measurement errors can be simulated on the computer. The use of random numbers in computer programs is often called the Monte Carlo method .

We begin this chapter with a discussion of the representation of numbers in a computer (Sect. 4.2), which is indispensable for an understanding of what follows. The best studied method for the creation of uniformly distributed random numbers is the subject of Sects. 4.3–4.7. Sections 4.8 and 4.9 cover the creation of random numbers that follow an arbitrary distribution and the especially common case of normally distributed numbers. In the last two sections one finds discussion and examples of the Monte Carlo method in applications of numerical integration and simulation.

In many examples and exercises we will simulate measurements with the Monte Carlo method and then analyze them. We possess in this way a computer laboratory, which allows us to study individually the influence of simulated measurement errors on the results of an analysis.

4.2 Representation of Numbers in a Computer

For most applications the representation of numbers used in a computation is unimportant. It can be of decisive significance, however, for the properties of computer-generated random numbers. We will restrict ourselves to the binary representation, which is used today in practically all computers. The elementary unit of information is the bit ,¹ which can assume the values of 0 or 1. This is realized physically by two distinguishably different electric or magnetic states of a component in the computer.

If one has k bits available for the representation of an integer, then 1 bit is sufficient to encode the sign. The remaining k − 1 bits are used for the binary representation of the absolute value in the form

(4.2.1)

Here each of the coefficients a (j) can assume only the values 0 or 1, and thus can be represented by a single bit.

The binary representation for non-negative integers is

One could simply use the first bit to encode the sign and represent the corresponding negative numbers such that in the first bit the 0 is replaced by a 1. That would give, however, two different representations for the number zero, or rather + 0 and − 0. In fact, one uses for negative numbers the “complementary representation”

Then using k bits, integers in the interval

(4.2.2)

can be represented.

In most computers 8 bit are grouped together into one byte . Four bytes are generally used for the representation of integers, i.e., k = 32, $${2}^{k-1} - 1 = 2\,147\,483\,647$$. In many small computers only two bytes are available, k = 16, $${2}^{k-1} - 1 = 32\,767$$. This constraint (4.2.2) must be taken into consideration when designing a program to generate random numbers.

Before turning to the representation of fractional numbers in a computer, let us consider a finite decimal fraction, which we can write in various ways, e.g.,

or in general

The quantities M and e are called the mantissa and exponent, respectively. One chooses the exponent such that the mantissa’s nonzero digits are all to the right of the decimal point, and the first place after the decimal point is not zero. If one has available n decimal places for the representation of the value M, then

is an integer. In our example, n = 4 and m = 1723. In this way the decimal fraction d is represented by the two integers m and e.

The representation of fractions in the binary system is done in a completely analogous way. One decomposes a number of the form

(4.2.3)

into a mantissa M and exponent e. If n m  bits are available for the representation of the mantissa (including sign), it can be expressed by the integer

(4.2.4)

If the exponent with its sign is represented by n e  bits, then it can cover the interval

(4.2.5)

In our Java classes we use floating-point numbers of the type double with 64 bit, n m  = 53 for the mantissa and n e  = 11 for the exponent.

For the interval of values in which a floating point number can be represented in a computer, the constraint (4.2.2) no longer applies but one has rather the weaker condition

(4.2.6)

Here e min and e max are given by (4.2.5). If 11 bit are available for representing the exponent (including sign), then one has $$e_{\mathrm{max}} = {2}^{10} - 1 = 1023$$. Therefore, one has the constraint $$\vert x\vert < {2}^{1023} \approx 1{0}^{308}$$.

When computing with floating point numbers, the concept of the relative precision of the representation is of considerable significance. There are a fixed number of binary digits corresponding to a fixed number of decimal places available for the representation of the mantissa M. If we designate by α the smallest possible mantissa, then two numbers x ₁ and x ₂ can still be represented as being distinct if

The absolute precision in the representation of x is thus

which depends on the exponent of x. The relative precision

is in contrast independent of x. If n binary digits are available for the representation of the mantissa, then one has M ≈ 2 n , since the exponent is chosen such that all n places for the mantissa are completely used. The smallest possible mantissa is α = 2⁰, so that the relative precision in the representation of x is

(4.2.7)

4.3 Linear Congruential Generators

Since, as mentioned, computers work in a strictly deterministic way, all (pseudo)"-random numbers generated in a computer are in the most general case a function of all of the preceding (pseudo)random numbers²

(4.3.1)

Programs for creating random numbers are called random number generators .

The best studied algorithm is based on the following rule,

(4.3.2)

All of the quantities in (4.3.2) are integer valued. Generators using this rule are called linear congruential generators (LCG). The symbol mod m or modulo m in (4.3.2) means that the expression before the symbol is divided by m and only the remainder of the result is taken, e.g., 6 mod 5 = 1. Each random number made by an LCG according to the rule (4.3.2) depends only on the number immediately preceding it and on the constant a (the multiplier), on c (the increment), and on m (the modulus). When these three constants and one initial value x ₀ are given, the infinite sequence of random numbers x ₀, x ₁, … is determined.

The sequence is clearly periodic. The maximum period length is m. Only partial sequences that are short compared to the period length are useful for computations.

Theorem on the maximum period of an LCG with $$c\neq 0$$: An LCG defined by the values m, a, c, and x ₀ has the period m if and only if

(a)

c and m have no common factors;

(b)

$$b = a - 1$$ is a multiple of p for every prime number p that is a factor of m;

(c)

b is a multiple of 4 if m is a multiple of 4.

The proof of this theorem as well as the theorems of Sect. 4.4 can be found in, e.g., KNUTH [2].

A simple example is c = 3, a = 5, m = 16. One can easily compute that x ₀ = 0 results in the sequence

Since the period m can only be attained when all m possible values are actually assumed, the choice of the initial value x ₀ is unimportant.

4.4 Multiplicative Linear Congruential Generators

If one chooses c = 0 in (4.3.2), then the algorithm simplifies to

(4.4.1)

Generators based on this rule are called multiplicative linear congruential generators (MLCG) . The computation becomes somewhat shorter and thus faster. The exact value zero, however, can no longer be produced (except for the unusable sequence 0, 0, …). In addition the period becomes shorter. Before giving the theorem on the maximum period length for this case, we introduce the concept of the primitive element modulo m.

Let a be an integer having no common factors (except unity) with m. We consider all a for which $${a}^{\lambda }\bmod m = 1$$ for integer λ. The smallest value of λ for which this relation is valid is called the order of a modulo m. All values a having the same largest possible order λ(m) are called primitive elements modulo m.

Theorem on the order λ(m) of a primitive element modulo m: For every integer e and prime number p

(4.4.2)

Theorem on primitive elements modulo p e : The number a is a primitive element modulo p e if and only if

(4.4.3)

For large values of p the primitive elements must be determined with computer programs with the aid of this theorem.

Theorem on the maximum period of an MLCG: The maximum period of an MLCG defined by the quantities m, a, c = 0, x ₀ is equal to the order λ(m). This is attained if the multiplier a is a primitive element modulo m and when the initial value x ₀ and the multiplier m have no common factors (except unity).

In fact, MLC generators with c = 0 are frequently used in practice. There are two cases of practical significance in choosing the multiplier m.

(i)

m = 2 e : Here m − 1 can be the largest integer that can be represented on the computer. According to (4.4.2) the maximum attainable period length is m∕4.

(ii)

m = p: If m is a prime number, the period of m − 1 can be attained according to (4.4.2).

4.5 Quality of an MLCG: Spectral Test

When producing random numbers, the main goal is naturally not just to attain the longest possible period. This could be achieved very simply with the sequence 0, 1, 2, …, m − 1, 0, 1, …. Much more importantly, the individual elements within a period should follow each other “randomly”. First the modulus m is chosen, and then one chooses various multipliers a corresponding to (4.4.3) that guaranty a maximum period. One then constructs generators with the constants a, m, and c = 0 in the form of computer programs and checks with statistical tests the randomness of the resulting numbers. General tests, also applicable to this particular question, will be discussed in Sect. 8. The spectral test was especially developed for investigating random numbers, in particular for detecting non-random dependencies between neighboring elements in a sequence.

In a simple example we first consider the case a = 3, m = 7, c = 0, x ₀ = 1 and obtain the sequence

We now form pairs of neighboring numbers

(4.5.1)

Here n is the period, which in our example is $$n = m - 1 = 6$$. In Fig. 4.1 the number pairs (4.5.1) are represented as points in a two-dimensional Cartesian coordinate system. We note – possibly with surprise – that they form a regular lattice. The surprise is somewhat less, however, when we consider two features of the algorithm (4.3.2):

(i)

All coordinate values x j are integers. In the accessible range of values 1 ≤ x j  ≤ n there are, however, only n ² number pairs (4.5.1) for which both elements are integer. They lie on a lattice of horizontal and vertical lines. Two neighboring lines have a separation of one.

(ii)

There are only n different pairs (4.5.1), so that only a fraction of the n ² points mentioned in (i) are actually occupied.

We now go from integer numbers x j to transformed numbers

(4.5.2)

with the property

(4.5.3)

For simplicity we assume that the sequence $$x_{0},x_{1},\ldots$$ has the maximum possible period m for an MLC generator. The pairs

Fig. 4.1:

Diagram of number pairs (4.5.1) for a = 3, m = 7.

(4.5.4)

lie in a square whose side has unit length. Because the x j are integers, the spacing between the horizontal or vertical lattice lines on which the points (4.5.4) must lie is 1∕m. By far not all of these points, however, are occupied. A finite family of lines can be constructed which pass through those points that are actually occupied. We consider now the spacing of neighboring lines within a family, look for the family for which this distance is a maximum, and call this d ₂.

If the distances between neighboring lattice lines for all families are approximately equal, we can then be certain of having a maximally uniform distribution of the occupied lattice points on the unit square. Since this distance is 1∕m for a completely occupied lattice (m ² points), we obtain for a uniformly occupied lattice with m points a distance of $$d_{2} \approx {m}^{-1/2}$$. With a very nonuniform lattice one obtains the considerably larger value $$d_{2} \gg {m}^{-1/2}$$.

If one now considers not only pairs (4.5.4), but t-tuples of numbers

(4.5.5)

one sees that the corresponding points lie on families of (t − 1)-dimensional hyperplanes in a t-dimensional cube whose side has unit length. Let us investigate as before the distance between neighboring hyperplanes of a family. We determine the family with the largest spacing and designate this by d t . One expects for a uniform distribution of points (4.5.5) a distance

(4.5.6)

If the lattice is nonuniform, however, we expect

(4.5.7)

The situations (4.5.6) and (4.5.7) are shown in Fig. 4.2. Naturally one tries to achieve as uniform a lattice as possible. One should note that there is at least a distance (4.5.6) between the lattice points. The lowest decimal places of random numbers are therefore not random, but rather reflect the structure of the lattice.

Fig. 4.2:

Diagram of number pairs (4.5.4) for various small values of a and m.

Theoretical considerations give an upper limit on the smallest possible lattice spacing,

(4.5.8)

The constants c t are of order unity. They have the numerical values [2]

(4.5.9)

The spectral test can now be carried out as follows. For given values (m, a) of the modulus and multiplier of an MLCG one determines with a computer algorithm [2] the values d t (m, a) for small t, e.g., t = 2, 3, …, 6. One constructs the test quantities

(4.5.10)

and accepts the generator as usable if the S t (m, a) do not exceed a given limit. Table 4.1 gives the results of extensive investigations by L’ECUYER [3]. The moduli m are prime numbers close to the maximum integer values representable by 16 or 32 bit. The multipliers are primitive elements modulo m. They fulfill the requirement $$a < \sqrt{m}$$ (see Sect. 4.6). The prime numbers were chosen such that a does not have to be much smaller than $$\sqrt{m}$$, but the condition (m, a) in Table 4.1 S t (m, a) > 0. 65, t = 2, 3, …, 6, still applies.

Table 4.1:

Suitable moduli m and multipliers a for portable MLC generators for computers with 32-bit (16-bit) integer arithmetic.

32 bit

16 bit

m

a

m

a

2 147 483 647

39 373

32 749

162

2 147 483 563

40 014

32 363

157

2 147 483 399

40 692

32 143

160

2 147 482 811

41 546

32 119

172

2 147 482 801

42 024

31 727

146

2 147 482 739

45 742

31 657

142

4.6 Implementation and Portability of an MLCG

By implementation of an algorithm one means its realization as a computer program for a specific type of computer. If the program can be easily transferred to other computer types and gives there (essentially) the same results, then the program is said to be portable . In this section we will give a portable implementation of an MLCG, as realized by WICHMANN and HILL [4] and L’ECUYER [3].

A program that implements the rule (4.4.1) is certain to be portable if the computations are carried out exclusively with integers. If the computer has k bits for the representation of an integer, then all numbers between $$-m - 1$$ and m for m < 2k−1 are available.

We now choose a multiplier a with

(4.6.1)

and define

(4.6.2)

so that

(4.6.3)

The expression m div a defined by (4.6.2) and (4.6.3) is the integer part of the quotient m∕a. We now compute the right-hand side of (4.4.1), where we leave off the index j and note that $$[(x\mbox{ div }q)m]\bmod m = 0$$, since x div q is an integer:

(4.6.4)

Since one always has 0 < x < m, it follows that

(4.6.5)

(4.6.6)

In this way both terms in square brackets in the last line of (4.6.4) are less than m, so that the bracketed expression remains in the interval between − m and m.

In the Java class DatanRandom we have implemented the expression (4.6.4) in the following three lines, in which all variables are integer:

   k = x / Q;

   x = A * (x - k * Q) - k * R;

   if(x < 0) x = x + M;

One should note that division of two integer variables results directly in the integer part of the quotient. The first line therefore yields x div q and the last line $$ax\bmod m$$, respectively.

The method DatanRandom.mlcg yields a partial sequence of random numbers of length N. Each time the subroutine is called, an additional partial sequence is produced. The period of the entire sequence is $$m - 1 = 2\,147\,483\,562$$. The computation is carried out entirely with integer arithmetic, ensuring portability. The output values are, however, floating point valued because of the division by m, and therefore correspond to a uniform distribution between 0 and 1.

Often one would like to interrupt a computation requiring many random numbers and continue it later starting from the same place. In this case one can read out and store the last computed (integer) random number directly before the interruption, and use it later for producing the next random number. In the technical terminology one calls such a number the seed of the generator.

It is sometimes desirable to be able to produce non-overlapping partial sequences of random numbers not one after the other but rather independently. In this way one can, for example, carry out parts of larger simulation problems simultaneously on several computers. As seeds for such partial sequences one uses elements of the total sequence separated by an amount greater than the length of each partial sequence. Such seeds can be determined without having to run through the entire sequence. From (4.4.1) it follows that

(4.6.7)

L’ECUYER [3] suggests setting n = 2 d and choosing some seed x ₀. The expression $${a}^{{2}^{d} }$$ mod m can be computed by beginning with a and squaring it d times modulo m. Then one computes x n using (4.6.7) and obtains correspondingly x 2n, x 3n, ….

4.7 Combination of Several MLCGs

Since the period of an MLCG is at most m − 1, and since m is restricted to $$m < {2}^{k-1} - 1$$ where k is the number of bits available in the computer for the representation of an integer, only a relatively short period can be attained with a single MLCG. WICHMANN and HILL [4] and L’ECUYER [3] have given a procedure for combining several MLCGs, which allows for very long periods. The technique is based on the following two theorems.

Theorem on the sum of discrete random variables, one of which comes from a discrete uniform distribution: If x ₁, …, x ℓ are independent random variables that can only assume integer values, and if x ₁ follows a discrete uniform distribution, so that

then

(4.7.1)

also follows this distribution.

We first demonstrate the proof for ℓ = 2, using the abbreviations min (x ₂) = a, max(x ₂) = b. One has

For ℓ = 3 we first construct the variable $$\mathsf{x}^{\prime}_{1} = \mathsf{x}_{1} + \mathsf{x}_{2}$$, which follows a discrete uniform distribution between 0 and d − 1, and then the sum x ₁′ + x ₃, which has only two terms and therefore possesses the same property. The generalization for ℓ > 3 is obvious.

Theorem on the period of a family of generators: Consider the random variables x j, i coming from a generator j with a period p j , so that the generator gives a sequence $$\mathsf{x}_{j,0},\mathsf{x}_{j,1},\ldots,\mathsf{x}_{j,p_{j-1}}$$. We consider now ℓ generators j = 1, 2, …, ℓ and the sequence of ℓ-tuples

(4.7.2)

Its period p is the smallest common multiple of the periods $$p_{1},p_{2},\ldots,p_{\ell}$$ of the individual generators. The proof is obtained directly from the fact that p is clearly a multiple of each p j .

We now determine the maximum value of the period p. If the ℓ individual MLCGs have prime numbers m j as moduli, then their periods are $$p_{j} = m_{j} - 1$$ and are therefore even. Therefore one has

(4.7.3)

Equality results if the quantities $$(m_{j} - 1)/2$$ possess no common factors.

The first theorem of this section can now be used to construct a sequence with period given by (4.7.3). One forms first the integer quantity

(4.7.4)

The alternating sign in (4.7.4), which simplifies the construction of the modulus function, does not contradict the prescription of (4.7.1), since one could also use in place of $$x_{2},x_{4},\ldots$$, the variables $$x^{\prime}_{2} = -x_{2},\:x^{\prime}_{4} = -x_{4},\:\ldots$$. The quantity z i can take on the values

(4.7.5)

The transformation to floating point numbers

(4.7.6)

gives values in the range 0 < u i  < 1.

In the method DatanRandom.ecuy we use the techniques, assembled above, to produce uniformly distributed random numbers with a long period. We combine two MLCGs with m ₁ = 2 147 483 563, a ₁ = 40 014, m ₂ = 2 147 483 399, a ₂ = 40 692. The numbers $$(m_{1} - 1)/2$$ and $$(m_{2} - 1)/2$$ have no common factor. Therefore the period of the combined generator is, according to (4.7.3),

The absolute values of all integers occurring during the computation remain in the range ≤ 2³¹ − 85. The resulting floating point values u are in the range 0 < u < 1. One does not obtain the values 0 or 1, at least if 23 or more bits are available for the mantissa, which is almost always the case when representing floating point numbers with 32 bit. The program with the given values of $$m_{1},m_{2},a_{1},a_{2}$$ has been subjected to the spectral test and to many other tests by L’ECUYER [3], who has provided a PASCAL version. He determined that it satisfied all of the requirements of the tests.

Figure 4.3 illustrates the difference between the simple MLCG and the combined generator. For the simple MLCG one can still recognize a structure in a scatter plot of the number pairs (4.5.4), although with an expansion of the abscissa by a factor of 1000. The corresponding diagram for the combined generator appears, in contrast, to be completely without structure. For each diagram one million pairs of random numbers were generated. The plots correspond only to a narrow strip on the left-hand edge of the unit square.

In order to initialize non-overlapping partial sequences one can use two methods:

(i)

One applies the procedure discussed in connection with (4.6.7) to both MLCGs, naturally with the same value n, in order to construct pairs of seeds for each partial sequence.

(ii)

It is considerably easier to use the same seed for the first MLCG for every partial sequence. For the second MLCG one uses an arbitrary seed for the first partial sequence, the following random number from the second MLCG for the second partial sequence, etc. In this way one obtains partial sequences that can reach a length of (m ₁ − 1) without overlapping.

Fig. 4.3:

Scatter plots of number pairs (4.5.4) from (a) a MLC generator and (b) a combined generator. The methods DatanRandom.mlcg and DatanRandom.ecuy, respectively, were used in the generation.

4.8 Generation of Arbitrarily Distributed Random Numbers

4.8.1 Generation by Transformation of the Uniform Distribution

If x is a random variable following the uniform distribution,

(4.8.1)

and y is a random variable described by the probability density g(y), the transformation (3.​7.​1) simplifies to

(4.8.2)

We use the distribution function G(y), which is related to g(y) through $$\mathrm{d}G(y)/\mathrm{d}y = g(y)$$, and write (4.8.2) in the form

(4.8.3)

or after integration,

(4.8.4)

This equation has the following meaning. If a random number x is taken from a uniform distribution between 0 and 1 and the function x = G(y) is inverted,

(4.8.5)

then one obtains a random number y described by the probability density g(y). The relationship is depicted in Fig. 4.4a. The probability to obtain a random number x between x and x + dx is equal to the probability to have a value y(x) between y and y + dy.

Fig. 4.4:

Transformation from a uniformly distributed variable x to a variable y with the distribution function G(y). The variable y can be continuous (a) or discrete (b).

The relationship (4.8.4) can be also be used to produce discrete probability distributions. An example is shown in Fig. 4.4b. The random variable y can take on the values $$y_{1},y_{2},\ldots,y_{n}$$ with the probabilities $$P(y_{1}),P(y_{2}),\ldots,P(y_{n})$$. The distribution function as given by (3.​2.​1) is G(y) = P(y < y).Theconstructionofastep function x = G(y) according to this equation gives the values

(4.8.6)

which lie in the range between 0 and 1. From this one can produce random numbers according to a discrete distribution G(y) by first producing random numbers x uniformly distributed between 0 and 1. Depending on the interval j in which x falls, x j−1 < x < x j , the number y j is then produced.

Example 4.1:

Exponentially distributed random numbers

We would like to generate random numbers according to the probability density

(4.8.7)

This is the probability density describing the time t of the decay of a radioactive nucleus that exists at time t = 0 and has a mean lifetime τ. The distribution function is

(4.8.8)

According to (4.8.4) and (4.8.5) we can obtain exponentially distributed random numbers t by first generating random numbers uniformly distributed between 0 and 1 and then finding the inverse function $$\mathsf{t} = {G}^{-1}(\mathsf{x})$$, i.e.,

Since 1 −x is also uniformly distributed between 0 and 1, it is sufficient to compute

(4.8.9)

 ■ 

Example 4.2:

Generation of random numbers following a Breit–Wigner distribution

To generate random numbers y which follow a Breit–Wigner distribution  (3.​3.​32),

we proceed as discussed in Sect. 4.8.1. We form the distribution function

and perform the integration using the substitution

Thus we obtain

By inversion we obtain

(4.8.10)

If x are random numbers uniformly distributed in the interval 0 < x < 1, then y follows a Breit–Wigner distribution. ■ 

Example 4.3:

Generation of random numbers with a triangular distribution

In order to generate random numbers y following a triangular distribution as in Problem 3.2 we form the distribution function

In particular we have

Inverting x = F(y) gives

If x is uniformly distributed with 0 < x < 1, then y follows a triangular distribution. ■ 

4.8.2 Generation with the von Neumann Acceptance–Rejection Technique

The elegant technique of the previous section requires that the distribution function x = G(y) be known and that the inverse function $$y = {G}^{-1}(x)$$ exists and be known as well.

Often one only knows the probability density g(y). One can then use the VON NEUMANN acceptance–rejection technique , which we introduce with a simple example before discussing it in its general form.

Example 4.4:

Semicircle distribution with the simple acceptance–rejection method

As a simple example we generate random numbers following a semicircular probability density,

(4.8.11)

Instead of trying to find and invert the distribution function G(y), we generate pairs of random numbers $$(\mathsf{y}_{i},\mathsf{u}_{i})$$. Here y i is uniformly distributed in the interval available to y, − R ≤ y ≤ R, and u i is uniformly distributed in the range of values assumed by the function g(y), 0 ≤ u ≤ R. For each pair we test if

(4.8.12)

If this inequality is fulfilled, we reject the random number y i . The set of random numbers y i that are not rejected then follow a probability density g(y), since each was accepted with a probability proportional to g(y i ). ■ 

The technique of Example 4.4 can easily be described geometrically. To generate random numbers in the interval a ≤ y ≤ b according to the probability density g(y), one must consider in the region a ≤ y ≤ b the curve

(4.8.13)

and a constant

(4.8.14)

which is greater than or equal to the maximum value of g(y) in that region. In the (y, u) plane this constant is described by the line u = d. Pairs of random numbers $$(\mathsf{y}_{i},\mathsf{u}_{i})$$ uniformly distributed in the interval $$a \leq \mathsf{y}_{i} \leq b,\:0 \leq \mathsf{u}_{i} \leq d$$ correspond to a uniform distribution of points in the corresponding rectangle of the (y, u)-plane. If all of the points for which (4.8.12) holds are rejected, then only points under the curve u = g(y) remain. Figure 4.5 shows this situation for the Example 4.4. [It is clear that the technique also gives meaningful results if the function is not normalized to one. In Fig. 4.5 we have simply set $$g(y) = \sqrt{{R}^{2 } - {y}^{2}}$$ and R = 1.]

For the transformation technique of Sect. 4.8.1, each random number y i required only that exactly one random number x i be generated from a uniform distribution and that it be transformed according to (4.8.5). In the acceptance–rejection technique, pairs $$\mathsf{y}_{i},\mathsf{u}_{i}$$ must always be generated, and a considerable fraction of the numbers y i – depending on the value of u i according to (4.8.12) – are rejected. The probability for y i to be accepted is

(4.8.15)

Fig. 4.5:

All the pairs (y i , u i ) produced are marked as points in the (y, u)-plane. Points above the curve u = g(y) (small points) are rejected.

We can call E the efficiency of the procedure. If the interval a ≤ y ≤ b includes the entire allowed range of y, then the numerator of (4.8.15) is equal to unity, and one obtains

(4.8.16)

The numerator and denominator of (4.8.15) are simply the areas contained in the region a ≤ y ≤ b under the curves (4.8.13) and (4.8.14), respectively. One distributes points (y i , u i ) uniformly under the curve (4.8.14) and rejects the random numbers y i if the inequality (4.8.12) holds. The efficiency of the procedure is certainly higher if one uses as the upper curve not the constant (4.8.14) but rather a curve that is closer to g(y).

With this in mind the acceptance–rejection technique can be stated in its general form:

(i)

One finds a probability density s(y) that is sufficiently simple that random numbers can be generated from it using the transformation method, and a constant c such that

(4.8.17)

holds.

(ii)

One generates one random number y uniformly distributed in the interval a < y < b and a second random number u uniformly distributed in the interval 0 < u < 1.

(iii)

One rejects y

(4.8.18)

After the points (ii) and (iii) have been repeated enough times, the resulting set of accepted random numbers y follows the probability density g(y), since

If the interval a ≤ y ≤ b includes the entire range of y for both g(y) as well as for s(y), then one obtains an efficiency

(4.8.19)

Example 4.5:

Semicircle distribution with the general acceptance–rejection method

One chooses for c ⋅ s(y) the polygon

The efficiency is clearly

in comparison to

as in Example 4.4. ■ 

4.9 Generation of Normally Distributed Random Numbers

By far the most important distribution for data analysis is the normal distribution, which we will discuss in Sect. 5.​7.​ We present here a program that can produce random numbers x i following the standard normal distribution with the probability density

(4.9.1)

The corresponding distribution function F(x) can only be computed and inverted numerically (Appendix C). Therefore the simple transformation method of Sect. 4.8.1 cannot be used. The polar method by BOX and MULLER [5] described here combines in an elegant way acceptance–rejection with transformation. The algorithm consists of the following steps:

(i)

Generate two independent random numbers u ₁, u ₂ from a uniform distribution between 0 and 1. Transform $$\mathsf{v}_{1} = 2\mathsf{u}_{1} - 1,\:\mathsf{v}_{2} = 2\mathsf{u}_{2} - 1$$.

(ii)

Compute $$\mathsf{s} = \mathsf{v}_{1}^{2} + \mathsf{v}_{2}^{2}$$.

(iii)

If s ≥ 1, return to step (i).

(iv)

$$\mathsf{x}_{1} = \mathsf{v}_{1}\sqrt{-(2/\mathsf{s} )\ln \mathsf{s}}\:$$ and $$\:\mathsf{x}_{2} = \mathsf{v}_{2}\sqrt{-(2/\mathsf{s} )\ln \mathsf{s}}$$ are two independent random numbers following the standard normal distribution.

The number pairs (v ₁, v ₂) obtained from step (i) are the Cartesian coordinates of a set of points uniformly distributed inside the unit circle. We can write them as $$\mathsf{v}_{1} = \mathsf{r}\cos \theta,\,\mathsf{v}_{2} = \mathsf{r}\sin \theta$$ using the polar coordinates $$\mathsf{r} = \sqrt{\mathsf{s}},\,\theta =\arctan (\mathsf{v}_{2}/\mathsf{v}_{1})$$. The point (x ₁, x ₂) then has the Cartesian coordinates

We now ask for the probability

Since s = r ² is by construction uniformly distributed between 0 and 1, one has

The probability density of r is

The joint distribution function of x ₁ and x ₂,

is the product of two distribution functions of the standard normal distribution. The procedure is implemented in the method DatanRandom.standardNormal and illustrated in Fig. 4.6.

Fig. 4.6:

Illustration of the Box–Muller procedure. (a) Number pairs (v ₁, v ₂) are generated that uniformly populate the square. Those pairs are then rejected that do not lie inside the unit circle (marked by small points). (b) This is followed by the transformation $$(\mathsf{v}_{1},\mathsf{v}_{2}) \rightarrow (\mathsf{x}_{1},\mathsf{x}_{2})\:$$.

Many other procedures are described in the literature for the generation of normally distributed random numbers. They are to a certain extent more efficient, but are generally more difficult to program than the BOX–MULLER procedure.

4.10 Generation of Random Numbers According to a Multivariate Normal Distribution

The probability density of a multivariate normal distribution of n variables $$\mathbf{x} = (x_{1},x_{2},\ldots,x_{n})$$ is according to (5.​10.​1)

Here $$\mathbf{a}$$ is the vector of expectation values and $$B = {C}^{-1}$$ is the inverse of the positive-definite symmetric covariance matrix. With the Cholesky decomposition B = D T D and the substitution $$\mathbf{u} = D(\mathbf{x} -\mathbf{a})$$ the exponent takes on the simple form

Thus the elements u i of the vectors $$\mathbf{u}$$ follow independent standard normal distributions [cf. (5.​10.​9)]. One obtains vectors $$\mathsf{\mathbf{x}}$$ of random numbers by first forming a vector $$\mathsf{\mathbf{u}}$$ of elements u i which follow the standard normal distribution and then performing the transformation

This procedure is implemented in the method DatanRandom.multivariateNormal.

4.11 The Monte Carlo Method for Integration

It follows directly from its construction that the acceptance–rejection technique, Sect. 4.8.2, provides a very simple method for numerical integration . If N pairs of random numbers $$(\mathsf{y}_{1},\mathsf{u}_{i}),\:i = 1,2,\ldots,N$$ are generated according to the prescription of the general acceptance–rejection technique, and if N − n of them are rejected because they fulfill condition (4.8.18), then the numbers N (or n) are proportional to the areas under the curves c ⋅ s(y) (or g(y)), at least in the limit of large N, i.e.,

(4.11.1)

Since the function s(y) is chosen to be particularly simple [in the simplest case one has $$s(y) = 1/(b - a)$$], the ratio n∕N is a direct measure of the value of the integral

(4.11.2)

Here the integrand g(y) does not necessarily have to be normalized, i.e., one does not need to require

as long as c is chosen such that (4.8.17) is fulfilled.

Example 4.6:

Computation of π

Referring to Example 4.4 we compute the integral using (4.8.11) with R = 1:

Choosing s(y) = 1 and c = 1 we obtain

We expect that when N points are distributed according to a uniform distribution in the square 0 ≤ y ≤ 1,  0 ≤ u ≤ 1, and when n of them lie inside the unit circle, then the ratio n∕N approaches the value $$I =\pi /4$$ in the limit N → ∞. Table 4.2 shows the results for various values of n and for various sequences of random numbers. The exact value of n∕N clearly depends on the particular sequence. In Sect. 6.​8 we will determine that the typical fluctuations of the number n are approximately $$\varDelta n = \sqrt{n}$$. Therefore one has for the relative precision for the determination of the integral (4.11.2)

(4.11.3)

We expect therefore in the columns of Table 4.2 to find the value of π with precisions of 10, 1, and 0. 1 %. We find in fact in the three columns fluctuations in the first, second, and third places after the decimal point. ■ 

Table 4.2:

Numerical values of 4 n∕N for various values of n. The entries in the columns correspond to various sequences of random numbers.

4 n∕N

n = 10²

n = 10⁴

n = 10⁶

3.419

3.122

3.141

3.150

3.145

3.143

3.279

3.159

3.144

3.419

3.130

3.143

The Monte Carlo method of integration can now be implemented by a very simple program. For integration of single variable functions it is usually better to use other numerical techniques for reasons of computing time. For integrals with many variables, however, the Monte Carlo method is more straightforward and often faster as well.

4.12 The Monte Carlo Method for Simulation

Many real situations that are determined by statistical processes can be simulated in a computer with the aid of random numbers. Examples are automobile traffic in a given system of streets or the behavior of neutrons in a nuclear reactor. The Monte Carlo method was originally developed for the latter problem by VON NEUMANN and ULAM. A change of the parameters of the distributions corresponds then to a change in the actual situation. In this way the effect of additional streets or changes in the reactor can be investigated without having to undertake costly and time consuming changes in the real system. Not only processes of interest following statistical laws can be simulated with the Monte Carlo method, but also the measurement errors which occur in every measurement.

Example 4.7:

Simulation of measurement errors of points on a line

We consider a line in the (t, y)-plane. It is described by the equation

(4.12.1)

If we choose discrete values of t

(4.12.2)

then they correspond to values of y

(4.12.3)

We assume that the values t ₀, t ₁, … of the “controlled variable” t can be set without error. Because of measurement errors, however, instead of the values y i , one obtains different values

(4.12.4)

Here $$\varepsilon _{i}$$ are the measurement errors, which follow a normal distribution with mean of zero and standard deviation σ y (cf. Sect. 5.​7). The method DatanRandom.line generates number pairs (t i , y′ i ). Figure 4.7 as an example displays 10 simulated points. ■ 

Fig. 4.7:

Line in the (t, y)-plane and simulated measured values with errors in y.

Example 4.8:

Generation of decay times for a mixture of two different radioactive substances

At time t = 0 a source consists of N radioactive nuclei of which aN decay with a lifetime τ ₁ and (a − 1)N with a mean lifetime τ ₂, with 0 ≤ a ≤ 1. Random numbers for two different problems must be used in the simulation the decay times occurring: for the choice of the type of nucleus and for the determination of the decay time of the nucleus chosen, cf. (4.8.9). The method DatanRandom.radio implements this example. ■ 

4.13 Java Classes and Example Programs

Java Class for the Generation of Random Numbers

DatanRandom contains methods for the generation of random numbers following various distributions, in particular DatanRandom.ecuy for the uniform, DatanRandom.standardNormal for the standard normal, and DatanRandom.multivariateNormal for the multivariate normal Distribution. Further methods are used to illustrate a simple MLC generator or to demonstrate the following examples.

Example Program 4.1:

The class E1Random demonstrates the generation of random numbers One can choose interactively between three generators. After clicking on Go 100 random numbers are generated and displayed. The seeds before and after generation are shown and can be changed interactively.

Example Program 4.2:

The class E2Random demonstrates the generation of measurement points, scattering about a straight lineExample 4.7 is realized. Parameter input is interactive, output both numerical and graphical.

Example Program 4.3:

The class E3Random demonstrates the simulation of decay timesExample 4.8 is realized. Parameter input is interactive, output in form of a histogram.

Example Program 4.4:

The class E4Random demonstrates the generation of random numbers from a multivariate normal distributionThe procedure of Sect. 4.10 is realized for the case of two variables. Parameter input is interactive. The generated number pairs are displayed numerically.

Literature

Literature Cited in the Text

[2]

D.E. Knuth, The Art of Computer Programming, vol. 2, Addison-Wesley, Reading MA 1981MATH

[3]

P. L’Ecuyer, Comm. ACM 31 (1988) 742MathSciNetCrossref

[4]

B.A. Wichmann and I.D. Hill, Appl. Stat. 31 (1982) 188Crossref

[5]

G.E.P. Box and M.E. Muller, Ann. Math. Stat. 29 (1958) 611Crossref

Footnotes

1

Abbreviation of binary digit.

2

Since the numbers are pseudorandom and not strictly random, we use the notation x in place of x.

© Springer International Publishing Switzerland 2014

Siegmund BrandtData Analysishttps://doi.org/10.1007/978-3-319-03762-2_5

5. Some Important Distributions and Theorems

Siegmund Brandt1 

(1)

Department of Physics, University of Siegen, Siegen, Germany

Keywords

CovarianceBlack BallConvolution

We shall now discuss in detail some specific distributions. This chapter could therefore be regarded as a collection of examples. These distributions, however, are of great practical importance and are often encountered in many applications. Moreover, their study will lead us to a number of important theorems.

5.1 The Binomial and Multinomial Distributions

Consider an experiment having only two possible outcomes. The sample space can therefore be expressed as

(5.1.1)

with the probabilities

(5.1.2)

One now performs n independent trials of the experiment defined by (5.1.1). One wishes to find the probability distribution for the quantity $$\mathsf{x} =\sum _{ i=1}^{n}\mathsf{x}_{i}$$, where one has x i  = 1 (or 0) when A (or $$\bar{A}$$) occurs as the result of the ith experiment.

The probability that the first k trials result in A and all of the rest in $$\bar{A}$$ is, using Eq. (2.​3.​8),

Using the rules of combinatorics, the event “outcome Ak times in n trials” occurs in $$\left (\begin{array}{c} n\\ k \end{array} \right ) = \frac{n!} {k!(n - k)!}$$ different ways, according to the order of the occurrences of A and $$\bar{A}$$ (see Appendix B). The probability of this event is therefore

(5.1.3)

We are interested in the mean value and variance of x. We first find these quantities for the variable x i of an individual event. According to (3.​3.​2) one has

(5.1.4)

and

(5.1.5)

From the generalization of (3.​5.​3) for x = ∑ x i it follows that

(5.1.6)

and from (3.​5.​10), since all of the covariances vanish because the x i are independent, one has

(5.1.7)

Figure 5.1 shows the distribution W k n for various n and for fixed p, and Fig. 5.2 shows it for fixed n and various values of p. Finally in Fig. 5.3 n and p are both varied but the product np is held constant. The figures will help us to see relationships between the binomial distribution  (5.1.3) and other distributions.

A logical extension of the binomial distribution deals with experiments where more than two different outcomes are possible. Equation (5.1.1) is then replaced by

(5.1.8)

Let the probability for the outcome A j be

(5.1.9)

We consider again n trials and ask for the probability that the outcome A j occurs k j times. This is given by

(5.1.10)

The proof is left to the reader. The probability distribution (5.1.10) is called the multinomial distribution.

Fig. 5.1:

Binomial distribution for various values of n, with p fixed.

We can define a random variable x ij that takes on the value 1 when the ith trial leads to the outcome A j , and is zero otherwise. In addition define $$\mathsf{x}_{j} =\sum _{ i=1}^{n}\mathsf{x}_{ij}$$. The expectation value of x j is then

(5.1.11)

Fig. 5.2:

Binomial distribution for various values of p, with n fixed.

The elements of the covariance matrix of the x j are

(5.1.12)

The off-diagonal elements are clearly not zero. This was to be expected, since from Eq. (5.1.9) the variables x j are not independent.

5.2 Frequency: The Law of Large Numbers

Usually the probabilities for the different types of events, e.g., p j in the case of the multinomial distribution, are not known but have to be obtained from experiment. One first measures the frequency of the events in n experiments,

(5.2.1)

Unlike the probability, the frequency is a random quantity, since it depends on the outcomes of the n individual experiments. By use of (5.1.11), (5.1.12), and (3.​3.​15) we obtain

(5.2.2)

and

(5.2.3)

Fig. 5.3:

Binomial distribution for various values of n but fixed product np. For higher values of n the distribution changes very little.

The product $$p_{j}(1 - p_{j})$$ in Eq. (5.2.3) is at most 1∕4. One sees that the expectation value of the frequency of an event is exactly equal to the probability that the event will occur, and that the variance of frequency about this expectation value can be made arbitrarily small as the number of trials increases. Since pq is at most 1∕4, one can always say that the standard deviation of h j is at most $$1/\sqrt{n}$$. This property of the frequency is known as the law of large numbers . It is clearly the reason for the frequency definition of probability given by Eq. (2.​2.​1).

Frequently the purpose of an experimental investigation is to determine the probability for the occurrence of a certain type of event. According to (5.2.2) we can use the frequency as an approximation of the probability. The square of the error of this approximation is then inversely proportional to the number of individual experiments. This kind of error, which originates from the fact that only a finite number of experiments can be performed, is called the statistical error . It is of prime importance for applications that are concerned with the counting of individual events, e.g., nuclear particles passing through a counter, animals with certain traits in heredity experiments, defective items in quality control, and so forth.

Example 5.1:

Statistical error

Suppose it is known from earlier experiments that a fraction R ≈ 1∕200 of a sample of fruit flies (Drosophila) develop a certain property A if exposed to a given dose of X-rays. An experiment is planned to determine the fraction R with an accuracy of 1%. How large must the original sample be in order to achieve this accuracy?

We use Eq. (5.2.3) and find $$p_{j} = 0.005,(1 - p_{j}) \approx 1$$. We must now choose n such that $$\sigma (h_{j})/h_{j} = 200\sigma (h_{j}) = 0.01$$. This gives σ(h j ) = 0. 000 05 and $${\sigma }^{2}(h_{j}) = 0.25 \times 1{0}^{-8}$$. Equation (5.2.3) gives

and therefore

A total of two million fruit flies would have to be used. This is practically impossible. To determine the fraction R with an accuracy of 10% would require 20 000 flies. ■ 

5.3 The Hypergeometric Distribution

Although we shall rigorously introduce the concept of random sampling at a later point, we will now discuss a typical problem of sampling. We consider a container – we shall not break with the habit of mathematicians of calling such a container an urn – with K white and $$L = N - K$$ black balls. We want to determine the probability that in drawing n balls (without replacing them) we will find exactly k white and $$l = n - k$$ black ones. The problem is rendered difficult by the fact that the drawing of a ball of a particular color changes the ratio of white and black balls and therefore influences the outcome of the next draw. One clearly has $$N\choose n$$ equally likely ways to choose n out of N balls. The probability that one of these possibilities will occur is therefore $$1/{N\choose n}$$. There are $$K\choose k$$ ways to choose k of the K white balls, and $$L\choose \ell$$ ways to choose ℓ of the L black ones. The required probability is therefore

(5.3.1)

As in Sect. 5.1 we define the random variable $$\mathsf{x} =\sum _{ i=1}^{n}\mathsf{x}_{i}$$ with x i  = 1 when the ith draw results in a black ball, and x i  = 0 otherwise. (In other words, we define k as the random variable x.)

To compute the expectation values of x we cannot simply add the expectation values of the x i , since these are no longer independent. Instead we must return to the definition (3.​3.​2),

If we substitute $$i - 1 = j$$, this gives

With Eq. (B.5) we obtain

(5.3.2)

The calculation of the variance follows along the same lines but is rather lengthy. The result is

(5.3.3)

Figures 5.4 and 5.5 depict several examples of the distribution. If n ≪ N, then drawing a white ball has little influence on the probabilities for the next draw. We therefore expect that in this case W k behaves in a manner similar to a binomial distribution with $$p = \frac{K} {N}$$ and $$q = \frac{N-K} {N}$$. This is also made clear by the similarity of Figs. 5.5 and 5.1. One obtains in fact the same expectation value,

as for the binomial distribution. The variance is then

which for the case n ≪ N becomes

Fig. 5.4:

Hypergeometric distribution for various values of n and small values of K and N.

Fig. 5.5:

Hypergeometric distribution for various values of n and for large values of K and N.

There are many applications of the hypergeometric distribution. Opinion polls, quality controls, and so forth are all based on the experimental scheme of taking (polling) an object without replacement back into the original sample or population. The distribution can be generalized in two ways. First we can of course consider more properties instead of just two (white and black balls). This leads us to a similar transition as the one from the binomial to the multinomial distribution. The original sample (population) contains N elements each of which possesses one of l properties,

The probability that n draws (without replacement) will be composed as

is, in analogy to Eq. (5.3.1),

(5.3.4)

Another extension of the hypergeometric distribution is obtained in the following way. We saw earlier that consecutive drawings ceased to be independent because the balls were not replaced. If now each time we draw a ball of one type we place more balls of that type back in the urn, this dependence can be enhanced. One then obtains the Polya distribution . It is of importance in the study of epidemic diseases, where the appearance of a case of the disease enhances the probability of future cases.

Example 5.2:

Application of the hypergeometric distribution fordetermination of zoological populations

From a pond K fish are taken and marked. They are then returned to the pond. After a short while n fish are caught, k of which are found to be marked. Before the second time that the fish are taken, the pond contains a total of N fish, of which K are marked. The probability of finding k marked out of n removed fish is given by Eq. (5.3.1). We will return to this problem in Example 7.3. ■ 

5.4 The Poisson Distribution

Looking at Fig. 5.3 it appears that if n tends to infinity, but at the same time np = λ is kept constant, the binomial distribution approaches a certain fixed distribution. We rewrite Eq. (5.1.3) as

In the limiting case all of the many individual factors of the term on the right approach unity. In addition one has

so that in the limit one has

(5.4.1)

The quantity f(k) is the probability of the Poisson distribution . It is plotted in Fig. 5.6 for various values of λ. As is the case for the other distributions we have encountered so far, the Poisson distribution is only defined for integer values of k.

The distribution satisfies the requirement that the total probability is equal to unity,

(5.4.2)

The expression in parentheses is in fact the Taylor expansion of $${\mathrm{e}}^{\lambda }$$.

We now want to determine the mean, variance, and skewness of the Poisson distribution. The definition (3.​3.​2) gives

and using this with (5.4.2),

(5.4.3)

We would now like to find E(k ²). One obtains in a corresponding way

and therefore

(5.4.4)

We will use Eqs. (5.4.3) and (5.4.4) to compute the variance. According to Eq. (3.​3.​16) one has

(5.4.5)

or

(5.4.6)

We now consider the skewness (3.​3.​13) of the Poisson distribution. Following Sect. 3.​3 we easily find that

Fig. 5.6:

Poisson distribution for various values of λ.

The skewness (3.​3.​13) is then

(5.4.7)

that is, the Poisson distribution becomes increasingly symmetric as λ increases. Figure 5.6 shows the distribution for various values of λ. In particular the distribution with λ = 3 should be compared with Fig. 5.3.

We have obtained the Poisson distribution from the binomial distribution with large n but constant λ = np, i.e., small p. We therefore expect it to apply to processes in which a large number of events occur but of which only very few have a certain property of interest to us (i.e., a large number of “trials” but few “successes”).

Example 5.3:

Poisson distribution and independence of radioactive decays

We consider a radioactive nucleus with mean lifetime τ and observe it for a time T ≪ τ. The probability that it decays within this time interval is W ≪ 1. We break the observation time T into n smaller time intervals of length t, so that T = nt. The probability for the nucleus to decay in a particular time interval is p ≈ W∕n. We now observe a radioactive source containing N nuclei which decay independently from each other for a total time T, and detect a ₁ decays in time interval 1, a ₂ decays in interval 2, etc. Let h(k) be the frequency of decays observed in the interval k, with ($$k = 0,1,\ldots$$). That is, if n k is the number of intervals with k decays, then $$h(k) = n_{k}/n$$. In the limit $$N \rightarrow \infty$$ and for large n the frequency distribution h(k) becomes the probability distribution (5.4.1). The statistical nature of radioactive decay was established in this way in a famous experiment by RUTHERFORD and GEIGER. ■ 

Similarly, the frequency of finding k stars per element of the celestial sphere or k raisins per volume element of a fruit cake is distributed according to the Poisson law, but not, however, the frequency of finding k animals of a given species per element of area, at least if these animals live in herds, since in this case the assumption of independence is not fulfilled.

As a quantitative example of the Poisson distribution many textbooks discuss the number of Prussian cavalrymen killed during a period of 20 years by horse kicks, an example originally due to VON BORTKIEWICZ [6]. We prefer to turn our attention to a somewhat less macabre example taken from a lecture of DE SOLLA PRICE [7].

Example 5.4:

Poisson distribution and the independence of scientificdiscoveries

The author first constructs the model of an apple tree with 1000 apples and 1000 pickers with blindfolded eyes who each try at the same time to pick an apple. Since we are dealing with a model, they do not hinder each other but it can happen that two or several of them will attempt to pick the same apple at the same time. The number of apples grabbed simultaneously by k people ($$k = 0,1,2,\ldots$$) follows a Poisson distribution. It was determined by DE SOLLA PRICE that the number of scientific discoveries made independently twice, three times, etc. is also distributed according to the Poisson law, in a way similar to the principle of the blindfolded apple pickers (Table 5.1). One gets the impression that scientists are not concerned with the activities of their colleagues. DE SOLLA PRICE believes that this can be explained by the assumption that scientists have a strong urge write papers, but feel only a relatively mild need to read them. ■ 

Table 5.1:

Simultaneous discovery and the Poisson distribution.

Number of

Cases of

Prediction

simultaneous

simultaneous

of Poisson

discoveries

discovery

distribution

0

Not defined

368

1

Not known

368

2

179

184

3

51

61

4

17

15

5

6

3

 ≥ 6

8

1

5.5 The Characteristic Function of a Distribution

So far we have only considered real random variables. In fact, in Sect. 3.​1 we have introduced the concept of a random quantity as a real number associated with an event. Without changing this concept we can formally construct a complex random variable from two real ones by writing

(5.5.1)

As its expectation value we define

(5.5.2)

By analogy with real variables, complex random variables are independent if the real and imaginary parts are independent among themselves.

If x is a real random variable with distribution function F(x) = P(x < x) and probability density f(x), we define its characteristic function to be the expectation value of the quantity exp(i ​t x):

(5.5.3)

That is, in the case of a continuous variable the characteristic function is a Fourier integral with its known transformation properties:

(5.5.4)

For a discrete variable we obtain instead from (3.​3.​2)

(5.5.5)

We now consider the moments of x about the origin,

(5.5.6)

and find that λ n can be obtained simply by differentiating the characteristic function n times at the point t = 0:

and therefore

(5.5.7)

If we now introduce the simple coordinate translation

(5.5.8)

and construct the characteristic function

(5.5.9)

then its nth derivative is (up to a power of i ​) equal to the nth moment of x about the expectation value [cf. (3.​3.​8)]:

(5.5.10)

and in particular

(5.5.11)

Inverting the Fourier transform (5.5.4) we see that it is possible to obtain the probability density from the characteristic function,

(5.5.12)

It is possible to show that a distribution is determined uniquely by its characteristic function. This is the case even for discrete variables where one has

(5.5.13)

since in this case the probability density is not defined. Often it is more convenient to use the characteristic function rather than the original distribution. Because of the unique relation between the two it is possible to switch back and forth at any place in the course of a calculation.

We now consider the sum of two independent random variables

Its characteristic function is

Generalizing relation (3.​5.​13) to complex variables we obtain

(5.5.14)

i.e., the characteristic function of a sum of independent random variables is equal to the product of their respective characteristic functions.

Example 5.5:

Addition of two Poisson distributed variables with use of the characteristic function

From Eqs. (5.5.5) and (5.4.1) one obtains for the characteristic function of the Poisson distribution

(5.5.15)

We now form the characteristic function of the sum of two independent Poisson distributed variables with mean values λ ₁ and λ ₂,

(5.5.16)

This is again of the form of Eq. (5.5.15). Therefore the distribution of the sum of two independent Poisson distributed variables is itself a Poisson variable. Its mean is the sum of the means of the individual distributions. ■ 

Fig. 5.7:

Probability density (a) and distribution function (b) of the standard normal distribution.

5.6 The Standard Normal Distribution

The probability density of the standard normal distribution is defined as

(5.6.1)

This function is depicted in Fig. 5.7a. It has a bell shape with the maximum at x = 0. From Appendix D.1 we have

(5.6.2)

so that ϕ ₀(x) is normalized to one as required. Using the symmetry of Fig. 5.7a, or alternatively, using the antisymmetry of the integrand we conclude that the expectation value is

(5.6.3)

By integrating by parts we can compute the variance to be

(5.6.4)

since the expression in the square brackets vanishes at the integral’s boundaries and the integral in curly brackets is given by Eq. (5.6.2).

The distribution function of the standard normal distribution

(5.6.5)

is shown in Fig. 5.7b. It cannot be expressed in analytic form. It is tabulated numerically in Appendix C.4.

5.7 The Normal or Gaussian Distribution

The standardized distribution of the last section had the properties $$\widehat{x} = E(\mathsf{x}) = 0{,\:\sigma }^{2}(\mathsf{x}) = 1$$, i.e., the variable x had the properties of the standardized variable u in Eq. (3.​3.​17). If we now replace x by $$(\mathsf{x} - a)/b$$ in (5.6.1), we obtain the probability density of the normal or Gaussian distribution ,

(5.7.1)

with

(5.7.2)

The characteristic function of the normal distribution (5.7.1) is, using Eq. (5.5.4),

(5.7.3)

With $$u = (x - a)/b$$ one obtains

(5.7.4)

By completing the square the integral can be rewritten as

(5.7.5)

With $$r = u -\mathrm{i \!}tb$$ the last integral takes on the form

The integrand does not have any singularities in the complex r plane. According to the residue theorem, therefore, the contour integral around any closed path vanishes. Consider a path that runs along the real axis from $$r = -L$$ to r = L, and then parallel to the imaginary axis from r = L to $$r = L -\mathrm{i \!}tb$$ and from there antiparallel to the real axis to $$r = -L -\mathrm{i \!}tb$$, and finally back to the starting point r = L. In the limit $$L \rightarrow \infty$$ the integrand vanishes along the parts of the path that run parallel to the imaginary axis. One then has

i.e., we can extend the integral to cover the entire real axis. The integral is computed in Appendix D.1 and has the value

(5.7.6)

Substituting this into Eqs. (5.7.5) and (5.7.4) we obtain finally the characteristic function of the normal distribution

(5.7.7)

For the case a = 0 one obtains from this the following interesting theorem:

A normal distribution with mean value zero has a characteristic function that has itself (up to normalization) the form of a normal distribution. The product of the variances of both functions is one.

If we now consider the sum of two independent normal distributions, then by applying Eq. (5.5.14) one immediately sees that the characteristic function of the sum is again of the form of Eq. (5.7.7). The sum of independent normally distributed quantities is therefore itself normally distributed. The Poisson distribution behaves in a similar way (cf. Example 5.5).

5.8 Quantitative Properties of the Normal Distribution

Figure 5.7a shows the probability density of the standard Gaussian distribution ϕ ₀(x) and the corresponding distribution function. By simple computation one can determine that the points of inflection of (5.6.1) are at x = ±1. [In the case of a general Gaussian distribution (5.7.1) they are at x = a ± b.] The distribution function ψ ₀(x) gives the probability for the random variable to take on a value smaller than x:

(5.8.1)

By symmetry one has

(5.8.2)

or conversely, the probability to obtain a random value within an interval of width 2x about zero (the expectation value) is

(5.8.3)

Since the integral (5.6.5) is not easy to evaluate, one typically finds the values of (5.8.1) and (5.8.3) from statistical tables, e.g., in Tables I.2 and I.3 of the appendix.

One can now extend this relation to the general Gaussian distribution given by Eq. (5.7.1). Its distribution function is

(5.8.4)

We are interested in finding the probability to obtain a random value inside (or outside) of a given multiple of σ = b about the mean value:

(5.8.5)

From Table I.3 we find

(5.8.6)

As we will see later in more detail, one can often assume that the measurement errors of a quantity are distributed according to a Gaussian distribution about zero. This means that the probability to obtain a value between x and x + dx is given by

The dispersion σ of the distribution ϕ(x) is called the standard deviation or standard error . If the standard error of an instrument is known and one carries out a single measurement, then Eq. (5.8.6) tells us that the probability that the true value is within an interval given by plus or minus the standard error about the measured value is 68. 3%. It is therefore a common practice to multiply the standard error with a more or less arbitrary factor in order to improve this percentage. (One obtains around 99. 8% for the factor 3.) This procedure is, however, misleading and often harmful. If this factor is not explicitly stated, a comparison of different measurements of the same quantity and especially the calculation of a weighted average (cf. Example 9.1) is rendered impossible or is liable to be erroneous.

The quantiles [see Eq. (3.​3.​25)] of the standard normal distribution are of considerable interest. For the distribution function (5.6.5) one obtains by definition

(5.8.7)

The quantile x p is therefore given by the inverse function

(5.8.8)

of the distribution function ψ ₀(x p ). This is computed numerically in Appendix C.4 and is given in Table I.4. Figure 5.8 shows a graphical representation.

We now consider the probability

(5.8.9)

for a quantity distributed according to the standard normal distribution to differ from zero in absolute value by less than x. Since

it is the inverse function and with it the quantiles of the distribution function P′(x) are obtained by substituting $$(P^{\prime}(x) + 1)/2$$ for the argument of the inverse function of P:

(5.8.10)

This function is tabulated in Table I.5.

Fig. 5.8:

Quantile of the standard normal distribution.

5.9 The Central Limit Theorem

We will now prove the following important theorem. If x i are independent random variables with mean values a and variances b ², then the variable

(5.9.1)

is normally distributed with

(5.9.2)

From Eq. (3.​3.​15) one then has that the variable

(5.9.3)

is normally distributed with

(5.9.4)

To prove this we assume for simplicity that all of the x i have the same distribution. If we denote the characteristic function of the x i by $$\varphi (t)$$, then the sum of n variables has the characteristic function $$\{\varphi {(t)\}}^{n}$$. We now assume that a = 0. (The general case can be related to this by a simple coordinate translation $$\mathsf{x}^{\prime}_{i} = \mathsf{x}_{i} - a$$.) From (5.5.10) we have the first two derivatives of $$\varphi (t)$$ at t = 0,

We can therefore expand,

Instead of x i let us now choose

as the variable. If we consider n to be fixed for the moment, then this implies a simple translation and a change of scale. The corresponding characteristic function is

or

The higher-order terms are at most of the order n ⁻². If we now consider the limiting case and use

(5.9.5)

we obtain

or

(5.9.6)

This, however, is exactly the characteristic function of the standard normal distribution ϕ ₀(u). One therefore has E(u) = 0, σ ²(u) = 1. Using Eqs. (5.9.5) and (3.​3.​15) leads directly to the theorem.

Example 5.6:

Normal distribution as the limiting case of the binomial

distribution

Suppose that the individual variables x i in (5.9.1) are described by the simple distribution given by (5.1.1) and (5.1.2), i.e., they can only take on the value 1 (with probability p) or 0 (with probability 1 − p). One then has E(x i ) = p, $${\sigma }^{2}(\mathsf{x}_{i}) = p(1 - p)$$. The variable

(5.9.7)

then follows the binomial distribution, $$P({\mathsf{x}}^{(n)} = k) = W_{k}^{n}$$ [see Eqs. (5.1.3), (5.1.6), (5.1.7)]. As done for (5.9.5) let us consider the distribution of

(5.9.8)

One clearly has $$P(\mathsf{x} = k) = P\left ({\mathsf{u}}^{(n)} = (k - np)/\sqrt{np(1 - p)}\right ) = W_{k}^{n}$$. These values, however, lie increasingly closer to each other on the u (n) axis as n increases. Let us denote the distance between two neighboring values of u (n) by Δ u (n). Then the distribution of a discrete variable $$P({\mathsf{u}}^{(n)})/\varDelta {\mathsf{u}}^{(n)}$$ finally becomes the probability density of a continuous variable. According to the Central Limit Theorem this must be a standard normal distribution. This is illustrated in Fig. 5.9, where $$P({\mathsf{u}}^{(n)})/\varDelta {\mathsf{u}}^{(n)}$$ is shown for various possible values of u (n). ■ 

Example 5.7:

Error model of Laplace

In 1783 LAPLACE made the following remarks concerning the origin of errors of an observation. Suppose the true value of a quantity to be measured is m ₀. Now let the measurement be disturbed by a large number n of independent causes, each resulting in a disturbance of magnitude $$\varepsilon$$. For each disturbance there exists an equal probability for a variation of the measured value in either direction, i.e., $$+\varepsilon$$ or $$-\varepsilon$$. The measurement error is then composed of the sum of the individual disturbances. It is clear that in this model the probability distribution of measurement errors will be given by the binomial distribution. It is interesting nevertheless to follow the model somewhat further, since it leads directly to the famous Pascal triangle .

Figure 5.10 shows how the probability distribution is derived from the model. The starting point is with no disturbance where the probability of measuring m ₀ is equal to one. With one disturbance this probability is split equally between the neighboring possibilities $$m_{0}+\varepsilon$$ and $$m_{0}-\varepsilon$$. The same happens with every further disturbance. Of course the individual probabilities leading to the same measured value must be added.

Fig. 5.9:

The quantity $$P({u}^{(n)})/\varDelta {u}^{(n)}$$ for various values of the discrete variable u (n) for increasing n.

Fig. 5.10:

Connection between the Laplacian error model and the binomial distribution.

Each line of the resulting triangle contains the distribution $$W_{k}^{n}(k = 0,1,\ldots,n)$$ of Eq. (5.1.3) for the case $$p = q = 1/2$$. Multiplied by $$1/({p}^{k}{q}^{n-k}) = {2}^{n}$$ it becomes a line of binomial coefficients of Pascal’s triangle (cf. Ap- pendix B).

It is easy to relate this to Example 5.6 by extending Eq. (5.9.8) and substituting $$p = 1/2$$. For n → ∞ the quantity

follows a normal distribution with expectation value zero and standard deviation $$\sqrt{n}\varepsilon /2$$. Thus Gaussian measurement errors can result from a large number of small independent disturbances. ■ 

The identification of the measurement error distribution as Gaussian is of great significance in many computations, particularly for the method of least squares. The normal distribution for measurement errors is, however, not a law of nature. The causes of experimental errors can be individually very complicated. One cannot, therefore, find a distribution function that describes the behavior of measurement errors in all possible experiments. In particular, it is not always possible to guaranty symmetry and independence. One must ask in each individual case whether the measurement errors can be modeled by a Gaussian distribution. This can be done, for example, by means of a χ ²-test, applied to the distribution of a measured quantity (see Sect. 8.​7). It is always necessary to check the distribution of experimental errors before more lengthy computations can be used whose results are only meaningful for the case of a Gaussian error distribution.

5.10 The Multivariate Normal Distribution

Consider a vector x of n variables,

We define the probability density of the joint normal distribution of the x i to be

(5.10.1)

with

(5.10.2)

Here $$\mathbf{a}$$ is an n-component vector and B is an n × n matrix, which is symmetric and positive definite. Since $$\phi (\mathbf{x})$$ is clearly symmetric about the point $$\mathbf{x} = \mathbf{a}$$, one has

(5.10.3)

that is,

or

(5.10.4)

The vector of expectation values is therefore given directly by $$\mathbf{a}$$.

We now differentiate Eq. (5.10.3) with respect to $$\mathbf{a}$$,

This means that the expectation value of the quantity in square brackets vanishes,

or

(5.10.5)

Comparing with Eq. (3.​6.​19) one sees that C is the covariance matrix of the variables $$\mathsf{\mathbf{x}} = (\mathsf{x}_{1},\mathsf{x}_{2},\ldots,\mathsf{x}_{n})$$.

Because of the practical importance of the normal distribution, we would like to investigate the case of two variables in somewhat more detail. In particular we are interested in the correlation of the variables. One has

(5.10.6)

By inversion one obtains for B

(5.10.7)

One sees that B is a diagonal matrix if the covariances vanish. One then has

(5.10.8)

If we substitute B ₀ into Eq. (5.10.1), we obtain – as expected – the joint probability density of two independently normally distributed variables as the product of two normal distributions:

(5.10.9)

In this simple case the constant k takes on the value

as can be determined by integration of (5.10.9) or simply by comparison with Eq. (5.7.1). In the general case of n variables with non-vanishing covariances, one has

(5.10.10)

Here det B is the determinant of the matrix B. If the variables are not independent, i.e., if the covariance does not vanish, then the expression for the normal distribution of two variables is somewhat more complicated.

Let us consider the reduced variables

and make use of the correlation coefficient

Equation (5.10.1) then takes on the simple form

(5.10.11)

with

(5.10.12)

Contours of equal probability density are characterized by a constant exponent in (5.10.11):

(5.10.13)

Let us take for the moment $$g(\mathbf{u}) = 1$$.

In the original variables Eq. (5.10.13) becomes

(5.10.14)

This is the equation of an ellipse centered around the point (a ₁, a ₂). The principal axes of the ellipse make an angle α with respect to the axes x ₁ and x ₂. This angle and the half-diameters p ₁ and p ₂ can be determined from Eq. (5.10.14) by using the known properties of conic sections:

(5.10.15)

(5.10.16)

(5.10.17)

The ellipse with these properties is called the covariance ellipse of the bivariate normal distribution. Several such ellipses are depicted in Fig. 5.11. The covariance ellipse always lies inside a rectangle determined by the point (a ₁, a ₂) and the standard deviations σ ₁ and σ ₂. It touches the rectangle at four points. For the extreme cases ρ = ±1 the ellipse becomes one of the two diagonals of this rectangle.

Fig. 5.11:

Covariance ellipses.

From (5.10.14) it is clear that other lines of constant probability (for g ≠ 1) are also ellipses, concentric and similar to the covariance ellipse and situated inside (outside) of it for larger (smaller) probability. The bivariate normal distribution therefore corresponds to a surface in the three-dimensional space (x ₁, x ₂, ϕ) (Fig. 5.12), whose horizontal sections are concentric ellipses. For the largest probability this ellipse collapses to the point (a ₁, a ₂). The vertical sections through the center have the form of a Gaussian distribution whose width is directly proportional to the diameter of the covariance ellipse along which the section extends. The probability of observing a pair x ₁, x ₂ of random variables inside the covariance ellipse is equal to the integral

Fig. 5.12:

Probability density of a bivariate Gaussian distribution (left) and the corresponding covariance ellipse (right). The three rows of the figure differ only in the numerical value of the correlation coefficient ρ.

(5.10.18)

where the region of integration A is given by the area within the covariance ellipse (5.10.14). The relation (5.10.18) is obtained by application of the transformation of variables $$\mathbf{y} = T\,\mathbf{x}$$ with $$T = {B}^{-1}$$ to the distribution $$\phi (\mathbf{x})$$. The resulting distribution has the properties $$\sigma (y_{1}) =\sigma (y_{2}) = 1$$ and $$\mathop{\mathrm{cov}}\nolimits (y_{1},y_{2}) = 0$$, i.e., it is of the form of (5.10.9). In this way the region of integration is transformed to a unit circle centered about $$(a_{1},a_{2})$$.

In our consideration of the normal distribution of measurement errors of a variable we found the interval $$a-\sigma \leq x \leq a+\sigma$$ to be the region in which the probability density f(x) exceeded a given fraction, namely $${\mathrm{e}}^{-1/2}$$ of its maximum value. The integral over this region was independent of σ. In the case of two variables, the role of this region is taken over by the covariance ellipse determined by σ ₁, σ ₂, and ρ, and not – as is sometimes incorrectly assumed – by the rectangle that circumscribes the ellipse in Fig. 5.11. The meaning of the covariance ellipse can also be seen from Fig. 5.13. Points 1 and 2, which lie on the covariance ellipse, correspond to equal probabilities ($$P(1) = P(2) = P_{\mathrm{e}}$$), although the distance of point 1 from the middle is less in both coordinate directions. In addition, point 3 is more probable, and point 4 less probable ($$P(4) <P_{\mathrm{e}}$$, $$P(3)> P_{\mathrm{e}}$$), even though point 4 even closer is to $$(a_{1},a_{2})$$ than point 3.

Fig. 5.13:

Relative probability for various points from a bivariate Gaussian distribution ($$P_{1} = P_{2} = P_{\mathrm{e}}$$, $$P_{3}> P_{\mathrm{e}}$$, $$P_{4} <P_{\mathrm{e}}$$).

For three variables one obtains instead of the covariance ellipse a covariance ellipsoid , for n variables a hyperellipsoid in an n-dimensional space (see also Sect. A.11). According to our construction, the covariance ellipsoid is the hypersurface in the n-dimensional space on which the function $$g(\mathbf{x})$$ in the exponent of the normal distribution (5.10.1) has the constant value $$g(\mathbf{x}) = 1$$. For other values $$g(\mathbf{x}) = \mbox{ const}$$ one obtains similar ellipsoids which lie inside (g < 1) or outside (g > 1) of the covariance ellipsoid. In Sect. 6.​6 it will be shown that the function $$g(\mathbf{x})$$ follows a χ ²-distribution with n degrees of freedom if $$\mathbf{x}$$ follows the normal distribution (5.10.1). The probability to find $$\mathbf{x}$$ inside the ellipsoid g = const is therefore

(5.10.19)

Here P is the incomplete gamma function given in Sect. D.5. For g = 1, that is, for the covariance ellipsoid in n dimensions, this probability is

(5.10.20)

Numerical values for small n are

The probability decreases rapidly as n increases. In order to be able to give regions for various n which correspond to equal probability content, one specifies a value W on the left-hand side of (5.10.19) and determines the corresponding value of g. Then g is the quantile with probability W of the χ ²-distribution with n degrees of freedom (see also Appendix C.5),

(5.10.21)

The ellipsoid that corresponds to the value of g that contains $$\mathbf{x}$$ with the probability W is called the confidence ellipsoid of probability W. This expression can be understood to mean that, e.g., for W = 0. 9 one should have 90% confidence that $$\mathbf{x}$$ lies within the confidence ellipsoid.

The variances σ i ² or the standard deviations Δ i  = σ i also have a certain meaning for n variables. The probability to observe the variable x i in the region $$a_{i} -\sigma _{i} <x_{i} <a_{i} +\sigma _{i}$$ is, as before, 68. 3%, independent of the number n of the variables. This only holds, however, when one places no requirements on the positions of any of the other variables x j , j ≠ i.

5.11 Convolutions of Distributions

5.11.1 Folding Integrals

On various occasions we have already discussed sums of random variables, and in the derivation of the Central Limit Theorem, for example, we found the characteristic function to be a useful tool in such considerations. We would now like to discuss the distribution of the sum of two quantities, but for greater clarity we will not make use of the characteristic function.

A sum of two distributions is often observed in experiments. One could be interested, for example, in the angular distribution of secondary particles from the decay of an elementary particle. This can often be used to determine the spin of the particle. The observed angle is the distribution of a sum of random quantities, namely the decay angle and its measurement error. One speaks of the convolution of two distributions.

Fig. 5.14:

Integration region for ( 5.11.4).

Let the original quantities be x and y and the sum

(5.11.1)

A requirement for further treatment is that the original variables must be independent. In this case, the joint probability density is the product of simple densities,

(5.11.2)

If we now ask for the distribution function of u, i.e., for

(5.11.3)

then this is obtained by integration of (5.11.2) over the hatched region A in Fig. 5.14;

(5.11.4)

By differentiation one obtains the probability density for u,

(5.11.5)

If x or y or both are only defined in a restricted region, then (5.11.5) is still true. The limits of integration, however, may be limited. We will consider various cases:

(a)

(5.11.6)

(Because $$y = u - x$$ and since for x min = 0 one has y max = u.)

(b)

(5.11.7)

(c)

(5.11.8)

We will demonstrate case (d) in the following example, in which both x and y are bounded from below and from above.

Example 5.8:

Convolution of uniform distributions

With

and Eq. (5.11.8) we obtain

We substitute $$v = u - x$$, $$\mathrm{d}v = -\mathrm{d}x$$ and obtain

(5.11.9)

Clearly one has 0 < u < 2. We now consider separately the two cases

(5.11.10)

Note that the lower (upper) limit of integration is not lower (higher) than the value 0 (1). The result is a triangular distribution (Fig. 5.15).

Fig. 5.15:

Convolution of uniform distributions. Probability density of the sum u of uniformly distributed random variables x (a) u = x, (b) $$\mathsf{u} = \mathsf{x} + \mathsf{x}$$, (c) $$\mathsf{u} = \mathsf{x} + \mathsf{x} + \mathsf{x}$$.

If this result is folded again with a uniform distribution, i.e., if u is the sum of three independent uniformly distributed variables, then one obtains

(5.11.11)

The proof is left to the reader. The distribution consists of three parabolic sections (Fig. 5.15) and is similar already to the Gaussian distribution predicted by the Central Limit Theorem. ■ 

5.11.2 Convolutions with the Normal Distribution

Suppose a quantity of experimental interest x can be considered to be a random variable with probability density f x (x). It is measured with a measurement error y, which follows a normal distribution with a mean of zero and a variance of σ ². The result of the measurement is then the sum

(5.11.12)

Its probability density is [see also (5.11.4)]

(5.11.13)

By carrying out many measurements, f(u) can be experimentally determined. The experimenter is interested, however, in the function f x (x). Unfortunately, Eq. (5.11.13) cannot in general be solved for f x (x). This is only possible for a restricted class of functions f(u). Therefore one usually approaches the problem in a different way. From earlier measurements or theoretical considerations one possesses knowledge about the form of f x (x), e.g., one might assume that f x (x) is described by a uniform distribution, without, however, knowing its boundaries a and b. One then carries out the convolution (5.11.13), compares the resulting function f(u) with the experiment and in this way determines the unknown parameters (in our example a and b).

In many cases it is not even possible to perform the integration (5.11.13) analytically. Numerical procedures, e.g., the Monte Carlo method, then have to be used. Sometimes approximations (cf. Example 5.11) give useful results. Because of the importance in many experiments of convolution with the normal distribution we will study some examples.

Example 5.9:

Convolution of uniform and normal distributions

Using Eqs. (3.​3.​26) and (5.11.8) and substituting $$v = (x - u)/\sigma$$ we obtain

(5.11.14)

The function ψ has already been defined in (5.6.5). Figure 5.16 shows the result for a = 0, b = 6, σ = 1. If one has $$\vert b - a\vert \gg \sigma$$ (as is the case in Fig. 5.16), one of the terms in parentheses in (5.11.14) is either 0 or 1. The rising edge of the uniform distribution at u = a is replaced by the distribution function of the normal distribution with standard deviation σ (see also Fig. 5.7). The falling edge at u = b is its “mirror image”. ■ 

Fig. 5.16:

Convolution of a uniform and Gaussian distribution.

Example 5.10:

Convolution of two normal distributions. “Quadratic addition of errors”

If one convolutes two normal distributions with mean values 0 and variances σ x ² and σ y ², one obtains

(5.11.15)

The proof has already been shown in Sect. 5.7 with the help of the characteristic function. It can also be obtained by computation of the folding integral (5.11.5). If the distributions f x (x) and f y (y) describe two independent sources of measurement errors, the result (5.11.15) is known as the “quadratic addition of errors”.  ■ 

Example 5.11:

Convolution of exponential and normal distributions

With

Eq. (5.11.6) takes on the following form:

We can rewrite the exponent

and obtain

We now require that σ ≪ τ, i.e., that the measurement error is much smaller than the typical value (width) of the exponential distribution. In addition, we only consider values of u for which $$u {-\sigma }^{2}/\tau \gg \sigma$$, i.e., $$u \gg \sigma$$. The integral is then approximately equal to $$\sqrt{2\pi }\sigma$$ or

In a semi-logarithmic representation, i.e., in a plot of lnf(u) versus u, the curve f(u) lies above the curve f x (x), by an amount $${\sigma }^{2}/{2\tau }^{2}$$, since

This is plotted in Fig. 5.17. The result can be qualitatively understood in the following way. For each small x interval of the exponential distribution, the convolution leads with equal probability to a shift to the left or to the right. Since, however, the exponential distribution for a given u is greater for small values of x, contributions to the convolution f(u) originate with greater probability from the left than from the right. This leads to an overall shift to the right of f(u) with respect to f x (x). ■ 

Fig. 5.17:

Convolution of exponential and normal distributions.

5.12 Example Programs

Example Program 5.1:

Class E1Distrib to simulate empirical frequency and demonstrate statistical fluctuations The program simulates the problem of Example 5.1. It allows input of values for n exp, n fly, and P(A), and then consecutively performs n exp simulated experiments. In each experiment n fly objects are analyzed. Each object has a probability P(A) to have the property A. For each experiment one line of output is produced containing the current number i exp of the experiment, the number N A of objects with the property A and the frequency $$h_{A} = N_{A}/n_{\mathrm{fly}}$$ with which the property A was found. The fluctuation of h A around the known input value P(A) in the individual experiments gives a good impression of the statistical error of an experiment.

Example Program 5.2:

Class E2Distrib to simulate the experiment of Rutherford and Geiger The principle of the experiment of Rutherford and Geiger is described in Example 5.3. It is simulated as follows. Input quantities are the number N of decays observed and the number n int of partial intervals Δ T of the total observation time T. For simplicity the length of each partial interval is set equal to one. A total of N random events are simulated by simply generating N random numbers uniformly distributed between 0 and T. They are entered into a histogram with n int intervals. The histogram is first displayed graphically and then analyzed numerically. For each number $$k = 0,1,\ldots,N_{\mathrm{int}}$$ the program determines how many intervals N(k) of the histogram have k entries. The numbers N(k) themselves are presented in the form of another histogram.

Show that for the process simulated in this example program one obtains in the limit $$N \rightarrow \infty$$

If N is increased step by step and at the same time $$\lambda = N_{p} = N/n_{\mathrm{int}}$$ is kept constant, then for large N one has

and, in the limit N → ∞,

Check the above statements by running the program with suitable pairs of numbers, e.g., $$(N,n_{\mathrm{int}}) = (4,2),\,(40,20),\,\ldots,(2000,1000)$$, by reading the numbers N(k) from the graphics display and by comparing them with the statements above.

Fig. 5.18:

Arrangement of nails in Galton’s board and one possible trajectory of a ball.

Example Program 5.3:

Class E3Distrib to simulate Galton’s boardGalton’s board is a simple implementation of Laplace’s model described in Example 5.7. The vertical board contains rows of horizontally oriented nails as shown in Fig. 5.18. The rows of nails are labeled $$j = 1,2,\ldots,n$$, and row j has j nails. One by one a total of N exp balls fall onto the nail in row 1. There each ball is deflected with probability p to the right and with probability (1 − p) to the left. (In a realistic board one has $$p = 1/2$$.) The distance between the nails is chosen in such a way that in each case the ball hits one of the two nails in row 2 and there again it is deflected with the probability p to the right. After falling through n rows each ball assumes one of n + 1 places, which we denote by k = 0 (on the left), k = 1, $$\ldots$$, k = n (on the right). After a total of N exp experiments (i.e., balls) one finds N(k) balls for each value k.

The program allows input of numerical values for N exp, n, and p. For each experiment the number k is first set to zero and n random numbers r j are generated from a uniform distribution and analyzed. For each r j  < p (corresponding to a deflection to the right in row j) the number k is increased by 1. For each experiment the value of k is entered into a histogram. After all experiments are simulated the histogram is displayed.

Show that in the limit N → ∞ one has

By choosing and entering suitable pairs of numbers (n, p), e.g., (n, p) = (1, 0. 5), (2, 0. 25), (10, 0. 05), (100, 0. 005) approximate the Poisson limit

and compare these predictions with the results of your simulations.

Literature

Literature Cited in the Text

[6]

L. von Bortkiewicz, Das Gesetz der kleinen Zahlen, Teubner, Leipzig 1898

[7]

D.J. De Solla Price, Little Science, Big Science, Columbia University Press, New York 1965

© Springer International Publishing Switzerland 2014

Siegmund BrandtData Analysishttps://doi.org/10.1007/978-3-319-03762-2_6

6. Samples

Siegmund Brandt1 

(1)

Department of Physics, University of Siegen, Siegen, Germany

Keywords

EuropeCovariance

In the last chapter we discussed a number of distributions, but we have not specified how they are realized in a particular case. We have only given the probability that a random variable x will lie within an interval with boundaries x and $$x + \mathrm{d}x$$. This probability depends on certain parameters describing its distribution (like λ in the case of the Poisson distribution) which are usually unknown. We therefore have no direct knowledge of the probability distribution and have to approximate it by a frequency distribution obtained experimentally. The number of measurements performed for this purpose, called a sample , is necessarily finite. To discuss the elements of sampling theory we first have to introduce a number of new definitions.

6.1 Random Samples. Distributionof a Sample. Estimators

Every sample is taken from a set of elements which correspond to the possible results of an individual observation. Such a set, which usually has infinitely many elements, is called a population . If a sample of n elements is taken from it, then we say that the sample has the size n. Let the distribution of the random variable x in the population be given by the probability density f(x). We are interested in the values of x assumed by the individual elements of the sample. Suppose that we take ℓ samples of size n and find the following values for x:

1st sample:

⋮

jth sample:

⋮

ℓth sample:

We group the result of one sample into an n-dimensional vector

(6.1.1)

which can be considered the position vector in an n-dimensional sample space (Sect. 2.​1). Its probability density is

(6.1.2)

This function must fulfill two conditions in order for the sample to be random :

(a)

The individual x i must be independent, e.g., one must have

(6.1.3)

(b)

The individual marginal distributions must be identical and equal to the probability density f(x) of the population,

(6.1.4)

Comparing with (6.1.2) it is clear that there is a simple relation between a population and a sample only if these conditions are fulfilled. In the following we will mean by the word sample a random sample unless otherwise stated.

It should be emphasized that in the actual process of sampling it is often quite difficult to ensure randomness. Because of the large variety of applications, a general prescription cannot be given. In order to obtain reliable results from sampling, we have to take the utmost precautions to meet the requirements (6.1.3) and (6.1.4). Independence (6.1.3) can be checked to a certain extent by comparing the frequency distributions of the first, second, … elements of a large number of samples. It is very difficult, however, to ensure that the samples in fact come from a population with the probability density f(x). If the elements of the population can be numbered, it is often useful to use random numbers to select the elements for the sample.

We now suppose that the n elements of a sample are ordered according to the value of the variable, e.g., marked on the x axis, and we ask for the number of elements of the sample n x for which x < x, for arbitrary x. The function

(6.1.5)

takes on the role of an empirical distribution function. It is a step function that increases by 1∕n as soon as x is equal to one of the values x of an element of the sample. It is called the sample distribution function . It is clearly an approximation for F(x), the distribution function of the population, which it approaches in the limit n → ∞.

A function of the elements of a sample (6.1.1) is called a statistic . Since x is a random variable, a statistic is itself a random variable. The most important example is the sample mean ,

(6.1.6)

A typical problem of data analysis is the following. The general mathematical form of the probability density of the population is known. In radioactive decay, for example, the number of nuclei which decay before the time t = τ is $$N_{\tau } = N_{0}(1 -\exp (-\lambda \tau ))$$, if N ₀ nuclei existed at time t = 0. Here, however, the decay constant λ is, in general, not known. By taking a finite sample (measuring a finite number of decay times of individual nuclei) we want to determine the parameter λ as accurately as possible. Since such a task cannot be exactly solved, because the sample is finite, one speaks of the estimation of parameters. To estimate a parameter λ of a distribution function one uses an estimator

(6.1.7)

An estimator is said to be unbiased if for arbitrary sample size the expectation value of the (random) quantity S is equal to the parameter to be estimated:

(6.1.8)

An estimator is said to be consistent if its variance vanishes for arbitrarily large sample size, i.e., if

(6.1.9)

Often one can give a lower limit for the variance of an estimator of a parameter. If one finds an estimator S ₀ whose variance is equal to this limit, then one apparently has the “best possible” estimator. S ₀ is then said to be an efficient estimator for λ.

6.2 Samples from Continuous Populations: Mean and Variance of a Sample

The case of greatest interest in applications concerns a sample from an infinitely large continuous population described by the probability density f(x). The sample mean (6.1.6) is a random variable, as are all statistics. Let us consider its expectation value

(6.2.1)

This expectation value is equal to the expectation value of x. Since Eq. (6.2.1) holds for all values of n, the arithmetic mean of a sample is, as one would expect, an unbiased estimator for the mean of the population. The characteristic function of the random variable $$\bar{\mathsf{x}}$$ is

(6.2.2)

Next we are interested in the variance of $$\bar{\mathsf{x}}$$,

Since all of the x i are independent, all of the cross terms of the type $$E\{(\mathsf{x}_{i} -\hat{ x})(\mathsf{x}_{j} -\hat{ x})\}\;,\;i\neq j$$ (i.e., all of the covariances) vanish, and we obtain

(6.2.3)

One thus shows that $$\bar{\mathsf{x}}$$ is a consistent estimator for $$\hat{x}$$. The variance (6.2.3) is itself, however, not a random variable, and is therefore not directly obtainable by experiment. As a definition for the sample variance we could try using the arithmetic mean of squared differences

(6.2.4)

Its expectation value is

(6.2.5)

Hence one sees that the sample variance defined in this way is a biased estimator for the population variance, having an expectation value smaller than σ ²(x). We can see directly from (6.2.5), however, the size of the bias. We therefore change our definition (6.2.4) and write for the sample variance

(6.2.6)

This is now an unbiased estimator for σ ²(x). The value (n − 1) in the denominator appears at first to be somewhat strange. One must consider, however, that for n = 1 the sample mean is equal to the value x of the sole element of the sample ($$\mathsf{x} =\bar{ \mathsf{x}}$$) and that therefore the quantity (6.2.4) would vanish. That is related to the fact that in (6.2.4) – and also in (6.2.6) – the sample mean $$\bar{\mathsf{x}}$$ was used instead of the population mean $$\hat{x}$$, since the latter was not known. Part of the information contained in the sample first had to be used and was not available for the calculation of the variance. The effective number of elements available for calculating the variance is therefore reduced. This is taken into consideration by reducing the denominator of the arithmetic mean (6.2.4). The same line of reasoning is repeated quantitatively in Sect. 6.5.

If we substitute the estimator for the population variance (6.2.6) into (6.2.3), we obtain an estimator for the variance of the mean

(6.2.7)

The corresponding standard deviation can be considered to be the error of the mean

(6.2.8)

Of course we are also interested in the error of the sample variance (6.2.6). In Sect. 6.6 we will show that this quantity can be determined under the assumption that the population follows a normal distribution. We will use the result here ahead of time. The variance of s ² is

(6.2.9)

If we substitute the estimator (6.2.6) into the right-hand side for σ ² and take the square root, we obtain for the error of the sample variance

(6.2.10)

Finally we give explicit expressions for estimators of the sample standard deviation and its error. The first is simply the square root of the sample variance

(6.2.11)

The error of the sample standard deviation is obtained from (6.2.10) by error propagation, which gives

(6.2.12)

Example 6.1:

Computation of the sample mean and variance from data

Suppose one has n = 7 measurements of a certain quantity (e.g., the length of an object). Their values are 10. 5, 10. 9, 9. 2, 9. 8, 9. 0, 10. 4, 10. 7. The computation is made easier if one uses the fact that all of the measured values are near a = 10, i.e., they are of the form $$\mathsf{x}_{i} = a +\delta _{i}$$. The relation (6.1.6) then gives

with

We thus have $$\bar{\mathsf{x}} = 10+\varDelta = 10.07$$.

The sample variance is computed according to (6.2.6) to be

The result can be obtained either by the first or last line of the relation above. In the last line only one difference is computed, not n. The numbers to be squared, however, are usually considerably larger, and one must consider the problem of rounding errors. We therefore use the original expression

The sample standard deviation is s ≈ 0. 75. From (6.2.8), (6.2.10), and (6.2.12) we obtain finally $$\varDelta \bar{\mathsf{x}} = 0.28$$, Δ s ² = 0. 32, and Δ s = 0. 21. ■ 

Naturally one does not usually compute the sample mean and variance by hand, but rather by the class Sample and its methods.

6.3 Graphical Representation of Samples: Histograms and Scatter Plots

Table 6.1:

Values of resistance R of 100 individual resistors of nominal value 200 k$$\varOmega$$. The data are graphically represented in Fig. 6.1.

193.199

195.673

195.757

196.051

196.092

196.596

196.679

196.763

196.847

197.267

197.392

197.477

198.189

198.650

198.944

199.070

199.111

199.153

199.237

199.698

199.572

199.614

199.824

199.908

200.118

200.160

200.243

200.285

200.453

200.704

200.746

200.830

200.872

200.914

200.956

200.998

200.998

201.123

201.208

201.333

201.375

201.543

201.543

201.584

201.711

201.878

201.919

202.004

202.004

202.088

202.172

202.172

202.297

202.339

202.381

202.507

202.591

202.633

202.716

202.884

203.051

203.052

203.094

203.094

203.177

203.178

203.219

203.764

203.765

203.848

203.890

203.974

204.184

204.267

204.352

204.352

204.729

205.106

205.148

205.231

205.357

205.400

205.483

206.070

206.112

206.154

206.155

206.615

206.657

206.993

207.243

207.621

208.124

208.375

208.502

208.628

208.670

208.711

210.012

211.394

After the theoretical considerations of the last sections we now turn to some simple practical aspects of the analysis of sample data. An important tool for this is the representation of the data in graphical form.

A sample

which depends on a single variable x can be represented simply by means of tick marks on an x axis. We will call such a representation a one-dimensional scatter plot . It contains all the information about the sample. Table 6.1 contains the values $$\mathsf{x}_{1},\mathsf{x}_{2},\ldots,\mathsf{x}_{n}$$ of a sample of size 100, obtained from measuring the resistance R of 100 individual resistors of nominal value 200 k$$\varOmega$$. After obtaining the sample the measurements were ordered.

Fig. 6.1:

Representation of the data from Table 6.1 as (a) a one-dimensional scatter plot, (b) a bar diagram, (c) a step diagram, and (d) a diagram of measured points with error bars.

Figure 6.1a shows the corresponding scatter plot. Qualitatively one can estimate the mean and variance from the position and width of the clustering of tick marks.

Another graphical representation is usually better suited to visualize the sample by using the second dimension available on the paper. The x axis is used as abscissa and divided into r intervals

of equal width Δ x. These intervals are called bins. The centers of the bins have the x-values

On the vertical axis one plots the corresponding numbers of sample elements

that fall into bins $$\xi _{1},\xi _{2},\ldots,\xi _{r}$$. The diagram obtained in this way is called a histogram of the sample. This can be interpreted as a frequency distribution, since $$h_{k} = n_{k}/n$$ is a frequency, i.e., a measure of the probability p k to observe a sample element in the interval ξ k . For the graphical form of histograms, various methods are used. In a bar diagram the values n k are represented as bars perpendicular to the x axis on the x k values (Fig. 6.1b). In a step diagram the n k are represented as horizontal lines that cover the entire width ξ k of the interval. Neighboring horizontal lines are connected by perpendicular lines (Fig. 6.1c). The fraction of the area covering each interval ξ k of the x axis is then proportional to the number n k of the sample elements in the interval. (If one uses the area in the interval for the graphical representation of n k , then the bins can also have different widths.) In economics bar diagrams are most commonly used. (Sometimes one also sees diagrams in which, instead of bars, line segments are used to connect the tips of the bars. In contrast to the step diagram, the resulting figure does not have an area proportional to the sample size n.) In the natural sciences, step diagrams are more common.

In Sect. 6.8 we will determine that as long as the values n k are not too small, their statistical errors are given by $$\varDelta n_{k} = \sqrt{n_{k}}$$. In order to plot them on a graph, the observed values n k can be drawn as points with vertical error bars ending at the points $$n_{k} \pm \sqrt{n_{k}}$$ (Fig. 6.1d).

It is clear that the relative errors $$\varDelta n_{k}/n_{k} = 1/\sqrt{n_{k}}$$ decrease for increasing n k , i.e., for a sample of fixed size n they decrease for increasing bin width of the histogram. On the other hand, by choosing a larger interval width, one loses any finer structure of the data with respect to the variable x. The ability of a histogram to convey information therefore depends crucially on the appropriate choice of the bin width, usually found only after several attempts.

Example 6.2:

Histograms of the same sample with various choices of bin width

In Fig. 6.2 four histograms of the same sample are shown. The population is a Gaussian distribution, which is also shown as a continuous curve. This was scaled in such a way that the area under the histogram is equal to the area under the Gaussian curve. Although the information contained in the plot is greater for a smaller bin width – for vanishing bin width the histogram becomes a one dimensional scatter plot – one notices the similarity between the histogram and Gaussian curve much more easily for the larger bin width. This is because for the larger bin width the relative statistical fluctuations of the contents of individual bins are smaller. The individual steps of the histogram differ less from the curve. ■ 

Fig. 6.2:

Histogram of the same sample from a Gaussian distribution represented with four different bin widths.

Constructing a histogram from a sample is a simple programming task. Suppose the histogram has n x bins of width Δ x, with the first interval extending from x = x ₀ to $$x = x_{0} +\varDelta x$$. The contents of the histogram is put into an array hist, with the first bin in hist[0], the second bin in hist[1], etc. The histogram is then specified in the computer by the array hist and the three values x ₀, Δ x, and n x . The class Histogram permits construction and administration of a histogram.

Graphical display of histograms can be accomplished using methods of the class DatanGraphics (Appendix F). With them the user can freely adjust all of the parameters that determine the appearance of the plot, such as the page format, scale factors, colors, line thickness, etc. Often it is convenient to use the class GraphicsWithHistogram, which does not allow for this freedom but by a single call gives rise to the graphical output of a histogram, stored in the computer.

A histogram allows a first direct look at the nature of the data. It answers questions such as “Are the data more or less distributed according to a Gaussian?” or “Do there exist points exceptionally far away from the average value?”. If the histogram leads one to conclude that the population is distributed according to a Gaussian, then the mean and standard deviation can be estimated directly from the plot. The mean is the center of gravity of the histogram. The standard deviation is obtained as in the following example.

Example 6.3:

Full width at half maximum (FWHM)

If the form of histogram allows one to assume that the represented sample originates from a Gaussian distribution, then one can draw by hand a Gaussian bell curve that follows the histogram as closely as possible. The position of the maximum is a good estimator for the mean of the sample. One then draws a horizontal line at half of the height of the maximum. This crosses the bell curve at the points x a and x b . The quantity

is called the full width at half maximum (FWHM). One can easily determine for a Gaussian distribution the simple relation

(6.3.1)

between the standard deviation and FWHM. This expression can be used to estimate the standard deviation of a sample when f is obtained from a histogram. ■ 

We now use the Monte Carlo method (Chap. 4) together with histograms in order to illustrate the concepts of mean, standard deviation, and variance of a sample, and their errors, as introduced in Sect. 6.2.

Example 6.4:

Investigation of characteristic quantities of samples from a Gaussian distribution with the Monte Carlo method

We generate successively 1000 samples of size N = 100 from the standard normal distribution, e.g., compute the mean $$\bar{\mathsf{x}}$$, variance s ², and standard deviation s for each sample as well as the errors $$\varDelta \bar{\mathsf{x}}$$, Δ s ², and Δ s, with the methods of the classSample. We then produce for each of the six quantities a histogram (Fig. 6.3), containing 1000 entries. Since each of the quantities is defined as the sum of many random quantities, we expect in all cases that the histograms should resemble Gaussian distributions. From the histogram for $$\bar{\mathsf{x}}$$, we obtain a full width at half maximum of about 0.25, and hence a standard deviation of approximately 0.1. Indeed the histogram for $$\varDelta \bar{\mathsf{x}}$$ shows an approximately Gaussian distribution with mean value $$\varDelta \bar{\mathsf{x}} = 0.1$$. (From the width of this histogram one could determine the error of the error $$\varDelta \bar{\mathsf{x}}$$ of the mean $$\bar{\mathsf{x}}$$!) From both histograms one obtains a very clear impression of the meaning of the error $$\varDelta \bar{\mathsf{x}}$$ of the mean $$\bar{\mathsf{x}}$$ for a single sample, as computed according to (6.2.8). It gives (within its error) the standard deviation of the population, from which the sample mean $$\bar{\mathsf{x}}$$ comes. If many samples are successively taken (i.e., if the experiment is repeated many times) then the frequency distribution of the values $$\bar{\mathsf{x}}$$ follows a Gaussian distribution about the population mean with standard deviation $$\varDelta \bar{\mathsf{x}}$$. The corresponding considerations also hold for the quantities s ², s, and their errors Δ s and Δ s ². ■ 

Fig. 6.3:

Histograms of the quantities $$\bar{\mathsf{x}}$$, $$\varDelta \bar{\mathsf{x}}$$, s, Δ s, s ², and Δ s ² from 1000 samples of size 100 from the standard normal distribution.

If the elements of the sample depend on two random variables x and y, then one can construct a scatter plot, where each element is represented as a point in a Cartesian coordinate system for the variables x and y. Such a two-dimensional scatter plot provides useful qualitative information about the relationship between the two variables.

The class GraphicsWith2DScatterDiagram generates such a diagram by a single call. (A plot in the format A5 landscape is generated, into which the scatter diagram, itself in square format, is fitted. If another plot format or edge ratio of the diagram is desired, the class has to be adapted accordingly.)

Example 6.5:

Two-dimensional scatter plot: Dividend versus price for industrial stocks

Table 6.2 contains a list of the first 10 of 226 data sets, in which the dividend in 1967 (first column) and share price on December 31, 1967 (second column) are given for a number of industrial stocks. The third column shows the company name for all German corporations worth more than 10 million marks. The scatter plot of the number pairs (share price, dividend) is shown in Fig. 6.4.

Fig. 6.4:

Scatter plot of dividend D versus share price P for industrial stocks.

As expected we see a strong correlation between dividend and share price. One can see, however, that the dividend does not grow linearly with the price. It appears that factors other than immediate profit determine the price of a stock.

Also shown are histograms for the share price (Fig. 6.5) and dividend (Fig. 6.6) which can be obtained as projections of the scatter plot onto the abscissa and ordinate. One clearly observes a non-statistical behavior for the dividends. It is given as a percent of the nominal value and is therefore almost always integer. One sees that even numbers are considerably more frequent than odd numbers. ■ 

Table 6.2:

Dividend, share price of a stock, and company name.

12.

133.

ACKERMANN-GOEGGINGEN

08.

417.

ADLERWERKE KLEYER

17.

346.

AGROB AG FUER GROB U. FEINKERAMIK

25.

765.

AG.F.ENERGIEWIRTSCHAFT

16.

355.

AG F. LICHT- U. KRAFTVERS.,MCHN.

20.

315.

AG.F. IND.U.VERKEHRSW.

08.

138.

AG. WESER

16.

295.

AEG ALLG.ELEKTR.-GES.

20.

479.

ANDREAE-NORIS ZAHN

10.

201.

ANKERWERKE

Fig. 6.5:

Histogram of the price of industrial stocks.

Fig. 6.6:

Histogram of the dividend of industrial stocks.

6.4 Samples from Partitioned Populations

It is often advantageous to divide a population G (e.g., all of the students in Europe) into various subpopulations G ₁, G ₂, …, G t (students at university 1, 2, …, t). Suppose a quantity of interest x follows in the various subpopulations the probability densities f ₁(x), f ₂(x), …, f t (x). The distribution function corresponding to f i (x) is then

(6.4.1)

This is equal to the conditional probability for x < x given that x is contained in the subpopulation G i . The rule of total probability (2.​3.​4) provides the relationship between the various F i (x) and the distribution function F(x) for G,

i.e.,

(6.4.2)

Correspondingly one has for the probability density

(6.4.3)

If we now abbreviate P(x ∈ G i ) by p i , then one has

(6.4.4)

The population mean is thus the mean of the individual means of the subpopulations, each weighted by probability of its subpopulation. For the population variance one obtains

All cross terms vanish since the x i are independent, leading to

or

(6.4.5)

One thus obtains the weighted mean of a sum of two terms. The first gives the dispersion of a subpopulation, the second gives the quadratic deviation of the mean of this subpopulation from the mean of the whole population.

Having discussed separating a population into parts, we now take from each subpopulation G i a sample of size n i (with $$\sum _{i=1}^{t}n_{i} = n$$) and examine the arithmetic mean of the total partitioned sample

(6.4.6)

with the expectation value and variance

(6.4.7)

(6.4.8)

Using (6.2.3) this is finally

(6.4.9)

One would obtain the same result by application of the law of error propagation (3.​8.​7) to Eq. (6.4.6).

It is clear that the arithmetic mean $$\bar{\mathsf{x}}_{p}$$ cannot in general be an estimator for the sample mean $$\hat{x}$$, since it depends on the arbitrary choice of the size n i of the samples from the subpopulations. A comparison of (6.4.7) with (6.4.4) shows that this is only true for the special case $$p_{i} = n_{i}/n$$.

The population mean $$\hat{x}$$ can be estimated in the following way. One first determines the means $$\bar{\mathsf{x}}_{i}$$ for the subpopulations, and constructs then the expression

(6.4.10)

in analogy to Eq. (6.4.4). By error propagation one obtains for the variance of $$\tilde{\mathsf{x}}$$

(6.4.11)

Example 6.6:

Optimal choice of the sample size for subpopulations

In order to minimize the variance $${\sigma }^{2}(\tilde{\mathsf{x}})$$, we cannot simply differentiate the relation (6.4.11) with respect to all n i , since the n i must satisfy a constraint, namely

(6.4.12)

We must therefore use the method of Lagrange multipliers , by multiplying Eq. (6.4.12) with a factor μ, adding this to Eq. (6.4.11), and finally setting the partial derivatives of the n i with respect to μ equal to zero:

(6.4.13)

(6.4.14)

From (6.4.13) we obtain

Together with (6.4.14) this gives

and therefore

(6.4.15)

The result (6.4.15) states that the sizes n i of the samples from the subpopulations i should be chosen in such a way that they are proportional to the probability p i of subpopulation i, weighted with the corresponding standard deviation.

As an example assume that a scientific publishing company wants to estimate the total amount spent for scientific books by two subpopulations: (1) students and (2) scientific libraries. Further, we will assume that there are 1000 libraries and 10⁶ students in the population and that the standard deviation of the money spent by students is $ 100, and for libraries (which are of greatly differing sizes) $ 3 ⋅ 10⁵. We then have

and from (6.4.15)

Note that the result does not depend on the means of the partial populations. The quantities p i , x i , and σ i are in general unknown. They must first be estimated from preliminary samples. ■ 

The discussion of subpopulations will be taken up again in Chap. 11.​

6.5 Samples Without Replacement from Finite Discrete Populations. Mean Square Deviation. Degrees of Freedom

We first encountered the concept a sample in connection with the hypergeometric distribution (Sect. 5.​3). There we determined that the independence of the individual sample elements was lost by the process of taking elements without replacing them from a finite (and hence discrete) population. We are therefore no longer dealing with genuine random sampling, even if no particular choice among the remaining elements is made.

To discuss this further let us introduce the following notation. Suppose the population consists of N elements y ₁, y ₂, …, y N . From it we take a sample of size n with the elements $$\mathsf{x}_{1},\mathsf{x}_{2},\ldots,\mathsf{x}_{n}$$. (In the hypergeometric distribution, the y j and hence the x i could only take on the values 0 and 1.)

Since it is equally probable for each of the remaining elements y j to be chosen, we obtain for the expectation value

(6.5.1)

Although $$\hat{y}$$ is not a random variable, this expression is the arithmetic mean of a finite number of elements of the population. A definition of the population variance encounters the difficulties discussed at the end of Sect. 6.2. We define it in analogy to (6.2.6) as

(6.5.2)

Let us now consider the sum of squares ,

(6.5.3)

Since we have not constrained the population in any way, the y j can take on all possible values. Therefore the first element in the sum in (6.5.2) can also take on any of the possible values. The same holds for the 2nd, 3rd, …, (N − 1)th terms summed. The Nth term in the sum is then, however, fixed, since

(6.5.4)

We say that the number of degrees of freedom of the sum of squares (6.5.3) is N − 1. One can illustrate this connection geometrically. We consider the case $$\bar{y} = 0$$ and construct an N-dimensional vector space with the y j . The quadratic sum (6.5.3) is then the square of the absolute value of the position vector in this space. Because of the equation of constraint (6.5.4) the tip of the position vector can only move in a space of dimension (N − 1). In mechanics the dimension of such a constrained space is called the number of degrees of freedom. This is sketched in Fig. 6.7 for the case N = 2. Here the position vector is constrained to lie on the line $$y_{2} = -y_{1}$$.

Fig. 6.7:

A sample of size two gives a sum of squares with one degree of freedom.

A sum of squares divided by the number of degrees of freedom, i.e., an expression of the form (6.5.2) is called a mean square or to be more complete, since we are dealing with the differences of the individual values from the expectation or mean value, mean square deviation . The square root of this expression, which is then a measure of the dispersion, is called the root mean square (RMS) deviation.

We now return to the sample $$\mathsf{x}_{1},\mathsf{x}_{2},\ldots,\mathsf{x}_{n}$$. For simplicity of notation we will introduce the Kronecker symbol , which describes the selection procedure for the sample. It is defined as

(6.5.5)

In particular one has

(6.5.6)

Since the selection of any of the y j as the ith element is equally probable, one has

(6.5.7)

Since δ i j describes a random procedure, it is clearly a random variable itself. Its expectation value is found from Eq. (3.​3.​2) (where n = 2, x ₁ = 0, x ₂ = 1) to be

(6.5.8)

If now one element x i of the sample is determined, one then has only (N − 1) selection possibilities out of the population for a further element, e.g., x k . That is,

(6.5.9)

Since the sample is taken without replacement, one has j ≠ ℓ, i.e.,

(6.5.10)

Similarly one has

(6.5.11)

since two different elements of the population cannot simultaneously occur as the ith element of the sample.

We consider now the expectation value of x ₁,

(6.5.12)

Since x ₁ is in not in any way special, the expectation values of all elements of the sample, and thus also of their arithmetic mean, have the same value

(6.5.13)

The arithmetic mean of the sample is thus an unbiased estimator for the population mean.

Next we consider the sample variance

(6.5.14)

By means of a somewhat longer calculation it can be shown that the expectation value is

(6.5.15)

The sample variance is thus an unbiased estimator for the population variance.

The variance of the mean is also of interest:

$$E(\bar{\mathsf{x}}) =\bar{ y}$$ is, however, a fixed, not a random quantity, whereas $$\bar{\mathsf{x}}$$ depends on the individual sample, and is hence a random variable. One therefore has

(6.5.16)

Comparing with the case of an infinite continuous sample (6.2.3) one sees the additional factor $$(1 - n/N)$$. This corresponds to the fact that the variance of $$\bar{\mathsf{x}}$$ vanishes in the case n = N, where the “sample” contains the entire population and where one has exactly $$\bar{\mathsf{x}} =\bar{ y}$$.

6.6 Samples from Gaussian Distributions: χ ²-Distribution

We return now to continuously distributed populations and consider in particular a Gaussian distribution with mean a and variance σ ². According to (5.​7.​7), the characteristic function of such a Gaussian distribution is

(6.6.1)

We now take a sample of size n from the population. The characteristic function of the sample mean was given in (6.2.2) in terms of the characteristic function of the population. From this we have

(6.6.2)

If we consider $$(\bar{\mathsf{x}} - a) = (\bar{\mathsf{x}} -\hat{ x})$$ in place of x, then one obtains

(6.6.3)

This is again the characteristic function of a normal distribution, but with a different variance,

(6.6.4)

For the simple case of a standard Gaussian distribution (a = 0, σ ² = 1) we have

(6.6.5)

We take a sample from this distribution

but we are interested in particular in the sum of the squares of the sample elements,

(6.6.6)

We want to show that the quantity x ² follows the distribution function¹

(6.6.7)

where

(6.6.8)

The quantity n is called the number of degrees of freedom.

We first introduce the abbreviation

(6.6.9)

and determine the probability density to be

(6.6.10)

For two degrees of freedom, the probability density is clearly an exponential function. First we want to prove what was claimed by (6.6.7) for one degree of freedom ($$\lambda = 1/2$$). Thus we ask for the probability that $${\mathsf{x}}^{2} {<\chi }^{2}$$, or rather, that $$-\sqrt{{\chi }^{2}} < \mathsf{x} < +\sqrt{{\chi }^{2}}$$. This is

Setting x ² = u, $$\mathrm{d}u = 2x\,\mathrm{d}x$$, we obtain directly

(6.6.11)

To prove the general case we first find the characteristic function of the χ ²-distribution to be

(6.6.12)

or with $${(1/2 -\mathrm{i \!}t)\chi }^{2} =\nu$$,

The integral on the right side is, according to (D.1.1), equal to Γ(λ). One therefore has

(6.6.13)

If we now consider the case of a second distribution with λ′, then one has

Since the characteristic function of a sum is equal to the product of the characteristic functions, one has the following important theorem:

The sum of two independent χ ² variables with

n ₁, n ₂ degrees of freedom follows itself a χ ²-distribution

with $$n = n_{1} + n_{2}$$ degrees of freedom.

This theorem can now be used to easily generalize the claim (6.6.7), proven up to now only for n = 1. The proof follows from the fact that the individual terms of the sum of squares are independent and therefore (6.6.6) can be treated as the sum of n different χ ² variables, each with one degree of freedom.

In order to obtain the expectation value and variance of the χ ²-distribution, we use the characteristic function, whose derivatives (5.​5.​7) give the central moments. We obtain

(6.6.14)

and

(6.6.15)

The expectation value of the χ ²-distribution is thus equal to the number of degrees of freedom, and the variance is two times larger. Figure 6.8 shows the probability density of the χ ² distribution for various values of n. One sees [as can be directly seen also from (6.6.10)] that for χ ² = 0, the function diverges for n = 1, is equal to 1∕2 for n = 2, and vanishes for n ≥ 3. A table of the χ ²-distribution is provided in the appendix (Table I.6).

Fig. 6.8:

Probability density of χ ² for the number of degrees of freedom n = 1, 2, …, 10. The expectation value E(χ ²) = n moves to the right as n increases.

The χ ²-distribution is of great significance in many applications, where the quantity χ ² is used as a measure of confidence in a certain result. The smaller the value of χ ², the greater is the confidence in the result. (After all, χ ² was defined as the sum of squares of deviations of elements of a sample from the population mean. See Sect. 8.​7.​) The distribution function

(6.6.16)

gives the probability that the random variable x ² is not larger than χ ². In practice, one frequently uses the quantity

(6.6.17)

as a measure of confidence in a result. W(χ ²) is often called the confidence level . W(χ ²) is large for small values of χ ² and falls with increasing χ ². The distribution function (6.6.16) is shown in Fig. 6.9 for various numbers of degrees of freedom n. The inverse function, which gives the quantiles of the χ ² distribution,

Fig. 6.9:

Distribution function for χ ² for the number of degrees of freedom n = 1, 2, …, 20. The function for n = 1 is the curve at the far left, and the function for n = 20 is at the far right.

(6.6.18)

is used especially often in “hypothesis testing” (see Sect. 8.​7). It is tabulated in the appendix (Table I.7).

Up to now we have restricted ourselves to the case where the population is described by the standard normal distribution. Usually, however, one has a normal distribution in general form with mean a and variance σ ². Then the sum of squares (6.6.6) is clearly no longer distributed according to the χ ²-distribution. One immediately obtains, however, a χ ²-distribution by considering the quantity

(6.6.19)

This result follows directly from Eq. (5.​8.​4).

If the expectation values a i and variances σ i of the individual variables are different, then one has

(6.6.20)

Finally if the n variables are not independent, but are described by a joint normal distribution (5.​10.​1) with the expectation values given by the vector $$\mathbf{a}$$ and the covariance matrix $$C = {B}^{-1}$$, then one has

(6.6.21)

6.7 χ ² and Empirical Variance

In Eq. (6.2.6) we found that

(6.7.1)

is a consistent, unbiased estimator for the variance σ ² of a population. Let the x i be independent and normally distributed with standard deviation σ. We want to show that the quantity

(6.7.2)

follows the χ ²-distribution with $$f = n - 1$$ degrees of freedom. We first carry out an orthogonal transformation of the n variables x i (see Sect. 3.​8):

(6.7.3)

One can verify that this transformation is in fact orthogonal, i.e., that

(6.7.4)

Since a sum or difference of independent normally distributed quantities is again itself normally distributed, all of the y i are normally distributed. The factors in (6.7.3) ensure that the y i have mean values of zero and standard deviations σ.

From (6.7.1) and (6.7.2) one then has

This expression is a sum of only (n − 1) independent squared terms. A comparison with (6.6.19) shows that the quantity (6.7.2) in fact follows a χ ²-distribution with (n − 1) degrees of freedom.

The squared terms $${(\mathsf{x}_{i} -\bar{\mathsf{x}})}^{2}$$ are not linearly independent. One has the following relation between them:

One can show that every additional relation between the squared terms reduces the number of degrees of freedom by one. Later we will make frequent use of this result, which we only state here without proof.

6.8 Sampling by Counting: Small Samples

Samples are often obtained in the following way. One draws n elements from a population, checks if they possess a given characteristic and accepts only those k elements into the sample that have the characteristic. The remaining n −k elements are rejected, i.e., their properties are not recorded. This approach thus becomes the counting of k out of n elements drawn.

This approach corresponds exactly to selecting a sample according to a binomial distribution. The parameters p and q of this distribution correspond then to the occurrence or non-occurrence of the property in question. As will be shown in Example 7.5,

(6.8.1)

is the maximum likelihood estimator of the parameter p. The variance of S is

(6.8.2)

By using (6.8.1) it can be estimated from the sample by

(6.8.3)

We define the error Δ k as

(6.8.4)

By using (6.8.3) we obtain

(6.8.5)

The error Δ k only depends on the number of elements counted and on the size of the sample. It is called the statistical error . A particularly important case is that of small k, or more precisely, the case k ≪ n. In this limit we can define λ = np and following Sect. 5.​4 consider the counted number k as a single element of a sample taken from a Poisson distributed population with parameter λ. From (6.8.1) and (6.8.5) we obtain

(6.8.6)

(6.8.7)

(This can be derived by using the result of Example 7.4 with N = 1.) The result (6.8.7) is often written in an actually incorrect but easy to remember form,

which is read: The statistical error of the counted number k is $$\sqrt{ \mathsf{k}}$$.

In order to interpret the statistical error $$\varDelta \lambda = \sqrt{\mathsf{k}}$$ we must examine the Poisson distribution somewhat more closely. Let us begin with the case where k is not too small (say, k > 20). For large values of λ the Poisson distribution becomes a Gaussian distribution with mean λ and variance σ ² = λ. This can be seen qualitatively from Fig. 5.6. As long as k is not too small, i.e., k ≫ 1, we can then treat the Poisson distribution in k with parameter λ as a normal distribution in x with mean λ and variance σ ² = λ. The discrete variable k is then replaced by the continuous variable x. The probability density of x is

(6.8.8)

The observation of k events corresponds to observing once the value of the random variable x = k.

With the help of the probability density (6.8.8) we now want to determine the confidence limits at a given confidence level $$\beta = 1-\alpha$$ in such a way that

(6.8.9)

That is, one requires that the probability that the true value of λ is contained within the confidence limits λ ₋ and λ ₊ be equal to the confidence level 1 −α. The limiting cases $$\lambda =\lambda _{-}$$ and $$\lambda =\lambda _{+}$$ are depicted in Fig. 6.10. They are determined such that

(6.8.10)

One clearly has

(6.8.11)

and correspondingly

(6.8.12)

Here ϕ ₀ and ψ ₀ are the probability density and distribution function of the standard normal distribution introduced in Sect. 5.​8.​ By using the inverse function Ω of the distribution function ψ ₀ [see Eq. (5.​8.​8)], one obtains

Fig. 6.10:

Normal distribution with mean λ and standard deviation σ for $$\lambda =\lambda _{-}$$ and $$\lambda =\lambda _{+}$$.

(6.8.13)

Because of (5.​8.​10) one has $$\varOmega (1 -\alpha /2) =\varOmega ^{\prime}(1-\alpha )$$ and because of the symmetry of the function Ω, $$\varOmega (1 -\alpha /2) = -\varOmega (\alpha /2)$$. Further, since α < 1, one has $$\varOmega (1 -\alpha /2) > 0$$, Ω(α∕2) < 0. From this we finally obtain

(6.8.14)

According to (6.8.6), k is the best estimator for λ. Since σ ² = λ, the best estimator for σ is given by $$\mathsf{s} = \sqrt{\mathsf{k}}$$. Since we have assumed that k ≫ 1, the uncertainty in s is significantly smaller than the uncertainty in k. We can therefore substitute x = k and $$\mathsf{s} = \sqrt{\mathsf{k}}$$ in (6.8.9) and obtain for the confidence interval with confidence level 1 −α

(6.8.15)

For $$1-\alpha = 68.3\,\%$$ we find from Sect. 5.​8 or Table I.5 that Ω′(α) = 1. What is usually reported,

which was already the result of (6.8.6) and (6.8.7), thus gives the confidence limits at the confidence level of 68. 3 %, but only for the case k ≫ 1. For the confidence level of 90 %, i.e., for α = 0. 1, we find Ω′(0. 1) = 1. 65 and for the confidence level 99 % one has Ω′(0. 01) = 2. 57.

For very small values of k, one can no longer replace the Poisson distribution with the normal distribution. We follow therefore reference [25]. We start again from Eq. (6.8.10), but use, instead of the probability density (6.8.8) for the continuous random variable x with fixed parameter λ, the Poisson probability for observing the discrete random variable n for a given λ,

(6.8.16)

For the observation k we now determine the confidence limits λ ₋ and λ ₊, which fulfill (6.8.10) with x = n (Fig. 6.11) and obtain in analogy to (6.8.11) and (6.8.12)

Fig. 6.11:

Poisson distribution with parameter λ for $$\lambda =\lambda _{-}$$ and $$\lambda =\lambda _{+}$$.

or

(6.8.17)

(6.8.18)

Here,

is the distribution function of the Poisson distribution.

In order to obtain numerical values for the confidence limits λ ₊ and λ ₋, we solve Eqs. (6.8.17) and (6.8.18). That is, we must construct the inverse function of the Poisson distribution for fixed k and given probability P (in our case α∕2 and $$1 -\alpha /2$$),

(6.8.19)

This is done numerically with the method StatFunct.quantile Poisson. The function (6.8.19) is given in Table I.1 for frequently occurring values of P.

For extremely small samples one is often only interested in an upper confidence limit at confidence level $$\beta = 1-\alpha$$. This is obtained by requiring

(6.8.20)

instead of (6.8.10). Thus one has

(6.8.21)

For the extreme case k = 0, i.e., for a sample in which no event was observed, one obtains the upper limit λ (up) by inverting α = F(1; λ (up)). The upper limit then has the following meaning. If the true value of the parameter were in fact λ = λ (up) and if one were to repeat the experiment many times, then the probability of observing at least one event is β. The observation k = 0 is then expressed in the following way: One has λ < λ (up) with a confidence level of 1 −α. From Table I.1 one finds that k = 0 corresponds to λ < 2. 996 ≈ 3 at a confidence level of 95 %.

Example 6.7:

Determination of a lower limit for the lifetime of the proton from the observation of no decays

As already mentioned, the probability for the decay of a radioactive nucleus with the time t is

Here τ is the mean lifetime of the nucleus. For t ≪ τ the expression simplifies to

For a total of N nuclei one expects that

nuclei will decay within the time t. The mean lifetime τ is obtained by counting the number of such decays. If one observes k decays from a total of N nuclei in a time t, then one obtains as the measured value of τ

Of particular interest is the mean lifetime of the proton, one of the primary building blocks of matter. In experiments recently carried out with great effort, one observes large numbers of protons with detectors capable of detecting each individual decay. Up to now, not a single decay has been seen. According to Table I, the true expected number of decays λ does not exceed three (at a confidence level of 95 %). One has therefore

at this confidence level. Typical experimental values are t = 0. 3 years, N = 10³³, i.e.,

The proton can therefore be considered as stable even over cosmological time scales, if one considers that the age of the universe is estimated to be only around 10¹⁰ years. ■ 

6.9 Small Samples with Background

In many experiments one is faced with the following situation. For the detected events one cannot determine whether they belong to the type that is actually of interest (signal events ) or to another type (background events ). For the expected number of events in the experiment one then has a Poisson distribution with the parameter $$\lambda =\lambda _{S} +\lambda _{B}$$. Here λ S is the sought after parameter of the number of signal events, and λ B is the parameter for the background events, which must of course be known if one wants to obtain information about λ S . (In an experiment as in Example 6.7, one might have, for example, an admixture of radioactive nuclei whose decays cannot be distinguished from those of the proton. If the number of such nuclei and their lifetime is known, then λ B can be computed.)

We are now tempted to simply take the results of the last section, to determine the confidence limits λ ± and the upper limit λ (up) and to set $$\lambda _{S\pm } =\lambda _{\pm }-\lambda _{B}$$, $$\lambda _{S}^{(\mathrm{up})} {=\lambda }^{(\mathrm{up})} -\lambda _{B}$$. This procedure can, however, lead to nonsensical results. (As seen in Example 6.7, one has λ (up) = 3 at a confidence level of 95 %, for k = 0. For λ B  = 4, k = 0 we would obtain $$\lambda _{S}^{(\mathrm{up})} = -1$$, although a value λ S  < 0 has no meaning.)

The considerations up to now are based on the following. The probability for observing n events, $$n = n_{S} + n_{B}$$, is

(6.9.1)

and the probabilities to observe n S signal events, and n B background events are

(6.9.2)

(6.9.3)

The validity of (6.9.1) was shown in Example 5.5 with the help of the characteristic function, starting from the independence of the two Poisson distributions (6.9.2) and (6.9.3). One can also obtain them directly by summation of all products of the probabilities (6.9.2) and (6.9.3) that lead to $$n = n_{S} + n_{B}$$, by application of (B.4) and (B.6),

The difficulties explained above are overcome with a method developed by ZECH [26]. In an experiment in which k events are recorded, the number of background events cannot simply be given by (6.9.3), since from the result of the experiment it is known that n B  ≤ k. One must therefore replace (6.9.3) by

(6.9.4)

This distribution is normalized to unity in the region 0 ≤ n B  ≤ k. In a corresponding way the distribution

(6.9.5)

takes the place of (6.9.1).

In this way one obtains in analogy to (6.8.17) and (6.8.18) for the limits of the confidence interval $$\lambda _{S-}\leq \lambda _{S} \leq \lambda _{S+}$$ at a confidence level of 1 −α

(6.9.6)

(6.9.7)

Here,

(6.9.8)

is the distribution function of the renormalized distribution (6.9.4). If only an upper limit at confidence level 1 −α is desired, then one clearly has in analogy to (6.8.21)

(6.9.9)

Table 6.3 gives some numerical values computed with the methods of the class SmallSample. Note that for k = 0, Eq. (6.9.7) has no meaning, so that λ S− cannot be defined. In this case Eq. (6.9.6) and also λ S+ are not meaningful. (In the table, however, the values for λ S− and λ S+ are shown as computed by the program. This sets $$\lambda _{S-} = 0$$ and computes λ S+ according to (6.9.6).)

Table 6.3:

Limits λ S− and λ S+ of the confidence interval and upper confidence limits λ S (up), for various values of λ B and various very small sample sizes k for a fixed confidence level of 90 %.

β = 0. 90

k

λ B

λ S−

λ S+

λ S (up)

0

0.0

0.000

2.996

2.303

0

1.0

0.000

2.996

2.303

0

2.0

0.000

2.996

2.303

1

0.0

0.051

4.744

3.890

1

1.0

0.051

4.113

3.272

1

2.0

0.051

3.816

2.995

2

0.0

0.355

6.296

5.322

2

1.0

0.100

5.410

4.443

2

2.0

0.076

4.824

3.877

3

0.0

0.818

7.754

6.681

3

1.0

0.226

6.782

5.711

3

2.0

0.125

5.983

4.926

4

0.0

1.366

9.154

7.994

4

1.0

0.519

8.159

7.000

4

2.0

0.226

7.241

6.087

5

0.0

1.970

10.513

9.275

5

1.0

1.009

9.514

8.276

5

2.0

0.433

8.542

7.306

6.10 Determining a Ratio of Small Numbers of Events

Often a number of signal events k is measured and compared to a number of reference events d. One is interested in the true value r of the ratio of the number of signal events to the number of reference events, or more precisely, the ratio of the probability to observe a signal event to that of a reference event. As an estimator for this ratio one clearly uses

We now ask for the confidence limits of r. If k and d are sufficiently large, then they may be approximated as Gaussian variables with standard deviations $$\sigma _{k} = \sqrt{\mathsf{k}}$$, $$\sigma _{d} = \sqrt{\mathsf{d}}$$. Then according to the law of error propagation one has

(6.10.1)

If in addition one has d ≫ k, then Δ r is simply

(6.10.2)

If the requirements for the validity of (6.10.1) or even of (6.10.2) are not fulfilled, i.e., if k and d are small numbers, then one must use considerations developed by JAMES and ROOS [23]. Clearly when one observes an individual event, the probability that it is a signal event is given by

(6.10.3)

and the probability that it is a reference event is

(6.10.4)

In an experiment in which a total of $$\mathsf{N} = \mathsf{k} + \mathsf{d}$$ are observed, the probability that exactly n signal events are present is given by a binomial distribution (5.​1.​3). This is

(6.10.5)

The probability to have n < k is then

(6.10.6)

with

(6.10.7)

i.e., the distribution function of the binomial distribution. To determine the limits r ₋ and r ₊ of the confidence interval at the confidence level $$\beta = 1-\alpha$$, we use in analogy to (6.8.17) and (6.8.18)

(6.10.8)

(6.10.9)

If one only seeks an upper limit at the confidence level $$\beta = 1-\alpha$$, it can be obtained from [see Eq. (6.8.21)]

(6.10.10)

The quantities r ₊, r ₋, and r (up) can be computed for given values of k, d, and β with the class SmallSample.

6.11 Ratio of Small Numbers of Events with Background

By combining as done by SWARTZ [24] the ideas of Sects. 6.9 and 6.10, one can deal with the following situation. In an experiment one has three types of events: signal, background, and reference. Signal and background events cannot be distinguished from each other. Suppose in the experiment one has detected a total of k signal and background events and d reference events. Let us label with r S and r B the true values (in the sense of the definition at the beginning of the previous section) of the ratios of the numbers of signal to reference and background to reference events. Then the probabilities that a randomly selected event is signal or background, p S and p B , are

(6.11.1)

The probability that it is a reference event is then

(6.11.2)

If one has a total of $$\mathsf{N} = \mathsf{k} + \mathsf{d}$$ events in the experiment, then the individual probabilities that one has exactly n S signal events, n B background events, and $$n_{R} = \mathsf{N} - n_{S} - n_{B}$$ reference events are

(6.11.3)

(6.11.4)

(6.11.5)

Since there are now three mutually exclusive types of events, one has instead of a binomial distribution (6.10.5) a trinomial distribution, i.e., a multinomial distribution (5.​1.​10) with ℓ = 3. The probability that in an experiment with a total of N events one has exactly n S signal, n B background, and $$\mathsf{N} - n_{S} - n_{B}$$ reference events is therefore

(6.11.6)

Here, however, we have not taken into consideration that the number of background events cannot be greater than k. In a manner similar to (6.9.4) one uses

(6.11.7)

in place of f B . This replacement must also be made in (6.11.6), which gives

(6.11.8)

The probability to have n S signal events regardless of the number of background events is

and finally the probability to have n S  ≤ k is

Since r B was assumed to be known, the quantity F′ for a given k depends only on r S . Similar to (6.9.6) and (6.9.7) one can determine the limits r S+ and r S− of the confidence region for r S with confidence level $$\beta = 1-\alpha$$ from the following requirement:

(6.11.9)

(6.11.10)

If one only wants an upper limit with confidence level $$\beta = 1-\alpha$$, this can be found according to (6.9.9) from

(6.11.11)

Table 6.4 contains some numerical values computed with methods of the class SmallSample. For k = 0, however, (6.11.10) and hence also r S− have no meaning. Similarly for (6.11.9), r S+ for k = 0 is not meaningful. (In the table, however, values for r S− and r S+ are given for k = 0 as computed by the program. For k = 0 this sets $$r_{S-} = 0$$ and determines r S+ according to (6.11.9).)

Table 6.4:

Limits r S− and r S+ of the confidence interval and upper confidence limit r S (up) for various values of r B and various very small values of k for a fixed number of reference events d and fixed confidence level of 90 %.

k

r B

r S−

r S+

r S (up)

0

0.0

0.000

0.349

0.259

0

0.1

0.000

0.349

0.259

0

0.2

0.000

0.349

0.259

1

0.0

0.005

0.573

0.450

1

0.1

0.005

0.502

0.382

1

0.2

0.005

0.464

0.348

2

0.0

0.034

0.780

0.627

2

0.1

0.010

0.686

0.535

2

0.2

0.007

0.613

0.467

3

0.0

0.077

0.979

0.799

3

0.1

0.020

0.880

0.701

3

0.2

0.012

0.788

0.612

4

0.0

0.127

1.174

0.968

4

0.1

0.044

1.074

0.869

4

0.2

0.019

0.976

0.771

5

0.0

0.180

1.367

1.135

5

0.1

0.085

1.267

1.035

5

0.2

0.034

1.167

0.936

6.12 Java Classes and Example Programs

Java Classes Referring to Samples

Sample contains methods computing characteristic parameters of a sample: mean, variance and standard deviation as well as the errors of these quantities.

SmallSample contains methods computing the confidence limits for small samples.

Histogram allows the construction and administration of a histogram.

Example Program 6.1:

The class E1Sample demonstrates the use of the class SampleThis short program generates a sample of size N taken from the standard normal distribution. It computes the six quantities: mean , error of the mean, variance , error of the variance, standard deviation, and error of the standard deviation, and outputs each quantity in a single line.

Example Program 6.2:

The class E2Sample demonstrates the use of the classes Histogram and GraphicsWithHistogramInitialization, filling and graphical representation of a histogram are demonstrated for a sample of N elements from the standardized normal distribution. Interactive input is provided for N as well as for the lower boundary x ₀, bin width Δ x and the number of bins n x of the histogram. The histogram is initialized, the sample elements are generated and entered into the histogram. Finally the histogram graphics is produced.

Example Program 6.3:

The class E3Sample demonstrates the use of class GraphicsWith2DScatterDiagramA scatter plot is created and later displayed graphically. The coordinates of the points making up the scatter plot are given as pairs of random numbers from a bivariate normal distribution (cf. Sect. 4.​10). The program asks for the parameters of the normal distribution (means a ₁, a ₂, standard deviations σ ₁, σ ₂, correlation coefficient ρ) and for the number of random number pairs to be generated. It generates the pairs and prepares the caption and the labeling of axes and scales and displays the plot (Fig. 6.12).

Example Program 6.4:

The class E4Sample demonstrates using the methods of the class SmallSample to compute confidence limitsThe program computes the limits λ S−, λ S+, and λ S (up) for the Poisson parameter of a signal. The user enters interactively the number of observed events k, the confidence level $$\beta = 1-\alpha$$, and the Poisson parameter λ B of the background. Suggestions: (a) Verify a few lines from Table 6.3.(b) Choose β = 0. 683, λ B  = 0 and compare for different values k the values λ S− and λ S+ with the naive statement $$\lambda = \mathsf{k} \pm \sqrt{\mathsf{k}}$$.

Fig. 6.12:

Pairs of random numbers taken from a bivariate normal distribution.

Example Program 6.5:

The class E5Sample demonstrates the use of methods of the class SmallSample to compute confidence limits of ratiosThe program computes the limits r S−, r S+, and r S (up) for the ratio r of the number of signal to reference events in the limit of a large number of events. More precisely, r is the ratio of the Poisson parameters λ S to λ R of the signal to reference events. The program asks interactively for the number k of observed (signal plus reference) events, for the number d of reference events, for the confidence level $$\beta = 1-\alpha$$, and for the expected ratio $$r_{B} =\lambda _{B}/\lambda _{R}$$ for background events.

Suggestion: Verify a few lines from Table 6.4.

Example Program 6.6:

The class E6Sample simulates experiments with few events and backgroundA total of n exp experiments are simulated. In each experiment N objects are analyzed. Each object yields with probability $$p_{S} =\lambda _{S}/N$$ a signal event and with probability $$p_{B} =\lambda _{B}/N$$ a background event. The numbers of events found in the simulated experiment are k S , k B , and $$\mathsf{k} = \mathsf{k}_{S} + \mathsf{k}_{B}$$. In the real experiment only k is known. The limits λ S−, λ S+, and λ S (up) for k are computed for a given confidence level $$\beta = 1-\alpha$$ and a given value of λ B and are displayed for each experiment.

Suggestion: Choose, e.g., n exp = 20, N = 1000, λ S  = 5, λ B  = 2, β = 0. 9 and find out whether, as expected, this simulation yields for 10 % of the experiments an interval $$(\lambda _{S-},\lambda _{S+})$$, which does not contain the value λ S . Keep in mind the meaning of the statistical error of your observation when using only 20 experiments.

Example Program 6.7:

The class E7Sample simulates experiments with few signal events and with reference eventsThe program asks interactively for the quantities n exp, N, λ S , λ B , λ R , and the confidence level $$\beta = 1-\alpha$$. It computes the probabilities $$p_{S} =\lambda _{S}/N$$, $$p_{B} =\lambda _{B}/N$$, $$p_{R} =\lambda _{R}/N$$, as well as the ratios $$r_{S} = p_{S}/p_{R}$$ and $$r_{B} = p_{B}/p_{R}$$ of a total of n exp simulated experiments, in each of which N objects are analyzed. Each object is taken to be a signal event with probability p S , a background event with probability p B , and a reference event with probability p R . (Here $$p_{S} + p_{B} + p_{R} \ll 1$$ is assumed.) The simulation yields the numbers k S , k B , and d for signal, background, and reference events, respectively. In a real experiment only the numbers $$\mathsf{k} = \mathsf{k}_{S} + \mathsf{k}_{B}$$ and d are known, since signal and background events cannot be distinguished. For the given values of β and r B and for the quantities k and d which were found in the simulated experiments, the limits r S−, r S+, and r S (up) are computed and displayed.

Suggestion: Modify the suggestion accompanying Example Program 6.6 by choosing an additional input parameter λ S  = 20. Find out in how many cases the true value r S used in the simulation is not contained in the interval $$(r_{S-},r_{S+})$$.

Literature

Literature Cited in the Text

[23]

F. James and M. Roos, Nuclear Physics B172 (1980) 475ADSCrossref

[24]

M.L. Swartz, Nuclear Instruments and Methods A294 (1966) 278

[25]

V.H. Regener, Physical Review 84 (1951) 161ADSCrossref

[26]

G. Zech, Nuclear Instruments and Methods A277 (1989) 608

Footnotes

1

The symbol χ ² (chi squared) was introduced by K. Pearson. Although it is written as something squared, which reminds one of its origin as a sum of squares, it is treated as a usual random variable.

© Springer International Publishing Switzerland 2014

Siegmund BrandtData Analysishttps://doi.org/10.1007/978-3-319-03762-2_7

7. The Method of Maximum Likelihood

Siegmund Brandt1 

(1)

Department of Physics, University of Siegen, Siegen, Germany

Keywords

Covariance

7.1 Likelihood Ratio: Likelihood Function

In the last chapter we introduced the concept of parameter estimation. We have also described the desirable properties of estimators, though without specifying how such estimators can be constructed in a particular case. We have derived estimators only for the important quantities expectation value and variance. We now take on the general problem.

In order to specify explicitly the parameters

we now write the probability density of the random variables

in the form

(7.1.1)

If we now carry out a certain number of experiments, say N, or we draw a sample of size N from a population, then we can give a number to each experiment j:

(7.1.2)

The number dP (j) has the character of an a posteriori probability , i.e., given after the experiment, how probable it was to find the result $${\mathsf{\mathbf{x}}}^{(j)}$$ (within a small interval). The total probability to find exactly all of the events

is then the product

(7.1.3)

This probability still clearly depends on $$\boldsymbol{\uplambda }$$. There are cases where the population is determined by only two possible sets of parameters, $$\boldsymbol{\uplambda }_{1}$$ and $$\boldsymbol{\uplambda }_{2}$$. Such cases occur, for example, in nuclear physics, where the parity of a state is necessarily “even” or “odd”. One can construct the ratio

(7.1.4)

and say that the values $$\boldsymbol{\uplambda }_{1}$$ are “Q times more probable” than the values $$\boldsymbol{\uplambda }_{2}$$. This factor is called the likelihood ratio .¹

A product of the form

(7.1.5)

is called a likelihood function . One must clearly distinguish between a probability density and a likelihood function, which is a function of a sample and is hence a random variable. In particular, the a posteriori nature of the probability in (7.1.5) is of significance in many discussions.

Example 7.1:

Likelihood ratio

Suppose one wishes to decide whether a coin belongs to type A or B by means of a number of tosses. The coins in question are asymmetric in such a way that A shows heads with a probability of 1∕3, and B shows heads with a probability of 2∕3.

A

B

Heads

1∕3

2∕3

Tails

2∕3

1∕3

If an experiment yields heads once and tails four times, then one has $$L_{A} = \frac{1} {3} \cdot {(\frac{2} {3})}^{4}$$ and $$L_{B} = \frac{2} {3} \cdot {(\frac{1} {3})}^{4}$$,

One would therefore tend towards the position that the coin is of type A. ■ 

7.2 The Method of Maximum Likelihood

The generalization of the likelihood ratio is now clear. One gives the greatest confidence to that choice of the parameters $$\boldsymbol{\uplambda }$$ for which the likelihood function (7.1.5) is a maximum. Figure 7.1 illustrates the situation for various forms of the likelihood function for the case of a single parameter λ.

Fig. 7.1:

Likelihood functions.

The maximum can be located simply by setting the first derivative of the likelihood function with respect to the parameter λ i equal to zero. The derivative of a product with many factors is, however, unpleasant to deal with. One first constructs therefore the logarithm of the likelihood function,

(7.2.1)

The function ℓ is also often called the likelihood function. Sometimes one says explicitly “log-likelihood function ”. Clearly the maxima of (7.2.1) are identical with those of (7.1.5). For the case of a single parameter we now construct

(7.2.2)

The problem of estimating a parameter is now reduced to solving this likelihood equation . By application of (7.2.1) we can write

(7.2.3)

where

(7.2.4)

is the logarithmic derivative of the density f with respect to λ.

In the general case of p parameters the likelihood equation (7.2.2) is replaced by the system of p simultaneous equations,

(7.2.5)

Example 7.2:

Repeated measurements of differing accuracy

If a quantity is measured with different instruments, then the measurement errors are in general different. The measurements x (j) are spread about the true value λ. Suppose the errors are normally distributed, so that a measurement corresponds to obtaining a sample from a Gaussian distribution with mean λ and standard deviation σ j . The a posteriori probability for a measured value is then

From all N measurements one obtains the likelihood function

(7.2.6)

with the logarithm

(7.2.7)

The likelihood equation thus becomes

It has the solution

(7.2.8)

Since $${\mathrm{d}}^{2}\ell/{\mathrm{d}\lambda }^{2} = -\sum \sigma _{j}^{-2} < 0$$, the solution is, in fact, a maximum. Thus we see that we obtain the maximum likelihood estimator as the mean of the N measurements weighted inversely by the variances of the individual measurements. ■ 

Example 7.3:

Estimation of the parameter N of the hypergeometricdistribution

As in the example with coins at the beginning of this chapter, sometimes parameters to be estimated can only take on discrete values. In Example 5.2 we indicated the possibility of estimating zoological population densities by means of tagging and recapture. According to (5.​3.​1), the probability to catch exactly n fish of which k are tagged out of a pond with an unknown total of N fish, out of which K are tagged, is given by

We must now find the value of N for which the function L is maximum. For this we use the ratio

The function L is thus maximum when N is the integer closest to nK∕k. ■ 

7.3 Information Inequality. Minimum Variance Estimators. Sufficient Estimators

We now want to discuss once more the quality of an estimator. In Sect. 6.​1 we called an estimator unbiased if for every sample the bias vanished,

(7.3.1)

Lack of bias is, however, not the only characteristic required of a “good” estimator. More importantly one should require that the variance

is small. Here one must often find a compromise, since there is a connection between B and σ ², described by the information inequality .²

One immediately sees that it is easy to achieve σ ²(S) = 0 simply by using a constant for S. We consider an estimator S(x ⁽¹⁾, x ⁽²⁾, …, x (N)) that is a function of the sample x ⁽¹⁾, x ⁽²⁾, …, x (N). According to (6.​1.​3) and (6.​1.​4) the joint probability density of the elements of the sample is

The expectation value of S is thus

(7.3.2)

According to (7.3.1), however, one also has

We now assume that we can differentiate with respect to λ in the integral. We then obtain

which is equivalent to

From (7.2.3) we have

and therefore

(7.3.3)

One clearly has

If we also compute the derivative with respect to λ, we obtain

By multiplying this equation by E(S) and subtracting the result of (7.3.3) one obtains

(7.3.4)

In order to see the significance of this expression, we need to use the Cauchy–Schwarz inequality in the following form:

If x and y are random variables and if x ² and y ² have finite expectation values, then

(7.3.5)

To prove this inequality we consider the expression

(7.3.6)

This is a non-negative number for all values of a. If we consider for the moment the case of equality, then this is a quadratic equation for a with the solutions

(7.3.7)

The inequality (7.3.6) is then valid for all a if the term under the square root is negative or zero. From this follows the assertion

If we now apply the inequality (7.3.5) to (7.3.4), one obtains

(7.3.8)

We now use (7.2.3) in order to rewrite the expression for E(ℓ′²),

All terms on the right-hand side vanish, since for i ≠ j

and

By differentiating the last line with respect to λ one obtains

Thus one has simply

Since the individual terms of the sum are independent, the expectation value of the sum is simply the sum of the expectation values. The individual expectation values do not depend on the elements of the sample. Therefore one has

This expression is called the information of the sample with respect to λ. It is a non-negative number, which vanishes if the likelihood function does not depend on the parameter λ.

It is sometimes useful to write the information in a somewhat different form. To do this we differentiate the expression

once more with respect to λ and obtain

The information can then be written as

or

(7.3.9)

The inequality (7.3.8) can now be written in the following way:

or

(7.3.10)

This is the information inequality . It gives the connection between the bias and the variance of an estimator and the information of a sample. It should be noted that in its derivation no assumption about the estimator was made. The right-hand side of the inequality (7.3.10) is therefore a lower bound for the variance of an estimator. It is called the minimum variance bound or Cramer–Rao bound. In cases where the bias does not depend on λ, i.e., particularly in cases of vanishing bias, the inequality (7.3.10) simplifies to

(7.3.11)

This relation justifies using the name information. As the information of a sample increases, the variance of an estimator can be made smaller.

We now ask under which circumstances the minimum variance bound is attained, or explicitly, when the equals sign in the relation (7.3.10) holds. In the inequality (7.3.6), one has equality if (a x + y) vanishes, since only then does one have $$E\{{(a\mathsf{x} + \mathsf{y})}^{2}\} = 0$$ for all values of a, x, and y. Applied to (7.3.8), this means that

or

(7.3.12)

Here A means an arbitrary quantity that does not depend on the sample x ⁽¹⁾, x ⁽²⁾, …, x (N), but may be, however, a function of λ. By integration we obtain

(7.3.13)

and finally

(7.3.14)

The quantities d and D do not depend on λ.

We thus see that estimators attain the minimum variance bound when the likelihood function is of the special form (7.3.14). They are therefore called minimum variance estimators .

For the case of an unbiased minimum variance estimator we obtain from (7.3.11)

(7.3.15)

By substituting (7.3.12) one obtains

or

(7.3.16)

If instead of (7.3.14) only the weaker requirement

(7.3.17)

holds, then the estimator S is said to be sufficient for λ. One can show [see, e.g., Kendall and Stuart, Vol. 2 (1967)], that no other estimator can contribute information about λ that is not already contained in S, if the requirement (7.3.17) is fulfilled. Hence the name “sufficient estimator” (or statistic).

Example 7.4:

Estimator for the parameter of the Poisson distribution

Consider the Poisson distribution  (5.​4.​1)

The likelihood function of a sample k ⁽¹⁾, k ⁽²⁾, …, k (N) is

and its derivative with respect to λ is

(7.3.18)

Comparing with (7.3.12) and (7.3.16) shows that the arithmetic mean $$\bar{k}$$ is an unbiased minimum variance estimator with variance λ∕N. ■ 

Example 7.5:

Estimator for the parameter of the binomial distribution

The likelihood function of a sample from a binomial distribution with the parameters p = λ, $$q = 1-\lambda$$ is given directly by (5.​1.​3),

(The result of the sample can be summarized by the statement that in n experiments, the event A occurred k times; see Sect. 5.​1.​) One then has

By comparing with (7.3.12) and (7.3.16) one finds k∕n to be a minimum variance estimator with variance $$\lambda (1-\lambda )/n$$. ■ 

Example 7.6:

Law of error combination (“Quadratic averaging of individual errors”)

We now return to the problem of Example 7.2 of repeated measurements of the same quantity with varying uncertainties, or expressed in another way, to the problem of drawing a sample from normal distributions with the same mean λ and different but known variances σ j . From (7.2.7) we obtain

We can rewrite this expression as

As in Example 7.2 we recognize

(7.3.19)

as an unbiased estimator for λ. Comparing with (7.3.12) shows that it is also a minimum variance estimator. From (7.3.16) one determines that its variance is

(7.3.20)

The relation (7.3.20) often goes by the name of the law of error combination or quadratic averaging of individual errors . It could have been obtained by application of the rule of error propagation (3.​8.​7) to (7.3.19). If we identify $$\sigma (\tilde{\lambda })$$ as the error of the estimator $$\tilde{\lambda }$$ and σ j as the error of the jth measurement, then we can write it in its usual form

(7.3.21)

If all of the measurements have the same error σ = σ j , Eqs. (7.3.19), (7.3.20) simplify to

which we have already found in Sect. 6.​2.​ ■ 

7.4 Asymptotic Properties of the Likelihood Function and Maximum-Likelihood Estimators

We can now show heuristically several important properties of the likelihood function and maximum-likelihood estimators for very large data samples, that is, for the limit N → ∞. The estimator $$\mathsf{S} =\tilde{\lambda }$$ was defined as the solution to the likelihood equation

(7.4.1)

Let us assume that ℓ′(λ) can be differentiated with respect to λ one more time, so that we can expand it in a series around the point $$\lambda =\tilde{\lambda }$$,

(7.4.2)

The first term on the right side vanishes because of Eq. (7.4.1). In the second term one can write explicitly

This expression has the form of the mean value of a sample. For very large N it can be replaced by the expectation value of the population (Sect. 6.​2),

(7.4.3)

Using Eq. (7.3.9) we can now write

(7.4.4)

In this way we can replace the expression for $$\ell^{\prime\prime}(\tilde{\lambda })$$, which is a function the sample x ⁽¹⁾, x ⁽²⁾, …, x (N), by the quantity $$-1/{b}^{2}$$, which only depends on the probability density f and the estimator $$\tilde{\lambda }$$. If one neglects higher-order terms, Eq. (7.4.2) can be expressed as

(7.4.5)

By integration one obtains

Inserting $$\lambda =\tilde{\lambda }$$ gives $$c =\ell (\tilde{\lambda })$$, or

(7.4.6)

By exponentiation one obtains

(7.4.7)

where k is a constant. The likelihood function L(λ) has the form of a normal distribution with mean $$\tilde{\lambda }$$ and variance b ². At the values $$\lambda =\tilde{\lambda } \pm b$$, where λ is one standard deviation from $$\tilde{\lambda }$$, one has

(7.4.8)

We can now compare (7.4.7) with Eqs. (7.3.12) and (7.3.16). Since we are estimating the parameter λ, we must write $$\mathsf{S} =\tilde{\lambda }$$ and thus E(S) = λ. The estimator $$\tilde{\lambda }$$ is therefore an unbiased minimum variance estimator with variance

(7.4.9)

Since the estimator $$\tilde{\lambda }$$ only possesses this property for the limiting case N → ∞, we call it asymptotically unbiased . This is equivalent to the statement that the maximum likelihood estimator is consistent (Sect. 6.​1). For the same reason the likelihood function is called asymptotically normal.

In Sect. 7.2 we interpreted the likelihood function L(λ) as a measure of the probability that the true value λ ₀ of a parameter is equal to λ. The result of an estimator is often represented in abbreviated form,

Since the likelihood function is asymptotically normal, at least in the case of large samples, i.e., many measurements, this can be interpreted by saying that the probability that the true value λ ₀ lies in the interval

is 68. 3 % (Sect. 5.​8). In practice the relation above is used for large but finite samples. Unfortunately one cannot construct any general rule for determining when a sample is large enough for this procedure to be reliable. Clearly if N is finite, (7.4.3) is only an approximation, whose accuracy depends not only on N, but also on the particular probability density f(x; λ).

Example 7.7:

Determination of the mean lifetime from a small numberof decays

The probability that a radioactive nucleus, which exists at time t = 0, decays in the time interval between t and t + dt is

For observed decay times t ₁, t ₂, …, t N the likelihood function is

Its logarithm is

with the derivative

Comparing with (7.3.12) we see that $$\tilde{\tau }=\bar{ t}$$ is the maximum likelihood solution, which has a variance of $${\sigma }^{2}(\tau ) {=\tau }^{2}/N$$. For $$\tau =\tilde{\tau }=\bar{ t}$$ one obtains $$\varDelta \tilde{\tau }=\bar{ t}/\sqrt{N}$$.

For $$\tilde{\tau }=\bar{ t}$$ one has

We can write

From this expression for the log-likelihood function one cannot so easily recognize the asymptotic form (7.4.6) for N → ∞. For small values of N it clearly does not have this form. Corresponding to (7.4.8), we want to use the values $$\tau _{+} =\tilde{\tau } +\varDelta _{+}$$ and $$\tau _{-} =\tilde{\tau } -\varDelta _{-}$$, where one has

for the asymmetric errors Δ ₊, Δ ₋. Clearly we expect for N → ∞ that Δ ₊, $$\varDelta _{-} \rightarrow \varDelta \tilde{\tau }=\sigma (\tilde{\tau })$$.

Fig. 7.2:

Data and log-likelihood function for Example 7.7.

In Fig. 7.2 the N observed decay times t i are marked as vertical tick marks on the abscissa for various small values of N. In addition the function $$-(\ell-\ell_{\max }) = -(\ell(\tau ) -\ell\left (\tilde{\tau })\right )$$ is plotted. The points τ ₊ and τ ₋ are found where a horizontal line intersects $$-(\ell-\ell_{\max }) = 1/2$$. The point $$\tilde{\tau }$$ is indicated by an additional mark on the horizontal line. One sees that with increasing N the function $$-(\ell-\ell_{\max })$$ approaches more and more the symmetric parabolic form and that the errors Δ ₊, Δ ₋, and $$\varDelta \tilde{\tau }$$ become closer to each other. ■ 

7.5 Simultaneous Estimation of Several Parameters: Confidence Intervals

We have already given a system of equations (7.2.5) allowing the simultaneous determination of p parameters λ = (λ ₁, λ ₂, …, λ p ). It turns out that it is not the parameter determination but rather the estimation of their errors that becomes significantly more complicated in the case of several parameters. In particular we will need to consider correlations as well as errors of the parameters.

We extend our considerations from Sect. 7.4 on the properties of the likelihood function to the case of several parameters. The log-likelihood function

(7.5.1)

can be expanded in a series about the point

(7.5.2)

to give

(7.5.3)

Since by the definition of $$\tilde{\boldsymbol{\uplambda }}$$ one has

(7.5.4)

which holds for all k, the series simplifies to

(7.5.5)

with

(7.5.6)

In the limit N → ∞ we can replace the elements of A, which still depend on the specific sample, by the corresponding expectation values,

(7.5.7)

If we neglect higher-order terms, we can give the likelihood function as

(7.5.8)

Comparing with (5.​10.​1) shows that this is a p-dimensional normal distribution with mean $$\tilde{\boldsymbol{\uplambda }}$$ and covariance matrix

(7.5.9)

The variances of the maximum likelihood estimators $$\tilde{\lambda }_{1},\tilde{\lambda }_{2},\ldots,\tilde{\lambda }_{p}$$ are given by the diagonal elements of the matrix (7.5.9). The off-diagonal elements are the covariances between all possible pairs of estimators,

(7.5.10)

(7.5.11)

For the correlation coefficient between the estimators $$\tilde{\lambda }_{j},\tilde{\lambda }_{k}$$ we can define

(7.5.12)

As in the case of a single parameter, the square roots of the variances are given as the error or standard deviations of the estimators,

(7.5.13)

In Sect. 7.4 we determined that by giving the maximum-likelihood estimator and its error one defines a region that contains the true value of the parameter with a probability of 68. 3 %. Since the likelihood function in the several parameter case is asymptotically a Gaussian distribution of several variables, this region is not determined only by the errors, but rather by the entire covariance matrix. In the special case of two parameters this is the covariance ellipse, which we introduced in Sect. 5.​10.​

The expression (7.5.8) has (with the replacement $$\mathbf{x} = \boldsymbol{\uplambda }$$) exactly the form of (5.​10.​1). We can therefore use it for all of the results of Sect. 5.​10.​ For the exponent one has

(7.5.14)

In the parameter space spanned by λ ₁, …, λ p , the covariance ellipsoid of the distribution (7.5.8) is then determined by the condition

(7.5.15)

For other values of $$g(\boldsymbol{\uplambda })$$ one obtains the confidence ellipsoids introduced in Sect. 5.​10.​ For smaller values of N, the series (7.5.3) cannot be truncated and the approximation (7.5.7) is not valid. Nevertheless, the solution (7.5.4) can clearly still be computed. For a given probability W one obtains instead of a confidence ellipsoid a confidence region , contained within the hypersurface

(7.5.16)

The value of g is determined in the same way as for the confidence ellipsoid as in Sect. 5.​10.​

In Example 7.7 we computed the region $$\tilde{\lambda }-\varDelta _{-} <\lambda <\tilde{\lambda } +\varDelta _{+}$$ for the case of a single variable. This corresponds to a confidence region with the probability content 68. 3 %.

Example 7.8:

Estimation of the mean and variance of a normal distribution

We want to determine the mean λ ₁ and standard deviation λ ₂ of a normal distribution using a sample of size N. This problem occurs, for example, in the measurement of the range of α-particles in matter. Because of the statistical nature of the energy loss through a large number of independent individual collisions, the range of the individual particles is Gaussian distributed about some mean value. By measuring the range x (j) of N different particles, the mean λ ₁ and “straggling constant” λ ₂ = σ can be estimated. We obtain the likelihood function

and

The system of likelihood equations is

Its solution is

For the estimator of the mean, the maximum-likelihood method leads to the arithmetic mean of the individual measurements. For the variance it gives the quantity s′² (6.​2.​4), which has a small bias, and not s ², the unbiased estimator (6.​2.​6).

Let us now determine the matrix B. The second derivatives are

We use the procedure of (7.5.7), substitute λ ₁, λ ₂ by $$\tilde{\lambda }_{1},\tilde{\lambda }_{2}$$ and find

or for the covariance matrix

We interpret the diagonal elements as the errors of the corresponding parameters, i.e.,

The estimators for λ ₁ and λ ₂ are not correlated. ■ 

Example 7.9:

Estimators for the parameters of a two-dimensional normaldistribution

To conclude we consider a population described by a two-dimensional normal distribution (Sect. 5.​10)

By constructing and solving a system of five simultaneous likelihood equations for the five parameters $$a_{1},a_{2},\sigma _{1}^{2},\sigma _{2}^{2},\varrho$$ we obtain for the maximum-likelihood estimators

(7.5.17)

Exactly as in Example 7.8, the estimators for the variances $$\mathsf{s}^{\prime {2}}_{1}$$ and $$\mathsf{s}^{\prime {2}}_{2}$$ are biased. This also holds for the expression (7.5.17), the sample correlation coefficient r. Like all maximum likelihood estimators, r is consistent, i.e., it provides a good estimation of $$\varrho$$ for very large samples. For N → ∞ the probability density of the random variable r becomes a normal distribution with mean $$\varrho$$ and variance

(7.5.18)

For finite samples the distribution is asymmetric. It is therefore important to have a sufficiently large sample before applying Eq. (7.5.17). As a rule of thumb, N ≥ 500 is usually recommended.  ■ 

7.6 Example Programs

Example Program 7.1:

The class E1MaxLike computes the mean lifetime and its asymmetric errors from a small number of radioactive decays The program performs the computations and the graphical display for the problem described in Example 7.7. First by Monte Carlo method a total of N decay times t i of radioactive nuclei with a mean lifetime of τ = 1 are simulated. The number N of decays and also the seeds for the random number generator are entered interactively.

Example Program 7.2:

The class E2MaxLike computes the maximum-likelihood estimates of the parameters of a bivariate normal distribution from a simulated sample The program asks interactively for the number n exp of experiments to simulate (i.e., of the samples to be treated consecutively), for the size n pt of each sample and for the means a ₁, a ₂, the standard deviations σ ₁, σ ₂, and the correlation coefficient ρ of a bivariate Gaussian distribution.

The covariance matrix C of the normal distribution is calculated and the generator of random numbers from a multivariate Gaussian distribution is initialized. Each sample is generated and then analyzed, i.e., the quantities $$\bar{x}_{1}$$, $$\bar{x}_{2}$$, $$s^{\prime}_{1}$$, $$s^{\prime}_{2}$$, and r are computed, which are estimates of a ₁, a ₂, σ ₁, σ ₂, and ρ [cf. (7.5.17)]. The quantities are displayed for each sample.

Suggestions: Choose n exp = 20, keep all other parameters fixed, and study the statistical fluctuations of r for n pt = 5, 50, 500. Use the values ρ = 0, 0. 5, 0. 95.

Footnotes

1

Although the likelihood ratio Q and the likelihood functions L and ℓ introduced below are random variables, since they are functions of a sample, we do not write them here with a special character type.

2

This inequality was independently found by H. Cramer, M. Fréchet, and C. R. Rao as well as by other authors. It is also called the Cramer–Rao or Fréchet inequality.

© Springer International Publishing Switzerland 2014

Siegmund BrandtData Analysishttps://doi.org/10.1007/978-3-319-03762-2_8

8. Testing Statistical Hypotheses

Siegmund Brandt1 

(1)

Department of Physics, University of Siegen, Siegen, Germany

Keywords

Preconceive

8.1 Introduction

Often the problem of a statistical analysis does not involve determining an originally unknown parameter, but rather, one already has a preconceived opinion about the value of the parameter, i.e., a hypothesis . In a sample taken for quality control, for example, one might initially assume that certain critical values are normally distributed within tolerance levels around their nominal values. One would now like to test this hypothesis. To elucidate such test procedures, called statistical tests , we will consider such an example and for simplicity make the hypothesis that a sample of size 10 originates from a standard normal distribution.

Suppose the analysis of the sample resulted in the arithmetic mean $$\bar{\mathsf{x}} = 0.154$$. Under the assumption that our hypothesis is correct, the random variable $$\bar{\mathsf{x}}$$ is normally distributed with mean 0 and standard deviation $$\frac{1} {\sqrt{10}}$$. We now ask for the probability to observe a value $$\vert \bar{\mathsf{x}}\vert \geq 0.154$$ from such a distribution. From (5.​8.​5) and Table I.3 this is

Thus we see that even if our hypothesis is correct, there is a probability of 62% that a sample of size 10 will lead to a sample mean that differs from the population mean by 0. 154 or more.

We now find ourselves in the difficult situation of having to answer the simple question: “Is our hypothesis true or false?” A solution to this problem is provided by the concept of the significance level : One specifies before the test a (small) test probability α. Staying with our previous example, if $$P(\vert \bar{\mathsf{x}}\vert \geq 0.154) <\alpha$$, then one would regard the occurrence of $$\bar{\mathsf{x}} = 0.154$$ as improbable. That is, one would say that $$\bar{\mathsf{x}}$$ differs significantly from the hypothesized value

and the hypothesis would be rejected. The converse is, however, not true. If P does not fall below α, we cannot say that “the hypothesis is true”, but rather “it is not contradicted by the result of the sample”. The choice of the significance level depends on the problem being considered. For quality control of pencils one might be satisfied with 1%. If, however, one wishes to determine insurance premiums such that the probability for the company to go bankrupt is less than α, then one would probably still regard 0. 01% as too high. In the analysis of scientific data α values of 5, 1, or 0. 1% are typically used. From Table I.3 we can obtain limiting values for $$\vert \bar{\mathsf{x}}\vert $$ such that a deviation in excess of these values corresponds to the given probabilities. These are

At these significance levels the value $$\vert \bar{\mathsf{x}}\vert $$ would have to exceed the values 0.62, 0.82, 1.04 before we could reject the hypothesis.

In some cases the sign of $$\bar{\mathsf{x}}$$ is important. In many production processes, deviations in a positive and negative direction are of different importance. (If a baker’s rolls are too heavy, this reduces profits; if they are too light they cost the baker his license.) If one sets, e.g.,

then this is referred to as a one-sided test in contrast to the two-sided test , which we have already considered (Fig. 8.1).

Fig. 8.1:

One-sided and two-sided tests

For many tests one does not construct the sample mean but rather a certain function of the sample called a test statistic , which is particularly suited for tests of certain hypotheses. As above one specifies a certain significance level α and determines a region U in the space of possible values of the test statistic T in which

The index H means that the probability was computed under the assumption that the hypothesis H is valid. One then obtains a sample, which results in a particular value T′ for the test statistic. If T′ is in the region U, the critical region of the test, then the hypothesis is rejected.

In the next sections we will discuss some important tests in detail and then turn to a more rigorous treatment of test theory.

8.2 F-Test on Equality of Variances

The problem of comparing two variances occurs frequently in the development of measurement techniques or production procedures. Consider two populations with the same expectation value; e.g., one measures the same quantity with two different devices without systematic error. One may then ask if they also have the same variance.

To test this hypothesis we take samples of size N ₁ and N ₂ from both populations, which we assume to be normally distributed. We construct the sample variance (6.​2.​6) and consider the ratio

(8.2.1)

If the hypothesis is true, then F will be near unity. It is known from Sect. 6.​6 that for every sample we can construct a quantity that follows a χ ²-distribution:

The two distributions have $$f_{1} = (N_{1} - 1)$$ and $$f_{2} = (N_{2} - 1)$$ degrees of freedom.

Under the assumption that the hypothesis ($$\sigma _{1}^{2} =\sigma _{ 2}^{2}$$) is true, one has

The probability density of a χ ²-distribution with f degrees of freedom is [see (6.​6.​10)]

We now compute the probability¹

that the ratio $$X_{1}^{2}/X_{2}^{2}$$ is smaller than Q:

Calculating the integral gives

(8.2.2)

where

Finally if one sets

then the distribution function of the ratio F can be obtained from (8.2.2),

This is called the Fisher F-distribution.² It depends on the parameters f ₁ and f ₂. The probability density for the F-distribution is

(8.2.3)

This is shown in Fig. 8.2 for fixed values of f ₁ and f ₂. The distribution is reminiscent of the χ ²-distribution; it is only non-vanishing for F ≥ 0, and has a long tail for F → ∞. Therefore it cannot be symmetric. One can easily show that for f ₂ > 2 the expectation value is simply

We can now determine a limit F′ α with the requirement

(8.2.4)

This expression means that the limit F′ α is equal to the quantile F 1−α of the F-distribution (see Sect. 3.​3) since

(8.2.5)

If this limit is exceeded, then we say that $$\sigma _{1}^{2} >\sigma _{ 2}^{2}$$ with the significance level α. The quantiles F 1−α for various pairs of values (f ₁, f ₂) are given in Table I.8. In general one applies a two-sided test, i.e., one tests whether the ratio F is between two limits F″ α and F‴ α , which are determined by

Fig. 8.2:

Probability density of the F-distribution for fixed values of f ₁ = 2, 4, …, 20. For f ₁ = 2 one has $$f(F) ={ \mathrm{e}}^{-F}$$. For f ₁ > 2 the function has a maximum which increases for increasing f ₁.

(8.2.6)

Because of the definition of F as a ratio, the inequality

clearly has the same meaning as

Here the first argument gives the number of degrees of freedom in the numerator, and the second in the denominator. The requirement (8.2.6) can also be written:

(8.2.7)

Table I.8 can therefore be used for the one-sided as well as the two-sided F-test.

A glance at Table I.8 also shows that $$F_{1-\alpha /2} > 1$$ for all reasonable values of α. Therefore one needs only to find the limit for the ratio

(8.2.8)

Here the indices g and k give the larger and smaller values of the two variances, i.e., $$s_{g}^{2} > s_{k}^{2}$$. If the inequality (8.2.8) is satisfied, then the hypothesis of equal variances must be rejected.

Example 8.1:

F-test of the hypothesis of equal variance of two series ofmeasurements

A standard length (100 μm) is measured using two traveling microscopes. The measurements and computations are summarized in Table 8.1. From Table I.8 we find for the two-sided F-test with a significance level of 10%,

The hypothesis of equal variances cannot be rejected. ■ 

Table 8.1:

F-test on the equality of variances. Data from Example 8.1.

Measurement with

Measurement

Instrument 1

Instrument 2

number

[μm]

[μm]

1

100

97

2

101

102

3

103

103

4

98

96

5

97

100

6

98

101

7

102

100

8

101

9

99

10

101

Mean

100

99.8

Degrees of freedom

9

6

s ²

8.3 Student’s Test: Comparison of Means

We now consider a population that follows a standard Gaussian distribution. Let $$\bar{\mathsf{x}}$$ be the arithmetic mean of a sample of size N. According to (6.​2.​3) the variance of $$\bar{\mathsf{x}}$$ is related to the population variance by

(8.3.1)

If N is sufficiently large, then from the Central Limit Theorem, $$\bar{\mathsf{x}}$$ will be normally distributed with mean $$\hat{x}$$ and variance $${\sigma }^{2}(\bar{\mathsf{x}})$$. That is,

(8.3.2)

will be described by a standard normal distribution. The quantity σ(x) is, however, not known. We only know the estimate for σ ²(x),

(8.3.3)

Then with (8.3.1) we can also estimate $${\sigma }^{2}(\bar{\mathsf{x}})$$ to be

(8.3.4)

We now ask to what extent (8.3.2) differs from the standard Gaussian distribution if $$\sigma (\bar{\mathsf{x}})$$ is replaced by $$\mathsf{s}_{\bar{\mathsf{x}}}$$. By means of a simple translation of coordinates we can always have $$\hat{x} = 0$$. We therefore only consider the distribution of

(8.3.5)

Since $$(N - 1)\,\mathsf{s}_{\mathsf{x}}^{2} = f\,\mathsf{s}_{\mathsf{x}}^{2}$$ follows a χ ²-distribution with $$f = N - 1$$ degrees of freedom, we can write

The distribution function of t is given by

(8.3.6)

After a somewhat lengthy calculation one finds

The corresponding probability density is

(8.3.7)

Fig. 8.3:

Student’s distribution f(t) for f = 1, 2, …, 10 degrees of freedom. For f = 1 the maximum is lowest and the tails are especially prominent.

Figure 8.3 shows the function f(t) for various degrees of freedom $$f = N - 1$$. A comparison with Fig. 5.7 shows that for f → ∞, the distribution (8.3.7) becomes the standard normal distribution ϕ ₀(t), as expected. Like ϕ ₀(t), f(t) is symmetric about 0 and has a bell shape. Corresponding to (5.​8.​3) one has

(8.3.8)

By requiring

(8.3.9)

we can again determine limits ± t′ α at a given significance level α, where

The quantiles $$t_{1-\frac{1} {2} \alpha }$$ are given in Table I for various values of α and f.

The application of Student’s test³ can be described in the following way: One has a hypothesis λ ₀ for the population mean of a normal distribution. A sample of size N yields the sample mean $$\bar{\mathsf{x}}$$ and sample variance $$\mathsf{s}_{\mathsf{x}}^{2}$$. If the inequality

(8.3.10)

is fulfilled for a given significance level α, then the hypothesis must be rejected.

This is clearly a two-sided test. If deviations only in one direction are important, then the corresponding test at the significance level α is

(8.3.11)

We can make the test more general and apply it to the problem of comparing two mean values. Suppose samples of size N ₁ and N ₂ have been taken from two populations X and Y. We wish to find a measure of correctness for the hypothesis that the expectation values are equal,

Because of the Central Limit Theorem, the mean values are almost normally distributed. Their variances are

(8.3.12)

and the estimates for these quantities are

(8.3.13)

According to the discussion in Example 5.10, the difference

(8.3.14)

also has an approximate normal distribution with

(8.3.15)

If the hypothesis of equal means is true, i.e., $$\hat{\varDelta }= 0$$, then the ratio

(8.3.16)

follows the standard normal distribution. If σ(Δ) were known one could immediately give the probability according to (5.​8.​2) for the hypothesis to be fulfilled. But only s Δ is known. The corresponding ratio

(8.3.17)

will in general be somewhat larger.

Usually the hypothesis $$\hat{x} =\hat{ y}$$ implies that $$\bar{\mathsf{x}}$$ and $$\bar{\mathsf{y}}$$ come from the same population. Then σ ²(x) and σ ²(y) are equal, and we can use the weighted mean of $$\mathsf{s}_{\mathsf{x}}^{2}$$ and $$\mathsf{s}_{\mathsf{y}}^{2}$$ as the corresponding estimator. The weights are given by (N ₁ − 1) and (N ₂ − 1):

(8.3.18)

From this we construct

and

(8.3.19)

It can be shown (see [8]) that the ratio (8.3.17) follows the Student’s t-distribution with $$f = N_{1} + N_{2} - 2$$ degrees of freedom. With this one can now perform Student’s difference test :

The quantity (8.3.17) is computed from the results of two samples. This value is compared to a quantile of Student’s distribution with $$f = N_{1} + N_{2} - 2$$ degrees of freedom with a significance level α. If

(8.3.20)

then the hypothesis of equal means must be rejected. Instead one would assume $$\hat{x} >\hat{ y}$$ or $$\hat{x} <\hat{ y}$$, depending on whether one has $$\bar{\mathsf{x}} >\bar{ \mathsf{y}}$$ or $$\bar{\mathsf{x}} <\bar{ \mathsf{y}}$$.

Example 8.2:

Student’s test of the hypothesis of equal means of two series of measurements

Column x of Table 8.2 contains measured values (in arbitrary units) of the concentration of neuraminic acid in the red blood cells of patients suffering from a certain blood disease. Column y gives the measured values for a group of healthy persons. From the mean values and variances of the two samples one finds

For α = 5% and f = 21 we find $$t_{1-\alpha /2} = 2.08$$. We must therefore conclude that the experimental data is not sufficient to determine an influence of the disease on the concentration. ■ 

Table 8.2:

Student’s difference test on the equality of means. Data from Example 8.2.

x

y

21

16

24

20

18

22

19

19

25

18

17

19

18

19

22

21

23

18

13

16

23

22

24

N ₁ = 16

N ₂ = 7

8.4 Concepts of the General Theory of Tests

The test procedures discussed so far have been obtained more or less intuitively and without rigorous justification. In particular we have not given any specific reasons for the choice of the critical region. We now want to deal with the theory of statistical tests in a somewhat more critical way. A complete treatment of this topic would, however, go beyond the scope of this book.

Each sample of size N can be characterized by N points in the sample space of Sect. 2.​1.​ For simplicity we will limit ourselves to a continuous random variable x, so that the sample can be described by N points $$({\mathsf{x}}^{(1)},{\mathsf{x}}^{(2)},\ldots,{\mathsf{x}}^{(N)})$$ on the x axis. In the case of r random variables we would have N points in an r-dimensional space. The result of such a sample, however, could also be specified by a single point in a space of dimension rN. A sample of size 2 with a single variable could, for example, be depicted as a point in a two-dimensional plane, spanned by the axes $${\mathsf{x}}^{(1)},{\mathsf{x}}^{(2)}$$. We will call such a space the E space . Every hypothesis H consists of an assumption about the probability density

(8.4.1)

The hypothesis is said to be simple if the function f is completely specified, i.e., if the hypothesis gives the values of all of the parameters λ i . It is said to be composite if the general mathematical form of f is known, but the exact value of at least one parameter remains undetermined. A simple hypothesis could, for example, specify a standard Gaussian distribution. A Gaussian distribution with a mean of zero but an undetermined variance, however, is a composite hypothesis. The hypothesis H ₀ is called the null hypothesis . Sometimes we will write explicitly

(8.4.2)

Other possible hypotheses are called alternative hypotheses , e.g.,

(8.4.3)

Often one wants to test a null hypothesis of the type (8.4.2) against a composite alternative hypothesis

(8.4.4)

Since the null hypothesis makes a statement about the probability density in the sample space, it also predicts the probability for observing a point $$X = ({\mathsf{x}}^{(1)},{\mathsf{x}}^{(2)},\ldots,{\mathsf{x}}^{(N)})$$ in the E space.⁴ We now define a critical region S c with the significance level α by the requirement

(8.4.5)

i.e., we determine S c such that the probability to observe a point X within S c is α, under the assumption that H ₀ is true. If the point X from the sample actually falls into the region S c , then the hypothesis H ₀ is rejected. One must note that the requirement (8.4.5) does not necessarily determine the critical region S c uniquely.

Although using the E space is conceptually elegant, it is usually not very convenient for carrying out tests. Instead one constructs a test statistic

(8.4.6)

and determines a region U of the variable T such that it corresponds to the critical region S c , i.e., one performs the mapping

(8.4.7)

The null hypothesis is rejected if T ∈ U.

Because of the statistical nature of the sample, it is clearly possible that the null hypothesis could be true, even though it was rejected since X ∈ S c . The probability for such an error, an error of the first kind , is equal to α. There is in addition another possibility to make a wrong decision, if one does not reject the hypothesis H ₀ because X was not in the critical region S c , even though the hypothesis was actually false and an alternative hypothesis was true. This is an error of the second kind . The probability for this,

(8.4.8)

depends of course on the particular alternative hypotheses H ₁. This connection provides us with a method to specify the critical region S c . A test is clearly most reasonable if for a given significance level α the critical region is chosen such that the probability β for an error of the second kind is a minimum. The critical region and therefore the test itself naturally depend on the alternative hypothesis under consideration.

Once the critical region has been determined, we can consider the probability for rejecting the null hypothesis as a function of the “true” hypothesis, or rather as a function of the parameters that describe it. In analogy to (8.4.5), this is

(8.4.9)

This probability is a function of S c and of the parameters $$\boldsymbol{\uplambda }$$. It is called the power function of a test. The complementary probability

(8.4.10)

is called the acceptance probability or the operating characteristic function of the test. It gives the probability to accept⁵ the null hypothesis. One clearly has

(8.4.11)

The most powerful test of a simple hypothesis H ₀ with respect to the simple alternative hypothesis is defined by the requirement

(8.4.12)

Sometimes there exists a uniformly most powerful test , for which the requirement (8.4.12) holds for all possible alternative hypotheses.

A test is said to be unbiased if its power function is greater than or equal to α for all alternative hypotheses:

(8.4.13)

This definition is reasonable, since the probability to reject the null hypothesis is then a minimum if the null hypothesis is true. An unbiased most powerful test is the most powerful of all the unbiased tests. Correspondingly one can define a unbiased uniformly most powerful test. In the next sections we will learn the rules which sometimes allow one to construct tests with such desirable properties. Before turning to this task, we will first give an example to illustrate the definitions just introduced.

Example 8.3:

Test of the hypothesis that a normal distribution with given variance σ ² has the mean λ = λ ₀

We wish to test the hypothesis $$H_{0}(\lambda =\lambda _{0})$$. As a test statistic we use the arithmetic mean $$\bar{\mathsf{x}} = \frac{1} {n}(\mathsf{x}_{1} + \mathsf{x}_{2} +\ldots +\mathsf{x}_{n})$$. (We will see in Example 8.4 that this is the most appropriate test statistic for our purposes.) From Sect. 6.​2 we know that $$\bar{\mathsf{x}}$$ is normally distributed with mean λ and variance σ ²∕n, i.e., that the probability density for $$\bar{\mathsf{x}}$$ for the case λ = λ ₀ is given by

(8.4.14)

This is shown in Fig. 8.4 together with four different critical regions, all of which have the same significance level α.

These are the regions

In order to obtain the power functions for each of these regions, we must vary the mean value λ. The probability density of $$\bar{\mathsf{x}}$$ for an arbitrary value of λ is, in analogy to (8.4.14), given by

(8.4.15)

The dashed curve in Fig. 8.4b represents the probability density for $$\lambda =\lambda _{1} =\lambda _{0} + 1$$. The power function (8.4.9) is now simply

(8.4.16)

The power functions obtained in this way for the critical regions U ₁, U ₂, U ₃, U ₄ are shown in Fig. 8.4c for n = 2 (solid curve) and n = 10 (dashed curve).

Fig. 8.4:

(a) Critical regions in E space, (b) critical region of the test function, and (c) power function of the test from Example 8.3.

We can now compare the effects of the four tests corresponding to the various critical regions. From Fig. 8.4c we immediately see that U ₁ corresponds to an unbiased test, since the requirement (8.4.13) is clearly fulfilled. On the other hand, the test with the critical region U ₂ is more powerful for the alternative hypothesis $$H_{1}(\lambda _{1} >\lambda _{0})$$, but is not good for $$H_{1}(\lambda _{1} <\lambda _{0})$$. For the test with U ₃, the situation is exactly the opposite. Finally, the region U ₄ provides a test for which the rejection probability is a maximum if the null hypothesis is true. Clearly this is very undesirable. The test was only constructed for demonstration purposes. If we compare the first three tests, we see that none of them are more powerful than the other two for all values of λ ₁. Thus we have not succeeded in finding a uniformly most powerful test. In Example 8.4, where we will continue the discussion of the present example, we will determine that for this problem there does not exist a uniformly most powerful test. ■ 

8.5 The Neyman–Pearson Lemma and Applications

In the last section we introduced the E space, in which a sample is represented by a single point X. The probability to observe a point X within the critical region S c – providing the null hypothesis H ₀ is true – was defined in (8.4.5),

(8.5.1)

We now define a conditional probability in E space,

One clearly has

(8.5.2)

The NEYMAN–PEARSON lemma states the following:

A test of the simple hypothesis H ₀ with respect to the simple alternative hypothesis H ₁ is a most powerful test if the critical region S c in E space is chosen such that

(8.5.3)

Here c is a constant which depends on the significance level.

We will prove this by considering another region S along with S c . It may partially overlap with S c , as sketched in Fig. 8.5. We choose the size of the region S such that it corresponds to the same significance level, i.e.,

Fig. 8.5:

The regions S and S c .

Using the notation of Fig. 8.5, we can write

Since A is contained in S c , we can use (8.5.3), i.e.,

Correspondingly, since B is outside of S c , one has

We can now express the power function (8.4.9) using these integrals:

or directly,

(8.5.4)

This is exactly the condition (8.4.12) for a uniformly most powerful test. Since we have not made any assumptions about the alternative hypothesis $$H_{1}(\lambda =\lambda _{1})$$ or the region S, we have proven that the requirement (8.5.3) provides a uniformly most powerful test when it is fulfilled by the alternative hypothesis.

Example 8.4:

Most powerful test for the problem of Example 8.3

We now continue with the ideas from Example 8.3, i.e., we consider tests with a sample of size N, obtained from a normal distribution with known variance σ ² and unknown mean λ. The conditional probability density of a point $$X = ({\mathsf{x}}^{(1)},{\mathsf{x}}^{(2)},\ldots,{\mathsf{x}}^{(N)})$$ in E space is the joint probability density of the x (j) for given values of λ, i.e.,

(8.5.5)

for the null hypothesis and

(8.5.6)

for the alternative hypothesis. The ratio (8.5.3) takes on the form

The expression

is a non-negative constant. The condition (8.5.3) thus has the form

This is the same as

(8.5.7)

Here c′ is a constant different from c. Equation (8.5.7) is, however, not only a condition for S c , but also specifies directly that $$\bar{\mathsf{x}}$$ should be used as the test variable. For each given λ ₁, i.e., for every simple alternative hypothesis $$H_{1}(\lambda =\lambda _{1})$$, (8.5.7) gives a clear prescription for the choice of S c or U, i.e., for the critical region and the test variable $$\bar{\mathsf{x}}$$.

For the case λ ₁ < λ ₀, the relation (8.5.7) becomes

This corresponds to the situation in Fig. 8.4b3 with c″ = λ IV . Similarly, for every alternative hypothesis with λ ₁ > λ ₀, the critical region of the most powerful test is given by

(see Fig. 8.4b2 with c‴ = λ‴). There does not exist a uniformly most powerful test, since the factor $$(\lambda _{0} -\lambda _{1})$$ in Eq. (8.5.7) changes sign at λ ₁ = λ ₀. ■ 

8.6 The Likelihood-Ratio Method

The Neyman–Pearson lemma gave the condition for a uniformly most powerful test. Such a test did not exist if the alternative hypothesis included parameter values that could be both greater and less than that of the null hypothesis. We determined this in Example 8.4; it can be shown, however, that it is true in general. The question thus arises as to what test should be used when no uniformly most powerful test exists. Clearly this question is not formulated precisely enough to allow a unique answer. We would like in the following to give a prescription that allows us to construct tests that have desirable properties and that have the advantage of being relatively easy to use.

We consider from the beginning the general case with p parameters $$\boldsymbol{\uplambda } = (\lambda _{1},\lambda _{2},\ldots,\lambda _{p})$$. The result of a sample, i.e., the point $$X = ({\mathsf{x}}^{(1)},{\mathsf{x}}^{(2)},\ldots,{\mathsf{x}}^{(N)})$$ in E space is to be used to test a given hypothesis. The (composite) null hypothesis is characterized by a given region for each parameter. We can use a p-dimensional space, with the $$\lambda _{1},\lambda _{2},\ldots,\lambda _{p}$$ as coordinate axes, and consider the region allowed by the null hypothesis as a region in this parameter space, called ω. We denote the region in this space representing all possible values of the parameters by Ω. The most general alternative hypothesis is then the part of Ω that does not contain ω. We denote this by Ω −ω. Recall now from Chap. 7 the maximum-likelihood estimator $$\tilde{\lambda }$$ for a parameter λ. It is that value of λ for which the likelihood function is a maximum. In Chap. 7 we tacitly assumed that one searched for the maximum in the entire allowable parameter space. In the following we will consider maxima in a restricted region (e.g., in ω). We write in this case $${\tilde{\lambda }}^{(\omega )}$$. The likelihood-ratio test defines a test statistic

(8.6.1)

Here $$f({\mathsf{x}}^{(1)},{\mathsf{x}}^{(2)},\ldots,{\mathsf{x}}^{(N)};\lambda _{1},\lambda _{2},\ldots,\lambda _{p})$$ is the joint probability density of the x (j) (j = 1, 2, …, N), i.e., the likelihood function (7.​1.​5). The procedure of the likelihood ratio test prescribes that we reject the null hypothesis if

(8.6.2)

Here T 1−α is defined by

(8.6.3)

and g(T | H ₀) is the conditional probability density for the test statistic T. The following theorem by WILKS [9] concerns the distribution function of T (or actually − 2lnT) in the limiting case of very large samples:

If a population is described by the probability density $$f(x;\lambda _{1},\lambda _{2},\ldots,\lambda _{p})$$ that satisfies reasonable requirements of continuity, and if p − r of the p parameters are fixed by the null hypothesis, while r parameters remain free, then the statistic − 2lnT follows a χ ²-distribution with p − r degrees of freedom for very large samples, i.e., for N → ∞.

We now apply this method to the problem of Examples 8.3 and 8.4, i.e., we consider tests with samples from a normally distributed population with known variance and unknown mean.

Example 8.5:

Power function for the test from Example 8.3

For the simple hypothesis $$H_{0}(\lambda =\lambda _{0})$$, the region ω of the parameter space is reduced to the point λ = λ ₀. We have thus

(8.6.4)

If we consider the most general alternative hypothesis $$H_{1}(\lambda =\lambda _{1}\neq \lambda _{0})$$, then we obtain as the maximum-likelihood estimator of λ the sample mean $$\bar{\mathsf{x}}$$. The likelihood ratio (8.6.1) thus becomes

(8.6.5)

The joint probability density is given by (7.​2.​6),

(8.6.6)

Therefore,

We must now calculate T 1−α and reject the hypothesis H ₀ if the inequality (8.6.2) is fulfilled. Since the logarithm of T is a monotonic function of T, we can use

(8.6.7)

as the test statistic and reject H ₀ if

with

In order to calculate the probability density h(T′ | H ₀) of T′, we start with the density $$f(\bar{x})$$ for the sample mean with the condition λ = λ ₀,

In order to carry out the transformation of variables (3.​7.​1), we need in addition the derivative,

which can be easily obtained from (8.6.7). One then has

(8.6.8)

This is indeed a χ ²-distribution for one degree of freedom. Thus in our example, WILKS’ theorem holds even for finite N. We see, therefore, that the likelihood-ratio test yields the unbiased test of Fig. 8.4b1. The test

is equivalent to

(8.6.9)

with

We can use this result to compute explicitly the power function of our test. For a given value of the population mean λ, the probability density for the sample mean is

Using (8.4.9) and (8.6.9) we obtain

(8.6.10)

Here ϕ ₀ and ψ ₀ are the probability density and distribution function of the standard normal distribution. The power function (8.6.10) is shown in Fig. 8.6 for α = 0. 05 and various values of N∕σ ². ■ 

Example 8.6:

Test of the hypothesis that a normal distribution of unknown variance has the mean value λ = λ ₀

In this case the null hypothesis $$H_{0}(\lambda =\lambda _{0})$$ is composite, i.e., it makes no statement about σ ². From Example 7.8 we know the maximum-likelihood estimator in the full parameter space,

Fig. 8.6:

Power function of the test from Example 8.5. The right-most curve corresponds to $$N{/\sigma }^{2} = 1$$.

In the parameter space of the null hypothesis we have

The likelihood ratio (8.6.1) is then

We transform again to a different test statistic T′ that is a monotonic function of T,

(8.6.11)

where

(8.6.12)

is Student’s test variable introduced in Sect. 8.3. From (8.6.11) we can compute a value of t for a given sample and reject the null hypothesis if

The very generally formulated method of the likelihood ratio has led us to Student’s test, which was originally constructed for tests with samples from a normal distribution with known mean and unknown variance. ■ 

8.7 The χ ²-Test for Goodness-of-Fit

8.7.1 χ ²-Test with Maximal Number of Degrees of Freedom

Suppose one has N measured values g i , i = 1, 2, …, N, each with a known measurement error σ i . The meaning of the measurement error is the following: g i is a measurement of the (unknown) true quantity h i . One has

(8.7.1)

Here the deviation $$\varepsilon _{i}$$ is a random variable that follows a normal distribution with mean 0 and standard deviation σ i .

We now want to test the hypothesis specifying the values h i on which the measurement is based,

(8.7.2)

If this hypothesis is true, then all of the quantities

(8.7.3)

follow the standard Gaussian distribution. Therefore,

(8.7.4)

follows a χ ²-distribution for N degrees of freedom. If the hypothesis (8.7.2) is false, then the individual deviations of the measured values g i from the values predicted by the hypothesis f i , normalized by the errors σ i , (8.7.3), will be greater. For a given significance level α, the hypothesis (8.7.2) is rejected if

(8.7.5)

i.e., if the quantity (8.7.4) is greater than the quantile $$\chi _{1-\alpha }^{2}$$ of the χ ²-distribution for N degrees of freedom.

8.7.2 χ ²-Test with Reduced Number of Degrees of Freedom

The number of degrees of freedom is reduced when the hypothesis to be tested is less explicit than (8.7.2). For this case we consider the following example. Suppose a quantity g can be measured as a function of an independent controlled variable t, which itself can be set without error,

The individual measurements g i correspond to given fixed values t i of the independent variable. The corresponding true quantities h i are given by some function

A particularly simple hypothesis for this function is the linear equation

(8.7.6)

The hypothesis can in fact include specifying the numerical values for the parameters a and b. In this case, all values f i in (8.7.2) are exactly known, and the quantity (8.7.4) follows – if the hypothesis is true – a χ ²-distribution for N degrees of freedom.

The hypothesis may, however, only state: There exists a linear relationship (8.7.6) between the controlled variable t and the variable h. The numerical values of the parameters a and b are, however, unknown. In this case one constructs estimators $$\tilde{a}$$, $$\tilde{b}$$ for the parameters, which are functions of the measurements g i and the errors σ i . The hypothesis (8.7.2) is then

Since, however, $$\tilde{a}$$ and $$\tilde{b}$$ are functions of the measurements g i , the normalized deviations u i in (8.7.3) are no longer all independent. Therefore the number of degrees of freedom of the χ ²-distribution for the sum of squares (8.7.4) is reduced by 2 to N − 2, since the determination of the two quantities $$\tilde{a}$$, $$\tilde{b}$$ introduces two equations of constraint between the quantities u i .

8.7.3 χ ²-Test and Empirical Frequency Distribution

Suppose we have a distribution function F(x) and its probability density f(x). The full region of the random variable x can be divided into r intervals

as shown in Fig. 8.7. By integrating f(x) over the individual intervals we obtain the probability to observe x in ξ i ,

(8.7.7)

We now take a sample of size n and denote by n i the number of elements of the sample that fall into the interval ξ i . An appropriate graphical representation of the sample is a histogram, as described in Sect. 6.​3.​

Fig. 8.7:

Dividing the range of the variable x into the intervals ξ k .

One clearly has

(8.7.8)

From the (hypothetical) probability density for the population we would have expected the value

for n i . For large values of n i , the variance of n i is equal to n i (Sect. 6.​8), and the distribution of the quantity u i with

(8.7.9)

becomes approximately – if the hypothesis is true – a standard Gaussian distribution. This holds also if one uses the expected variances n p i instead of the observed quantities n i in the denominator of (8.7.9),

(8.7.10)

If we now construct the sum of squares of the u i for all intervals,

(8.7.11)

then we expect (for large n) that this follows a χ ²-distribution if the hypothesis is true. The number of degrees of freedom is r − 1, since the u i are not independent because of (8.7.8). The number of degrees of freedom is reduced to $$r - 1 - p$$ if, in addition, p parameters are determined from the observations.

Example 8.7:

χ ²-test for the fit of a Poisson distribution to an empirical frequency distribution

In an experiment investigating photon-proton interactions, a beam of high energy photons (γ-quanta) impinge on a hydrogen bubble chamber. The processes by which a photon materializes in the chamber, electron-positron pair conversion, are counted in order to obtain a measure of the intensity of the photon beam. The frequency of cases in which 0,1,2,… pairs are observed simultaneously, i.e., in the same bubble-chamber photograph, follows a Poisson distribution (see Example 5.3). Deviations from the Poisson distribution provide information about measurement losses, which are important for uncovering systematic errors. The results of observing n = 355 photographs are given in column 2 of Table 8.3 and in Fig. 8.8. From Example 7.4, we know that the maximum-likelihood estimator of the parameter of the Poisson distribution is given by $$\tilde{\lambda }=\sum _{k}k\,n_{k}/\sum _{k}n_{k}$$. We find $$\tilde{\lambda }= 2.33$$. The values p k of the Poisson distribution with this parameter multiplied by n are given in column 3. By summing the squared terms in column 4 one obtains the value X ² = 10. 44. The problem has six degrees of freedom, since r = 8, p = 1. We chose α = 1% and find $$\chi _{0.99}^{2} = 16.81$$ from Table I.7. We therefore have no reason to reject the hypothesis of a Poisson distribution. ■ 

Table 8.3:

Data for the χ ²-test from Example 8.7.

Number of electron pairs per photograph

Number of photographs with k electron pairs

Prediction of Poisson distribution

k

n k

n p k

0

47

34.4

4.61

1

69

80.2

1.56

2

84

93.7

1.00

3

76

72.8

0.14

4

49

42.6

0.96

5

16

19.9

0.76

6

11

7.8

1.31

7

3

2.5

0.10

8

—

(0.7)

X ² = 10. 44

Fig. 8.8:

Comparison of the experimental distribution n k (histogram with solid line) from Example 8.7 with the Poisson distribution np k (dashed line).

8.8 Contingency Tables

Suppose n experiments have been carried out whose results are characterized by the values of two random variables x and y. We consider the two variables as discrete, being able to take on the values x ₁, x ₂, …, x k ; y ₁, y ₂, …, y ℓ . Continuous variables can be approximated by discrete ones by dividing their range into intervals, as shown in Fig. 8.7. Let the number of times the result x = x i and y = y j is observed be n ij . One can arrange the numbers n ij in a matrix, called a contingency table (Table 8.4).

Table 8.4:

Contingency table.

y ₁

y ₂

…

y ℓ

x ₁

n ₁₁

n ₁₂

…

n 1ℓ

x ₂

n ₂₁

n ₂₂

…

n 2ℓ

⋮

⋮

⋮

⋮

x k

n k1

n k2

…

n kℓ

We denote by p i the probability for x = x i to occur, and by q j the probability for y = y j . If the variables are independent, then the probability to simultaneously observe x = x i and y = y j is equal to the product p i  q j . The maximum-likelihood estimators for p and q are

Since

one has $$k +\ell -2$$ independent estimators $$\tilde{p}_{i}$$, $$\tilde{q}_{j}$$. We can now organize the elements of the contingency table into a single line,

and carry out a χ ²-test. For this we must compute the quantity

(8.8.1)

and compare it to the quantile $$\chi _{1-\alpha }^{2}$$ of the χ ²-distribution corresponding to a given significance level α. The number of degrees of freedom is still obtained from the number of intervals minus the number of estimated parameters minus one,

If the variables are not independent, then n ij will not, in general, be near $$n\,\tilde{p}_{i}\,\tilde{q}_{j}$$, i.e., one will find

(8.8.2)

and the hypothesis will be rejected.

8.9 2 × 2 Table Test

The simplest nontrivial contingency table has only two rows and two columns, and is called a 2 × 2 table , as shown in Table 8.5. It is often used in medical studies. (The variables x ₁ and x ₂ could represent, for example, two different treatment methods, and y ₁ and y ₂ could represent success and failure of the treatment. One wishes to determine whether success is independent of the treatment.)

Table 8.5:

2 × 2 table.

y ₁

y ₂

x ₁

n ₁₁ = a

n ₁₂ = b

x ₂

n ₂₁ = c

n ₂₂ = d

One computes the quantity X ² either according to (8.8.1) or from the formula

which is obtained by rearranging (8.8.1). If the variables x and y are independent, then X ² follows a χ ²-distribution with one degree of freedom. One rejects the hypothesis of independence at the significance level α if

In order for the quantity X ² to actually follow a χ ²-distribution is again necessary that the individual n ij are sufficiently large, (and the hypothesis of independence must be true).

8.10 Example Programs

Example Program 8.1:

The class E1Test generates samples and tests the equality of their variances The program performs a total of n exp simulated experiments. Each experiment consists of the simulation of two samples of sizes N ₁ and N ₂ from normal distributions with standard deviations σ ₁ and σ ₂. The variance of each of the samples is computed using the class Sample. The sample variances are called s g ² and $$\mathsf{s}_{k}^{2}$$, so that

From the corresponding sample sizes the numbers of degrees of freedom $$f_{g} = N_{g} - 1$$ and $$f_{k} = N_{k} - 1$$ are computed. Finally, the ratio $$\mathsf{s}_{g}^{2}/\mathsf{s}_{k}^{2}$$ is compared with the quantile $$F_{1-\alpha /2}(f_{g},f_{k})$$ at a given confidence level $$\beta = 1-\alpha$$. If the ratio is larger than the quantile, then the hypothesis of equal variances has to be rejected. The program asks for the quantities n exp, N ₁, N ₂, σ ₁, σ ₂, and β. For each simulated experiment one line of output is displayed.

Suggestions: Choose n exp = 20 and β = 0. 9. (a) For $$\sigma _{1} =\sigma _{2}$$ you would expect the hypothesis to be rejected in 2 out of 20 cases because of an error of the first kind. Note the large statistical fluctuations, which obviously depend on N ₁ and N ₂, and choose different pairs of values N ₁, N ₂ for $$\sigma _{1} =\sigma _{2}$$. (b) Check the power of the test for different variances $$\sigma _{1}\neq \sigma _{2}$$.

Example Program 8.2:

The class E2Test generates samples and tests the equality of their means with a given value using Student’s Test This short program performs n exp simulation experiments. In each experiment a sample of size N is drawn from a normal distribution with mean x ₀ and width σ. Using the class Sample the sample mean $$\bar{\mathsf{x}}$$ and the sample variance $$\mathsf{s}_{\mathsf{x}}^{2}$$ are determined. If λ ₀ is the population mean specified by the hypothesis, then the quantity

can be used to test the hypothesis. At a given confidence level $$\beta = 1-\alpha$$ the hypothesis is rejected if

Here $$t_{1-\alpha /2}$$ is the quantile of Student’s distribution with $$f = N - 1$$ degrees of freedom. The program asks for the quantities n exp, N, x ₀, σ, λ ₀, and β. For each simulated experiment one line of output is displayed.

Suggestion: Modify the suggestions at the end of Sect. 8.1 to apply to Student’s test.

Example Program 8.3:

The class E3Test generates samples and computes the test statistic χ ² for the hypothesis that the samples are taken from a normal distribution with known parametersFor samples of size N the hypothesis H ₀ that they stem from a normal distribution with mean a ₀ and standard deviation σ ₀ is tested. A total of n exp samples are drawn in simulated experiments from a normally distributed population with mean a and standard deviation σ. For each sample the quantity

(8.10.1)

is computed. Here x i are the elements of the sample. If a = a ₀ and σ = σ ₀, then the quantity X ² follows a χ ²-distribution for N degrees of freedom. This quantity can therefore be used to perform a χ ²-test on the hypothesis H ₀. The program does not, however, perform the χ ²-test , but rather it displays a histogram of the quantity X ² together with the χ ²-distribution. One observes that for a = a ₀ and σ = σ ₀ the histogram and χ ²-distribution indeed coincide within statistical fluctuations. If, however, $$a\neq a_{0}$$ and/or $$\sigma \neq \sigma _{0}$$, then deviations appear. These deviations become particularly clear if instead of X ² the quantity

(8.10.2)

is displayed. Here F(X ², N) is the distribution function (C.5.2) of the χ ²-distribution for N degrees of freedom. F(X ², N) is equal to the probability that a random variable drawn from a χ ²-distribution is smaller than X ². Thus, P is the probability that it is greater than or equal to X ². If the hypothesis H ₀ is true, then F and therefore also P follow uniform distributions between 0 and 1. If, however, H ₀ is false, then the distribution of the X ² is not a χ ²-distribution, and the distribution of the P is not a uniform distribution. The test statistic X ² often (not completely correctly) is simply called “χ ²” and the quantity P is then called the “χ ²-probability”. Large values of X ² obviously signify that the terms in the sum (8.10.1) are on the average large compared to unity, i.e., that the x i are significantly different from a ₀. For large values of X ², however, P becomes small, cf. (8.10.2). Large values of “χ ²” therefore correspond to small values of the “χ ²-probability”. The hypothesis H ₀ is rejected at the confidence level $$\beta = 1-\alpha$$ if $${\mathsf{X}}^{2} >\chi _{ 1-\alpha }^{2}(N)$$. That is equivalent to $$F({\mathsf{X}}^{2},N) >\beta$$ or P < α.

The program allows for interactive input of the quantities n exp, N, a, a ₀, σ, σ ₀ and displays the distributions of both X ² and P(X ²) in the form of histograms.

Suggestions: (a) Choose n exp = 1000; $$a = a_{0} = 0$$, $$\sigma =\sigma _{0} = 1$$ and for N = 1, N = 2, and N = 10 display both X ² and P(X ²). (b) Repeat (a), keeping a = 0 and choosing a ₀ = 1 and a ₀ = 5. Explain the shift of the histogram for P(X ²). (c) Repeat (a), keeping σ = 1 fixed and choosing σ ₀ = 0. 5 and σ ₀ = 2. Discuss the results. (d) Modify the program so that instead of a ₀ and $$\sigma _{0}^{2}$$, the sample mean $$\bar{\mathsf{x}}$$ sample variance s ² are used for the computation of X ². The quantity X ² can be used for a χ ²-test of the hypothesis that the samples were drawn from a normal distribution. Display histograms of X ² and P(X ²) and show that X ² follows a χ ²-distribution with N − 2 degrees of freedom.

Literature

Literature Cited in the Text

[8]

M.G. Kendall and A. Stuart, The Advanced Theory of Statistics, vol. 2, Charles Griffin, London 1968MATH

[9]

S.S. Wilks, Ann. Math. Stat., 9 (1938) 60Crossref

Footnotes

1

We use here the symbol W for a distribution function in order to avoid confusion with the ratio F.

2

This is also often called the ν ²-distribution, ω ²-distribution, or Snedecor distribution.

3

The t-distribution was introduced by W. S. Gosset and published under the pseudonym “Student”.

4

Although X and the function T(x) introduced below are random variables, we do not use for them a special character type.

5

We use here the word “acceptance” of a hypothesis, although more precisely one should say, “There is no evidence to reject the hypothesis.”

© Springer International Publishing Switzerland 2014

Siegmund BrandtData Analysishttps://doi.org/10.1007/978-3-319-03762-2_9

9. The Method of Least Squares

Siegmund Brandt1 

(1)

Department of Physics, University of Siegen, Siegen, Germany

Keywords

Covariance

The method of least squares was first developed by LEGENDRE and GAUSS. In the simplest case it consists of the following prescription:

The repeated measurements y j can be treated as the sum of the (unknown) quantity x and the measurement error $$\varepsilon _{j}$$,

The quantity x should be determined such that the sum of squares of the errors $$\varepsilon _{j}$$ is a minimum,

We will see that in many cases this prescription can be derived as a result of the principle of maximum likelihood, which historically was developed much later, but that in other cases as well, it provides results with optimal properties. The method of least squares, which is the most widely used of all statistical methods, can also be used in the case where the measured quantities y j are not directly related to the unknown x, but rather indirectly, i.e., as a linear (or also nonlinear) combination of several unknowns x ₁, x ₂, …. Because of the great practical significance of the method we will illustrate the various cases individually with examples before turning to the most general case.

9.1 Direct Measurements of Equal or Unequal Accuracy

The simplest case of direct measurements with equal accuracy has already been mentioned. Suppose one has carried out n measurements of an unknown quantity x. The measured quantities y j have measurement errors $$\varepsilon _{j}$$. We now make the additional assumption that these are normally distributed about zero:

(9.1.1)

This assumption can be justified in many cases by the Central Limit Theorem. The probability to observe the value y j as the result of a single measurement is thus proportional to

The log-likelihood function for all n measurements is thus (see Example 7.2)

(9.1.2)

The maximum likelihood condition,

is thus equivalent to

(9.1.3)

This is exactly the least-squares prescription. As we have shown in Examples 7.2 and 7.6, this leads to the result that the best estimator for x is given by the arithmetic mean of the y j

(9.1.4)

The variance of this estimator is

(9.1.5)

or, if we set the measurement errors and standard deviations equal,

(9.1.6)

The more general case of direct measurements of different accuracy has also already been treated in Example 7.6. Let us assume again a normal distribution centered about zero for the measurement errors, i.e.,

(9.1.7)

Comparing with (7.​2.​7) gives the requirement of the maximum-likelihood method

(9.1.8)

The individual terms in the sum are now weighted with the inverse of the variances. The best estimator for x is then [cf. (7.​2.​8)]

(9.1.9)

i.e., the weighted mean of the individual measurements. One sees that an individual measurement thus contributes to the final result less when its measurement error is greater. From (7.​3.​20) we know the variance of $$\tilde{x}$$; it is

(9.1.10)

We can use the result (9.1.9) in order to compute the best estimates $$\tilde{\varepsilon }_{j}$$ of the original measurement errors $$\varepsilon _{j}$$ from (9.1.1) to obtain

We expect these quantities to be normally distributed about zero with the variance σ j ². That is, the quantities $$\tilde{\varepsilon }_{j}/\sigma _{j}$$ should follow a standard Gaussian distribution. According to Sect. 6.​6, the sum of squares

(9.1.11)

then follows a χ ²-distribution with n − 1 degrees of freedom.

This property of the quantity M can now be used to carry out a χ ²-test on the validity of the assumption (9.1.7). If, for a given significance level α, the quantity M exceeds the value $$\chi _{1-\alpha }^{2}$$, then we would have to recheck the assumption (9.1.7). Usually one does not doubt that the y j are in fact measurements of the unknown x. It may be, however, that the errors $$\varepsilon _{j}$$ are not normally distributed. In particular, the measurements may also be biased, i.e., the expectation value of the errors $$\varepsilon _{j}$$ may be different from zero. The presence of such systematic errors can often be inferred from the failure of the χ ²-test.

Example 9.1:

Weighted mean of measurements of different accuracy

The best values for constants of fundamental significance, such as important constants of nature, are usually obtained as weighted averages of measurements obtained by different experimental groups. For the properties of elementary particles, such mean values are compiled at regular intervals. We will consider as an example somewhat older measurements of the mass of the neutral K-meson (K⁰), taken from such a compilation from 1967 [10]. An average was computed from the results of four experiments, all carried out with different techniques. The calculation can be carried out following the scheme of Table 9.1. The resulting value of M is 7.2. If we choose a significance level of 5 %, we find from Table I.7 for three degrees of freedom χ 0. 95 ² = 7. 82. At the time of the averaging, one could therefore assume that the result $$m_{{\mathrm{K}}^{0}} = (497.9 \pm 0.2)$$ MeV represented the best value for the mass of the K-meson, as long as no further experiments were carried out. (More than 40 years later the weighted mean of all measurements was $$m_{{\mathrm{K}}^{0}} = (497.614 \pm 0.024)$$ MeV [11]). ■ 

Table 9.1:

Construction of the weighted mean from four measurements of the mass of the neutral K meson. The y j are the measured values in MeV.

j

y j

σ j

y j  g j

1

498.1

0.4

to 1cm 6.3

3038.0

0.2

0.3

2

497.44

0.33

to 1cm10

4974.4

− 0. 46

2.1

3

498.9

0.5

to 1cm 4

1995.6

1.0

4.0

4

497.44

0.5

to 1cm 4

1989.8

− 0. 46

0.8

∑

24.3

11 997.8

7.2

Let us now consider the case where the χ ²-test fails. As mentioned above, one usually assumes that at least one of the measurements has a systematic error. By investigation of the individual measurements, one can sometimes determine that one or two measurements deviate from the others by a large amount. Such a case is illustrated in Fig. 9.1a, where several different measurements are shown with their errors. (The measured value is plotted along the vertical axis; the horizontal axis merely distinguishes between the different measurements.) Although the χ ²-test would fail if all measurements from Fig. 9.1a are used, this is not the case if measurements 4 and 6 are excluded.

Fig. 9.1:

Averaging of 10 measurements where the χ ²-test fails. (a) Anomalous deviation of certain measurements: (1) Averaging of all measurements; (2) averaging without y ₄ and y ₆. (b) Errors of individual measurements clearly too small: (1) Error of the mean according to (9.1.10); (2) error of the mean according to (9.1.13).

Unfortunately the situation is not always so clear. Figure 9.1b shows a further example where the χ ²-test fails. (According to Chap.​ 8 one would reject the hypothesis that the measurements are determinations of the same quantity.) There is no single measurement, however, that is responsible for this fact. It would now be mathematically correct to not give any average value at all, and to make no statement about a best value, as long as no further measurements are available. In practice, this is clearly not satisfactory. ROSENFELD et al. [10] have suggested that the individual measurement errors should be increased by a scale factor $$\sqrt{ M/(n - 1)}$$ , i.e., one replaces the σ j  by

(9.1.12)

The weighted mean $$\tilde{x}$$ obtained by using these measurement errors does not differ from that of expression (9.1.9). The variance, however, is different from (9.1.10). It becomes

(9.1.13)

We now compute the analogous expression to (9.1.11),

(9.1.14)

This is the expectation value of χ ² for n − 1 degrees of freedom. Equation (9.1.14) clearly provided the motivation for relation (9.1.12). We repeat that this relation has no rigorous mathematical basis. It should be used with caution, since it hides the influence of systematic errors. On the other hand it provides reasonable errors for the mean for cases such as those in Fig. 9.1b, while direct application of Eq. (9.1.10) leads to an error that is far to small to reflect the actual dispersion of the individual measurements about the mean. Both solutions for the error of the mean are shown in Fig. 9.1b.

9.2 Indirect Measurements: Linear Case

Let us now consider the more general case of several unknown quantities x i  (i = 1, 2, …, r). The unknowns are often not measured directly. Instead, only a set of linear functions of the x i are measurable,

(9.2.1)

We now write this relation in a somewhat different form,

(9.2.2)

We define a column vector,

(9.2.3)

and write (9.2.2) in the more compact form,

(9.2.4)

If we define in addition

(9.2.5)

then the system of equations (9.2.4) can be written as a matrix equation,

(9.2.6)

Of course the measured quantities still have measurement errors $$\varepsilon _{j}$$, which we assume to be normally distributed. We then have¹

(9.2.7)

Since the y j are independent measurements, we can arrange the variances σ j ² in a diagonal covariance matrix for y j or $$\varepsilon _{j}$$,

(9.2.8)

In analogy to (9.1.7) we call the inverse of the covariance matrix a weight matrix ,

(9.2.9)

If we now put the measurements and errors together into vectors, we obtain from (9.2.7)

(9.2.10)

From (9.2.6) one then has

(9.2.11)

We want to solve this system of equations for the unknowns $$\mathbf{x}$$ with the maximum-likelihood method. With our assumption (9.2.7), the measurements y j are normally distributed with the probability density

(9.2.12)

Thus for all measurements one obtains likelihood functions

(9.2.13)

(9.2.14)

This expression is clearly a maximum when

(9.2.15)

Using (9.2.9) and (9.2.11), we can rewrite this expression as

(9.2.16)

or

(9.2.17)

or in abbreviated form

(9.2.18)

(9.2.19)

We will simplify this expression further by using the Cholesky decomposition (cf. Sect. A.9) of the positive-definite symmetric weight matrix G y ,

(9.2.20)

In the frequently occurring case of uncorrelated measurements (9.2.9) one has

(9.2.21)

Using the notation

(9.2.22)

Eq. (9.2.19) takes on the simple form

(9.2.23)

The method for solving this equation for $$\mathbf{x}$$ is described in detail in Appendix A, in particular in Sects. A.5 through A.14. The solution can be written in the form

(9.2.24)

[See (A.10.3).] Here A′⁺ is the pseudo-inverse of the matrix A′ (see Sect. A.10). In Sect. A.14, a solution of the form of (9.2.24) is indeed found with the help of the singular value decomposition of the matrix A′. This procedure is particularly accurate numerically.

To compute by hand one uses instead of Eq. (9.2.24) the mathematically equivalent expression for the solution of the normal equations [see (A.5.17)],

(9.2.25)

or, with (9.2.22) in terms of the quantities $$\mathbf{c}$$ and A,

(9.2.26)

The solution includes, of course, the special case of Sect. 9.1. In the case of direct measurements of different accuracy, $$\mathbf{x}$$ has only one element, $$\mathbf{a}_{0}$$ vanishes, and A is simply an n-component column vector whose elements are all equal to − 1. One then has

and (9.2.25) becomes

which is identical to (9.1.9).

The solution of (9.2.26) represents a linear relation between the solution vector $$\tilde{\mathbf{x}}$$ and the vector of measurements $$\mathbf{y}$$, since $$\mathbf{c} = \mathbf{y} + \mathbf{a}_{0}$$. We can thus apply the error propagation techniques of Sect. 3.​8.​ Using (3.​8.​2) and (3.​8.​4) one immediately obtains

The matrices G y , G y ⁻¹, and $$({A}^{\mathrm{T}}G_{y}A)$$ are symmetric, i.e., they are identical to their transposed matrices. Using the rule (A.1.8), this expression simplifies to

(9.2.27)

We have thus obtained a simple expression for the covariance matrix of the estimators $$\tilde{\mathbf{x}}$$ for the unknowns $$\mathbf{x}$$. The square roots of the diagonal elements of this matrix can be viewed as “measurement errors”, although the quantities x were not directly measured.

We can also use the result (9.2.26) to improve the original measurements y. By substituting (9.2.26) into (9.2.11) one obtains a vector of estimators of the measurement errors $$\boldsymbol{\upvarepsilon }$$,

(9.2.28)

These measurement errors can now be used to compute improved measured values,

(9.2.29)

The $$\tilde{\boldsymbol{\upeta }}$$ are again linear in y. We can again use error propagation to determine the covariance matrix of the improved measurements,

(9.2.30)

The improved measurements $$\tilde{\eta }$$ satisfy (9.2.1) if the unknowns are replaced by their estimators $$\tilde{x}$$.

9.3 Fitting a Straight Line

We will examine in detail the simple but in practice frequently occurring task of fitting a straight line to a set of measurements y j at various values t j of a so-called controlled variable t. The values of these variables will be assumed to be known exactly, i.e., without error. The variable t i could be, for example, the time at which an observation y i is made, or a temperature or voltage that is set in an experiment. (If t also has an error, then fitting a line in the (t, y) plane becomes a nonlinear problem. It will be treated in Sect. 9.10, Example 9.11.)

In the present case the relation (9.2.1) has the simple form

or using vector notation,

We will attempt to determine the unknown parameters

from the measurements in Table 9.2.

A comparison of our problem to Eqs. (9.2.2) through (9.2.6) gives a ₀ = 0,

The matrices G y and H are found by substitution of the last line of Table 9.2 into (9.2.9) and (9.2.21),

One thus has

To invert the 2 × 2 matrix we use (A.6.8). The solution (9.2.25) is then

The covariance matrix of for $$\tilde{\mathbf{x}}$$ is

Its diagonal elements are the variances of $$\tilde{x}_{1}$$ and $$\tilde{x}_{2}$$, and their square roots are the errors,

Table 9.2:

Data for fitting a straight line.

j

1

2

3

4

t j

0.0

1.0

2.0

3.0

y j

1.4

1.5

3.7

4.1

σ j

0.5

0.2

1.0

0.5

For the correlation coefficient between $$\tilde{x}_{1}$$ and $$\tilde{x}_{2}$$ one finds

The improved measurements are

They lie on a line given by $$\tilde{\boldsymbol{\upeta }} = -A\tilde{\mathbf{x}}$$, which of course is different in general from the “true” solution. The “residual errors” of $$\tilde{\boldsymbol{\upeta }}$$ can be obtained with (9.2.30),

The square roots of the diagonal elements are

The fit procedure, in which more measurements (four) were used than were necessary to determine the two unknowns, has noticeably reduced the individual errors of the measurements in comparison to the original values σ j .

Finally we will compute the value of the minimum function (9.2.16),

With this result we can carry out a χ ²-test on the goodness-of-fit of a straight line to our data. Since we started with n = 4 measured values and have determined r = 2 unknown parameters, we still have $$n - r = 2$$ degrees of freedom available (cf. Sect. 9.7). If we choose a significance level of 5 %, we find from Table I.7 for two degrees of freedom χ 0. 95 ² = 5. 99. There is thus no reason to reject the hypothesis of a straight line.

Fig. 9.2:

Fit of a straight line to data from Table 9.2. (a) Original measured values and errors; (b) improved measurement values and residual errors; (c) covariance ellipse for the fitted quantities x ₁, x ₂; (d) various lines corresponding to individual points on the covariance ellipse.

The results of the fit are shown in Fig. 9.2. The measurements y j are shown as functions of the variables t. The vertical bars give the measurement errors. They cover the range y j ±σ j . The plotted line corresponds to the result $$\tilde{x}_{1}$$, $$\tilde{x}_{2}$$. The improved measurements lie on this line. They are shown in Fig. 9.2b together with the residual errors $$\varDelta \tilde{\eta }_{j}$$. In order to illustrate the accuracy of the estimates $$\tilde{x}_{1}$$, $$\tilde{x}_{2}$$, we consider the covariance matrix $$C_{\tilde{x}}$$. It determines a covariance ellipse (Sect. 5.​10) in a plane spanned by the variables x ₁, x ₂. This ellipse is shown in Fig. 9.2c. Points on the ellipse correspond to fits of equal probability. Each of these points determines a line in the (t, y) plane. Some of the points are indicated in Fig. 9.2c and the corresponding lines are plotted in Fig. 9.2d. The points on the covariance ellipse thus correspond to a bundle of lines. The line determined by the “true” values of the unknowns lies in this bundle with the probability $$1 -{\mathrm{e}}^{-1/2}$$; cf. (5.​10.​18).

9.4 Algorithms for Fitting Linear Functions of the Unknowns

The starting point of the ideas in Sect. 9.2 was the assumption (9.2.6) of a linear relationship between the “true” values $$\boldsymbol{\upeta }$$ of the measured quantities y and the unknowns x. We will write this relation in the form

(9.4.1)

or in terms of components,

(9.4.2)

Often it is useful to consider the index j as specifying that the measurement y j corresponds to a value t j of a controlled variable, which is taken to be known without error. Then (9.4.2) can be written in the form

(9.4.3)

This relation describes a curve in the (t, η) plane. It is characterized by the parameters x. The determination of the parameters is thus equivalent to fitting a curve to the measurements y j  = y(t j ). Usually the individual measurements y j are uncorrelated. The weight matrix G y is diagonal; it has the Cholesky decomposition (9.2.20). One then has simply

see (9.2.22).

9.4.1 Fitting a Polynomial

As a particularly simple but useful example of a function for (9.4.3), let us consider the relation

(9.4.4)

This is a polynomial in t j , but is linear in the unknowns x. The special case r = 2 has been treated in detail in Sect. 9.3.

A comparison of (9.4.4) with (9.4.2) yields directly

or more completely,

The class LsqPol performs the fit of a polynomial to data.

Example 9.2:

Fitting of various polynomials

It is often interesting to fit polynomials of various orders to measured data, as long as the number of degrees of freedom of the fit is at least one, i.e., n > ℓ + 1. As a numerical example we will use measurements from an elementary particle physics experiment. Consider an investigation of elastic scattering of negative K mesons on protons with a fixed K meson energy. The distribution of the cosine of the scattering angle Θ in the center-of-mass system of the collision is characteristic of the angular momentum of possible intermediate states in the collision process. If, in particular, the distribution is considered as a polynomial in cosΘ, the order of the polynomial can be used to determine the spin quantum numbers of such intermediate states.

The measured values y j (j = 1, 2, …, 10) are simply the numbers of collisions for which cosΘ was observed in a small interval around t j  = cosΘ j . As measurement errors the statistical errors were used, i.e., the square roots of the number of observations. The data are given in Table 9.3. The results of the fit of polynomials of various orders are summarized in Table 9.4 and Fig. 9.3. With the χ ²-test we can check successively whether a polynomial of order zero, one, … gives a good fit to the data.

Table 9.3:

Data for Example 9.2. One has $$\sigma _{j} = \sqrt{y_{j}}$$.

j

t j  = cosΘ j

y j

1

to 1.5cm − 0. 9

81

2

t o 1.5cm − 0. 7

50

3

to 1.5cm − 0. 5

35

4

t o 1.5cm − 0. 3

27

5

to 1.5cm − 0. 1

26

6

to 1.5cm 0.1

60

7

to 1.5cm 0.3

106

8

to 1.5cm 0.5

189

9

to 1.5cm 0.7

318

10

to 1.5cm 0.9

520

Table 9.4:

Summary of results from Example 9.2 (n = 10 measured points, r parameters, $$f = n - r$$ degrees of freedom).

r

f

M

1

57.85

9

833.55

2

82.66

99.10

8

585.45

3

47.27

185.96

273.61

7

36.41

4

37.94

126.55

312.02

137.59

6

2.85

5

39.62

119.10

276.49

151.91

52.60

5

1.68

6

39.88

121.39

273.19

136.58

56.90

16.72

4

1.66

Fig. 9.3:

Fit of polynomials of various orders (0, 1, …, 5) to the data from Example 9.2.

We see that the first two hypotheses (a constant and a straight line) are not in agreement with the experimental data. This can be seen in Fig. 9.3 and is also reflected in the values of the minimum function. The hypothesis r = 3, a second-order polynomial, gives qualitative agreement. Most of the measurements, however, do not fall on the fitted parabola within the error bars. The χ ²-test fails with a significance level of 0.0001. For the hypotheses r = 4, 5, and 6, however, the agreement is very good. The fitted curves go through the error bars and are almost identical. The χ ²-test does not call for a rejection of the hypothesis even at α = 0. 5. We can therefore conclude that a third order polynomial is sufficient to describe the data. An even more careful investigation of the question as to what order a polynomial must be used to describe the data is possible with orthogonal polynomials; cf. Sect. 12.​1.​ ■ 

9.4.2 Fit of an Arbitrary Linear Function

The matrix A and the vector c enter into the solution of the problem of Sect. 9.2. They depend on the form of the function to be fitted and must therefore be provided by the user. (In Sect. 9.4.1 the function was known, so the user did not have to worry about computing A and c.) The class LsqLin performs the fit of an arbitrary linear function to data.

Example 9.3:

Fitting a proportional relation

Suppose from the construction of an experiment it is known that the true value η j of the measurement y j is directly proportional to the value of the controlled variable t j :

The constant of proportionality x ₁ is to be determined from the measurements. This relation is simpler than a first-order polynomial, which contains two constants. A comparison with (9.4.2) gives

and thus

Fig. 9.4:

Fit of a proportional relation (above) and a first-order polynomial (below) to data.

Shown in Fig. 9.4 are the results of the fit of a line through the origin, i.e., a proportionality, and the fit of a first-order polynomial. The value of the minimum function is clearly smaller in the case of the general line, and the fit is visibly better. The number of degrees of freedom, however, is less, and the desired constant of proportionality is not determined. ■ 

9.5 Indirect Measurements: Nonlinear Case

If the relation between the n-vector $$\boldsymbol{\upeta }$$ of true values of the measured quantities y and the r-vector of unknowns is given by a function,

that is not – as (9.4.1) – linear in x, then our previous procedure fails in the determination of the unknowns.

Instead of (9.2.2) one has

(9.5.1)

or in vector notation,

(9.5.2)

We can, however, relate this situation to the linear case if we expand the f j in a Taylor series and keep only the first term. We carry out the expansion about the point $$\mathbf{x}_{0} = (x_{10},x_{20},\ldots,x_{r0})$$, which is a first approximation for the unknowns, which has been obtained in some way,

(9.5.3)

If we now define

(9.5.4)

(9.5.5)

(9.5.6)

and use the relation (9.2.10), then we obtain

(9.5.7)

Thus we can now write the system of equations (9.5.2) in the form

(9.5.8)

The least-squares condition (9.2.16) is then

(9.5.9)

in complete analogy to (9.2.19). Using the quantities defined in (9.2.20) through (9.2.22) we can obtain the solution directly from (9.2.24),

(9.5.10)

Corresponding to (9.5.4) one can use $$\tilde{\boldsymbol{\upxi }}$$ to find a better approximation

(9.5.11)

and compute new values of A and c at the point x ₁. A solution $$\tilde{\boldsymbol{\upxi }}$$ (9.5.10) for (9.5.9) with these values gives $$\mathbf{x}_{2} = \mathbf{x}_{1} + \tilde{\boldsymbol{\upxi }}$$, etc. This iterative procedure can be terminated when the minimum function in the last step has not significantly decreased in comparison to the result of the previous step.

There is no guaranty, however, for the convergence of this procedure. Heuristically it is clear, however, that the expectation for convergence is greater when the Taylor series, truncated after the first term, is a good approximation in the region over which x is varied in the procedure. This is at least the region between x ₀ and the solution $$\tilde{\mathbf{x}}$$. (Intermediate steps can also lie outside of this region.) It is therefore important, particularly in highly nonlinear problems, to start from a good first approximation.

If the solution $$\tilde{\mathbf{x}} = \mathbf{x}_{n} = \mathbf{x}_{n-1} + \tilde{\boldsymbol{\upxi }}$$ is reached in n steps, then it can be expressed as a linear function of $$\tilde{\boldsymbol{\upxi }}$$. Using error propagation one finds that the covariance matrices of $$\tilde{\mathbf{x}}$$ and $$\tilde{\boldsymbol{\upxi }}$$ are then identical and one finds that

(9.5.12)

The covariance matrix loses its validity, however, if the linear approximation (9.5.3) is not a good description in the region $$\tilde{x}_{i} \pm \varDelta x_{i}$$, i = 1, …, r. (Here $$\varDelta x_{i} = \sqrt{C_{ii}}$$.)

9.6 Algorithms for Fitting Nonlinear Functions

It is sometimes useful to set one or several of the r unknown parameters equal to given values, i.e., to treat them as constants and not as adjustable parameters. This can clearly be done by means of a corresponding definition of f in (9.5.1). For the user, however, it is more convenient to write only one subprogram that computes the function f for the r original parameters and when needed, to communicate to the program that the number of adjustable parameters is to be reduced from r to r′. Of course a list with r elements ℓ i must also be given, in which one specifies which parameters x i are to be held constant (ℓ i  = 0) and which should remain adjustable (ℓ i  = 1).

Two more difficulties come up when implementing the considerations of the previous section in a program:

On the one hand, the elements of the matrix A must be found by constructing the function to be fitted and differentiating it with respect to the parameters. Of course it is particularly convenient for the user if he does not have to program these derivatives himself, but rather can turn this task over to a routine for numerical differentiation. In Sect. E.1 we provide such a subroutine, which we will call from the program discussed below. Numerical differentiation implies, however, a loss of precision and an increase in computing time. In addition, the method can fail. Our programs communicate such an occurrence by means of an output parameter. Thus in some cases the user will be forced to program the derivatives by hand.

The second difficulty is related to the fact that the minimum function

(9.6.1)

is no longer a simple quadratic form of the unknowns like (9.2.17) and (9.2.23). One consequence of this is that the position of the minimum $$\tilde{\mathbf{x}}$$ cannot be reached in a single step. In addition, the convergence of the iterative procedure strongly depends on whether the first approximation x ₀ is in a region where the minimum function is sufficiently similar to a quadratic form. The determination of a good first approximation must be handled according to the given problem. Some examples are given below. If one constructs the iterative procedure as indicated in the previous section, then it can easily happen that the minimum function does not decrease with every step. In order to ensure convergence despite this, two methods can be applied, which are described in Sects. 9.6.1 and 9.6.2. The first (reduction of step size) is simpler and faster. The second (the Marquardt procedure) has, however, a larger region of convergence. We give programs for both methods, but recommend applying Marquardt procedure in cases of doubt.

9.6.1 Iteration with Step-Size Reduction

As mentioned, the inequality

(9.6.2)

does not hold in every case for the result x i of step i. The following consideration helps us in handling such steps. Let us consider the expression $$M(\mathbf{x}_{i-1} + s\tilde{\boldsymbol{\upxi }})$$ as a function of the quantity s with 0 ≤ s ≤ 1. If we replace $$\tilde{\boldsymbol{\upxi }}$$ by $$s\tilde{\boldsymbol{\upxi }}$$ in (9.5.9), then we obtain

Differentiating with respect to s gives

or with $$\mathbf{c}' = -A'\tilde{\boldsymbol{\upxi }}$$, cf. (9.5.10),

and thus

if $${A'}^{\mathrm{T}}A' = {A}^{\mathrm{T}}G_{y}A$$ is positive definite. (This is always the case near the minimum.) The matrix A′T A′ gives the curvature of the function M in the space spanned by the unknowns x ₁, …, x r . For only one such unknown, the region of positive curvature is the region between the points of inflection around the minimum (cf. Fig. 10.1). Since the function M is continuous in s, there exists a value λ > 0 such that

After an iteration i + 1, for which (9.6.2) does not hold, one multiplies $$\tilde{\boldsymbol{\upxi }}$$ by a number s, e.g., $$s = 1/2$$, and checks whether

If this is the case, then one sets $$\mathbf{x}_{i} = \mathbf{x}_{i-1} + s\tilde{\boldsymbol{\upxi }}$$. If it is not the case, then one multiplies again with s, and so forth.

The class LsqNon operates according to this procedure of iteration with step-size reduction. As in the linear case we consider the measurements as being dependent on a controlled variable t, i.e., $$y_{j} = y_{j}(t_{j})$$. For the true values η j corresponding to the measurements one has

or [cf. (9.6.1)]

This function has to be programmed by the user. That is done within an extension of the abstract class DatanUserFunction, see the example programs in Sect. 9.14. The matrix A is computed by numerical differentiation with the class AuxDri. Should the accuracy of that method not suffice, the user has to provide a method with the same name and the same method declarations, computing the matrix A by analytic differentiation.

Example 9.4:

Fitting a Gaussian curve

In many experiments one has signals y(t) that have the form of a Gaussian curve,

(9.6.3)

One wishes to determine the parameters x ₁, x ₂, x ₃ that give the amplitude, position of the maximum, and the width of the signal.

Fig. 9.5:

Measured values and fitted Gaussian curve (above); logarithm of the measured values (below).

Figure 9.5 shows the result of the fit to data. The values x ₁ = 0. 5, x ₂ = 1. 5, x ₃ = 0. 2 were used as a first approximation. In fact we could have estimated significantly better initial values directly from the plot of the measurements, i.e., x ₁ ≈ 1 for the amplitude, x ₂ ≈ 1. 25 for the position of the maximum, and x ₃ ≈ 0. 4 for the width.

Here we have just mentioned a particularly useful aid in determining the initial approximations: the analysis of a plot of the data by eye. One can also proceed more formally, however, and consider the logarithm of (9.6.3),

This is a polynomial in t, which is linear in the three terms in parentheses,

By taking the logarithm, however, the distribution for the measurement errors, originally Gaussian, has been changed, so that strictly speaking one cannot fit a polynomial to determine a ₁, a ₂, and a ₃. For purposes of determining the first approximation we can disregard this difficulty and set the errors of the lny(t i ) equal to one. We then determine the quantities a ₁, a ₂, a ₃, and from them the values x ₁, x ₂, x ₃, and use these as initial values.

The logarithms of the measured values y i are shown in the lower part of Fig. 9.5. One can see that fit range for the parabola must be restricted to points in the bell-shaped region of the curve, since the points in the tails are subject to large fluctuations. ■ 

Example 9.5:

Fit of an exponential function

In studies of radioactivity, for example, a function of the form

(9.6.4)

must be fitted to measured values y i (t i ). The program to be provided here by the user can have the following form.

Fig. 9.6:

Measured values and fitted exponential function (above); logarithms of the measured values (below).

In Fig. 9.6 the result of a fit to data is shown. The determination of the first approximation for the unknowns can again be obtained by fitting a straight line (graphically or numerically) to the logarithm of the function y(t),

Here one usually uses only the values at small t, since these points have smaller fluctuations. ■ 

Example 9.6:

Fitting a sum of exponential functions

A radioactive substance often consists of a mixture of components with different decay times. One must therefore fit a sum of several exponential functions. We will consider the case of two functions

(9.6.5)

Fig. 9.7:

Measured values with the fitted sum of two exponential functions (above); logarithm of the measured values (below).

Figure 9.7 shows the result of fitting a sum of two exponential functions to the data. A first approximation can be determined by fitting two different lines to lny i (t i ) for the regions of smaller and larger t i . ■ 

9.6.2 Marquardt Iteration

The procedure with step-size reduction discussed in Sect. 9.6.1 leads to a minimum when x is already in a region where A′T A′ is positive definite, i.e., in a region around the minimum where the function M(x) has a positive curvature. (In the one-dimensional case of Fig. 10.1, this is the region between the two points of inflection on either side of the minimum.) It is clear, however, that it must be possible to extend the region of convergence to the region between the two maxima surrounding the minimum. This possibility is offered by the Marquardt procedure, which is presented in Sect. 10.​15 in a somewhat more general form. The class LsqMar treats the nonlinear case of least squares; the user’s task is much the same as in the case of LsqNon. Readers who want to became familiar with this class should first study Sect. 10.​15 and the introductory sections of Chap.​ 10 and finally Sect. A.17

Example 9.7:

Fitting a sum of two Gaussian functions and a polynomial

In practice it is usually not so easy to find the amplitude, position, and width of signals as was shown in Example 9.4. One usually has more than one signal, lying on a background that varies slowly with the controlled variable t. Since in general this background is not well known, it is approximated by a line or a second-order polynomial. We will consider the sum of such a polynomial and two Gaussian distributions, i.e., a function of nine unknown parameters,

(9.6.6)

The derivatives (9.5.5) are

If the numerical differentiation fails, the derivatives must be computed with a specially written version of the routine AuxDri. For the numerical example shown in Fig. 9.8, however, this is not necessary. The user must supply, of course, an extension of DatanUserFunction to compute (9.6.6).

Figure 9.8 shows the result of fitting to a total of 50 measurements. It is perhaps interesting to look at some of the intermediate steps in Fig. 9.9. As a first approximation the parameters of the polynomial were set to zero ($$x_{1} = x_{2} = x_{3} = 0$$). For both of the clearly visible signals, rough estimates of the amplitude, position, and width were used for the initial values. The function (9.6.6) with the parameters of this first approximation is shown in the first frame (STEP 0). In the following frames one sees the change of the function after 2, 4, 6, and 8 steps, and finally at convergence of the class LsqMar after a total of 9 steps. ■ 

Fig. 9.8:

Measured values and fitted sum of a second-order polynomial and two Gaussian functions.

Fig. 9.9:

Successive approximations of the fit function to the measurements.

9.7 Properties of the Least-Squares Solution: χ ²-Test

Up to now the method of least squares has merely been an application of the maximum-likelihood method to a linear problem. The prescription of least squares (9.2.15) was obtained directly from the maximization of the likelihood function (9.2.14). In order to be able to specify this likelihood function in the first place, it was necessary to know the distribution of the measurement errors. We assumed a normal distribution. But also when there is no exact knowledge of the error distribution, one can still apply the relation (9.2.15) and with it the remaining formulas of the last sections. Such a procedure seems to lack a theoretical justification. The Gauss–Markov theorem, however, states that in this case as well, the method of least squares provides results with desirable properties. Before entering into this, let us list once more the properties of the maximum-likelihood solution.

(a)

The solution $$\tilde{\mathbf{x}}$$ is asymptotically unbiased, i.e., for very large samples,

(b)

It is a minimum variance estimator, i.e.,

(c)

The quantity (9.2.16)

follows a χ ²-distribution with n − r degrees of freedom.

The properties (a) and (b) are familiar from Chap.​ 7.​ We will demonstrate the validity of (c) for the simple case of direct measurements (r = 1), for which the matrix G y is diagonal:

The quantity M then simply becomes a sum of squares,

(9.7.1)

Since each $$\varepsilon _{j}$$ comes from a normally distributed population with mean of zero and variance σ j ², the quantities $$\varepsilon _{j}^{2}/\sigma _{j}^{2}$$ are described by a standard Gaussian distribution. Thus the sum of squares follows a χ ²-distribution with n − 1 degrees of freedom.

If the distribution of the errors $$\varepsilon _{j}$$ is not known, then the least-squares solution has the following properties:

(a)

The solution is unbiased.

(b)

Of all solutions $${\mathbf{x}}^{{\ast}}$$ that are unbiased estimators for x and are linear combinations of the measurements y, the least-squares solution has the smallest variance. (This is the GAUSS–MARKOV theorem .)

(c)

The expectation value of

is

(This is exactly the expectation value of a χ ² variable for n − r degrees of freedom.)

The quantity M is often simply called χ ², although it does not necessarily follow a χ ²-distribution. Together with the matrices $$C_{\tilde{x}}$$ and $$C_{\tilde{\eta }}$$ it provides a convenient measure of the quality of a fit with the least-squares method. If the value of M obtained from the measurements comes out much larger than n − r, then the assumptions of the calculation must be carefully checked. The result should not simply be accepted without critical examination.

The number $$f = n - r$$ is called the number of degrees of freedom of a fit or the number of equations of constraint of the fit . It is clear from the beginning (see Appendix A), that the problem of least squares can only be solved for f ≥ 0. Only for f > 0, however, is the quantity M meaningfully defined and usable for testing the quality of a fit.

If it is known that the errors are normally distributed, then a χ ²-test can be done in conjunction with the fit. One rejects the result of the fit if

(9.7.2)

i.e., if the quantity M exceeds the quantile of the χ ²-distribution with n − r degrees of freedom at a significance level of α. A larger value of M can be caused be the following reasons (and also by an error of the first kind):

(a)

The assumed functional dependence $$\mathbf{f}(\mathbf{x},\boldsymbol{\upeta }) = 0$$ between the measured values η and the unknown parameters x is false. Either the function $$\mathbf{f}(\mathbf{x},\boldsymbol{\upeta })$$ is completely wrong, or some of the parameters taken to be known are not correct.

(b)

The function $$\mathbf{f}(\mathbf{x},\boldsymbol{\upeta })$$ is correct, but the series expansion with only one term is not a sufficiently good approximation in the region of parameter space covered in the computation.

(c)

The initial approximation x ₀ is too far from the true value x. Better values for x ₀ could lead to more acceptable values of M. This point is clearly related to (b).

(d)

The covariance matrix of the measurements C y , which is often only based on rough estimates or assumptions, is not correct.

These four points must be carefully taken into consideration if the method of least squares is to be applied successfully. In many cases the least-squares computation is repeated many times for different data sets. One can then look at the empirical frequency distribution of the quantity M and compare it to a χ ²-distribution with the corresponding number of degrees of freedom. Such a comparison is particularly useful for a good estimate of C y . At the start of a new experiment, the apparatus that provides the measured values y is usually checked by measuring known quantities. In this way the parameters x for several data sets are determined. The covariance matrix of the measurements can then be determined such that the distribution (i.e., the histogram) of M agrees as well as possible with a χ ²-distribution. This investigation is particularly illustrative when one considers the distribution of F(M) instead of that of M, where F is the distribution function (6.​6.​11) of the χ ²-distribution. If M follows a χ ²-distribution, then F(M) follows a uniform distribution.

9.8 Confidence Regions and Asymmetric Errors in the Nonlinear Case

We begin this section with a brief review of the meaning of the covariance matrix $$C_{\tilde{x}}$$ of the unknown parameters for the linear least-squares case. The probability that the true value of the unknowns is given by x is given by a normal distribution of the form (5.​10.​1),

(9.8.1)

where $$B = C_{\tilde{x}}^{-1}$$. The exponent (multiplied by − 2),

(9.8.2)

for $$g = 1 = \mbox{ const}$$ describes the covariance ellipsoid (cf. Sect. 5.​10). For other values of g(x) = const one obtains confidence ellipsoids for a given probability W [see (5.​10.​19)]. In this way confidence ellipsoids can easily be constructed that have the point $$\mathbf{x} = \tilde{\mathbf{x}}$$ at the center and which contain the true value x of the unknown with probability W = 0. 95.

We can now find a relation between the confidence ellipsoid and the minimum function. For this we use the expression (9.2.19), which is exactly true in the linear case, and we compute the difference between the minimum function M(x) at a point x and the minimum function $$M(\tilde{\mathbf{x}})$$ at the point $$\tilde{\mathbf{x}}$$ where it is a minimum,

(9.8.3)

According to (9.2.27), one has

(9.8.4)

So the difference (9.8.3) is exactly the function introduced above, g(x).

Thus the covariance ellipsoid is in fact the hypersurface in the space of the r variables x ₁, …, x r , where the function M(x) has the value $$M(\mathbf{x}) = M(\tilde{\mathbf{x}}) + 1$$. Correspondingly, the confidence ellipsoid with probability W is the hypersurface for which

(9.8.5)

where the constant g is the quantile of the χ ²-distribution with $$f = n - r$$ degrees of freedom according to (5.​10.​21),

(9.8.6)

In the nonlinear case of least squares, our considerations remain approximately valid. The approximation becomes better when the nonlinear deviations of the expression (9.5.3) are less in the region where the unknown parameters are varied. The deviations are not only small when the derivatives are almost constant, i.e., the function is almost linear, but also when the variation of the unknown parameters is small. A measure of the variation of an unknown is, however, its error. Because of error propagation, the errors of the unknowns are small when the original errors of the measurements are small. In the nonlinear case, therefore, the covariance matrix retains the same interpretation as in the linear case as long as the measurement errors are small.

If the measurement errors are large we retain the interpretation (9.8.5), i.e., we determine the hypersurface with (9.8.5) and state that the true value x lies with probability W within the confidence region around the point. This region is, however, no longer an ellipsoid.

For only one parameter x, the curve M = M(x) can easily be plotted. The confidence region is simply a segment of the x axis. For two parameters x ₁, x ₂ the boundary of the confidence region is a curve in the (x ₁, x ₂) plane. It is the contour line (9.8.5) of the function M = M(x). The graphical representation of these contours is performed by the method DatanGraphics.drawContour, cf. Appendix F.5. For more then two parameters, a graphical representation is only possible in the form of slices of the confidence region in the (x i , x j ) plane in the $$(x_{1},x_{2},\ldots,x_{r})$$ space through the point $$\tilde{\mathbf{x}} = (\tilde{x}_{1},\tilde{x}_{2},\ldots,\tilde{x}_{r})$$, where x i , x j can be any possible pair of variables.

Example 9.8:

The influence of large measurement errors on the confidence region of the parameters for fitting an exponential function

The results of fitting an exponential function as in Example 9.5 to data points with errors of different sizes are shown in Fig. 9.10. The two fitted parameters x ₁ and x ₂ and their errors and correlation coefficient are also shown in the figure. The quantities Δ x ₁, Δ x ₂, and ρ were computed directly from the elements of the covariance matrix $$C_{\tilde{x}}$$. As expected, these errors increase for increasing measurement errors.

Fig. 9.10:

Fit of an exponential function $$\eta = x_{1}\exp (-x_{2}t)$$ to measurements with errors of different sizes.

Fig. 9.11:

Results of the fit from Fig. 9.10 in the parameter space x ₁, x ₂. Solution $$\tilde{\mathbf{x}}$$ (small circle), symmetric errors (error bars), covariance ellipse, asymmetric error limits (horizontal and vertical lines), confidence region corresponding to the probability of the covariance ellipse (dark contour).

The covariance ellipses for this fit are shown as thin lines in the various frames of Fig. 9.11. The small circle in the center indicates the solution $$\tilde{\mathbf{x}}$$. In addition the vertical bars through the point $$\tilde{\mathbf{x}}$$ mark the regions $$\tilde{x}_{1} \pm \varDelta x_{1}$$ and $$\tilde{x}_{2} \pm \varDelta x_{2}$$. As a thicker line a contour is indicated which surrounds the confidence region $$M(\mathbf{x}) = M(\tilde{\mathbf{x}}) + 1$$. We see that for small measurement errors the confidence region and covariance ellipse are practically identical, while for large measurement errors they clearly differ from each other. ■ 

Computing and plotting the confidence regions requires, to the extent that they differ from the confidence ellipsoids, considerable effort. It is therefore important to be able to decide whether a clear difference between the two exists. For clarity we will stay with the case of two variables and consider again Example 9.8 and in particular Fig. 9.11. The errors $$\varDelta x_{i} =\sigma _{i} = \sqrt{C_{\tilde{x} } (i, i)}$$ obtained from the covariance matrix $$C_{\tilde{x}}$$ have the following property. The lines $$x_{i} =\tilde{ x}_{i} \pm \varDelta x_{i}$$ are tangent to the covariance ellipse. If the covariance ellipse must be replaced by a confidence region of less regular form, then we can nevertheless find the horizontal and vertical tangents, and at those places in Fig. 9.11 one has

(9.8.7)

Because of the loss of symmetry, the errors Δ x ₊ and Δ x ₋ are in general different. One speaks of asymmetric errors . For r variables one has tangent hypersurfaces of dimension r − 1 instead of tangent lines.

We now give a procedure to compute the asymmetric errors Δ x i±. One is only interested in asymmetric confidence regions when the asymmetric errors are significantly different from the symmetric ones. The values $$x_{i\pm } =\tilde{ x}_{i} \pm \varDelta x_{i\pm }$$ have the property that the minimum of the function M for a fixed value of x i  = x i± has the value

(9.8.8)

with g = 1. For other values of g one obtains corresponding asymmetric confidence boundaries. If we bring all the terms in (9.8.8) to the left-hand side,

(9.8.9)

then we see the problem to be solved is a combination of minimization and zero-finding. We find the zeros using an iterative procedure as in Sect. E.2. The zero, i.e., the point x i fulfilling (9.8.9), is first bracketed by the values x small and x big, corresponding to negative and positive values of (9.8.9). Next, this interval is reduced by successively dividing it in half until the expression (9.8.9) differs from zero by no more than g∕100. The minimum in (9.8.9) is found with LsqNon or LsqMar. The classes, by which the asymmetric errors are determined, are LsqAsn, if LsqNon is used, and LsqAsm for the use of LsqMar. The asymmetric errors for Example 9.8 are shown in Fig. 9.11.

9.9 Constrained Measurements

We now return to the case of Sect. 9.1, where the quantities of interest were directly measured. The N measurements are, however, no longer completely independent, but rather are related to each other by q equations of constraint. One could measure, for example, the three angles of a triangle. The equation of constraint says that their sum is equal to $$18{0}^{\circ }$$. We again ask for the best estimates $$\tilde{\eta }_{j}$$ for the quantities η j . The measurements give instead the values

(9.9.1)

As above let us assume a normal distribution about zero for the measurement errors $$\varepsilon _{j}$$:

The q equations of constraint have the form

(9.9.2)

Let us first consider the simple case of linear equations of constraint. The Eqs. (9.9.2) are then of the form

(9.9.3)

or in matrix notation,

(9.9.4)

9.9.1 The Method of Elements

Although not well suited for automatic processing of data, an illustrative procedure is the method of elements . We can use the q equations (9.9.3) to eliminate q of the n quantities η. The remaining n − q quantities $$\alpha _{i}\,(i = 1,2,\ldots,n - q)$$ are called elements. They can be chosen arbitrarily from the original η or they can be a linear combination of them. We can then express the full vector η as a set of linear combinations of these elements,

(9.9.5)

or

(9.9.6)

Equation (9.9.6) is of the same type as (9.2.2). The solution must thus be of the form of (9.2.26), i.e.,

(9.9.7)

describes the estimation of the elements $$\boldsymbol{\upalpha }$$ according to the method of least squares. The corresponding covariance matrix is

(9.9.8)

cf. (9.2.27). The improved measurements are obtained by substituting (9.9.7) into (9.9.6):

(9.9.9)

By error propagation the covariance matrix is found to be

(9.9.10)

Example 9.9:

Constraint between the angles of a triangle

Suppose measurements of the angles of a triangle have yielded the values $$y_{1} = 8{9}^{\circ }$$, $$y_{2} = 3{1}^{\circ }$$, $$y_{3} = 6{1}^{\circ }$$, i.e.,

The linear equation of constraint is

It can be written as

with

As elements we choose η ₁ and η ₂. The system (9.9.5) then becomes

or

i.e.,

We assume a measurement error for the angle of $${1}^{\circ }$$, i.e.,

Using these in (9.9.7) gives

Using (9.9.9) one finally obtains

This result was clearly to be expected. The “excess” in the measured sum of angles of 2∘ is subtracted equally from the three measurements. This would not have been the case, however, if the individual measurements had errors of different magnitudes. The reader can easily repeat the computation for such a case. The residual errors of the improved measurements can be determined by application of (9.9.10),

The residual error of each angle is thus equal to $$\sqrt{ 2/3} \approx 0.82$$. ■ 

At this point we want to make a general statement on measurements related by equations of constraint. Although in the statistical methods used so far we have found no way of dealing with systematic errors, equations of constraint offer such a possibility in many cases. If, for example, the sum of angles in many measurements is observed to be greater than 180∘ more frequently than less, then one can conclude that the measurement apparatus has a systematic error.

9.9.2 The Method of Lagrange Multipliers

Instead of elements, one more frequently uses the method of Lagrange multipliers . Although both methods clearly give the same results, the latter has the advantage that all unknowns are treated in the same way and thus the user is spared having to choose elements. The method of Lagrange multipliers is a well-known procedure in differential calculus for determination of extrema in problems with additional constraints.

We begin again from the linear system of equations of constraint (9.9.4)

and we recall that the measured quantities are the sum of the true value η and the measurement error $$\varepsilon$$,

Thus one has

(9.9.11)

Since y is known from the measurement, and b ₀ and B are also constructed from known quantities, we can construct a column vector with q elements,

(9.9.12)

which does not contain any unknowns. Thus (9.9.11) can be written in the form

(9.9.13)

We now introduce an additional column vector with q elements, whose elements, not yet known, are the Lagrange multipliers ,

(9.9.14)

Using this we extend the original minimum function (9.2.16)

to

(9.9.15)

The function L is called the Lagrange function . The requirement

with the constraint

is then fulfilled when the total differential of the Lagrange function vanishes, i.e., when

This is equivalent to

(9.9.16)

The system (9.9.16) consists of n equations containing a total of n + q unknowns, $$\varepsilon _{1}$$, $$\varepsilon _{2}$$, …, $$\varepsilon _{n}$$ and μ ₁, μ ₂, …, μ q . In addition we have the q equations of constraint (9.9.13). We transpose (9.9.16) and obtain

(9.9.17)

By substitution into (9.9.13) we obtain

which can easily be solved for $$\boldsymbol{\upmu }$$:

(9.9.18)

With (9.9.17) we thus have the least-squares estimators of the measurement errors,

(9.9.19)

The estimators of the unknowns are then given by (9.9.1),

(9.9.20)

With the abbreviation

this becomes

(9.9.21)

The covariance matrices of $$\tilde{\boldsymbol{\upmu }}$$ and $$\tilde{\boldsymbol{\upeta }}$$ are easily obtained by applying error propagation to the linear system of equations (9.9.18) and (9.9.19),

(9.9.22)

(9.9.23)

Example 9.10:

Application of the method of Lagrange multipliers to Example 9.9

We apply the method of Lagrange multipliers to the problem of Example 9.9. We then have

and in addition,

and

We can now compute (9.9.21),

The covariance matrices are then

 ■ 

We now generalize the method of Lagrange multipliers for the case of nonlinear equations of constraint having the general form (9.9.2), i.e.,

These equations can be expanded in a series about $$\boldsymbol{\upeta }_{0}$$,

(9.9.24)

Here the $$\boldsymbol{\upeta }_{0}$$ are the first approximations for the true values $$\boldsymbol{\upeta }$$. Using the definitions

we can write (9.9.24) in the form

(9.9.25)

Except for a sign this relation corresponds to (9.9.13). The solution $$\tilde{\boldsymbol{\delta }}$$ therefore can be read off (9.9.19),

(9.9.26)

As first approximations $$\boldsymbol{\eta }_{0}$$ we use the measured values y,

(9.9.27)

and obtain

(9.9.28)

For linear constraint equations this already is the solution.

If the equations are nonlinear an iteration is performed. The prescription for step i of the iteration is described for the general case at the end of the next section. If, in the formulas given there, all terms containing the matrix A are set to zero, one obtains the iteration procedure for the case of constraint measurements.

For each step i one computes

(9.9.29)

The procedure is terminated as convergent, if a further step leads to no appreciable reduction of M i . We call the result $$\tilde{\boldsymbol{\eta }}$$.

The covariance matrix is still given by (9.9.23), i.e.,

(9.9.30)

if the elements of B are computed using at $$\tilde{\boldsymbol{\eta }}$$. The best estimates of the measurement errors are computed using

(9.9.31)

With them the minimum function is found to be

(9.9.32)

This quantity again can be used for a χ ² test with q degrees of freedom.

Although the method of Lagrange multipliers is mathematically elegant, in programs provided here we use the method of orthogonal transformations (see Sect. 9.12).

9.10 The General Case of Least-Squares Fitting

After the preparation of the previous sections we can now take up the general case of fitting with the method of least squares.

We first recall the notation. The r unknown parameters are placed in a vector x. The quantities to be measured form an n-vector $$\boldsymbol{\upeta }$$. The values y actually measured differ from $$\boldsymbol{\upeta }$$ by the errors $$\boldsymbol{\upvarepsilon }$$. We will assume a normal distribution for the individual errors $$\varepsilon _{j}$$ (j = 1, 2, …, n), i.e., a normal distribution for the n variables $$\varepsilon _{j}$$ with the null vector as the vector of expectation values, and a covariance matrix $$C_{y} = G_{y}^{-1}$$. The vectors x and $$\boldsymbol{\upeta }$$ are related by m functions

(9.10.1)

We will further assume that we have already obtained in some way a first approximation for the unknowns x ₀. As a first approximation for $$\boldsymbol{\upeta }$$ we use $$\boldsymbol{\upeta }_{0} = \mathbf{y}$$ as in Sect. 9.9. Finally we require that the functions f k can be approximated by linear functions in the range of variability of our problem, i.e., in the region around ($$\mathbf{x}_{0},\boldsymbol{\upeta }_{0}$$), which is given by the differences x −x ₀ and $$\boldsymbol{\upeta } -\boldsymbol{\upeta }_{0}$$. We can then write

(9.10.2)

With the abbreviations

(9.10.3)

(9.10.4)

(9.10.5)

(9.10.6)

the system of equations (9.10.2) can be written as follows:

(9.10.7)

The Lagrange function is

(9.10.8)

Here $$\boldsymbol{\upmu }$$ is an m-vector of the Lagrange multipliers. We require that the total derivative of (9.10.8) with respect to $$\boldsymbol{\updelta }$$ vanishes. This is equivalent to requiring

or

(9.10.9)

Substitution into (9.10.7) gives

(9.10.10)

or

(9.10.11)

where

(9.10.12)

With (9.10.9) we can now write

(9.10.13)

Since the Lagrange function L is a minimum also with respect to $$\boldsymbol{\upxi }$$, the total derivative of (9.10.8) with respect to $$\boldsymbol{\upxi }$$ must also vanish, i.e.,

By transposing and substituting (9.10.11) one obtains

or

(9.10.14)

Substituting (9.10.14) into (9.10.13) and (9.10.11) immediately gives the estimates of the deviations $$\boldsymbol{\updelta }$$ and the Lagrange multipliers $$\boldsymbol{\upmu }$$,

(9.10.15)

(9.10.16)

The estimates for the parameters x and for the improved measurements $$\boldsymbol{\upeta }$$ are

(9.10.17)

(9.10.18)

From (9.10.14), (9.10.4), and (9.10.5) we obtain for the matrix of derivatives of the elements of $$\tilde{\boldsymbol{\upxi }}$$ with respect to the elements of y

Using error propagation one obtains the covariance matrix

(9.10.19)

Correspondingly one finds

(9.10.20)

One can show that under the assumed conditions, i.e., sufficient linearity of (9.10.2) and normally distributed measurement errors, the minimum function M, which can also be written in the form

(9.10.21)

follows a χ ²-distribution with m − r degrees of freedom.

If the Eqs. (9.10.1) are linear, then the relations (9.10.17) to (9.10.20) already are the solutions. In nonlinear cases one can perform an iterative procedure, which we now discuss in detail for step i with i = 1, 2, …. For the functions f k the following holds:

With $${A}^{(i)},{B}^{(i)},{\mathbf{c}}^{(i)}$$ we denote the quantities A, B, c, evaluated at $$\mathbf{x}_{i-1},\boldsymbol{\eta }_{i-1}$$. Furthermore let

Then

We now denote with

the sum of the contributions of all previous steps to improve the measurements and find for the difference between the measurements y and the approximation $$\boldsymbol{\eta }_{i}$$

since $$\boldsymbol{\eta }_{0} = \mathbf{y}$$. The first term of the Lagrangian function is

and the full Lagrangian is

We can now proceed as above and get, with $$G_{B}^{(i)} = {({B}^{(i)}G_{y}^{-1}{B}^{(i)\mathrm{T}})}^{-1}$$,

and

For every step i we compute

(9.10.22)

The procedure is terminated as convergent, if a new step does not yield an appreciable further reduction of M i . The results of the iteration are denoted by $$\tilde{\mathbf{x}},\tilde{\boldsymbol{\eta }}$$. The corresponding covariance matrices are given by (9.10.19) and (9.10.20), if the matrices A and B are evaluated at $$\tilde{\mathbf{x}},\tilde{\boldsymbol{\eta }}$$. It is, of course, possible for the iteration process to diverge. In this case points (a)–(d), raised at the end of Sect. 9.7, should be considered.

In the following section we describe a different way to determine the solutions $$\tilde{\mathbf{x}},\tilde{\boldsymbol{\eta }}$$. Also for that procedure the formulas (9.10.19), (9.10.20), and (9.10.21) for the computation of the covariance matrices $$G_{\tilde{x}}^{-1},G_{\tilde{\eta }}^{-1}$$ and the minimum function M remain valid, if the matrices A and B are evaluated at the position of the solution.

9.11 Algorithm for the General Case of Least Squares

In the Java class LsqGen, treating the general case of least squares, we do not use the method of Lagrange multipliers but rather the procedure of Sect. A.18, which is based on orthogonal transformations.

At every step of the iteration we must determine the r-vector $$\boldsymbol{\upxi }$$ and the $$n$$-vector $$\boldsymbol{\updelta }$$. We combine both into an (r + n)-vector u,

(9.11.1)

The m × r matrix A and the m × n matrix B are also combined into an m × (r + n) matrix E,

(9.11.2)

The vector containing the solutions u must satisfy the constraint (9.10.7), i.e.,

(9.11.3)

We now consider the minimum function in the ith iterative step. It depends on

(9.11.4)

where

(9.11.5)

is the result of all of the previous steps that changed y. One then has

(9.11.6)

We now extend the n × n matrix G y to an $$(r + n) \times (r + n)$$ matrix

(9.11.7)

for which we find a Cholesky decomposition according to Sect. A.9,

(9.11.8)

Then (9.11.6) becomes

or

(9.11.9)

with

(9.11.10)

Now one must merely solve the problem (9.11.9) with the constraint (9.11.3), e.g., with the procedure of Sect. A.18. With the solution

or rather with the vectors $$\tilde{\boldsymbol{\upxi }}$$, $$\tilde{\boldsymbol{\updelta }}$$ one finds improved values for x [cf. (9.10.17)], $$\boldsymbol{\upeta }$$ [cf. (9.10.18)], and for s, as well as for t [cf. (9.11.5) and (9.11.10)], with which an additional iterative step can be carried out.

The procedure can be regarded as having converged and terminated when the minimum function (9.11.9) in two successive steps only changes by an insignificant amount, or it can be terminated without success if after a given number of steps convergence has not been reached. In the case of convergence, the covariance of the unknowns can be computed according to (9.10.19). The calculation of the covariance matrix of the “improved” measurements $$\tilde{\boldsymbol{\upeta }}$$ according to (9.10.20) is possible as well. It is, however, rarely of interest. Finally, the value of M obtained in the last step can be used for a χ ²-test of the goodness-of-fit with m − r degrees of freedom.

All these operations are performed by the class LsqGen. This includes the numerical computation of the derivatives for the matrix E. The user only has to program the relation (9.10.1), which depends on the problem at hand. That is done by an extension of the abstract class DatanUserFunction. There exist example programs (Sect. 9.14) for the following examples in this chapter, including realizations of such classes.

Example 9.11:

Fitting a line to points with measurement errors in both the abscissa and ordinate

Suppose a number of measured points (t i , s i ) in the (t, s) plane are given. Each point has measurement errors Δ t i , Δ s i , which can, in general, be correlated. The covariance between the measurement errors Δ t i and Δ s i is c i . We identify t i and s i with elements of the n-vector y of measured quantities

The covariance matrix is

A straight line in the (t, s) plane is described by the equation $$s = x_{1} + x_{2}t$$. For the assumption of such a line through the measured points, the equations of constraint (9.10.1) take on the form

(9.11.11)

Because of the term x ₂ η 2k−1, the equations are not linear. The derivatives with respect to η 2k−1 depend on x ₂ and those with respect to x ₂ depend on η 2k−1.

Fig. 9.12:

Fitting a line to four measured points in the (t, s) plane. The points are shown with measurement errors (in t and s) and covariance ellipses. The individual plots show the results of the fit, the errors, and correlation coefficients.

The results of fitting to four measured points are shown in Fig. 9.12. The examples in the two individual plots differ only in the correlation coefficient of the third measurement, ρ ₃ = 0. 5 and $$\rho _{3} = -0.5$$. One can see a noticeable effect of the sign of ρ ₃ on the fit result and in particular on the value of the minimum function. ■ 

Example 9.12:

Fixing parameters

In Fig. 9.13 the results of fitting a line to measured points with errors in the abscissa and ordinate are shown, where in each plot one parameter of the line was held fixed. In the upper plot the intercept of the vertical axis x ₁ was fixed, and in the lower plot the slope x ₂ was fixed. ■ 

Fig. 9.13:

Fitting a line to the same points as in the first plot of Fig. 9.12. The intercept with the vertical axis x ₁ was held constant in the upper plot, and below the slope x ₂ was held constant.

9.12 Applying the Algorithm for the General Case to Constrained Measurements

If all of the variables x ₁, …, x r are fixed, then there are no more unknowns in the least-squares problem.

In the equations of constraint (9.10.2) only the components of $$\boldsymbol{\upeta }$$ are variable. Thus all terms containing the matrix A vanish from the formulas of the previous sections. As previously, however, the improved measurements $$\tilde{\boldsymbol{\upeta }}$$ can be computed. The quantity M can in addition be used for a χ ²-test with m degrees of freedom for how well the equations of constraint are satisfied by the measurements. Finally, the covariance matrix of the improved measurements $$C_{\tilde{\eta }}$$ can be determined.

Mathematically – as well as when using the program LsqGen – it does not matter whether all variables are fixed (r ≠ 0, r′ = 0) or whether from the start the equations of constraint do not depend on the variables x (r = 0). In both cases LsqGen gives the same solution.

Example 9.13:

χ ²-test of the description of measured points with errors in abscissa and ordinate by a given line

We use the same measurements as used already in Examples 9.11 and 9.12. The results of the analysis of these measurements with LsqGen with fixed parameters are shown in Fig. 9.14. For the upper plot, x ₁ and x ₂ were fixed to the values obtained from fitting with two adjustable parameters in Example 9.11, Fig. 9.12. Clearly we also obtain the same value of M as previously. For the lower plot, arbitrarily crude estimates (x ₁ = 0, x ₂ = 0. 5) were used. They give a significantly higher value of M. This value would lead to rejection of the hypothesis with a confidence level of 99 % that the data points are described by a linear relation with these parameter values (χ 0. 99 ² = 13. 28 for four degrees of freedom).

Fig. 9.14:

Constrained measurements. The hypothesis was tested whether the true values of the measured points, indicated by their covariance ellipses, lie on the line $$s = x_{1} + x_{2}t$$. With the numerical values of M, a χ ²-test with four degrees of freedom can be carried out. Shown as well are the improved measurements, which lie on the line, and their errors.

It is also interesting to consider the improved measurements $$\tilde{\boldsymbol{\upeta }}$$ and their errors, which are shown in Fig. 9.14. The improved measurements naturally lie on the line. The measurement errors are the square roots of the diagonal elements of $$C_{\tilde{\eta }}$$. The correlations between the errors of a measured point in s and t are obtained from the corresponding off-diagonal elements $$C_{\tilde{\eta }}$$. They are exactly equal to one. The covariance ellipses of the individual improved measurements collapse to line segments which lie on the line given by x ₁, x ₂. ■ 

9.13 Confidence Region and Asymmetric Errors in the General Case

The results obtained in Sect. 9.8 on confidence regions and asymmetric errors are also valid for the general case. We therefore limit ourselves to stating that asymmetric errors are computed by the class LsqAsg and to presenting an example.

Example 9.14:

Asymmetric errors and confidence region for fitting a straight line to measured points with errors in the abscissa and ordinate

Figure 9.15 shows the result of fitting to four points with large measurement errors. From Sect. 9.8 we already know that large measurement errors can lead to asymmetric errors in the fitted parameters. In fact, we see highly asymmetric errors and large differences between the covariance ellipse and the corresponding confidence region. ■ 

Fig. 9.15:

(Above) Measured points with covariance ellipses and fitted line. (Below) Result of the fit, given in the plane spanned by the parameters x ₁, x ₂. Shown are the fitted parameter values (circle), symmetric errors (crossed bars), covariance ellipse, asymmetric errors (horizontal and vertical lines), and the confidence region (dark contour).

9.14 Java Classes and Example Programs

Java Classes for Least-Squares Problems

LsqPol

handles the fitting of a polynomial (Sect. 9.4.1).

LsqLin

handles the linear case of indirect measurements (Sect. 9.4.2).

LsqNon

handles the nonlinear case of indirect measurements (Sect. 9.6.1).

LsqMar

handles the nonlinear case using Marquardt’s method (Sect. 9.6.2).

LsqAsn

yields asymmetric errors or confidence limits in the nonlinear case (Sect. 9.8).

LsqAsm

yields asymmetric errors or confidence limits using Marquardt’s method.

LsqGen

handles the general case of least squares (Sect. 9.11).

LsqAsg

yields asymmetric errors or confidence limits in the general case (Sect. 9.13).

Example Program 9.1:

The class E1Lsq demonstrates the use of LsqPolThe short program uses the data of Table 9.3 and computes vectors $$\tilde{\mathbf{x}}$$ of coefficients and their covariance matrix C x for r = 1, 2, 3, 4. Here r − 1 is the degree of the polynomial , i.e., r is the number of elements in x. The results are presented numerically

Suggestions: (a) Modify the program (by modifying a single statement) so that the cases r = 1, 2, …, 10 are treated. Which peculiarity do you expect for r = 10? (b) Instead of the data of Table 9.3 use different data determined without error by a polynomial, e.g., y = t ², and let the program determine the parameters of the polynomial from the data. Use different (although obviously incorrect) sets of values for the errors Δ y i , e.g., $$\varDelta y_{i} = \sqrt{y_{i}}$$ for one run through the program and Δ y i  = 1 for another run. What is the influence of the choice of Δ y i on the coefficients $$\tilde{\mathbf{x}}$$, on the minimum function, and on the covariance matrix?

Example Program 9.2:

The class E2Lsq demonstrates the use of LsqLinThe program uses the data of Fig. 9.4 and sets up the matrix A and the vector c; these are needed for the fit of a proportional relation y = x ₁ t to the data. Next the fit is performed by calling LsqLin. The results are displayed numerically.

Suggestions: (a) Modify the program so that in addition a first degree polynomial is fitted. Set up the matrix A yourself, i.e., do not use LsqPol. Compare your result with Fig. 9.4. (b) Display the results graphically as in Fig. 9.4.

Example Program 9.3:

The class E3Lsq demonstrates the use of LsqNonThe program solves the following problem. First 20 pairs of values (t i , y i ) are generated. The values t i of the controlled variables are $$1/21,\,2/21,\,\ldots,\,20/21$$. The values y i are given by

Here $$\varepsilon _{i}$$ is an error taken from a normal distribution with expectation value zero and width σ i . The width σ i is different from point to point. It is taken from a uniform distribution with the limits σ∕2 and 3σ∕2. Thus, the y i are points scattered within their errors around a Gaussian curve defined by the parameters $$\mathbf{x} = (x_{1},x_{2},x_{3})$$. The widths of the error distributions are known, i.e., Δ y i  = σ i . The data points are generated for the parameter values x ₁ = 1, x ₂ = 1. 2, x ₃ = 0. 4. (They are identical with those shown in Fig. 9.5.) The program now successively performs four different fits of a Gaussian curve to the data. In a first step all three parameters in the fit procedure are considered variable. Then successively one, two, and finally all parameters are fixed. Before each fit, first approximations for the variables are set which, of course, are not modified during the fitting procedure for the fixed parameters. The results are presented numerically.

Suggestions: (a) Choose different first approximations for the non-fixed parameters and observe the influence of the choice on the results. (b) Obtain first approximations by computation, e.g., by the procedure described in Example 9.4, in which a parabola is fitted to the logarithms of the data. (c) Modify the program by adding graphical output corresponding to Fig. 9.5.

Example Program 9.4:

The class E4Lsq demonstrates the use of LsqMar The program solves the problem of Example 9.7. First a set of 50 data points (t i , y i ) is generated. They are scattered according to their errors around a curve corresponding to the sum of a second degree polynomial and two Gaussians , cf. (9.6.6). The nine parameters of this function are combined to form the vector x. The measurement errors Δ y i are generated by the procedure described in Sect. 9.3. The data points are generated for predefined values of x. Next, by calling LSQMAR (with significantly different values x as first approximations) solutions $$\tilde{\mathbf{x}}$$ are obtained by a fit to the data points. The results are presented numerically and also in graphical form.

Suggestions: (a) Fix all parameters except for x ₅ and x ₈, which determine the mean values of the two Gaussians used for generating the data. (In the program, as customary in Java, where indexing begins with 0, they are denoted by x[4] and x[7].) Allow for interactive input of x ₅ and x ₈. For different input values of (x ₅, x ₈), e.g., (10, 30), (19, 20), (19, 15), (10, 11), try to determine whether you can still separate the two Gaussians by the fit, i.e., whether you obtain significantly different values for $$\tilde{x}_{5}$$ and $$\tilde{x}_{8}$$, considering their errors $$\varDelta \tilde{x}_{5}$$, $$\varDelta \tilde{x}_{8}$$. (b) Repeat (a), but for smaller measurement errors. Choose, e.g., σ = 0. 1 or 0.01 instead of σ = 0. 4.

Example Program 9.5:

The class E5Lsq demonstrates the use of LsqAsnThe program solves the problem of Example 9.8. First, pairs (t i , y i ) of data are produced. The y i are scattered according to their measurement errors around a curve given by the function $$y_{i}(t_{i}) = x_{1}\exp (-x_{2}t)$$, cf. (9.6.4). The measurement errors Δ y i are generated by the procedure already used in Sect. 9.3. Starting from a given first approximation for the parameters x ₁, values $$\tilde{\mathbf{x}}$$ of these parameters are fitted to the data using LsqNon. Finally the asymmetric errors are found using LsqAsn. Two plots are produced. One shows the measurements and the fitted curve. The second displays, in the (x ₁, x ₂)plane, the fitted parameters with symmetric and asymmetric errors, covariance ellipse, and confidence region.

Example Program 9.6:

The class E6Lsq demonstrates the use of LsqGen

The program fits a straight line to points with measurement errors in the abscissa and ordinate, i.e., it solves the problem of Example 9.11. From the measured values, their errors and covariances the vector y and the covariance matrix C y are set up. Determination of the two parameters x ₁ (ordinate intercept) and x ₂ (slope) is done with LsqGen. For this the function (9.11.11) is needed. It is implemented in the method getValue of the subclass StraightLine of E6Lsq, which itself is an extension of the abstract class DatanUserFunction. The first approximations of x ₁ and x ₂ are obtained by constructing a straight line through the outermost two points. A loop extends over the two cases of Fig. 9.12. The results are shown numerically and graphically.

Example Program 9.7:

The class E7Lsq demonstrates the use of LsqGen with some variables fixedThe program also treats Example 9.11. Again the first approximations of x ₁ and x ₂ are obtained by constructing a straight line through the outer two measured points. A loop extends over two cases. In the first one x ₁ is fixed at x ₁ = 0. 2, in the second x ₂ is fixed at x ₂ = 0. 5. The results correspond to Fig. 9.13.

Example Program 9.8:

The class E8Lsq demonstrates the use of LsqGen with all variables are fixed and produces a graphical representation of improved measurementsThe problem of Example 9.13 is solved. The results are those of Fig. 9.14.

Example Program 9.9:

The class E9Lsq demonstrates the use of LsqAsg and draws the confidence-region contour for the fitted variablesHere, the problem of Example 9.14 is solved. The results are those of Fig. 9.15.

Literature

Literature Cited in the Text

[10]

A.H. Rosenfeld, H.A. Barbero-Galtieri, W.J. Podolski, L.R. Price, P. Söding, Ch.G. Wohl, M. Roos, and W.J. Willis, Rev. Mod. Phys. 33 (1967) 1ADSCrossref

[11]

Particle Data Group, Physics Letters B204 (1988) 1

Footnotes

1

For simplicity of notation we no longer write random variables with a special character type. From context it will always be evident which variables are random.

© Springer International Publishing Switzerland 2014

Siegmund BrandtData Analysishttps://doi.org/10.1007/978-3-319-03762-2_10

10. Function Minimization

Siegmund Brandt1 

(1)

Department of Physics, University of Siegen, Siegen, Germany

Keywords

Covariance

Locating extreme values (maxima and minima) is particularly important in data analysis. This task occurs in solving the least-squares problem in the form $$M(\mathbf{x},\mathbf{y}) =\min$$ and in the maximum likelihood problem as L = max. By means of a simple change of sign, the latter problem can also be treated as locating a minimum. We always speak therefore of minimization .

10.1 Overview: Numerical Accuracy

We consider first the simple quadratic form

(10.1.1)

It has an extremum where the first derivative vanishes,

(10.1.2)

that is, at the value

(10.1.3)

With M(x m) = M m, Eq. (10.1.1) can easily be put into the form

(10.1.4)

Although the function whose minimum we want to find does not usually have the simple form of (10.1.1), it can nevertheless be approximated by a quadratic form in the region of the minimum, where one has the Taylor expansion around the point x ₀,

(10.1.5)

where

(10.1.6)

In this approximation the minimum is given by the point where the derivative M′(x) is zero, i.e.,

(10.1.7)

This holds only if x ₀ is sufficiently close to the minimum so that terms of order higher than quadratic in (10.1.5) can be neglected. The situation is depicted in Fig. 10.1. The function M(x) has a minimum at x m, maxima at x M, and points of inflection at x s. For the second derivative in the region x > x m one has $$M''(x)$$ > 0 for x < x s and M(x″) < 0 for x > x s. If we now choose x ₀ to be in the region $$x_{\mathrm{m}} < x_{0} < x_{\mathrm{M}}$$, then the first derivative $$M'(x)$$ there is always positive. Therefore x mp lies closer to x m only if x ₀ < x s. Clearly the point x ₀ is not in general chosen arbitrarily, but rather as close as possible to where the minimum is expected to be. We can call this estimated value the zeroth approximation of x m. Various strategies are available to obtain successively better approximations:

Fig. 10.1:

The function M(x) has a minimum at x m, maxima at the points x M, and points of inflection at x s.

(i)

Use of the function and its first and second derivatives at x ₀.

One computes x mp according to (10.1.7), takes x mp as a first approximation, i.e., x ₀ is replaced by x mp, and obtains by repeated application of (10.1.7) a second approximation. The procedure is repeated until two successive approximations differ by less than a given value $$\varepsilon$$. From the discussion above it follows that this procedure does not converge if the zeroth approximation lies outside the points of inflection in Fig. 10.1.

(ii)

Use of the function and its first derivative at x ₀.

The sign of the derivative $$M'(x_{0}) = -b$$ at the point x ₀ determines the direction in which the function increases. It is assumed that one should search for the minimum in the direction in which the function decreases. The quantity

(10.1.8)

is computed, i.e., one replaces the second derivative by the value unity.

Instead of x ₀ one now uses x ₁ as the approximation, and so forth. Alternatively one can also use instead of (10.1.8) the rule

(10.1.9)

where c is an arbitrary positive constant. Both rules ensure that the step from x ₀ to x ₁ proceeds in the direction of the minimum. If in addition one chooses c to be small, then the step is small, so that it does not go beyond (or not far beyond) the minimum.

(iii)

Use of the function at various points.

The procedure (i) can be carried out without knowing the derivative of the function if the function itself is known at three points. One can then uniquely fit a parabola through these three points and take the extreme value as an approximation of the minimum of the function. One speaks of locating the minimum by quadratic interpolation. It is, however, by no means certain that the extreme value of the parabola is a minimum and not a maximum. As in procedure (i) it is therefore important that the three chosen points are already in the region of the minimum of the function.

(iv)

Successive reduction of an interval containing the minimum.

In none of the procedures discussed up to this point were we able to guaranty that the minimum of the function would actually be found. The minimum can be found with certainty provided one knows an interval x a  < x < x b containing the minimum. If such an interval is known, one can locate the minimum with arbitrary accuracy by successively subdividing and checking in which subinterval the minimum is located.

In Sects. 10.2–10.7 we shall examine the minimization of a function of only one variable. In Sect. 10.2 the formula for a parabola determined by three points will be given. In Sect. 10.3 it is shown that the minimization of a function of one variable is equivalent to the minimization of a function of n variables on a line in an n-dimensional space. Section 10.4 describes a procedure for locating an interval containing the minimum. In Sect. 10.5 a minimum search by means of interval division is described. This is combined in Sect. 10.5 with the procedure of quadratic interpolation in a way that the interpolation is only used when it leads quickly to the minimum of the function. If this is not the case, one continues with interval division. In this way we possess a procedure with the certainty of interval division combined with – as much as possible – the speed of quadratic interpolation. The same procedure can also be used for a function of n variables if the search for the minimum is restricted to a line in the n-dimensional space. This problem is addressed in Sect. 10.7.

Next we turn to the task of searching for the minimum of a function of n variables. We begin in Sect. 10.8 with the particularly elegant simplex method. This is followed by the discussion of various procedures of successive minimization along fixed directions in the n-dimensional space. These directions can simply be the directions of the coordinates (Sect. 10.9), or for a function that depends only quadratically on the variables they can be chosen such that the minimum is reached in at most n steps (Sects. 10.10 and 10.11).

Finally we discuss a procedure of n-dimensional minimization which employs aspects of methods (i) and (ii) of the one-dimensional case. If x is the n-dimensional vector of the variables, then the general quadratic form, i.e., the generalization of (10.1.1) to n variables, is

(10.1.10)

Here A is a symmetric matrix, A kℓ  = A ℓ k . The partial derivative with respect to x i is

(10.1.11)

Expressing all of the partial derivatives as a vector ∇ M gives

(10.1.12)

At the minimum point the vector of derivatives vanishes. The minimum is therefore located at

(10.1.13)

in analogy to (10.1.3).

Clearly the function M(x) does not in general have the simple form of (10.1.10). We can, however, expand it in a series around the point x ₀,

(10.1.14)

with the negative gradient

(10.1.15)

and the Hessian matrix of second derivatives

(10.1.16)

The series (10.1.14) is the starting point for various minimization procedures:

(i)

Minimization in the direction of the gradient.

Starting from the point x ₀ one searches for the minimum along the direction given by the gradient ∇ M(x ₀) and calls the point where it is found x ₁. Starting from x ₁ one looks for the minimum along the direction ∇ M(x ₁), and so forth. We will discuss this procedure, called minimization in the direction of steepest descent , in Sect. 10.12.

(ii)

Step of given size in the gradient direction.

One computes in analogy to (10.1.9)

(10.1.17)

with a given positive c. That is, one takes a step in the direction of steepest descent of the function, without, however, searching exactly for the minimum in this direction. Next one computes the gradient at x ₁, steps from x ₁ in the direction of this gradient, etc. In Sect. 10.13 we will combine this method with the following one.

(iii)

Use of the gradient and the Hessian matrix at x ₀.

If we truncate (10.1.14) after the quadratic term, we obtain a function whose minimum is, according to (10.1.13), given by

(10.1.18)

We take x ₁ = x mp as the first approximation, compute for this point the gradient and Hessian matrix, obtain by corresponding use of (10.1.18) the second approximation, and so forth. This procedure is discussed in Sect. 10.14. It converges quickly if the zeroth approximation x ₀ is sufficiently close to the minimum. If that is not the case, however, then it gives – as for the corresponding one dimensional procedure – no reasonable solution. We will combine it, therefore, in Sect. 10.15 with method (ii), in order to obtain, when possible, the speed of (iii), but when necessary, the certainty of (ii).

In Sects. 10.8 through 10.15 very different methods for solving the same problem, the minimization of a function of n variables, will be discussed. In Sect. 10.16 we give information on how to choose one of the methods appropriate for the problem in question. Section 10.17 is dedicated to considerations of errors. In Sect. 10.18 several examples are discussed in detail.

Before we find the minimum x m of a function, we would like to inquire briefly about the numerical accuracy we expect for x m. The minimum is after all almost always determined by a comparison of values of the function at points close in x. If we solve (10.1.4) for (x − x m), we obtain

We assume that A, i.e., the second derivative of the function M, is of order of magnitude unity close to the minimum. (This need only be true approximately. In fact, in numerical calculations one always scales all of the quantities such that they are of order unity, i.e., not something like 10⁶ or 10⁻⁶.) If we compute the function M with the precision δ then the difference M(x) − M(x m) is also known at best with a precision of δ, i.e., two function values can not be considered as being significantly different if they differ by only δ. For the corresponding x values one then has

(10.1.19)

If the computer has n binary places available for representing the mantissa, then a value x can be represented with the precision (4.​2.​7)

For computing a value x one chooses therefore a relative precision

since it is clearly pointless to try to compute a value with a higher precision than that with which it can be represented. If x is computed iteratively, i.e., one computes a series x ₀, x ₁, … of approximations for x, then one can truncate this series as soon as

or

(10.1.20)

for a given $$\varepsilon$$. With this prescription we will have difficulties, however, if x k  = 0. We introduce, therefore, in addition to $$\varepsilon$$ a constant t ≠ 0 and extend (10.1.20) to

(10.1.21)

The last task remaining is to choose the numerical values for $$\varepsilon$$ and t. If x is the position of the minimum, then by (10.1.19) a value for $$\varepsilon$$ must be chosen that is greater than or equal to the square root of the relative precision for the representation of a floating point number. With computations using “double precision” in Java there are n = 53 binary places available for the representation of the mantissa. Then only the values

are reasonable. The quantity t corresponds to an absolute precision. Therefore it can be chosen to be considerably smaller.

10.2 Parabola Through Three Points

If three points $$(x_{a},y_{a}),(x_{b},y_{b}),(x_{c},y_{c})$$ of a function are known, then we can determine the parabola

(10.2.1)

that passes through these points. Instead of (10.2.1) we can also represent the parabola by

(10.2.2)

This relationship is naturally valid for three given points, i.e.,

and

From this we obtain

(10.2.3)

(10.2.4)

with

and for the extremum of the parabola

(10.2.5)

Fig. 10.2:

Parabola through three points (small circles) and its extremum (large circle). In the left figure there is a minimum, and on the right a maximum.

The class MinParab performs this simple calculation. We must still determine whether the extremum of the parabola is a minimum or a maximum (cf. Fig. 10.2). One has a minimum if the second derivative of (10.2.2) with respect to x − x b is positive, i.e., if c ₂ > 0. We now order the three given points such that

and find that then

With this one has for the sign of c ₂

Both expressions (x c − x b ) and (x b − x a ) are positive. Therefore for the extremum to be a minimum it is sufficient that

(10.2.6)

The condition is not necessary, but has the advantage of greater clarity. In the interval x a  < x < x c one clearly has a minimum if there is a point (x b , y b ) in the interval where the function has a value smaller than at the two end points. Clearly this statement is also valid if the function is not a parabola. We will make use of this fact in the next section.

10.3 Function of n Variables on a Line in an n-Dimensional Space

Locating the minimum of the function M(x) of a single variable x in an interval of the x axis is equivalent to locating the minimum of a function M(x) of an n-dimensional vector of variables $$\mathbf{x} = (x_{1},x_{2},\ldots,x_{n})$$ with respect to a given line in the n-dimensional space. If x ₀ is a fixed point and d is a fixed vector, then

(10.3.1)

describes a fixed line (see Fig. 10.3), and

(10.3.2)

is the value of the function at the point a on this line. For n = 1, x ₀ = 0, d = 1 and with the change of notation a = x, that is, f(x) = M(x), one recovers the original problem.

Fig. 10.3:

The line given by (10.3.1) in two dimensions.

The class FunctionOnLine computes the value (10.3.2); it makes use of an extension of the abstract class DatanUserFunction, to be provided by the user, which defines the function M(x). In Sects. 10.4 through 10.6 we consider the minimum of a function of a single variable. The programs also treat, however, the case of a minimum of a function of n variables on a line in the n-dimensional space.

10.4 Bracketing the Minimum

For many minimization procedures it is important to know ahead of time that the minimum x m is located in a specific interval,

(10.4.1)

By systematically reducing the interval, the position of the minimum can then be further constrained until finally for a given precision $$\varepsilon$$ one has

(10.4.2)

There is, in fact, a minimum in the interval (10.4.1) if there is an x value x b such that

(10.4.3)

The class MinEnclose attempts to bracket the minimum of a function by giving values $$x_{a},x_{b},x_{c}$$ with the property (10.4.3). The program is based on a similar subroutine by PRESS et al. [12]. Starting from the input values x a , x b , which, if necessary, are relabeled so that y b  ≤ y a , a value $$x_{c} = x_{b} + p(x_{b} - x_{a})$$ is computed that presumably lies in the direction of decreasing function value, i.e., closer to the minimum. The factor p in our program is set to p = 1. 618 034. In this way the original interval (x a , x b ) is enlarged by the ratio of the golden section (cf. Sect. 10.5). The goal is reached if y c  > y b . If this is not the case, a parabola is constructed through the three points $$(x_{a},y_{a}),(x_{b},y_{b}),(x_{c},y_{c})$$, whose minimum is at x m.

We now examine the point (x m, y m). Here one must distinguish between various cases:

( a)

$$x_{b} < x_{\mathrm{m}} < x_{c}$$:

( a1)

$$y_{\mathrm{m}} < y_{c}: (x_{b},x_{\mathrm{m}},x_{c})$$ is the desired interval.

( a2)

$$y_{b} < y_{\mathrm{m}}: (x_{a},x_{b},x_{\mathrm{m}})$$ is the desired interval.

( a3)

y m > y c and y m < y b : There is no minimum. The interval will be extended further to the right.

( b)

$$x_{c} < x_{\mathrm{m}} < x_{\mathrm{end}}$$ and $$x_{\mathrm{end}} = x_{b} + f(x_{c} - x_{b})$$ and f = 10 in our program.

( b1)

$$y_{\mathrm{m}} > y_{c}: (x_{b},x_{c},x_{\mathrm{m}})$$ is the desired interval.

( b2)

y m < y c : There is no minimum. The interval will be extended further to the right.

( c)

x end < x m: As a new interval $$(x_{b},x_{c},x_{\mathrm{end}})$$ is used.

( d)

x m < x b : This result is actually impossible. It can, however, be caused by a rounding error. The interval will be extended further to the right.

If the goal is not reached in the current step, a further step is taken with the new interval. Figure 10.4 shows an example of the individual steps carried out until the bracketing of the minimum is reached.

Fig. 10.4:

Bracketing of a minimum with three points a, b, c according to (10.4.3). The initial values are shown as larger circles, and the results of the individual steps are shown as small circles.

10.5 Minimum Search with the Golden Section

As soon as the minimum has been enclosed by giving three points $$x_{a},x_{b},x_{c}$$ with the property (10.4.3), the bracketing can easily be tightened further. One chooses a point x inside the larger of the two subintervals (x a , x b ) and (x b , x c ). If the value of the function at x is smaller than at x b , then the subinterval containing x is taken as the new interval. If the value of the function is greater, then x is taken as the endpoint of the new interval.

A particularly clever division of the intervals is possible with the golden section . Let us assume (see Fig. 10.5) that

(10.5.1)

is the length of the subinterval (x a , x b ) (to be determined later) measured in units of the length of the full interval (x a , x c ). We now want to be able to divide the subinterval (x a , x b ) again with a point x corresponding to a fraction g,

(10.5.2)

In addition, the points x and x b should be situated symmetrically with respect to each other in the interval (x a , x c ), i.e.,

Fig. 10.5:

The golden section.

(10.5.3)

It follows that

that is,

and

(10.5.4)

As shown at the beginning of this section (for the case shown in Fig. 10.5 $$x_{b} - x_{a} > x_{c} - x_{b}$$), the minimum, which was originally only constrained to be in the interval (x a , x c ), now lies either in the interval (x a , x b ) or in the interval (x, x c ). By subdividing by the golden section one obtains intervals of equal size.

Fig. 10.6:

Stepwise bracketing of a minimum by subdividing according to the golden section. The original interval (larger circles) is reduced with every step (small circles).

Figure 10.6 shows the first six steps of an example of minimization with the golden section.

10.6 Minimum Search with Quadratic Interpolation

From the example shown in Fig. 10.6 one sees that the procedure of interval subdivision is quite certain, but works slowly. We now combine it, therefore, with quadratic interpolation.

The class MinCombined is based on a program developed by BRENT [13] , who first combined the two methods. The meanings of the most important variable names in the program are as follows. a and b denote the x values x a and x b that contain the minimum. xm is their mean value. m is the point at which the function has its lowest value up to this step, w is the point with the second lowest, and v is the point with the third lowest value of the function. u is the point where the function was last computed. One begins with the two initial values, x a and x b , that contain the minimum and adds a point x, that divides the interval (x a , x b ) according to the golden section. Then in each subsequent iteration parabolic interpolation is attempted as in Sect. 10.2. The result is accepted if it lies in the interval defined by the last step and if in this step the change in the minimum is less than half as much as in the previous one. By this condition it is ensured that the procedure converges, i.e., that the steps become smaller on the average, although a temporary increase in step size is tolerated. If both conditions are not fulfilled, a reduction of the interval is carried out according to the golden section.

Fig. 10.7:

Stepwise bracketing of a minimum with the combined method of Brent. The initial interval (large circles) is reduced with each step (small circles). Steps are carried out according to the golden section (GS) or by quadratic interpolation (QI).

Numerical questions are handled in a particularly careful way of Brent. Starting from the two parameters $$\varepsilon$$ and t, which define the relative precision, and the current value x for the position of the minimum, an absolute precision $$\varDelta x =\varepsilon x + t$$ = tol is computed according to (10.1.21). The iterations are continued until the half of the interval width falls below the value tol (i.e., until the distance from x to xm is not greater than tol) or until the maximum number of steps is reached. In addition the function is not computed at points that are separated by a distance less than tol, since such function values would not differ significantly.

The first six steps of an example of minimization according to Brent are shown in Fig. 10.7. Steps according to the golden section are marked by GS, and those from quadratic interpolation with QI. The comparison with Fig. 10.6 shows the considerably faster convergence achieved by quadratic interpolation.

10.7 Minimization Along a Direction in n Dimensions

The class MinDir computes the minimum of a function of of n variables along the line defined in Sect. 10.3 by x ₀ and d. It first uses MinEnclose to bracket the minimum and then MinCombined to locate it exactly.

The class MinDir is the essential tool used to realize a number of different strategies to find a minimum in n dimensional space, which we discuss in Sects. 10.9–10.15. A different type of strategy is the basis of the simplex method presented in the next section.

10.8 Simplex Minimization in n Dimensions

A simple and very elegant (although relatively slow) procedure for determining the minimum of a function of several variables is the simplex method by NELDER and MEAD [14]. The variables $$x_{1},x_{2},\ldots,x_{n}$$ define an n-dimensional space. A simplex is defined in this space by n + 1 points x i ,

(10.8.1)

A simplex in two dimensions is a triangle with the corner points $$\mathbf{x}_{1},\mathbf{x}_{2},\mathbf{x}_{3}$$.

We use y i to designate the value of the function at x i and use particular indices for labeling special points x i . At the point x H the value of the function is highest, i.e., y H  > y i , i ≠ H. At the point x h it is higher than at all other points except x H (y h  > y i , i ≠ H, i ≠ h) and at the point x ℓ it is lowest (y ℓ  < y i , i ≠ ℓ). The simplex is now changed step by step. In each substep one of four operations takes place, these being reflection, stretching, flattening, or contraction of the simplex (cf. Fig. 10.8).

Fig. 10.8:

Transformation of a simplex (triangle with thick border) into a new form (triangle with thin border) by means of a reflection (a), stretching (b), flattening (c), and contraction (d).

Using $$\overline{\mathbf{x}}$$ to designate the center-of-gravity of the (hyper)surface of the simplex opposite to x H ,

(10.8.2)

the reflection of x H is

(10.8.3)

with α > 0 as the coefficient of reflection. The reflected simplex differs from the original one only in that x H is replaced by x r .

A stretching of the simplex consists of replacing x H by

(10.8.4)

with the coefficient of stretching γ. One therefore chooses (for γ > 1) a point along the line joining x H and x r , which is still further than the point x r .

In a flattening, x H is replaced by a point lying on the line joining x H and $$\overline{\mathbf{x}}$$. This point is located between the two points,

(10.8.5)

The coefficient of flattening β is in the range 0 < β < 1.

In the three operations discussed up to now only one point of the simplex is changed, that being the point x H , which corresponds to the highest value of the function. The point is displaced along the line by x H and $$\overline{\mathbf{x}}$$. After the displacement it either lies on the same side of $$\overline{\mathbf{x}}$$ (flattening) or on the other side of $$\overline{\mathbf{x}}$$ (reflection) or even far beyond $$\overline{\mathbf{x}}$$ (stretching). In contrast to these operations, in a contraction, all points but one are replaced. The point x ℓ where the function has its lowest value is retained. All remaining points are moved to the midpoints of the line segments joining them with x ℓ

(10.8.6)

For the original simplex and for each one created by an operation, the points $$\overline{\mathbf{x}}$$ and x r and the corresponding function values $$\overline{y}$$ and y r are computed. The next operation is determined as follows:

Fig. 10.9:

Determining the minimum of a function of two variables with the simplex method. The function is shown by contour lines on which the function is constant. The function is highest on the outermost contour. The minimum is at the position of the small circle within the innermost contour. Each simplex is a triangle. The initial simplex is marked by thicker lines.

(a)

If y r  < y ℓ , a stretching is attempted. If this gives y e  < y ℓ , then the stretching is carried out. Otherwise one performs a reflection.

(b)

For y r  > y h a reflection is carried out if y r  < y H . Otherwise the simplex is unchanged. In each case this is followed by a flattening. If one obtains as a result of the flattening a point x a for which the function value is not less than both y H and $$\overline{y}$$, the flattening is rejected, and instead a contraction is carried out.

After every step we examine the quantity

(10.8.7)

If this falls below a given value, the procedure is terminated and we regard x ℓ as the point where the function has its minimum.

The class MinSim determines the minimum of a function of n variables by the simplex method. The program is illustrated in the example of Fig. 10.9. The triangle with the dark border is the initial simplex. The sequence of triangles with thin borders starting from it correspond to the individual transformations. One clearly recognizes as the first steps: stretching, stretching, reflection, reflection, flattening …. The simplex first finds its way into the “valley” of the function and then runs along the bottom of the valley towards the minimum. It is reshaped in the process so that it is longest in the direction in which it is progressing, and in this way it can also pass through narrow valleys. It is worth remarking that this method, which is obtained by considering the problem in two and three dimensions, also works in n dimensions and also in this case possesses a certain descriptiveness.

10.9 Minimization Along the Coordinate Directions

Some of the methods for searching for a minimum in an n-dimensional space are based on the following principle. Starting from a point x ₀ one searches for the minimum along a given direction in the space. Next one minimizes from there along another direction and finds a new minimum, etc. Within this general framework various strategies can be developed to choose the individual directions.

The simplest strategy consists in the choice of the coordinate directions in the space of the n variables x i . We can label the corresponding basis vectors with $$\mathbf{e}_{1},\mathbf{e}_{2},\ldots,\mathbf{e}_{n}$$, and they are then chosen in order as directions. After e n one begins again with e ₁, e ₂, …. A partial sequence of minimizations along all coordinate directions starting from x ₀ gives the point x ₁. After a new partial sequence one obtains x ₂, etc.

The procedure is ended successfully when for the values of the function M(x n ) and M(x n−1) for two successive steps one has

(10.9.1)

i.e., a condition corresponding to (10.1.21) for given values of $$\varepsilon$$ and t. We compare here, however, the value of the function M and not the independent variables x. Otherwise we would have to compute the distance between two points in an n-dimensional space.

Figure 10.10 shows the minimization of the same function as in Fig. 10.9 with the coordinate-direction method. After the first comparatively large step, which leads into the “valley” of the function, the following steps are quite small. The individual directions are naturally perpendicular to each other. The “best” point at each step moves along a staircase-like path along the floor of the valley towards the minimum.

Fig. 10.10:

Minimization along the coordinate directions. The starting point is shown by the larger circle, and the end by the smaller circle. The line shows the results of the individual steps.

10.10 Conjugate Directions

The slow convergence seen in Fig. 10.10 clearly stems from the fact that when minimizing along one direction, one loses the result of the minimization with respect to another direction. We will now try to choose the directions in such a way that this does not happen. For this we suppose for simplicity that the function is a quadratic form (10.1.10). Its gradient at the point x is then given by (10.1.12),

(10.10.1)

The change of the gradient in moving by Δ x is

(10.10.2)

This expression is a vector that gives the direction of change of the gradient when the argument is moved in the direction Δ x.

If a minimization has been carried out along a direction p, then the reduction in M with respect to p is retained when one moves in a direction q such that the gradient only changes perpendicular to p, i.e., when one has

(10.10.3)

The vectors p and q are said to be conjugate to each other with respect to the positive-definite matrix A. If one has n variables, i.e., if A is an n × n matrix, then one can find in general n conjugate linearly independent vectors.

POWELL [15] has given a method to find a set of conjugate directions for a function described by a quadratic form. For this one chooses as a first set of directions n linearly independent unit vectors p i , e.g., the coordinate directions p i  = e i . Starting from a point x ₀ one finds successively the minima in the directions p i . The results can be labeled by

Here a ₁ is the vector representing all n substeps, a ₂ contains all of the steps except the first one, and a n is just the last substep. The sum of the n substeps leads then from the point x ₀ to

The direction a ₁ describes the average direction of the first n substeps. Therefore we carry out a step in the direction a ₁, call the result again x ₀, determine a new set of directions

then call these q i again p i and proceed as above. As was shown by POWELL [15], the directions after n steps, i.e., n(n + 1) individual minimizations, are individually conjugate to each other if the function is a quadratic form.

10.11 Minimization Along Chosen Directions

The procedure given at the end of the last section contains, however, the danger that the directions p ₁, …, p n can become almost linearly dependent, because at every step p ₁ is rejected in favor of a ₁∕ | a ₁ | , and these directions need not be very different from step to step. Powell has therefore suggested not to replace the direction p ₁ by a ₁∕ | a ₁ | , but rather to replace the direction p max, along which the greatest reduction of the function took place. This sounds at first paradoxical, since what is clearly the best direction is replaced by another. But since out of all the directions p max gave the largest contribution towards reducing the function, a ₁∕ | a ₁ | has a significant component in the direction p max. By retaining these two similar directions one would increase the danger of a linear dependence.

In some cases, however, after completing a step we will retain the old directions without change. We denote by x ₀ the point before carrying out the step in progress, by x ₁ the point obtained by this step, by

(10.11.1)

an extrapolated point that lies in the new direction x ₁ −x ₀ from x ₀ but is still further than x ₁, and by M ₀, M ₁, and M e the corresponding function values. If

(10.11.2)

then the function no longer decreases significantly in the direction (x ₀ −x ₁). We then remain with the previous directions.

Further we denote by Δ M the greatest change in M along a direction in the step in progress and compute the quantity

(10.11.3)

If

then we retain the old directions. This requirement is satisfied if either the first or second factor in the first term of (10.11.3) becomes large. The first factor $$(M_{0} - 2M_{1} + M_{\mathrm{e}})$$ is proportional to a second derivative of the function M. If it is large (compared to the first derivative in meaningful units), then we are already close to the minimum. The second factor $${(M_{0} - M_{1} -\varDelta M)}^{2}$$ is large when the reduction of the function M ₀ − M ₁, with the contribution Δ M, does not stem mainly from a single direction.

The class MinPow determines the minimum of a function of n variables by successive minimization along chosen directions according to Powell. Figure 10.11 shows the advantages of that method. One can see a significantly faster convergence to the minimum compared to that of Fig. 10.10.

10.12 Minimization in the Direction of Steepest Descent

In order to get from a point x ₀ to the minimum of a function M(x), it is sufficient to always follow the negative gradient $$\mathbf{b}(\mathbf{x}) = -\boldsymbol{\nabla }M(\mathbf{x})$$. It is thus obvious that one should look for the minimum along the direction ∇ M(x ₀). One calls this point x ₁, searches then for the minimum along ∇ M(x ₁), and so forth, until the termination requirement (10.9.1) is satisfied.

Fig. 10.11:

Minimization along a chosen direction with the method of Powell.

Fig. 10.12:

Minimization in the direction of steepest descent.

A comparison of the steps taken with this method, shown in Fig. 10.12, with those from minimization along a coordinate direction (Fig. 10.10) shows, however, a surprising similarity. In both cases, successive steps are perpendicular to each other. Thus the directions cannot be fitted to the function in the course of the procedure and convergence is slow.

The initially surprising fact that successive gradient directions are perpendicular to each other stems from the construction of the procedure. Searching from x ₁ for the minimum in the direction b ₀ means that the derivative in the direction b ₀ vanishes at the point x ₁,

and thus the gradient is perpendicular to b ₀.

10.13 Minimization Along Conjugate Gradient Directions

We take up again the idea of conjugate directions from Sect. 10.10. We construct, starting from an arbitrary vector g ₁ = h ₁, two sequences of vectors,

(10.13.1)

(10.13.2)

with

(10.13.3)

Thus one has

(10.13.4)

(10.13.5)

This means that successive vectors g are orthogonal, and successive vectors h are conjugate to each other.

Rewriting the relation (10.13.3) gives

(10.13.6)

(10.13.7)

We now try to construct the vectors g i and h i without explicit knowledge of the Hessian matrix A. To do this we again assume that the function to be minimized is a quadratic form (10.1.10). At a point x i we define the vector $$\mathbf{g}_{i} = -\boldsymbol{\nabla }M(\mathbf{x}_{i})$$. If we now search for the minimum starting from x i along the direction h i , we find it at x i+1 and construct there $$\mathbf{g}_{i+1} = -\boldsymbol{\nabla }M(\mathbf{x}_{i+1})$$. Then g i+1 and g i are orthogonal, since from (10.1.12) one has

and

(10.13.8)

Here λ i has been chosen such that x i+1 is the minimum along the direction h i . This means that there the gradient is perpendicular to h i ,

(10.13.9)

Substituting into (10.13.8) gives indeed

in agreement with (10.13.7).

From these results we can now construct the following algorithm. We begin at x ₀, construct there the gradient ∇ M(x ₀), and set its negative equal to the two vectors

We minimize along h ₁. At the point of the minimum x ₁ we construct $$\mathbf{g}_{2} = -\boldsymbol{\nabla }M(\mathbf{x}_{1})$$ and compute from (10.13.6)

and from (10.13.2)

From x ₁ we then minimize along h ₂, and so forth.

The class MinCjg determines the minimum of a function of n variables by successive minimization along conjugate gradient directions. A comparison of Fig. 10.13 with Fig. 10.12 shows the superiority of the method of conjugate gradients over determination of direction according to steepest descent, especially near the minimum.

Fig. 10.13:

Minimization along conjugate gradient directions.

10.14 Minimization with the Quadratic Form

If the function to be minimized M(x) is of the simple form (10.1.10), then the position of the minimum is given directly by (10.1.13). Otherwise one can always expand M(x) about a point x ₀,

(10.14.1)

with

(10.14.2)

and obtain as an approximation for the minimum

(10.14.3)

One can now compute again b and A as derivatives at the point x ₁ and from this obtain a further approximation x ₂ according to (10.14.3), and so forth.

For the case where the approximation (10.14.2) gives a good description of the function M(x), the procedure converges quickly, since it tries to jump directly to the minimum. Otherwise it might not converge at all. We have already discussed the difficulties for the corresponding one-dimensional case in Sect. 10.1 with Fig. 10.1.

The class MinQdr finds the minimum of a function of n variables with the quadratic form. Figure 10.14 illustrates the operation of the method. One can observe that the minimum is in fact reached in very few steps.

Fig. 10.14:

Minimization with quadratic form.

10.15 Marquardt Minimization

MARQUARDT [16] has given a procedure that combines the speed of minimization with the quadratic form near the minimum with the robustness of the method of steepest descent, where one finds the minimum even starting from a point far away. It is based on the following simple consideration.

The prescription (10.14.3), written as a computation of the ith approximation for the position of the minimum,

(10.15.1)

means that one obtains x i from the point x i−1 by taking a step along the vector A ⁻¹ b. Here $$\mathbf{b} = -\boldsymbol{\nabla }M(\mathbf{x}_{i-1})$$ is the negative gradient, i.e., a vector in the direction of steepest descent of the function M at the point x i−1. If in (10.15.1) one had the unit matrix multiplied by a constant instead of the matrix A, i.e., if instead of (10.15.1) one were to use the prescription

(10.15.2)

then one would step by the vector b∕λ from x i−1. This is a step in the direction of steepest descent of the function, which is smaller for larger values of the constant λ. A sufficiently small step in the direction of steepest descent is, however, always a step towards the minimum (at least when one is still in the “approach” to the minimum, i.e., in the one-dimensional case of Fig. 10.1, between the two maxima). The Marquardt procedure consists of interpolating between the prescriptions (10.15.1) and (10.15.2) in such a way that the function M is reduced with every step, and such that the fast convergence of (10.15.1) is exploited as much as possible.

In place of (10.15.1) or (10.15.2) one computes

(10.15.3)

Here λ is determined in the following way. One first chooses a fixed number ν > 1 and denotes by λ (i−1) the value of λ from the previous step. As an initial value one chooses, e.g., λ ⁽⁰⁾ = 0. 01. The value obtained from (10.15.3) of x i clearly depends on λ. One computes two points x i (λ (i−1)) and $$\mathbf{x}_{i}{(\lambda }^{(i-1)}/\nu )$$, where for λ one chooses the values λ (i−1) and $${\lambda }^{(i-1)}/\nu$$, and the corresponding function values $$M_{i} = M(\mathbf{x}_{i}{(\lambda }^{(i-1)}))$$ and $$M_{i}^{(\nu )} = M(\mathbf{x}_{i}{(\lambda }^{(i-1)}/\nu ))$$. These are compared with the function value $$M_{i-1} = M(\mathbf{x}_{i-1})$$. The result of the comparison determines what happens next. The following cases are possible:

(i)

$$M_{i}^{(\nu )} \leq M_{i-1}$$: One sets $$\mathbf{x}_{i} = \mathbf{x}_{i}{(\lambda }^{(i-1)}/\nu )$$ and $${\lambda }^{(i)} ={\lambda }^{(i-1)}/\nu$$.

(ii)

$$M_{i}^{(\nu )} > M_{i-1}$$ and M i  ≤ M i−1: One sets $$\mathbf{x}_{i} = \mathbf{x}_{i}{(\lambda }^{(i-1)})$$ and $${\lambda }^{(i)} ={\lambda }^{(i-1)}$$.

(iii)

$$M_{i}^{(\nu )} > M_{i-1}$$ and M i  > M i−1: One replaces λ (i−1) by λ (i−1) ν and repeats the computation of $$\mathbf{x}_{i}{(\lambda }^{(i-1)}/\nu )$$ and x i (λ (i−1)) and the corresponding function values, and repeats the comparisons.

In this way one ensures that the function value in fact decreases with each step and that the value of λ is always as small as possible when adjusted to the local situation. Clearly (10.15.3) becomes (10.15.1) for λ → 0, i.e., it describes minimization with the quadratic form. For very large values of λ, Eq. (10.15.3) becomes the relation (10.15.2), which prescribes a small but sure step in the direction of steepest descent.

Fig. 10.15:

Minimization with the Marquardt procedure.

The class MinMar finds the minimum of a function of n variables by Marquardt minimization. In Fig. 10.15 one sees the operation of the Marquardt method. It follows a systematic path to the minimum. Comparing with Fig. 10.14 one sees that the method of the quadratic form converged in fewer steps. The Marquardt procedure, however, leads to the minimum in many cases where the method of the quadratic form would fail, e.g., in cases with poorly determined initial values.

10.16 On Choosing a Minimization Method

Given the variety of minimization methods, the user is naturally faced with the question of choosing a method appropriate to the task. Before we give recommendations for this, we will first recall the various methods once again.

The simplex method (routine MinSim) is particularly robust. Only function values M(x) are computed. The method is slow, however. Faster, but still quite robust is minimization along a chosen direction (routine MinPow). This also requires only function values.

The method of conjugate gradients (routine MinCjg) requires, as the name indicates, the computation of not only the function, but also its gradient. The number of iteration steps is, however, approximately equal to that of MinPow.

For minimization with the quadratic form (routine MinQdr) and Marquardt minimization (routine MinMar) one requires in addition the Hessian matrix of second derivatives. The derivatives are computed numerically with utility routines. The user can replace these utility routines by routines in which the analytic formulas for the derivatives are programmed. If the starting values are sufficiently accurate, MinQdr converges after just a few steps. The convergence is slower for MinMar. The method is, however, more robust; it often converges when starting from values from which MinQdr would fail.

From these characteristics of the methods we arrive at the following recommendations:

1.

For problems that need only to be solved once or not many times, that is, in which the computing time does not play an important role, one should choose MinSim or MinPow.

2.

For problems occurring repeatedly (with different numerical values) one should use MinMar. If one always has an accurate starting approximation, MinQdr can be used.

3.

For repeated problems the derivatives needed for MinMar or MinQdr should be calculated analytically. Although this entails additional programming work, one gains precision compared to numerical derivatives and saves in many cases computing time.

At this point we should make an additional remark about the comparison of the minimization methods of this chapter with the method of least squares from Chap. 9.​ The method of least squares is a special case of minimization. The function to be minimized is a sum of squares, e.g., (9.​1.​8), or the generalization of a sum of squares, e.g., (9.​5.​9). In this generalized sum of squares there appears the matrix of derivatives A. These are not, however, the derivatives of the function to be minimized, but rather derivatives of a function f, which characterizes the problem under consideration; cf. (9.​5.​2). Second derivatives are never needed. In addition, if one uses the singular value decomposition, as has been done in our programs in Chap. 9 to solve the least-squares problem, one works in numerically critical cases with a considerably higher accuracy than in the computation of sums of squares (cf. Sect. A.13, particularly Example A.4).

Least-squares problems should therefore always be carried out with the routines of Chap. 9.​ This applies particularly to problems of fitting of functions like those in the examples of Sect. 9.​6, when one has many measured points. The matrix A then contains many rows, but few columns. In computing the function to be minimized the product A T A is encountered, and one is threatened with the above mentioned loss of accuracy in comparison to the singular value decomposition.

10.17 Consideration of Errors

In data analysis minimization procedures are used for determining best estimates $$\widetilde{\mathbf{x}}$$ for unknown values x. The function to be minimized M(x) is here usually a sum of squares (Chap. 9) or a log-likelihood function (multiplied by − 1) as in Chap. 7.​ In using the equations from Chap. 7 one must note, however, that there the n-vector that was called λ is now denoted by x. The variables in Chap. 7 that were called x (i) are the measured values (usually called y in Chap. 9).

Information on the error of $$\widetilde{\mathbf{x}}$$ is obtained directly from results of Sects. 9.​7, 9.8, and 9.13 by defining

(10.17.1)

as the elements of the symmetric matrix of second derivatives (the Hessian matrix) of the minimization function. Here one must note that the factor f QL takes on the numerical value

(10.17.2)

when the function being minimized is a sum of squares. If the function is a log-likelihood function (times − 1) then this must be set equal to

(10.17.3)

1. Covariance matrix. Symmetric errors. The covariance matrix of $$\widetilde{\mathbf{x}}$$ is

(10.17.4)

The square roots of the diagonal elements are the (symmetric) errors

(10.17.5)

It is only meaningful to give the covariance matrix if the measurement errors are small and/or there are many measurements, i.e., the requirements for (7.​5.​8) are fulfilled.

2. Confidence ellipsoid . Symmetric confidence limits . The covariance matrix defines the covariance ellipsoid; cf. Sects. 5.​10 and A.11. Its center is given by $$\mathbf{x} = \widetilde{\mathbf{x}}$$. The probability that the true value of x is contained inside the ellipsoid is given by (5.​10.​20). The ellipsoid for which this probability has a given value W, the confidence level, is given by the confidence matrix

(10.17.6)

Here $$\chi _{W}^{2}(n_{f})$$ is the quantile of the χ ²-distribution for n f degrees of freedom and probability P = W; cf. (5.​10.​19) and (C.5.3). The number of degrees of freedom n f is equal to the number of measured values minus the number of parameters determined by the minimization. The square roots of the diagonal elements of $$C_{\widetilde{x}}^{(W)}$$ are the distances from the symmetric confidence limits,

(10.17.7)

The class MinCov yields the covariance or confidence matrix, respectively, for parameters determined by minimization.

3. Confidence region . If giving a covariance or confidence ellipsoid is not meaningful, it is still possible to give a confidence region with confidence level W. It is determined by the hypersurface

(10.17.8)

With the help of the following routine a contour of a cross section through this hypersurface is drawn in a plane that contains the point $$\widetilde{\mathbf{x}}$$ and is parallel to (x i , x j ). Here x i and x j are two components of the vector of parameters x. The boundary of the confidence region in a plane spanned by two parameters, which were found by minimization, can be graphically shown with the method DatanGraphics.drawContour, cf. Examples 10.1–10.3 and Example Programs 10.2–10.4.

4. Asymmetric errors and confidence limits. If the confidence region is not an ellipsoid, the asymmetric confidence limits for the variable x i can still be determined by

(10.17.9)

The differences

(10.17.10)

are the (asymmetric) distances from the confidence limits. If one takes $$\chi _{W}^{2}(n_{f}) = 1$$, the asymmetric errors Δ x i+ and Δ x i− are obtained. The class MinAsy yields asymmetric errors or the distances from the confidence limits, respectively, for parameters, determined by minimization.

10.18 Examples

Example 10.1:

Determining the parameters of a distribution from the elements of a sample with the method of maximum likelihood

Suppose one has N measurements y ₁, y ₂, …, y n that can be assumed to come from a normal distribution with expectation value a = x ₁ and standard deviation σ = x ₂. The likelihood function is

(10.18.1)

and its logarithm is

(10.18.2)

The task of determining the maximum-likelihood estimators $$\widetilde{x}_{1}$$, $$\widetilde{x}_{2}$$ has already been solved in Example 7.8 by setting the analytically calculated derivatives of the function ℓ(x) to zero. Here we will accomplish this by means of numerical minimization of ℓ(x). For this we must first provide a function that computes the function to be minimized,

This simple example is implemented in Example Programs 10.2 and 10.3. The results of the minimization of this user function are shown in Fig. 10.16 for two samples. Confidence regions and covariance ellipses agree reasonably well with each other. The agreement is better for the larger sample. ■ 

Fig. 10.16:

Determination of the parameters x ₁ (mean) and x ₂ (width) of a Gaussian distribution by maximization of the log-likelihood function of a sample. The plots on the left show two different samples, marked as one-dimensional scatter plots (tick marks) on the y axis. The curves f(y) are Gaussian distributions with the fitted parameters. The right-hand plots show in the (x ₁, x ₂) plane the values of the parameters obtained with symmetric errors and covariance ellipses, as well as the confidence region for χ W ² = 1 and the corresponding confidence boundaries (horizontal and vertical lines).

Example 10.2:

Determination of the parameters of a distribution from the histogram of a sample by maximizing the likelihood

Instead of the original sample y ₁, y ₂, …, y N as in Example 10.1, one often uses the corresponding histogram. Denoting by n i the number of observations that fall into the interval centered about the point t i with width Δ t,

(10.18.3)

the histogram is given by the pairs of numbers

(10.18.4)

If the original sample is taken from a normal distribution with expectation value x ₁ = a and standard deviation x ₂ = σ, i.e., from the probability density

(10.18.5)

then one might expect the $$n_{i}(t_{i})$$ (at least in the limit N → ∞) to be given by

(10.18.6)

The quantities $$n_{i}(t_{i})$$ are integers, which are clearly not equal in general to g i . We can, however, regard each $$n_{i}(t_{i})$$ as a sample of size one from a Poisson distribution with the expectation value

(10.18.7)

The a posteriori probability to observe the value $$n_{i}(t_{i})$$ is clearly

(10.18.8)

The likelihood function for the observation of the entire histogram is

(10.18.9)

and its logarithm is

(10.18.10)

If we use for λ i the notation of (10.18.7) and find the minimum of − ℓ with respect to x ₁, x ₂, then this determines the best estimates of the parameters x ₁, x ₂. Of course the same procedure can be applied not only in the case of a simple Gaussian distribution, but for any distribution that depends on parameters. One must simply use the corresponding probability density in (10.18.5) and (10.18.6). In the user function given below one must change only one instruction in order to replace the Gaussian by another distribution. This example is implemented in Example Program 10.4. There the user function carries the name MinLogLikeHistPoisson.

Fig. 10.17:

Determination of the parameters x ₁ (mean) and x ₂ (width) of a Gaussian distribution by maximizing the log-likelihood function of a histogram. The left-hand plots show two histograms corresponding to the samples from Fig.  10.16. The curves are the Gaussian distributions normalized to the histograms. The right-hand plots show the symmetric errors, covariance ellipse, and confidence region represented as in Fig. 10.16.

The results of the minimization with this user function are shown in Fig. 10.17 for two histograms, which are based on the samples from Fig. 10.16. The results are very similar to those from Example 10.1. The errors of the parameters, however, are somewhat larger. This is to be expected, since some information is necessarily lost when constructing a histogram from the sample.

Fig. 10.18:

As in Fig.  10.17, but for histograms with different interval widths of the same sample.

A histogram can be viewed as a sample in a compressed representation. The compression becomes greater as the bin width of the histogram increases. This is made clear in Fig. 10.18. One sees that for the same sample, the errors of the determined parameters increase for greater bin width. The effect is relatively small, however, for the relatively large sample size in this case. ■ 

Example 10.3:

Determination of the parameters of a distribution from the histogram of a sample by minimization of a sum of squares

If the bin contents n i of a histogram (10.18.4) are sufficiently large, then the statistical fluctuations of each n i can be approximated by a Gaussian distribution with the standard deviation

(10.18.11)

(See Sect. 6.​8.​) The weighted sum of squares describing the deviation of the histogram contents n i from the expected values g i from (10.18.6) is then

(10.18.12)

When carrying out the sum one must take care (in contrast to Example 10.2) that empty bins (n i  = 0) are not included. Even better is to not include bins in the sum even with low bin contents, e.g., n i  < 4.

Also this example is implemented in Example Program 10.4. Here g i is given by (10.18.6) as in the previous example. The sum is carried out over all bins with n i  > 0. The user function is called MinHistSumOfSquares. The statistical fluctuations of the bin contents are taken as approximately Gaussian.

Figure 10.19 shows the results obtained by minimizing the quadratic sum for the histogram in Fig. 10.18. Since the histograms are based on the same sample, the values n i decrease for decreasing bin width, whereby the requirement for using the sum of squares becomes less well fulfilled. Thus we cannot trust as much the results nor the errors for smaller bin width. One sees, however, that the errors given by the procedure in fact increase for decreasing bin width. ■ 

Fig. 10.19:

Determination of the parameters x ₁ (mean) and x ₂ (width) of a Gaussian distribution by minimization of a weighted sum of squares. The histograms on the left are the same as in Fig. 10.18. The fit results and errors, covariance ellipses, and confidence regions are represented as in Figs. 10.17 and 10.18.

We emphasize that the determination of parameters from a histogram by quadratic-sum minimization gives less exact results than those obtained by likelihood maximization. This is because the assumption of a normal distribution of the n i with the width (10.18.11) is only an approximation, which often requires large bin widths for the histogram and thus implies a loss of information. If, however, enough data, i.e., sufficiently large samples, are available, then the difference between the two procedures is small. One should compare, e.g., Fig. 10.18 (upper right-hand plot) with Fig. 10.19 (lower right-hand plot).

10.19 Java Classes and Example Programs

Java Classes for Minimization Problems

MinParab finds the extremum of a parabola through three given points.

FunctionOnLine computes the value of a function on a straight line in n-dimensional space.

MinEnclose brackets the minimum on a straight line in n-dimensional space.

MinCombined finds the minimum in a given interval along a straight line with a combined method according tho Brent.

MinDir finds the minimum on a straight line in n-dimensional space.

MinSim finds the minimum in n-dimensional space using the simplex method.

MinPow finds the minimum in n-dimensional space with Powell’s method of chosen directions.

MinCjg finds the minimum in n-dimensional space with the method of conjugate directions.

MinQdr finds the minimum in n-dimensional space with quadratic form.

MinMar finds the minimum in n-dimensional space with Marquardt’s method.

MinCov finds the covariance matrix of the coordinates of the minimum.

MinAsy finds the asymmetric errors of the coordinates of the minimum.

Example Program 10.1:

The class E1Min demonstrates the use of MinSim, MinPow, MinCjg, MinQdr, and MinMarThe program calls one of the classes, as requested by the user, in order to solve the following problem. One wants to find the minimum of the function $$f = f(\mathbf{x}) = f(x_{1},x_{2},x_{3})$$. The search is started at the point $${\mathbf{x}}^{(\mathrm{in})} = (x_{1}^{(\mathrm{in})},x_{2}^{(\mathrm{in})},x_{3}^{(\mathrm{in})})$$, which is also input by the user. The program treats consecutively four different cases:

No variables are fixed,

x ₃ is fixed,

x ₂ and x ₃ are fixed,

All variables are fixed.

The user can choose one of the following functions to be minimized:

Suggestions: Discuss the functions f ₁ through f ₇. All of them have a minimum at $$x_{1} = x_{2} = x_{3} = 0$$. Some possess additional minima. Study the convergence behavior of the different minimization methods for these functions using different starting points and explain this behavior qualitatively.

Example Program 10.2:

The class E2Min determines the parameters of a distribution from the elements of a sample and demonstrates the use of MinCovThe program solves the problem in Example 10.1. First a sample is drawn from the standard normal distribution. Next the sample is used to estimate the parameters x ₁ (mean) and x ₂ (standard deviation) of the population by minimizing the negative of the likelihood function (10.18.2). This is done with MinSim. The covariance matrix of the parameters is determined by calling MinCov. The results are presented numerically. The rest of the program performs the graphical display of the sample as a one-dimensional scatter plot and of the fitted function as in Fig. 10.16 (left-hand plots).

Suggestion: Run the program for samples of different size.

Example Program 10.3:

The class E3Min demonstrates the use of MinAsy and draws the boundary of a confidence regionThe class solves the same problem as the previous example. In addition it computes the asymmetric errors of the parameters using MinAsy. Then the solution, the symmetric errors , the covariance ellipse , and the asymmetric errors are displayed graphically in the (x ₁, x ₂) plane, and with the help of the method DatanGraphics.drawContour, the contour of the confidence region is shown as well. The plot corresponds to Fig. 10.16 (right-hand plots).

Example Program 10.4:

The class E4Min determines the parameters of a distribution from the histogram of a sample The program solves the problem of Examples 10.2 and 10.3. First a sample of size n ev is drawn from a standard normal distribution and a histogram with n t bins between $$t_{0} = -5.25$$ and t max = 5. 25 is constructed from the sample. The bin centers are t i (i = 1, 2, …, n t ), and the bin contents are n i . As chosen by the user either the likelihood function ℓ given by (10.18.10) is maximized (i.e., − ℓ is minimized) by MinSim and the user function MinLogLikeHistPoisson, or the sum of squares Q given by (10.18.12) is minimized, again with MinSim, but using MinHistSumOfSquares.

The results are presented in graphical form. One plot contains the histogram and the fitted Gaussian (i.e., a plot corresponding to the plots on the left-hand side of Figs. 10.17 or 10.18), a second one presents the solution in the (x ₁, x ₂) plane with symmetric and asymmetric errors, covariance ellipse, and confidence region (corresponding to the plots on the right-hand side of those figures).

Suggestions: (a) Choose n ev = 100. Show that for likelihood maximization the errors Δ x ₁, Δ x ₂ increase if you decrease the number of bins beginning with n t  = 100, but that for the minimization of the sum of squares the number of bins has to be small to get meaningful errors. (b) Show that for n ev = 1000 and n t  = 50 or n t  = 20 there is practically no difference between the results of the methods.

Literature

Literature Cited in the Text

[12]

W.H. Press, B.P. Flannery, S.A. Teukolsky and W.T. Vetterling, Numerical Recipes, Cambridge University Press, Cambridge 1986MATH

[13]

R.P. Brent, Algorithms for Minimization without Derivatives, Prentice-Hall, Englewood Cliffs NJ 1973MATH

[14]

J.A. Nelder and R. Mead, Computer Journal 7 (1965) 308MathSciNetCrossref

[15]

M.J.D. Powell, Computer Journal 7 (1965) 155Crossref

[16]

D.W. Marquardt, J. Soc. Ind. Appl. Math. 11 (1963) 431Crossref

© Springer International Publishing Switzerland 2014

Siegmund BrandtData Analysishttps://doi.org/10.1007/978-3-319-03762-2_11

11. Analysis of Variance

Siegmund Brandt1 

(1)

Department of Physics, University of Siegen, Siegen, Germany

The analysis of variance (or ANOVA ), originally developed by R. A. FISHER, concerns testing the hypothesis of equal means of a number of samples. Such problems occur, for example, in the comparison of a series of measurements carried out under different conditions, or in quality control of samples produced by different machines. One tries to discover what influence the changing of external variables (e.g., experimental conditions, the number of a machine) has on a sample. For the simple case of only two samples, this problem can also be solved with Student’s difference test (Sect. 8.​3).

We speak of one-way analysis of variance, or also one-way classification, when only one external variable is changed. The evaluation of a series of measurements of an object micrometer performed with different microscopes can serve as an example. One has a two- (or more) way analysis of variance (two-way classification) when several variables are changed simultaneously. In the example above, if different observers carry out the series of measurements with each microscope, then a two-way analysis of variance can investigate influences of both the observer and the instrument on the result.

11.1 One-Way Analysis of Variance

Let us consider a sample of size n, which can be divided into t groups according to a certain criterion A. Clearly the criterion must be related to the sampling or measuring process. We say that the groups are constructed according to the classification A. We assume that the populations from which the t subsamples are taken are normally distributed with the same variance σ ². We now want to test the hypothesis that the mean values of these populations are also equal. If this hypothesis is true, then all of the samples come from the same population. We can then apply the results of Sect. 6.​4 (samples from subpopulations). Using the same notation as there, we have t groups of size n i with

and we write the jth element of the ith group as x ij . The sample mean of the ith group is

(11.1.1)

and the mean of the entire sample is

(11.1.2)

We now construct the sum of squares

The last term vanishes because of (11.1.1) and (11.1.2). One therefore has

(11.1.3)

The first term is the sum of squares between the groups obtained with the classification A. The second term is a sum over the sums of squares within a group. The sum of squares Q is decomposed into a sum of two sums of squares corresponding to different “sources” – the variation of means within the classification A and the variation of measurements within the groups. If our hypothesis is correct, then Q is a sum of squares from a normal distribution, i.e., Q∕σ ² follows a χ ²-distribution with n − 1 degrees of freedom. Correspondingly, for each group the quantity

follows a χ ²-distribution with n i − 1 degrees of freedom. The sum

is then described by a χ ²-distribution with $$\sum _{i}(n_{i} - 1) = n - t$$ degrees of freedom (see Sect. 6.​6). Finally, $$Q_{A}{/\sigma }^{2}$$ follows a χ ²-distribution with t − 1 degrees of freedom.

The expressions

(11.1.4)

are unbiased estimators of the population variances. (In Sect. 6.​5 we called such expressions mean squares.) The ratio

(11.1.5)

can thus be used to carry out an F-test.

If the hypothesis of equal means is false, then the values $$\bar{x}_{i}$$ of the individual groups will be quite different. Thus $$s_{A}^{2}$$ will be relatively large, while $$s_{W}^{2}$$, which is the mean of the variances of the individual groups, will not change much. This means that the ratio (11.1.5) will be large. Therefore one uses a one-sided F-test. The hypothesis of equal means is rejected at the significance level α if

(11.1.6)

The sums of squares can be computed according to two equivalent formulas,

(11.1.7)

The expression on the right of each line is usually easier to compute. Since each sum of squares is obtained by computing the difference of two relatively large numbers, one must pay attention to possible problems with errors in rounding. Although one only needs the sums Q A and Q W in order to compute the ratio F, it is recommended to compute Q as well, since one can then perform a check using (11.1.3), i.e., $$Q = Q_{A} + Q_{W}$$. The check is only meaningful when Q is computed with the left-hand form of (11.1.3). Usually the results of an analysis of variance are summarized in a so-called analysis of variance table , (or ANOVA table) as shown in Table 11.1.

Before carrying out an analysis of variance one must consider whether the requirements are met under which the procedure has been derived. In particular, one must check the assumption of a normal distribution for the measurements within each group. This is by no means certain in every case. If, for example, the measured values

Table 11.1:

ANOVA table for one-way classification.

Source

SS

DF

MS

F

(sum of

(degrees of

(mean

squares)

freedom)

square)

Between the groups

Q A

t − 1

Within the groups

Q W

n − t

Sum

Q

n − 1

are always positive (e.g., the length or weight of an object) and if the standard deviation is of a magnitude comparable to the measured values, then the probability density can be asymmetric and thus not Gaussian. If, however, the original measurements (let us denote them for the moment by x′) are transformed using a monotonic transformation such as

(11.1.8)

where a and b are appropriately chosen constants, then a normal distribution can often be sufficiently well approximated. Other transformations sometimes used are $$x = \sqrt{x'}$$ or $$x = 1/x'$$.

Example 11.1:

One-way analysis of variance of the influence of various drugs

The spleens of mice with cancer are often attacked particularly strongly. The weight of the spleen can thus serve as a measure of the reaction to various drugs. The drugs (I–III) were used to treat ten mice each. Table 11.2 contains the measured spleen weights, which have already been transformed according to x = logx′, where x′ is the weight in grams. Most of the calculation is presented in Table 11.2. Table 11.3 contains the resulting ANOVA table. Since even at a significance level of 50 % the F-test gives F 0. 5(2, 24) = 3. 4, one cannot reject the hypothesis of equal mean values. The experiment thus showed no significant difference in the effectiveness of the three drugs. ■ 

Table 11.2:

Data for Example 11.1.

Experiment

Group

number

I

II

III

1

19

40

32

2

45

28

26

3

26

26

30

4

23

15

17

5

36

24

23

6

23

26

24

7

26

36

29

8

33

27

20

9

22

28

—

10

—

19

—

253

269

201

n i

9

10

8

n = 27

28.11

26.90

25.13

790.23

723.61

631.52

Table 11.3:

ANOVA table for Example 11.1.

Source

SS

DF

MS

F

Between the groups

38

2

19.0

0.377

Within the groups

1209

24

50.4

Sum

1247

26

47.8

11.2 Two-Way Analysis of Variance

Before we turn to analysis of variance with two external variables, we would like to examine more carefully the results obtained for one-way classification . We denoted the jth measurement of the quantity x in group i by x ij . We now assume for simplicity that each group contains the same number of measurements, i.e., n i  = J. In addition we denote the total number of groups by I. The classification into individual groups was done according to the criterion A, e.g., the production number of a microscope, by which the groups can be distinguished. The labeling according to measurement and group is illustrated in Table 11.4.

We can write the individual means of the groups in the form

(11.2.1)

Table 11.4:

One-way classification.

Measurement

Classification A

number

A ₁

A ₂

…

A i

…

A I

1

x ₁₁

x ₂₂

x i1

x I1

2

x ₁₂

x ₂₂

x i2

x I2

⋮

j

x 1j

x 2j

x ij

x Ij

⋮

J

x 1J

x 2J

x iJ

x IJ

Here a point denotes summation over the index that it replaces. This notation allows a simple generalization to a larger number of indices. The analysis of variance with one-way classification is based on the assumption that the measurements within a group only differ by the measurement errors, which follow a normal distribution with a mean of zero and variance σ ². That is, we consider the model

(11.2.2)

The goal of an analysis of variance was to test the hypothesis

(11.2.3)

By choosing measurements out of a certain group i and by applying the maximum-likelihood method to (11.2.2) one obtains the estimator

(11.2.4)

If H ₀ is true, then one has

(11.2.5)

The (composite) alternative hypothesis is that not all of the μ i are equal. We want, however, to retain the concept of the overall mean and we write

The model (11.2.2) then has the form

(11.2.6)

Between the quantities a i , which represent a measure of the deviation of the mean for the ith group from the overall mean, one has the relation

(11.2.7)

The maximum-likelihood estimators for the a i are

(11.2.8)

The one-way analysis of variance of Sect. 11.1 was derived from the identity

(11.2.9)

which describes the deviation of the individual measurements from the overall mean. The sum of squares Q of these deviations could then be decomposed into the terms Q A and Q W ; cf. (11.1.3).

After this preparation we now consider a two-way classification , where the measurements are divided into groups according to two criteria, A and B. The measurements x ijk belong to class A i , which is given by the classification according to A, and also to class B j . The index k denotes the measurement number within the group that belongs to both class A i and class B j .

A two-way classification is said to be crossed , when a certain classification B j has the same meaning for all classes A. If, for example, microscopes are classified by A and observers by B, and if each observer carries out a measurement with each microscope, then the classifications are crossed. If, however, one compares the microscopes in different laboratories, and if therefore in each laboratory a different group of J observers makes measurements with a certain microscope i, then the classification B is said to be nested in A . The index j then merely counts the classes B within a certain class A.

The simplest case is a crossed classification with only one observation. Since then k = 1 for all observations x ijk , we can drop the index k. One uses the model

(11.2.10)

where $$\varepsilon$$ is normally distributed with mean zero and variance σ ². The null hypothesis says that by classification according to A or B, no deviation from the overall mean occurs. We write this in the form of two individual hypotheses,

(11.2.11)

The least-squares estimators for a i and b j are

In analogy to Eq. (11.2.9) we can write

(11.2.12)

In a similar way the sum of squares can be written

(11.2.13)

where

(11.2.14)

When divided by the corresponding number of degrees of freedom, these sums are estimators of σ ², providing that the hypotheses (11.2.11) are correct. The hypotheses $$H_{0}^{(A)}$$ and $$H_{0}^{(B)}$$ can be tested individually by using the ratio

(11.2.15)

Here one uses one-sided F-tests as in Sect. 11.1. The overall situation can be summarized in an ANOVA table (Table 11.5).

If more than one observation is made in each group, then the crossed classification can be generalized in various ways. The most important generalization involves interaction between the classes. One then has the model

(11.2.16)

The quantity (ab) ij is called the interaction between the classes A i and B j . It describes the deviation from the group mean that occurs because of the specific interaction of A i and B j . The parameters a i , b j , (ab) ij are related by

(11.2.17)

Their maximum-likelihood estimators are

(11.2.18)

Table 11.5:

Analysis of variance table for crossed two-way classification with only one observation.

Source

SS

DF

MS

F

Class. A

Q A

I − 1

Class. B

Q B

J − 1

Within groups

Q W

Sum

Q

The null hypothesis can be divided into three individual hypotheses,

(11.2.19)

which can then be tested individually. The analysis of variance is based on the identity

(11.2.20)

which allows the decomposition of the sum of squares of deviations into four terms,

(11.2.21)

The degrees of freedom and mean squares as well as the F-ratio, which can be used for testing the hypotheses, are given in Table 11.6.

Table 11.6:

Analysis of variance table for crossed two-way classification.

Source

SS

DF

MS

F

Class. A

Q A

I − 1

Class. B

Q B

J − 1

Interaction

Q AB

Within groups

Q W

IJ(K − 1)

Sum

Q

IJK − 1

Finally, we will give the simplest case of a nested two-way classification . Because the classification B is only defined within the individual classes of A, the terms b j and (ab) ij from Eq. (11.2.10) are not defined, since they imply a sum over i for fixed j. Therefore one uses the model

(11.2.22)

with

The term b ij is a measure of the deviation of the measurements of class B j within class A i from the overall mean of class A i . The null hypothesis consists of

(11.2.23)

An analysis of variance for testing these hypotheses can be carried out with the help of Table 11.7. Here one has

In a similar way one can construct various models for two-way or multiple classification. For each model the total sum of squares is decomposed into a certain sum of individual sums of squares, which, when divided by the corresponding number of degrees of freedom, can be used to carry out an F-test. With this, the hypotheses implied by the model can be tested.

Table 11.7:

Analysis of variance table for nested two-way classification.

Source

SS

DF

MS

F

Class. A

Q A

I − 1

Within A

Q B(A)

I(J − 1)

Within groups

Q W

IJ(K − 1)

Sum

Q

IJK − 1

Some models are, at least formally, contained within others. For example one finds by comparing Tables 11.6 and 11.7 the relation

(11.2.24)

A similar relation holds for the corresponding number of degrees of freedom,

(11.2.25)

Example 11.2:

Two-way analysis of variance in cancer research

Two groups of rats are injected with the amino acid thymidine containing traces of tritium, a radioactive isotope of hydrogen. In addition, one of the groups receives a certain carcinogen. The incorporation of thymidine into the skin of the rats is investigated as a function of time by measuring the number of tritium decays per unit area of skin. The classifications are crossed since the time dependence is controlled in the same way for both series of test animals. The measurements are compiled in Table 11.8. The numbers are already transformed from the original counting rates x′ according to $$x = 50\log x' - 100$$. The results, obtained with the class AnalysisOfVariance, are shown in Table 11.9. There is no doubt that the presence or absence of the carcinogen (classification A) has an influence on the result, since the ratio F (A) is very large. We now want to test the existence of a time dependence (classification B) and of an interaction between A and B at a significance level of α = 0. 01. From Table I.8 we find F 0. 99 = 2. 72. The hypotheses of time independence and vanishing interaction must therefore be rejected. Table 11.9 also contains the values of α for which the hypothesis would not need to be rejected. They are very small. ■ 

Table 11.8:

Data for Example 11.2.

Obs.

Injection

Time after injection (h)

no.

4

8

12

16

20

24

28

32

36

48

1

34

54

44

51

62

61

59

66

52

52

2

Thymidine

40

57

52

46

61

70

67

59

63

50

3

38

40

53

51

54

64

58

67

60

44

4

36

43

51

49

60

68

66

58

59

52

1

28

23

42

43

31

32

25

24

26

26

2

Thymidine

32

23

41

48

45

38

27

26

31

27

3

and

34

29

34

36

41

32

27

32

25

27

4

Carcinogen

27

30

39

43

37

34

28

30

26

30

Table 11.9:

Printout from Example 11.2.

Analysis of variance table

Source

Sum of

Degrees of

Mean

F Ratio

Alpha

squares

freedom

square

A

9945.80

1

9945.80

590.547 25

0.00E−10

B

1917.50

9

213.06

12.650 50

0.54E−10

INT.

2234.95

9

248.33

14.744 85

0.03E−10

W

1010.50

60

16.84

TTL.

15 108.75

79

191.25

11.3 Java Class and Example Programs

11.3.1 Java Class

AnalysisOfVariance

performs a crossed as well as a nested two-way analysis of variance.

Example Program 11.1:

The class E1Anova demonstrates the use of AnalysisOfVarianceThe short program analyses the data of Example 11.2. Data and output are presented as in Table 11.9.

Example Program 11.2:

The class E2Anova simulates data and performs an analysis of variance on themThe program allows interactive input of numerical values for σ, the quantities I, J, K and three further parameters: $$\varDelta _{i},\varDelta _{j},\varDelta _{k}$$. It generates data of the simple form

Here the quantities $$\varepsilon _{ijk}$$ are taken from a normal distribution with zero mean standard deviation σ. An analysis of variance is performed on the data and the results are presented for crossed and for nested two-way classification.

Suggestion: Take a value ≠ 0 for only one of the parameters $$\varDelta _{i},\varDelta _{j},\varDelta _{k}$$ and interpret the resulting analysis of variance table.

© Springer International Publishing Switzerland 2014

Siegmund BrandtData Analysishttps://doi.org/10.1007/978-3-319-03762-2_12

12. Linear and Polynomial Regression

Siegmund Brandt1 

(1)

Department of Physics, University of Siegen, Siegen, Germany

Keywords

Covariance

The fitting of a linear function (or, more generally, of a polynomial) to measured data that depend on a controlled variable is probably the most commonly occurring task in data analysis. This procedure is also referred to as linear (or polynomial) regression . Although we have already treated this problem in Sect. 9.​4.​1, we take it up again here in greater detail. Here we will use different numerical methods, emphasize the most appropriate choice for the order of the polynomial, treat in detail the question of confidence limits, and also give a procedure for the case where the measurement errors are not known.

12.1 Orthogonal Polynomials

In Sect. 9.​4.​1 we have already fitted a polynomial of order r − 1,

(12.1.1)

to measured values y i (t i ), which corresponded to a given value t i of the controlled variable t. Here the quantities η(t) were the true values of the measured quantities y(t). In Example 9.2 we saw that there can be a maximum reasonable order for the polynomial, beyond which a further increase in the order gives no significant improvement in the fit. We want here to pursue the question of how to find the optimal order of the polynomial. Example 9.2 shows that when increasing the order, all of the coefficients x ₁, x ₂, $$\ldots$$ change and that all of the coefficients are in general correlated. Because of this the situation becomes difficult to judge. These difficulties are avoided by using orthogonal polynomials.

Instead of (12.1.1) one describes the data by the expression

(12.1.2)

Here the quantities f j are polynomials of order j − 1,

(12.1.3)

These are chosen to satisfy the condition of orthogonality (or more precisely condition of orthonormality) with respect to the values t i and the measurement weights $$g_{i} = 1/\sigma _{i}^{2}$$,

(12.1.4)

Defining

(12.1.5)

and using the matrix notation (9.​2.​9)

(12.1.6)

Eq. (12.1.4) has the simple form

(12.1.7)

The least-squares requirement,

(12.1.8)

is of the form (9.​2.​19) and thus has the solution (9.​2.​26)

(12.1.9)

where we have used (9.​2.​18) and (12.1.7). Because of (9.​2.​27) and (12.1.7) one has

(12.1.10)

i.e., the covariance matrix of the coefficients $$\widetilde{x}_{1}$$, …, $$\widetilde{x}_{r}$$ is simply the unit matrix. In particular, the coefficients are uncorrelated.

We now discuss the procedure for determining the matrix elements

(12.1.11)

For j = 1, the orthogonality condition gives

(12.1.12)

For j = 2 there are two orthogonality conditions. First we obtain

and from this

(12.1.13)

where

(12.1.14)

is the weighted mean of the values of the controlled variable t i at which the measurements were made. The second orthogonality condition for j = 2 gives

or

(12.1.15)

Substitution into (12.1.13) gives b ₂₁.

For j > 2 one can obtain the values of $$A_{ij} = f_{j}(t_{i})$$ recursively from the quantities for j − 1 and j − 2. We make the ansatz

(12.1.16)

multiply by $$g_{i}f_{j-1}(t_{i})$$, sum over i, and obtain

Because of the orthogonality condition, the second term on the right-hand side is equal to unity, and the third term vanishes. Thus one has

(12.1.17)

By multiplication of (12.1.16) by $$g_{i}f_{j-2}(t_{i})$$ and summing one obtains in the same way

(12.1.18)

Finally by computing the expression

and substituting f j (t i ) from (12.1.16) one obtains

(12.1.19)

Once the quantities α, β, γ have been computed for a given j, then the coefficients b jk are determined to be

(12.1.20)

With (12.1.3) and (12.1.5) one thus obtains the quantities $$A_{ij} = f_{j}(t_{i})$$. Since the procedure is recursive, the column j of the matrix A is obtained from the elements of the columns to its left. Be extending the matrix to the right, the original matrix elements are not changed. This has the consequence that by increasing the number of terms in the polynomial (12.1.1) from r to r′, the coefficients $$\widetilde{x}_{1}$$, …, $$\widetilde{x}_{r}$$ are retained, and one merely obtains additional coefficients. All of the $$\widetilde{x}_{j}$$ are uncorrelated and have the standard deviation $$\sigma _{\widetilde{x}_{j}} = 1$$. The order of the polynomial sufficient to describe the data can now be determined by the requirement

With this the contribution of all higher coefficients $$\widetilde{x}_{j}$$ is smaller than $$c\sigma _{\widetilde{x}_{j}}$$. The simplest choice of c is clearly c = 1.

For a given r one can now substitute the values b jk into (12.1.3) and $$f_{j}(t_{i})$$ into (12.1.2) in order to obtain the best estimates $$\widetilde{\eta }_{i}(t_{i})$$ of the true values $$\eta _{i}(t_{i})$$ corresponding to the measurements $$y_{i}(t_{i})$$. One can also compute the quantity

If the data are in fact described by the polynomial of the chosen order, then the quantity M follows a χ ²-distribution with $$f = N - r$$ degrees of freedom. It can be used for a χ ²-test for the goodness-of-fit of the polynomial.

Example 12.1:

Treatment of Example 9.2 with Orthogonal Polynomials

Application of polynomial regression to the data of Example 9.2 yields the results shown in Table 12.1. The numerical values of the minimization function are, of course, exactly the same as in Example 9.2. The fitted polynomials are also the same. The numerical values $$\widetilde{x}_{1}$$, $$\widetilde{x}_{2}$$, $$\ldots$$ are different from those in Example 9.2 since these quantities are now defined differently. We have emphasized that the covariance matrix for $$\widetilde{\mathbf{x}}$$ is the r × r unit matrix. We see that the quantities x ₆, x ₇, …, x ₁₀ have magnitudes less than unity and thus are no longer significantly different from zero. It is more difficult to judge the significance of x ₅ = 1. 08. In many cases one would not consider this value as being clearly different from zero. This means that a third order polynomial (i.e., r = 4) is sufficient to describe the data. ■ 

Table 12.1:

Results of the application of polynomial regression using the data from Example 9.2.

r

M

Degrees of freedom

1

24.05

833.55

9

2

15.75

585.45

8

3

23.43

36.41

7

4

5.79

2.85

6

5

1.08

1.69

5

6

0.15

1.66

4

7

0.85

0.94

3

8

− 0. 41

0.77

2

9

− 0. 45

0.57

1

10

0.75

0.00

0

12.2 Regression Curve: Confidence Interval

Once the order of the polynomial (12.1.2) in one way or another has been fixed, then every point of the regression polynomial or – in reference to the graphical representation – the regression curve can be computed. This is done by first substituting the recursively computed values b jk into (12.1.3) for a given t, and using the thus computed f j (t) and the parameters $$\widetilde{\mathbf{x}}$$ in (12.1.2):

(12.2.1)

Here $$\mathbf{d}$$ is an r-vector with the elements

(12.2.2)

After error propagation, (3.​8.​4), the variance of $$\widetilde{\eta }(t)$$ is

(12.2.3)

since $$C_{\widetilde{x}} = I$$. The reduced variable

(12.2.4)

thus follows a standard Gaussian distribution. This also means that the probability for the position of the true value η(t) is distributed according to a Gaussian of width $$\sigma _{\widetilde{\eta }}(t)$$ centered about the value $$\widetilde{\eta }(t)$$.

We can now easily give a confidence interval for η(t). According to Sect. 5.​8 one has with probability P that

(12.2.5)

e.g., Ω′(0. 95) = 1. 96. Thus the confidence limits at the confidence level P (e.g., P = 0. 95) are

(12.2.6)

12.3 Regression with Unknown Errors

If the measurement errors $$\sigma _{i} = 1/\sqrt{g_{i}}$$ are not known, but it can be assumed that they are equal,

(12.3.1)

then one can easily obtain an estimate s for σ directly from the regression itself. The quantity

(12.3.2)

follows a χ ²-distribution with $$f = N - r$$ degrees of freedom. More precisely, if many similar experiments with N measurements each were to be carried out, then the values of M computed for each experiment would be distributed like a χ ²-variable with $$f = N - r$$ degrees of freedom. Its expectation value is simply $$f = N - r$$. If the value of σ in (12.3.2) is not known, then it can be replaced by an estimate s such that on the right-hand side one has the expectation value of M,

(12.3.3)

Here all of the steps necessary to compute $$\widetilde{\eta }(t_{i})$$ must be carried out using the formulas of the last section and with the value σ i  = 1. (In fact, for the case (12.3.1) the quantities σ i and g i could be removed completely from the formulas for computing $$\widetilde{\mathbf{x}}$$ and $$\widetilde{\eta }(t)$$.)

If we define

as the value of the variance of $$\widetilde{\eta }(t)$$ obtained for σ i  = 1, then

(12.3.4)

is clearly the estimate of the variance of $$\widetilde{\eta }(t)$$, which is based on the estimate (12.3.3) for the variance of the measured quantities. Replacing $$\sigma _{\widetilde{\eta }(t)}$$ by $$s_{\widetilde{\eta }(t)}$$ in (12.2.4), we obtain the variable

(12.3.5)

which no longer follows a standard normal distribution but rather Student’s t-distribution for $$f = N - r$$ degrees of freedom (cf. Sect. 8.​3). For the confidence limits at the confidence level $$P = 1-\alpha$$ one now has

(12.3.6)

where $$t_{1-\alpha /2}$$ is the quantile of the t-distribution for $$f = N - r$$ degrees of freedom.

It must be emphasized at this point that in the case where the errors are not known, one loses the possibility of applying the χ ²-test. The goodness-of-fit for such a polynomial cannot be tested. Thus the procedure described at the end of Sect. 12.1 for determining the order of the polynomial is no longer valid. One must rely on a priori knowledge about the order of the polynomial. Therefore one almost always refrains from treating anything beyond linear dependences between η and t, i.e., the case r = 2.

Example 12.2:

Confidence limits for linear regression

Fig. 12.1:

(Above) Measurements with errors and regression line with 95 % confidence limits. (Below) Measured points with errors assumed to be of unknown but equal magnitude, regression line, and 95 % confidence limits.

In the upper plot of Fig. 12.1, four measured points with their errors, the corresponding regression line, and the limits at a confidence level of 95 % are shown. The measured points are taken from the example of Sect. 9.​3.​ The confidence limits clearly become wider the more one leaves the region of the measured points. The narrowest region is close to the measured point with the smallest error. The plot is similar to Fig.​ 9.​2d. It is easy to convince oneself that the envelope of the lines of Fig.​ 9.​2d corresponds to the 68. 3 % confidence limit. In the lower plot of Fig. 12.1 the same measured points were used, but the errors were treated as equal but of unknown magnitude. The regression line and 95 % confidence limits are shown for this assumption. ■ 

12.4 Java Class and Example Programs

12.4.1 Java Class

Regression performs a polynomial regression.

Example Program 12.1:

The class E1Reg demonstrates the use of RegressionThe short program contains the data of Examples 9.2 and 12.1. As measurement errors the statistical errors $$\varDelta y_{i} = \sqrt{y_{i}}$$ are taken. A polynomial regression with r = 10 parameters is performed. The results are presented numerically (see also Table 12.1).

Example Program 12.2:

The class E2Reg demonstrates the use of Regression and presents the results in graphical formThe same problem as in E1Reg is treated. An additional input variable is r max, the maximum number of terms in the polynomials to be presented graphically. Graphs of these polynomials are shown together with the data points.

Suggestion: Choose consecutively r max = 2, 3, 4, 6, 10. Try to explain why the curves for r max = 3 or 4 seem to be particularly convincing although for r max = 10 all data points lie exactly on the graph of the polynomial.

Example Program 12.3:

The class E3Reg demonstrates the use of Regression and graphically represents the regression line and its confidence limitsThis program again treats the same problem as E1Reg. Additional input parameters are r, the order of the polynomial to be fitted, and the probability P, which determines the confidence limits. Presented in a plot are the data points with their errors, the regression line, and – in a different color – its confidence limits.

Example Program 12.4:

The class E4Reg demonstrates the linear regression with known and with unknown errorsThe program uses the data of Example 12.2. The user can declare the errors to be either known or unknown and chooses a probability P which determines the confidence limits. The graphics – corresponding to Fig. 12.1 – contains data points, regression line, and confidence limits.

© Springer International Publishing Switzerland 2014

Siegmund BrandtData Analysishttps://doi.org/10.1007/978-3-319-03762-2_13

13. Time Series Analysis

Siegmund Brandt1 

(1)

Department of Physics, University of Siegen, Siegen, Germany

Keywords

Covariance

13.1 Time Series: Trend

In the previous chapter we considered the dependence of a random variable y on a controlled variable t. As in that case we will assume here that y consists of two parts, the true value of the measured quantity η and a measurement error $$\varepsilon$$,

(13.1.1)

In Chap. 12 we assumed that η i was a polynomial in t. The measurement error $$\varepsilon _{i}$$ was considered to be normally distributed about zero.

We now want to make less restrictive assumptions about η. In this chapter we call the controlled variable t “time”, although in many applications it could be something different. The method we want to discuss is called time series analysis and is often applied in economic problems. It can always be used where one has little or no knowledge about the functional relationship between η and t. In considering time series problems it is common to observe the y i at equally spaced points in time,

(13.1.2)

since this leads to a significant simplification of the formulas.

An example of a time series is shown in Fig. 13.1. If we look first only at the measured points, we notice strong fluctuations from point to point. Nevertheless they clearly follow a certain trend. In the left half of the plot they are mostly positive, and in the right half, mostly negative. One could qualitatively obtain the average time dependence by drawing a smooth curve by hand through the points. Since, however, such curves are not free from personal influences, and are thus not reproducible, we must try to develop an objective method.

Fig. 13.1:

Data points (circles) and moving average (joined by line segments).

We use the notation from (13.1.1) and call η i the trend and $$\varepsilon _{i}$$ the random component of the measurement y i . In order to obtain a smoother function of t, one can, for example, construct for every value of y i the expression

(13.1.3)

i.e., the unweighted mean of the measurements for the times

The expression (13.1.3) is called a moving average of y.

13.2 Moving Averages

Of course the moving average (13.1.3) is a very simple construction. We will show later (in Example 13.1) that the use of a moving average of this form is equivalent to the assumption that η is a linear function of time in the interval considered,

(13.2.1)

Here α and β are constants. They can be estimated from the data by linear regression.

Instead of restricting ourselves to the linear case, we will assume more generally that η can be a polynomial of order ℓ.

In the averaging interval, t takes on the values

(13.2.2)

Because η is a polynomial in t,

(13.2.3)

it is also a polynomial in j,

(13.2.4)

since (13.2.2) describes a linear transformation between t j and j, i.e., it is merely a change of scale. We now want to obtain the coefficients x ₁, x ₂, …, x ℓ+1 from the data by fitting with least squares. This task has already been treated in Sect. 9.​4.​1.​ We assume (in the absence of any better knowledge) that all of the measurements are of the same accuracy. Thus the matrix G y  = aI is simply a multiple of the unit matrix I. According to (9.​2.​26), the vector of coefficients is thus given by

(13.2.5)

where A is a $$(2k + 1) \times (\ell+1)$$ matrix,

(13.2.6)

Table 13.1:

Components of the vector $$\mathbf{a}$$ for computing moving averages.

ℓ = 2 and ℓ = 3

k

A

α ₋₇

α ₋₆

α ₋₅

α ₋₄

α ₋₃

α ₋₂

α ₋₁

α ₀

2

35

− 3

12

17

3

21

− 2

3

6

7

4

231

− 21

14

39

54

59

5

429

− 36

9

44

69

84

89

6

143

− 11

0

9

16

21

24

25

7

1105

− 78

− 13

42

87

122

147

162

167

ℓ = 4 and ℓ = 5

k

A

α ₋₇

α ₋₆

α ₋₅

α ₋₄

α ₋₃

α ₋₂

α ₋₁

α ₀

3

231

5

− 30

75

131

4

429

15

− 55

30

135

179

5

429

18

− 45

− 10

60

120

143

6

2431

110

− 198

− 135

110

390

600

677

7

46 189

2145

− 2860

− 2937

− 165

3755

7500

10 125

11 063

For the trend $$\widetilde{\eta }_{0}$$ at the center of the averaging interval (j = 0) we obtain from (13.2.4) the estimate

(13.2.7)

It is equal to the first coefficient of the polynomial. According to (13.2.5), $$\widetilde{x}_{1}$$ is obtained by multiplication of the column vector of measurements $$\mathbf{y}$$ on the left with the row vector

(13.2.8)

i.e., with the first row of the matrix $$-{({A}^{\mathrm{T}}A)}^{-1}{A}^{\mathrm{T}}$$. We obtain

(13.2.9)

This is a linear function of the measurements within the averaging interval. Here the vector $$\mathbf{a}$$ does not depend on the measurements, but rather only on ℓ and k, i.e., on the order of the polynomial and on the length of the interval. Clearly one must choose

since otherwise there would not remain any degrees of freedom for the least-squares fit. The components of $$\mathbf{a}$$ for small values of ℓ and k can be obtained from Table 13.1.

Equation (13.2.9) describes the moving average corresponding to the assumed polynomial (13.2.4). Once the vector $$\mathbf{a}$$ is determined, the moving averages

(13.2.10)

can easily be computed for each value of i.

Example 13.1:

Moving average with linear trend

In the case of a linear trend function,

the matrix A becomes simply

One then has

In this case the moving average is simply the unweighted mean (13.1.3). ■ 

For more complicated models one can obtain the vectors $$\mathbf{a}$$ either by solving (13.2.8) or simply from Table 13.1. Because of the symmetry of A, one can show that polynomials of odd order (i.e., ℓ = 2n with n an integer) have the same values of $$\mathbf{a}$$ as those of polynomials of the next lower order $$\ell= 2n - 1$$. One can also easily show that $$\mathbf{a}$$ has the symmetry

(13.2.11)

13.3 Edge Effects

Of course the moving average (13.2.10) can be used to estimate the trend only for points i that have at least k neighboring points both to the right and left, since the averaging interval covers 2k + 1 points. This means that for the first and last k points of a time series one must use a different estimator. One obtains the most obvious generalization of the estimator by extrapolating the polynomial (13.2.4) rather than using it only at the center of an interval. One then obtains the estimators

(13.3.1)

Here the notation $$\widetilde{{x}}^{(k+1)}$$ and $$\widetilde{{x}}^{(n-k)}$$ indicates that the coefficients $$\widetilde{x}$$ were determined for the first and last intervals of the time series for which the centers are at (k + 1) and (n − k).

The estimators are now defined even for i < 1 and i > n. They thus offer the possibility to continue the time series (e.g., into the future). Such extrapolations must be treated with great care for two reasons:

(i)

Usually there is no theoretical justification for the assumption that the trend is described by a polynomial. It merely simplifies the computation of the moving average. Without a theoretical understanding for a trend model, the meaning of extrapolations is quite unclear.

(ii)

Even in cases where the trend can rightly be described by a polynomial, the confidence limits quickly diverge from the estimated polynomial in the extrapolated region. The extrapolation becomes very inaccurate.

Whether point (i) is correct must be carefully checked in each individual case. The more general point (ii) is already familiar from the linear regression (cf. Fig. 12.1). We will investigate this in detail in the next section.

13.4 Confidence Intervals

We first consider the confidence interval for the moving average u i from Eq. (13.2.10). The errors of the measurements y i are unknown and must therefore first be estimated. From (12.​3.​2) one obtains for the sample variance of the y j in the interval of length 2k + 1,

(13.4.1)

where $$\widetilde{\eta }_{j}$$ is given by

(13.4.2)

The covariance matrix for the measurements can then be estimated by

(13.4.3)

The covariance matrix of the coefficients $$\mathbf{x}$$ is then given by (9.​2.​27),

(13.4.4)

Since $$u_{i} =\widetilde{\eta } _{0} =\widetilde{ x}_{1}$$, we thus have for an estimator of the variance of u i

(13.4.5)

From (13.2.6), (13.2.7), and (13.2.8) one easily obtains that $$({A}^{\mathrm{T}}A)_{11}^{-1} = a_{0}$$, since the middle row of A is $$-(1,0,0,\ldots,0)$$.

Using the same reasoning as in Sect. 12.​3 we obtain at a confidence level of 1 −α

(13.4.6)

For a given α we can give the confidence limits as

(13.4.7)

Here $$t_{1-\frac{1} {2} \alpha }$$ is a quantile of Student’s distribution for 2k − ℓ degrees of freedom. The true value of the trend lies within these limits with a confidence level of 1 −α.

Completely analogous, although more difficult computationally, is the determination of confidence limits at the ends of the time series. The moving average is now given by (13.3.1). Labeling the arguments in the expressions (13.3.1) $$j = i - k - 1$$ and $$j = i + k - n$$, we obtain

(13.4.8)

Here $$\mathbf{T}$$ is a row vector of length ℓ + 1,

(13.4.9)

According to the law of error propagation (3.​8.​4) we obtain

(13.4.10)

With (13.4.4) we finally have

(13.4.11)

where s y ² is again given by (13.4.1).

The quantity $$s_{\widetilde{\eta }}^{2}$$ can now be computed for every value of j, even for values lying outside of the time series itself. Thus we obtain the confidence limits

(13.4.12)

Caution is always recommended in interpreting the results of a time series analysis. This is particularly true for two reasons:

1.

There is usually no a priori justification for the mathematical model on which the time series analysis is based. One has simply chosen a convenient procedure in order to “separate out” statistical fluctuations.

2.

The user has considerable freedom in choosing the parameters k and ℓ, which, however, can have a significant influence on the results. The following example gives an impression of the magnitude of such influences.

Example 13.2:

Time series analysis of the same set of measurements using different averaging intervals and polynomials of different orders

Fig. 13.2:

Time series analyses of the same data with fixed ℓ and various values of k.

Fig. 13.3:

Time series analyses of the same data with fixed k and various values of ℓ.

Figures 13.2 and 13.3 contain time series analyses of the average number of sun spots observed in the 36 months from January 1962 through December 1964. Various values of k and ℓ were used. The individual plots in Fig. 13.2 show ℓ = 1 (linear averaging) but different interval lengths ($$2k + 1 = 5$$, 7, 9, and 11). One can see that the curve of moving averages becomes smoother and the confidence interval becomes narrower when k increases, but that then the mean deviation of the individual observations from the curve also increases. The extrapolation outside the range of measured points is, of course, a straight line. (For ℓ = 0 we would have obtained the same moving averages for the inner points. The outer and extrapolated points would lie, however, on a horizontal line, since a polynomial of order zero is a constant.) The plots in Fig. 13.3 correspond to the interval lengths $$2k + 1 = 7$$ and ℓ = 1, 2, 3, 4. The moving averages lie closer to the data points and the confidence interval becomes larger when the value of ℓ increases. ■ 

From these observations we can derive the following qualitative rules:

1.

The averaging interval should not be chosen larger than the region where one expects that the data can be well described by a polynomial of the given order. That is, for ℓ = 1 the interval 2k + 1 should be chosen such that the expected nonlinear effects within the interval remain small.

2.

On the other hand, the smoothing effect becomes stronger as the length of the averaging interval increases. As a rule of thumb, the smoothing becomes more effective for increasing $$2k + 1-\ell$$.

3.

Caution is required in extrapolation of a time series, especially in nonlinear cases.

The art of time series analysis is, of course, much more highly developed than what we have been able to describe in this short chapter. The interested reader is referred to the specialized literature, where, for example, smoothing functions other than polynomials or multidimensional analyses are treated.

13.5 Java Class and Example Programs

13.5.1 Java Class for Time Series Analysis

TimeSeries performs a time series analysis.

Example Program 13.1:

The class E1TimSer demonstrates the use of TimeSeriesThe uses the data of Example 13.2. After setting some parameters it performs a time series analysis by a call of TimeSeries with k = 2, ℓ = 2. The data, moving averages, and distances to the confidence limits (at a confidence level of 90 %) are output numerically.

Example Program 13.2:

The class E2TimSer performs a time series analysis and yields graphical outputThe program starts works on the same data as E1TimSer. It allows interactive input for the parameters k and ℓ and for the confidence level P and then performs a time series analysis. Subsequently a plot is produced in which the data are displayed as small circles. The floating averages are shown as a polyline. Polylines in a different color indicate the confidence limits.Suggestion: Produce the individual plots of Figs. 13.2 and 13.3.

Matrix Calculations

The solution of the over-determined linear system of equations A x  ≈  b is of central significance in data analysis. This can be solved in an optimal way with the singular value decomposition, first developed in the late 1960s. In this appendix the elementary definitions and calculation rules for matrices and vectors are summarized in Sects.  A.1 and  A.2 . In Sect.  A.3 orthogonal transformations are introduced, in particular the Givens and Householder transformations, which provide the key to the singular value decomposition.

After a few remarks on determinants (Sect.  A.4 ) there follows in Sect.  A.5 a discussion of various cases of matrix equations and a theorem on the orthogonal decomposition of an arbitrary matrix, which is of central importance in this regard. The classical procedure of normal equations, which, however, is inferior to the singular value decomposition, is described here.

Sections  A.6 – A.8 concern the particularly simple case of exactly determined, non-singular matrix equations. In this case the inverse matrix A ⁻¹ exists, and the solution to the problem A x  =  b is $$\mathbf{x} = {A}^{-1}\mathbf{b}$$ . Methods and programs for finding the solution are given. The important special case of a positive-definite symmetric matrix is treated in Sect.  A.9 .

In Sect.  A.10 we define the pseudo-inverse matrix A ⁺ of an arbitrary matrix A . After introducing eigenvectors and eigenvalues in Sect.  A.11 , the singular value decomposition is presented in Sects.  A.12 and  A.13 . Computer routines are given in Sect.  A.14 . Modifications of the procedure and the consideration of constraints are the subject of Sects.  A.15 through  A.18 .

It has been attempted to make the presentation illustrative rather than mathematically rigorous. Proofs in the text are only indicated in a general way, or are omitted entirely. As mentioned, the singular value decomposition is not yet widespread. An important goal of this appendix is to make its use possible. For readers with a basic knowledge of matrix calculations, the material covered in Sects.  A.3 , A.12 , A.13 , A.14.1 , and A.18 is sufficient for this task. Sections  A.14.2 through A.14.5 contain technical details on carrying out the singular value decomposition and can be omitted by hurried users.

All procedures described in this Appendix are implemented as methods of the classes DatanVector or DatanMatrix , respectively. Only in a few cases will we refer to these methods explicitly, in oder to establish the connection with some of the more complicated algorithms in the text.

Definitions: Simple Operations

By a vector in m dimensions (an m -vector) a we mean an m -tuple of real numbers, which are the components of a . The arrangement of the components in the form

(A.1.1)

is called a column vector .

A m × n matrix is a rectangular arrangement of m × n numbers in m rows and n columns,

(A.1.2)

It can be viewed to be composed of n column vectors. By transposition of the m × n matrix A one obtains an n × m matrix A T with elements

(A.1.3)

Under transposition a column vector becomes a row vector ,

(A.1.4)

A column vector is an m × 1 matrix; a row vector is a 1 × m matrix.

For matrices one has the following elementary rules for addition , subtraction and multiplication by a constant

(A.1.5)

(A.1.6)

The product !by a |anhi AB of two matrices is only defined if the number of columns of the first matrix is equal to the number of rows of the second, e.g., A  =  A m×ℓ and B  =  B ℓ×m . One has then

(A.1.7)

Since

one has

(A.1.8)

With ( A.1.7 ) one can also define the product of a row vector a T with a column vector b , if both have the same number of elements m ,

(A.1.9)

The result is then a number, i.e., a scalar . The product ( A.1.9 ) is called the scalar product . It is usually written without indicating the transposition simply as a ⋅  b . The vectors a and b are orthogonal to each other if their scalar product vanishes. Starting from ( A.1.9 ) one obtains the following useful property of the matrix product ( A.1.7 ). The element C ik , which is located at the intersection of the i th row and the k th column of the product matrix C , is equal to the scalar product of the i th row of the first matrix A with the k th column of the second matrix B .

The diagonal elements of a matrix ( A.1.2 ) are the elements A ii . They form the main diagonal of the matrix A . If all of the non-diagonal elements vanish, A ij  = 0,  i ≠ j , then A is a diagonal matrix . An n × n diagonal matrix all of whose diagonal elements are unity is the n -dimensional unit matrix I n  =  I ,

(A.1.10)

The null matrix has only zeros as elements:

(A.1.11)

A null matrix with only one column is the null vector 0 .

We will now mention several more special types of square matrices.

A square matrix is symmetric if

(A.1.12)

If

(A.1.13)

then the matrix is antisymmetric .

A bidiagonal matrix B possesses non-vanishing elements only on the main diagonal ( b ii ) and on the parallel diagonal directly above it ( b i, i+1 ).

A tridiagonal matrix possesses in a addition non-vanishing elements directly below the main diagonal. A lower triangular matrix has non-vanishing elements only on and below the main diagonal, an upper triangular matrix only on and above the main diagonal.

The Euclidian norm or the absolute value of a vector is

(A.1.14)

A vector with unit norm is called a unit vector . We write this in the form

More general vector norms are

(A.1.15)

For every vector norm $$\|\mathbf{x}\|$$ one defines a matrix norm $$\|A\|$$ as

(A.1.16)

Matrix norms have the following properties:

(A.1.17)

(A.1.18)

(A.1.19)

(A.1.20)

Vector Space, Subspace, Rank of a Matrix

An n-dimensional vector space is the set of all n -dimensional vectors. If u and v are vectors in this space, then α u and u + v are also in the space, i.e., the vector space is closed under vector addition and under multiplication with a scalar α . The vectors a ₁ ,  a ₂ ,  … ,  a k are linearly independent if

(A.2.1)

for all α j except for $$\alpha _{1} =\alpha _{2} = \cdots =\alpha _{k} = 0$$ . Otherwise they are linearly dependent . The maximum number k max of vectors that can be linearly independent is equal to the dimension of the vector space n . An arbitrary set of n linearly independent vectors a ₁ ,  a ₂ ,  … ,  a n forms a basis of the vector space. Any vector a can be expressed as a linear combination of the basis vectors,

(A.2.2)

A special basis is

(A.2.3)

These basis vectors are orthonormal , i.e.,

(A.2.4)

The component a j of the vector a is the scalar product of a with the basis vector e j ,

(A.2.5)

cf. ( A.1.1 ) and ( A.1.9 ).

For n  ≤ 3 vectors can be visualized geometrically. A vector a can be represented as an arrow of length a . The basis vectors ( A.2.3 ) are perpendicular to each other and are of unit length. The perpendicular projections of a onto the directions of the basis vectors are the components ( A.2.5 ), as shown in Fig.  A.1 .

Fig. A.1:

The vector a in the system of orthonormal basis vectors e ₁ , e ₂ .

A subset T of a vector space S is called a subspace if it is itself closed under vector addition and multiplication with a scalar. The greatest possible number of linearly independent vectors in T is the dimension of T . The product of an m × n matrix A with an n -vector a is an m -vector b ,

(A.2.6)

The relation ( A.2.6 ) can be regarded as a mapping or transformation of the vector a onto the vector b .

The span of a set of vectors a ₁ ,  … ,  a k is the vector space defined by the set of all linear combinations u of these vectors,

(A.2.7)

It has a dimension m  ≤  k . The column space of an m × n matrix A is the span of the n column vectors of A ; in this case the m -vectors u have the form u  =  A x with arbitrary n -vectors x . Clearly the dimension of the column space is ≤ min( m ,  n ). Similarly, the row space of A is the span of the m row vectors.

The null space or kernel of A consists of the set of vectors x for which

(A.2.8)

Column and row spaces of an m × n matrix have the same dimension. This is called the rank of the matrix. An m × n has full rank if

(A.2.9)

Otherwise it has reduced rank . An n × n matrix with Rang( A ) <  n is said to be singular .

A vector a is orthogonal to a subspace T if it is orthogonal to every vector t  ∈  T . (A trivial example is $$\mathbf{t} = t_{1}\mathbf{e}_{1} + t_{2}\mathbf{e}_{2}$$ , a  =  a e ₃ , a ⋅  t  = 0.) A subspace U is orthogonal to the subspace T if for every pair of vectors u  ∈  U , t  ∈  T one has u ⋅  t  = 0. The set of all vectors u + t forms a vector space V , called the direct sum of T and U ,

(A.2.10)

Its dimension is

(A.2.11)

If ( A.2.10 ) holds, then T and U are subspaces of V . They are called orthogonal complements , T  =  U  ⊥  , U  =  T  ⊥  . If T is a subspace of S , then there always exists an orthogonal complement T  ⊥  , such that S  =  T ⊕ T  ⊥  . Every vector a  ∈  S can then be uniquely decomposed into the form $$\mathbf{a} = \mathbf{t} + \mathbf{u}$$ with t  ∈  T and u  ∈  T  ⊥  . For the norms of the vectors the relation $${a}^{2} = {t}^{2} + {u}^{2}$$ holds.

If T is an ( n − 1)-dimensional subspace of an n -dimensional vector space S and if s is a fixed vector in S , then the set of all vectors $$\mathbf{h} = \mathbf{s} + \mathbf{t}$$ with t  ∈  T forms an ( n − 1)-dimensional hyperplane H in S . If H is given and h ₀ is an arbitrary fixed vector in H , then T is the set of all vectors $$\mathbf{t} = \mathbf{h} -\mathbf{h}_{0}$$ , h  ∈  H , as shown in Fig.  A.2 . If $$\widehat{\mathbf{u}}$$ is a unit vector in the one-dimensional subspace T  =  H  ⊥  orthogonal to H , then the scalar product

(A.2.12)

has the same value for all h  ∈  H , where d is the distance of the hyperplane from the origin (see Fig.  A.3 ).

Fig. A.2:

Hyperplane H in a two-dimensional vector space.

Fig. A.3:

Hyperplane H and complementary one-dimensional vector space T .

For a given $$\widehat{\mathbf{u}}$$ and d , Eq. ( A.2.12 ) defines a hyperplane H . It divides the n -dimensional vector space into two half spaces , which consist of the set of vectors x for which $$\widehat{\mathbf{u}} \cdot \mathbf{x} < 0$$ and $$\widehat{\mathbf{u}} \cdot \mathbf{x} > 0$$ .

Orthogonal Transformations

According to ( A.2.6 ), the mapping of an n -vector a onto an n -vector b is performed by multiplication by a square n × n matrix, b  =  Q a . If the length of the vector ( A.1.14 ) remains unchanged, one speaks of an orthogonal transformation . For such a case one has b  =  a or b ²  =  a ² , i.e.,

and thus

(A.3.1)

A square matrix Q that fulfills ( A.3.1 ) is said to be orthogonal .

It is clear that transformations are orthogonal when the transformed vector b is obtained from a by means of a spatial rotation and/or reflection. We will examine some orthogonal transformations that are important for the applications of this appendix.

Givens Transformation

The Givens rotation is a transformation that affects only components in a plane spanned by two orthogonal basis vectors. For simplicity we will first consider only two-dimensional vectors.

Fig. A.4:

Application of the Givens transformation ( a ) to the vector v that defines the transformation and ( b ) to an arbitrary vector u .

A vector v can be represented as

(A.3.2)

A rotation by an angle $$-\vartheta$$ transforms the vector v into the vector $$\mathbf{v}^{\prime} = G\mathbf{v}$$ , whose second component vanishes, as in Fig.  A.4 a. Clearly one has

(A.3.3)

Of course the transformation thus defined with the vector v can also be applied to any other vector u , as shown in Fig.  A.4 b. In n dimensions the Givens rotation leads to the transformation

such that v ′ k  = 0. The components v ′ ℓ  =  v ℓ , ℓ ≠ k , ℓ ≠ i , remain unchanged and v ′ i is determined such that the norm of the vector remains unchanged, v ′ =  v . This is clearly given by

(A.3.4)

In practical applications, however, the full matrix is not needed. The method DatanMatrix . defineGivensTransformation defines a Givens transformation by the input of the two components v ₁ ,  v ₂ , DatanMatrix . applyGivensTransformation applies that transformation to two components of another vector. The method DatanMatrix .defineAndApplyGivensTransformation defines a transformation and directly applies it to the defining vector.

Householder Transformation

The Givens rotation is used to transform a vector in such a way that a given vector component vanishes. A more general transformation is the Householder transformation . If the original vector is

(A.3.5)

then for the transformed vector we want to have

(A.3.6)

That is, the components v ℓ ′, v ′ ℓ+1 , … , v n ′ should vanish. The remaining components should (with the exception of v p ′) remain unchanged. The component v p ′ must be changed in such a way that one has v  =  v ′. From the Pythagorean theorem in $$n -\ell +1$$ dimensions one has

or

(A.3.7)

with σ  = ±1. We choose

(A.3.8)

We now construct the matrix H of ( A.3.6 ). To do this we decompose the vector v into a sum,

(A.3.9)

Here the vector v H is in the subspace spanned by the basis vectors e p , e ℓ , e ℓ+1 , … , e n , and $$\mathbf{v}_{{H}^{\perp }}$$ is in the subspace orthogonal to it. We now construct

(A.3.10)

and

(A.3.11)

If we now decompose an arbitrary vector a into a sum of vectors parallel and perpendicular to u ,

with

then one has

Thus we see that the transformation is a reflection in the subspace that is orthogonal to the vector u , as in Fig.  A.5 . One can easily verify that H in fact yields the transformation ( A.3.6 ).

Fig. A.5:

The vector v is mapped onto $$\mathbf{v}^{\prime}$$ according to a Householder transformation such that v ₂ ′ = 0. The mapping corresponds to a reflection in the subspace U that is orthogonal to the auxiliary vector u .

The transformation is uniquely determined by the vector u . According to ( A.3.10 ) one has for the components of this vector $$u_{p} = v_{p} +\sigma v_{H}$$ , u ℓ  =  v ℓ , $$u_{\ell+1} = v_{\ell+1}$$ , … , u n  =  v n , and u i  = 0 for all other i . If the vector v and the indices p and ℓ are given, then only u p must be computed. The quantity u ² appearing in ( A.3.11 ) is then

We can thus write ( A.3.11 ) in the form

(A.3.12)

The matrix H , however, is not needed explicitly to compute a transformed vector,

It is sufficient to know the vector u and the constant b . Since, however, u only differs from v in the element u p (and in the vanishing elements), it is sufficient, starting from v , to first compute the quantities u p and b and when applying the transformation to use in addition the elements v ℓ , v ℓ+1 , … , v n . These are at the same time the corresponding elements of u .

By the method DatanMatrix . defineHouseholderTransformation a transformation is defined; with DatanMatrix . applyHouseholderTransformation it is applied to a vector.

Sign Inversion

If the diagonal element I ii of the unit matrix is replaced by − 1, then one obtains a symmetric orthogonal matrix,

Applying this to the vector a ,

changes the sign of the element a i and leaves all the other elements unchanged. Clearly R (i) is a Householder matrix which produces a reflection in the subspace orthogonal to the basis vector e i . This can be seen immediately by substituting u  =  e i in ( A.3.11 ).

Permutation Transformation

The n × n unit matrix ( A.1.10 ) can easily be written as an arrangement of the basis vectors ( A.2.3 ) in a square matrix,

It is clearly an orthogonal matrix. The orthogonal transformation I a  =  a leaves the vector a unchanged. If we now exchange two of the basis vectors e i and e k , then we obtain the symmetric orthogonal matrix P ik . As an example we show this for n  = 4, i  = 2, k  = 4,

The transformation $$\mathbf{a}^{\prime} = {P}^{(ik)}\mathbf{a}$$ leads to an exchange of the elements a i and a k . All other elements remain unchanged:

Multiplication of an n × m matrix A on the left with P (ik) permutes the lines i and k of A . Multiplication of an m × n matrix A from the right exchanges the columns i and k . If D is an n × n diagonal matrix, then the elements D ii and D kk are exchanged by the operation

Determinants

To every n × n matrix A one can associate a number, its determinant , det A . A determinant, like the corresponding matrix, is written as a square arrangement of the matrix elements, but is enclosed, however, by vertical lines. The determinants of orders two and three are defined by

(A.4.1)

and

(A.4.2)

or, written in another way,

(A.4.3)

A general determinant of order n is written in the form

(A.4.4)

The cofactor A ij † of the element A ij of a matrix is a determinant of order ( n − 1), calculated from the matrix obtained by deleting the i th row and j th column of the original matrix, and multiplying by $${(-1)}^{i+j}$$ ,

(A.4.5)

Determinants of higher order can be written as the sum of all elements of any row or column multiplied with the corresponding cofactors,

(A.4.6)

One can easily show that the result is independent of the choice of row i or column j . Equation ( A.4.3 ) already shows that ( A.4.6 ) is correct for n  = 3. Determinants of arbitrary order can be computed by decomposing them according to their cofactors until one reaches, for example, the order two. A singular matrix , i.e., a square matrix whose rows or columns are not linearly independent, has determinant zero .

From A we can construct a further matrix by replacing each element ij by the cofactor of the element ji . In this way we obtain the adjoint matrix of  A ,

(A.4.7)

For determinants the following rules hold:

(A.4.8)

(A.4.9)

For an orthogonal matrix Q one has Q   Q T  =  I , i.e.,

and thus

(A.4.10)

Matrix Equations: Least Squares

A system of m linear equations with n unknowns x ₁ , x ₂ , … , x n has the general form

(A.5.1)

or in matrix notation

(A.5.2)

In finding the solution to this equation x we must distinguish between various cases, which can be characterized by the values of m , n , and

The vector A x is in the column space of A , which is of dimension k . Since b is an m -vector, the equation can in general only be fulfilled if k  =  m , i.e., for $$k = n = m$$ and for k  =  m  <  n , since k  ≤ min( m ,  n ). For $$k = n = m$$ one has n independent equations ( A.5.1 ) with n unknowns, which have a unique solution. If k  =  m  <  n , then there are arbitrarily many n -vectors x that can be mapped on to the m -vector A x  =  b such that ( A.5.2 ) is fulfilled. The system of equations is underdetermined . The solution is not unique.

For k  = Rang( A ) ≠ m there is, in general, no solution of ( A.5.2 ). In this case we look for a vector $$\widetilde{\mathbf{x}}$$ , for which the left-hand side of ( A.5.2 ) is a vector of minimum Euclidian norm. That is, we replace Eq. ( A.5.2 ) by

(A.5.3)

i.e., we look for a vector $$\widetilde{\mathbf{x}}$$ for which the mapping $$A\widetilde{\mathbf{x}}$$ differs as little as possible from b .

Given a m -vector c , only for $$k =\mathop{ \mathrm{Rang}}\nolimits (A) = n$$ does there exist only one n -vector x , such that A x  =  c . Therefore, only for the case k  =  n is there a unique solution $$\widetilde{\mathbf{x}}$$ . Thus there exists for Rang( A ) =  n and m  ≥  n a unique solution $$\widetilde{\mathbf{x}}$$ of ( A.5.3 ). For n  =  m one has r  = 0.

The relation ( A.5.3 ) is also often written simply in the form

(A.5.4)

or even, not entirely correctly, in the form ( A.5.2 ). One calls the solution vector $$\widetilde{\mathbf{x}}$$ the least-squares solution of ( A.5.4 ). In Table  A.1 we list again the various cases that result from different relationships between m , n , and k .

Table A.1:

Behavior of the solutions of ( A.5.3 ) for various cases. m is the row number, n is the column number and k is the rank of the matrix A .

Case

Rang( A )

Residual

Solution unique

1a

m  =  n

k  =  n

r  = 0

Yes

1b

k  <  n

r  ≥ 0

No

2a

m  >  n

k  =  n

r  ≥ 0

Yes

2b

k  <  n

r  ≥ 0

No

3a

m  <  n

k  =  m

r  = 0

No

3b

k  <  m

r  ≥ 0

No

We now want to state more formally what we have determined in this section with respect to the solution of ( A.5.4 ).

Theorem on the orthogonal decomposition of a matrix: Every m × n matrix A of rank k can be written in the form

(A.5.5)

where H is an m × m orthogonal matrix, K is an n × n orthogonal matrix, and R is an m × n matrix of the form

(A.5.6)

and where R ₁₁ is a k × k matrix of rank k .

Substituting ( A.5.5 ) into ( A.5.4 ) and multiplying from the left by H T leads to

(A.5.7)

We define

(A.5.8)

and

(A.5.9)

so that ( A.5.7 ) takes on the form

(A.5.10)

Because of ( A.5.6 ), this breaks into two independent relations

(A.5.11)

and

(A.5.12)

If m  =  k and/or n  =  k , then the corresponding lower partial vectors are absent in ( A.5.8 ) and/or ( A.5.9 ) and the corresponding lower matrices are absent in ( A.5.6 ). Since in ( A.5.11 ) R ₁₁ is a k × k matrix of rank k , and p ₁ and g ₁ are k -vectors, there exists a solution vector $$\widetilde{\mathbf{p}}_{1}$$ for which equality holds. Because of the null matrix on the left-hand side of ( A.5.12 ), we cannot derive any information about p ₂ from this relation.

Theorem on the solutions of Ax ≈ b : If $$\widetilde{\mathbf{p}}_{1}$$ is the unique solution vector of ( A.5.11 ), then the following statements hold:

(i)

All solutions of ( A.5.3 ) have the form

(A.5.13)

Here $$\widetilde{\mathbf{p}}_{2}$$ is an arbitrary ( n − k )-vector, i.e., the solution is unique for k  =  n . There is always, however, a unique solution of minimum absolute value :

(A.5.14)

(ii)

All solutions $$\widehat{\mathbf{x}}$$ have the same residual vector

(A.5.15)

with the absolute value $$r = g_{2} = \vert \mathbf{g}_{2}\vert $$ . The residual vanishes for k  =  m .

The problem A x  ≈  b should always be handled with orthogonal decompositions, preferably with the singular value decomposition and singular value analysis described in Sects.  A.12 and  A.13 . Numerically the results are at least as accurate as with other methods, and are often more accurate (cf. Sect.  A.13 , Example A.4).

Nevertheless we will briefly present as well the method of normal equations . The method is very transparent compared to the orthogonal decomposition and is therefore always described in textbooks.

We consider the square ( A.5.3 ) of the residual vector

The requirement r ²  = min leads to

These n linear equations are called normal equations. They can be arranged in a matrix,

(A.5.16)

This is a system of n equations with n unknowns. If the equations are linearly independent, then the n × n matrix ( A T A ) is of full rank n ; it is not singular. According to Sect.  A.6 there then exists an inverse ( A T A ) ⁻¹ , such that $${({A}^{\mathrm{T}}A)}^{-1}({A}^{\mathrm{T}}A) = I$$ . Thus the desired solution to ( A.5.3 ) is

(A.5.17)

This simple prescription is, however, useless if A T A is singular or nearly singular (cf. Example A.4).

Inverse Matrix

For every non-singular n × n matrix A , the inverse matrix A ⁻¹ is defined by

(A.6.1)

It is also an n × n matrix.

If A ⁻¹ is known, then the solution of the matrix equation

(A.6.2)

is given simply by

(A.6.3)

As we will show, A ⁻¹ only exists for non-singular square matrices, so that ( A.6.3 ) only gives the solution for the case 1a of Table  A.1 .

In order to determine A ⁻¹ , we set $${A}^{-1} = X$$ and express the column vectors of X as x ₁ , x ₂ , … , x n . Equation ( A.6.1 ) is then decomposed into n equations:

(A.6.4)

The right-hand sides are the basis vectors ( A.2.3 ). For the case n  = 2 we can write the system ( A.6.4 ) in various equivalent ways, e.g.,

(A.6.5)

or alternatively

or as a system of four equations with four unknowns,

(A.6.6)

By elimination and substitution one easily finds

(A.6.7)

or in matrix notation

(A.6.8)

The matrix on the right-hand side is the adjoint of the original matrix, i.e.,

(A.6.9)

One can show that this relation holds for square matrices of arbitrary order. From ( A.6.9 ) it is clear that the inverse of a singular matrix, i.e., of a matrix with vanishing determinant, is not determined.

In practice the inverse matrix is not computed using ( A.6.9 ) but rather as in our example of the 2 × 2 matrix by elimination and substitution from the system ( A.6.4 ). This system consists of n sets of equations of the form

(A.6.10)

each consisting of n equations with n unknowns. Here A is a non-singular n × n matrix. We will first give the solution algorithm for square non-singular matrices A , but we will later remove these restrictions.

Gaussian Elimination

We will write Eq. ( A.6.10 ) for an n × n matrix A in components,

(A.7.1)

and we will solve the system by Gaussian elimination . For this we define n − 1 multipliers

(A.7.2)

multiply the first equation by m ₂₁ , and subtract it from the second. We then multiply the first equation by m ₃₁ and subtract it from the third, and so forth. We obtain the system

(A.7.3)

where the unknown x ₁ has disappeared from all the equations except the first. The coefficients A ij ⁽²⁾ , b i ⁽²⁾ are given by the equations

The procedure is then repeated with the last n − 1 equations by defining

multiplying the second equation of the system ( A.7.3 ) with the corresponding m i2 , and then subtracting it from the third, fourth, … , n th equation. In the k th step of the procedure, the multipliers

(A.7.4)

are used and the new coefficients

(A.7.5)

are computed. After n − 1 steps we have produced the following triangular system of equations :

(A.7.6)

The last equation contains only x n . By substituting into the next higher equation one obtains x n−1 , i.e., in general

(A.7.7)

One should note that the change of the elements of the matrix A does not depend on the right-hand side b . Therefore one can reduce several systems of equations with different right-hand sides simultaneously. That is, instead of A x  =  b , one can solve the more general system

(A.7.8)

at the same time, where B and X are n × m matrices, the matrix X being unknown.

Example A.1:

Inversion of a 3 × 3 matrix

As a numerical example let us consider the inversion of a 3 × 3 matrix, i.e., B  =  I . The individual computations for the example

are shown in Table  A.2 . The result is

 ■ 

Table A.2:

Application of Gaussian elimination to Example A.1.

Following the individual steps of the calculation one sees that division is carried out in two places, namely in Eqs. ( A.7.4 ) and ( A.7.7 ). The denominator is in both cases a coefficient

i.e., the upper left-hand coefficient of the system, the so-called pivot for the step i − 1 of the reduction process. Our procedure fails if this coefficient is equal to zero. In such a case one can simply exchange the i th line of the system with some other lower line whose first coefficient is not zero. The system of equations itself is clearly not changed by exchanging two equations. The procedure still fails if all of the coefficients of a column vanish. In this case the matrix A is singular, and there is no solution. In practice (at least when using computers where the extra work is negligible) it is advantageous to always carry out a reshuffling so that the pivot is the coefficient with the largest absolute value among those in the first column of the reduced system. One then always has the largest possible denominator. In this way rounding errors are kept as small as possible. The procedure just described is called Gaussian elimination with pivoting .

Following it, the method DatanMatrix . matrixEquation solves Eq. ( A.7.8 ). In the same way the method DatanMatrix .inverse determines the inverse of a square nonsingular matrix.

LR-Decomposition

For the elements A ij (k) transformed by Gaussian elimination [cf. ( A.7.6 )], one has on and above the main diagonal

(A.8.1)

and below the main diagonal

(A.8.2)

The transformation ( A.7.5 ) is thus only carried out for k  = 1, 2,  … ,  r with $$r =\min (i - 1,\,j)$$ ,

(A.8.3)

Summation over k gives

Thus the elements $$A_{ij}^{(1)} = A_{ij}$$ of the original matrix A can be written in the form

(A.8.4)

Noting that the multipliers m ik have only been defined up to now for i  >  k , we can in addition set

and reduce the two relations ( A.8.4 ) to one,

The equation shows directly that the matrix A can be represented as the product of two matrices L and R . Here L is a lower and R an upper triangular matrix,

(A.8.5)

The original system of equations ( A.6.10 )

is thus equivalent to two triangular systems of equations,

Instead of resorting to the formulas from Sect.  A.7 , we can compute the elements of L and R directly from ( A.8.5 ). We obtain

(A.8.6)

i.e., for k  = 1 one computes the first row of R (which is equal to the first row of A ) and the first column of L ; for k  = 2 one computes the second row of R and the second column of L . In a computer program the elements of R and L can overwrite the original matrix A with the same indices, since when computing r kj one only needs the element A kj and other elements of R and L that have already been computed according to ( A.8.6 ). The corresponding consideration holds for the computation of m ik .

The algorithm ( A.8.6 ) is called the Doolittle LR-decomposition . By including pivoting this is clearly equivalent to Gaussian elimination.

Cholesky Decomposition

If A is a real symmetric positive-definite matrix (cf. Sect.  A.11 ), then it can be uniquely expressed as

(A.9.1)

Here U is a real upper triangular matrix with positive diagonal elements. The n × n matrix A has the required property for the validity of ( A.9.1 ), in particular in those cases where it is equal to the product of an arbitrary real n × m matrix B (with m  ≥  n and full rank) with its transpose B T,A = B BT.

To determine U we first carry out the Doolittle LR-decomposition, define a diagonal matrix D whose diagonal elements are equal to those of R ,

(A.9.2)

and introduce the matrices

(A.9.3)

and

(A.9.4)

We can then write

(A.9.5)

Because of the assumed symmetry one has

(A.9.6)

Comparing with ( A.9.5 ) gives L T  =  R ′, i.e.,

With

(A.9.7)

Eq. ( A.9.1 ) is indeed fulfilled. The relation ( A.9.7 ) means for the elements u ij of U

Thus the values r and m can be eliminated in favor of u from ( A.8.6 ), and we obtain with

the algorithm for the Cholesky decomposition of a real positive-definite symmetric matrix A . Since for such matrices all of the diagonal elements are different from zero, one does not need in principle any pivoting. One can also show that it would bring no advantage in numerical accuracy. The method DatanMatrix .choleskyDecomposition performs the Cholesky decomposition of a symmetric positive definite square matrix.

Positive-definite symmetric matrices play an important role as weight and covariance matrices. In such cases one often requires the Cholesky decomposition A  =  U T U as well as multiplication of U by a matrix. It is easier computationally if this multiplication is done by the method DatanMatrix .choleskyMultiply, which takes the triangular form of U into account.

Of particular interest is the inversion of a symmetric positive-definite n × n matrix A . For this we will solve the n matrix equations ( A.6.4 ) with the previously described Cholesky decomposition of A ,

(A.9.8)

We denote the ℓ th component of the three vectors x i , y i , and e i by x i ℓ , y i ℓ , and e i ℓ . Clearly one has $$x_{i\ell} = ({A}^{-1})_{i\ell}$$ and e i ℓ  =  δ i ℓ , since e i ℓ is the element ( i ,  ℓ ) of the unit matrix.

We now determine y i ℓ by means of forward substitution . From ( A.9.8 ) it follows directly that

i.e.,

and thus

Since, however, δ i ℓ  = 0 for i ≠ ℓ , this expression simplifies to

We can now obtain x i ℓ by means of backward substitution of y i ℓ into

or, written in components,

or

We obtain

If we compute x i ℓ only for ℓ  ≥  i , then by backward substitution one only encounters the elements y i ℓ for ℓ  ≥  i , i.e., only non-vanishing y i ℓ . The vanishing elements y i ℓ thus do not need to be stored. The elements x i ℓ for ℓ  <  i follow simply from the symmetry of the original matrix, x i ℓ  =  x ℓ i .

The method DatanMatrix .choleskyInversion first performs the Cholesky decomposition of A . Then a loop is carried out over the various right-hand sides e i , i  = 1,  … ,  n , of ( A.9.8 ). As the result of running through the loop once one obtains the elements x in , x i, n−1 , … , x ii of row i . They are stored as the corresponding elements of the output matrix A . The elements of the vector y , which is only used for intermediate results, can be stored in the last row of A . Finally the elements below the main diagonal are filled by copies of their mirror images.

Pseudo-inverse Matrix

We now return to the problem of Sect.  A.5 , that of solving the Eq. ( A.5.4 )

(A.10.1)

for an arbitrary m × n matrix A of rank k . According to ( A.5.14 ), the unique solution of minimum norm is

The vector p ₁ is the solution of Eq. ( A.5.11 ) and therefore

since R ₁₁ is non-singular. Because of ( A.5.8 ) one has finally

(A.10.2)

In analogy to ( A.6.3 ) we write

(A.10.3)

and call the n × m matrix

(A.10.4)

the pseudo-inverse of the m × n matrix A .

The matrix A ⁺ is uniquely determined by A and does not depend on the particular orthogonal decomposition ( A.5.5 ). This can easily be seen if one denotes the j th column vector of A ⁺ by $$\mathbf{a}_{j}^{+} = {A}^{+}\mathbf{e}_{j}$$ , with e j the j th column vector of the m -dimensional unit matrix. According to ( A.10.3 ) the vector a j ⁺ is the minimum-length solution of the equation $$A\mathbf{a}_{j}^{+} = \mathbf{e}_{j}$$ , and is therefore unique.

Eigenvalues and Eigenvectors

We now consider the eigenvalue equation

(A.11.1)

for the n × n matrix G . If this is fulfilled, then the scalar λ is called the eigenvalue and the n -vector x the eigenvector of G . Clearly the eigenvector x is only determined up to an arbitrary factor. One can choose this factor such that |  x  |  = 1.

We consider first the particularly simple case where G is a diagonal matrix with non-negative diagonal elements,

(A.11.2)

which can be expressed as the product of an arbitrary diagonal matrix S with itself,

(A.11.3)

The eigenvalue equation S ² x  =  λ x then has the eigenvalues $$s_{i}^{2} =\lambda _{i}$$ and the normalized eigenvectors are the basis vectors x i  =  e i .

In place of S we now set

(A.11.4)

with orthogonal matrices U and V and

(A.11.5)

We can write the eigenvalue equation of G in the form

(A.11.6)

or

Multiplying on the left with V T ,

and comparing with ( A.11.1 ) and ( A.11.2 ) shows that G has the same eigenvalues $$\lambda _{i} = s_{i}^{2}$$ as S ² , but has the orthogonally transformed eigenvectors

(A.11.7)

One can clearly find the eigenvalues and eigenvectors of G if one knows the orthogonal matrix V that transforms G into a diagonal matrix,

(A.11.8)

The transformation ( A.11.8 ) is called a principal axis transformation . The name becomes clear by considering the equation

(A.11.9)

We are interested in the geometrical position of all points r that fulfill ( A.11.9 ).

For the vector r the following two representations are completely equivalent:

(A.11.10)

and

(A.11.11)

with the components r i and r ′ i taken with respect to the basis vectors e i and $$\mathbf{e}^{\prime}_{i}$$ , respectively. With the representation ( A.11.11 ) and by using ( A.11.5 ) and ( A.11.7 ) we obtain

and finally

(A.11.12)

This is clearly the equation of an ellipsoid in n dimensions with half-diameters in the directions $$\mathbf{e}^{\prime}_{i}$$ and having the lengths

(A.11.13)

The vectors

(A.11.14)

are the principal axes of the ellipsoid. They have the directions of the eigenvectors of G . Their lengths $$a_{i} = 1/\sqrt{s_{i }^{2}}$$ are determined by the eigenvalues  s i ² .

The matrix

clearly has the same eigenvectors as G , but has the eigenvalues 1∕ s i ² . The lengths of the half-diameters of the ellipsoid described above are then directly equal to the square roots of the eigenvalues of C .

The matrix C is called the unweighted covariance matrix of A . The ellipsoid is called the unweighted covariance ellipsoid .

Please note that all considerations were done for matrices of the type ( A.11.5 ) which, by construction, are symmetric and non-negative definite , i.e., they have real, non-negative eigenvalues. Ellipses with finite semiaxes are obtained only with positive-definite matrices. If the same eigenvalue occurs several times then the ellipsoid has several principal axes of the same length. In this case there is a certain ambiguity in the determination of the principal axes which, however, can always be chosen orthogonal to each other.

Up to now we have not given any prescription for finding the eigenvalues. The eigenvalue equation is G x  =  λ x or

(A.11.15)

Written in this way it can be considered as a linear system of equations for determining x , for which the right-hand side is the null vector. Corresponding to ( A.6.5 ) and ( A.6.9 ) there is a non-trivial solution only for

(A.11.16)

This is the characteristic equation for determining the eigenvalues of A . For n  = 2 this is

and has the solutions

If G is symmetric, as previously assumed, i.e., g ₁₂  =  g ₂₁ , then the eigenvalues are real. If G is positive definite, they are positive.

The characteristic equation of an n × n matrix has n solutions. In practice, however, for n  > 2 one does not find the eigenvalues by using the characteristic equation, but rather by means of an iterative procedure such as the singular value decomposition (see Sect.  A.12 ).

Singular Value Decomposition

We now consider a particular orthogonal decomposition of the m × n matrix  A ,

(A.12.1)

Here and in Sects.  A.13 and A.14 we assume that m  ≥  n . If this is not the case, then one can simply extend the matrix A with further rows whose elements are all zero until it has n rows, so that m  =  n . The decomposition ( A.12.1 ) is a special case of the decomposition ( A.5.5 ). The m × n matrix S , which takes the place of R , has the special form

(A.12.2)

and D is a k × k diagonal matrix with k  = Rang( A ). The diagonal elements of S are called the singular values of A . If A is of full rank, then k  =  n and all s i ≠ 0. For reduced rank k  <  n one has s i  = 0 for i  >  k . We will see below that U and V can be determined such that all s i are positive and ordered to be non-increasing,

(A.12.3)

The singular values of A have a very simple meaning. From Sect.  A.11 one has directly that the s i are the square roots of the eigenvalues of G  =  A T A . Thus the half-diameters of the covariance ellipsoid of G are $$a_{i} = 1/s_{i}$$ . If G is singular, then A has the reduced rank k  <  n , and the n − k singular values s k+1 , … , s n vanish. In this case the determinant

(A.12.4)

also vanishes.

With the substitutions H  →  U , K  →  V , R  →  S , R ₁₁  →  D we obtain from Sect.  A.5

(A.12.5)

(A.12.6)

With

(A.12.7)

one has

(A.12.8)

i.e.,

(A.12.9)

and

(A.12.10)

These have the solutions

(A.12.11)

i.e.,

for arbitrary p ₂ . The solution with minimum absolute value is

(A.12.12)

The residual vector has the form

(A.12.13)

and the absolute value

(A.12.14)

Singular Value Analysis

The rank k of the matrix A plays a decisive role in finding the solution of A x  ≈  b . Here k characterizes the transition from non-zero to vanishing singular values, s k  > 0, $$s_{k+1} = 0$$ . How should one judge the case of very small values of s k , i.e., $$s_{k} <\varepsilon$$ for a given small $$\varepsilon$$ ?

Example A.2:

Almost vanishing singular values

As a simple example we consider the case $$m = n = 2$$ , $$U = V = I$$ . One then has

(A.13.1)

that is,

(A.13.2)

If we now take s ₂  → 0 in ( A.13.2 ), then |  x ₂  | →  ∞ . At first glance one obtains a completely different picture if one sets s ₂  = 0 in ( A.13.1 ) directly. This gives

(A.13.3)

Thus $$x_{1} = b_{1}/s_{1}$$ as in ( A.13.2 ), but x ₂ is completely undetermined. The solution $$\widetilde{\mathbf{x}}$$ of minimum absolute value is obtained by setting x ₂  = 0. The question is now, “What is right? x ₂  =  ∞ or x ₂  = 0?” The answer is that one should set

and choose the parameter $$\varepsilon$$ such that the expression $$b_{2}/\varepsilon$$ is still numerically well-defined. This means that one must have $$\varepsilon /\vert b_{2}\vert \gg {2}^{-m}$$ if m binary digits are available for the representation of a floating point number (cf. Sect.  4.​2 ). Thus a finite value of b ₂ is computed as long as the numerical determination of this value is reasonable. If this is not the case then one approaches the situation s ₂  = 0, where b ₂ is completely undetermined, and one sets b ₂  = 0. ■ 

Example A.3:

Point of intersection of two almost parallel lines

We consider the two lines shown in Fig.  A.6 , which are described by

For the vector x of the intersection point one has A x  =  b with

One can easily check that A  =  U   S   V T holds with

i.e., $$s_{1} = \sqrt{2}$$ , $$s_{2} =\alpha \sqrt{2}$$ . Using

one obtains

independent of α .

If, however, one has $$s_{2} =\alpha \sqrt{2} <\varepsilon$$ and then sets s ₂  = 0, then one obtains

Fig. A.6:

Two lines intersect at the point (1,1) with an angle 2 γ  = 2 arctan  α .

From Fig.  A.6 one can see that for α  → 0 the two lines come together to a single line described by x ₂  = 1. The x ₁ -coordinate of the “intersection point” is completely undetermined. It is set equal to zero, since the solution vector x has minimum length [cf. ( A.5.14 )].

As in the case of “indirect measurements” in Chap.  9 we now assume that the vector b is equal to the vector of measurements y , which characterize the two lines, and that their measurement errors are given by the covariance matrix $$C_{y} = G_{y}^{-1}$$ . In the simplest case of equal uncorrelated errors one has

The covariance matrix for the unknowns $$\widetilde{\mathbf{x}}$$ is according to ( 9.​2.​27 )

and thus in the case of uncorrelated measurement errors,

Up to the factor σ ² it is thus equal to the unweighted covariance matrix $$C = {({A}^{\mathrm{T}}A)}^{-1}$$ for the matrix A . For our matrix A one has

The corresponding ellipse has the half-diameters $$\mathbf{e}_{1}/\alpha \sqrt{2}$$ and $$\mathbf{e}_{2}/\sqrt{2}$$ . The covariance ellipse of $$\widetilde{\mathbf{x}}$$ then has for the case of equal uncorrelated measurements equal half-diameters, multiplied, however, by the factor σ . They have the lengths $$\sigma _{x_{1}} =\sigma /\alpha \sqrt{2}$$ , $$\sigma _{x_{2}} =\sigma /\sqrt{2}$$ . Clearly one then sets x ₁  = 0 if the inequality $$x_{1} \ll \sigma _{x_{1}}$$ would hold for a finite fixed x ₁ , i.e., $$\alpha \sqrt{2} \ll \sigma$$ . ■ 

The decision as to whether a small singular value should be set equal to zero thus depends on numerical considerations and on a consideration of the measurement errors. The following fairly general procedure of singular value analysis has proven to be useful in practice.

1.

With a computer program one carries out the singular value decomposition, which yields, among other things, the ordered singular values

2.

Depending on the problem at hand one chooses a positive factor f  ≪ 1.

3.

All singular values for which s i  <  fs ₁ are set equal to zero. In place of k one has thus ℓ  ≤  k such that s i  = 0 for i  >  ℓ .

4.

With the replacements described above ( k  →  ℓ , $$s_{\ell+1} = \cdots = s_{k} = 0$$ ) the formulas of Sect.  A.12 retain their validity.

In place of the residual ( A.12.14 ) one obtains a somewhat larger value, since in the sum in the expression

(A.13.4)

one has more terms than in ( A.12.14 ).

The procedure implies that in some cases one has an effective reduction of the rank k of the matrix A to a value ℓ  <  k . If A has its full rank, then this can be reduced. This has the great advantage that numerical difficulties with small singular values are avoided. In contrast to, for example, the Gaussian or Cholesky procedures, the user does not have to worry about whether G  =  A T A is singular or nearly singular.

Although the singular value analysis always gives a solution of minimum absolute value $$\widetilde{\mathbf{x}}$$ for the problem A x  ≈  b , caution is recommended for the case ℓ  <  n (regardless of whether k  <  n or ℓ  <  k  =  n ). In such a case one has an (almost) linear dependence of the unknowns x ₁ , … , x n . The solution $$\widetilde{\mathbf{x}}$$ is not the only solution of the problem, but rather is simply the solution with the smallest absolute value out of the many possible solutions.

We have already remarked in Sect.  A.5 that the singular value decomposition is advantageous also with respect to numerical precision compared to other methods, especially to the method of normal equations. A detailed discussion (see, e.g., [ 18 ]) is beyond the scope of this book. Here we limit ourselves to giving an example.

Example A.4:

Numerical superiority of the singular value decomposition

compared to the solution of normal equations

Consider the problem A x  ≈  b for

The singular values of A are the square roots of the eigenvalues of

and are determined by ( A.11.16 ) to be

This was done with singular value decomposition without using the matrix G . If the computing precision is $$\varepsilon$$ and if $${\beta }^{2} <\varepsilon$$ but $$\beta >\varepsilon$$ , then one obtains

i.e., both singular values remain non-zero. If instead of the singular value decomposition one uses the normal equations

then the matrix G appears explicitly. With $${\beta }^{2} <\varepsilon$$ it is numerically represented as

This matrix is singular, det G  = 0, and cannot be inverted, as foreseen in ( A.5.17 ). This is also reflected in its singular values

 ■ 

Algorithm for Singular Value Decomposition

Strategy

We will now give an algorithm for the singular value decomposition and follow for the most part the presentation of LAWSON and HANSON [ 18 ], which is based on the work of GOLUB and KAHN [ 19 ], BUSINGER and GOLUB [ 20 ], and GOLUB and REINSCH [ 21 ]. The strategy of the algorithm is based on the successive application of orthogonal transformations.

In the first step , the matrix A is transformed into a bidiagonal matrix C ,

(A.14.1)

The orthogonal matrices Q and H are themselves products of Householder"=transformation matrices.

In step 2 the matrix C is brought into diagonal form by means of an iterative procedure:

(A.14.2)

Here the matrices U ′ and V ′ are given by products of Givens-transformation matrices and if necessary reflection matrices. The latter ensure that all diagonal elements are non-negative.

In step 3 , further orthogonal matrices U ″ and V ″ are determined. They are products of permutation matrices and ensure that the diagonal elements of

(A.14.3)

are ordered so as to be non-increasing as in ( A.12.3 ). As the result of the first three steps one obtains the matrices

(A.14.4)

which produce the singular value decomposition ( A.12.1 ) as well as the diagonal elements s ₁ , s ₂ , … , s k .

In step 4 the singular value analysis is finally carried out. Stated simply, all singular values are set to zero for which

(A.14.5)

Thus one can finally give the solution vector x to the equation

(A.14.6)

In practice, Eq. ( A.14.6 ) is often solved simultaneously for various right-hand sides, which can then be arranged in an m × ℓ matrix,

Instead of the solution vector $$\widetilde{\mathbf{x}}$$ we thus obtain the n × ℓ solution matrix

of the equation

(A.14.7)

The method DatanMatrix .sigularValueDecomposition computes this solution. It merely consists of four calls to other methods that carry out the four steps described above. These are explained in more detail in the following sections.

Usually one is only interested in the solution matrix $$\widetilde{X}$$ of the problem A   X  ≈  B and possibly in the number of singular values not set to zero. Sometimes, however, one would like explicit access to the matrices U and V and to the singular values. This is made possible by the method DatanMatrix .pseudoInverse.

Bidiagonalization

The method DatanMatrix .sv1 performs the procedure described below. (It is declared as private within the class DatanMatrix as are the further methods referred to in Sect.  A.14 ; of course their source code can be studied.) The goal is to find the m × n matrix C in ( A.14.1 ). Here C is of the form

(A.14.8)

and C ′ is an n × n bidiagonal matrix

(A.14.9)

The goal is achieved by multiplication of the matrix A with appropriate Householder matrices (alternating on the right and left):

(A.14.10)

The matrix Q ₁ is computed from the elements of the first column of A and is applied to all columns of A (and B ). It results in only the element (1, 1) of A remaining non-zero. The matrix H ₂ is computed from the first row of the matrix Q ₁ A . It results in the element (1, 1) remaining unchanged, the element (1, 2) being recomputed and the elements (1, 3), … , (1,  n ) being set equal to zero. It is applied to all rows of the matrix ( Q ₁ A ). One thus obtains

Here “•” denotes an element of the final bidiagonal matrix C ′, and “.” an element that will be changed further. Now Q ₂ is determined from the second column of Q ₁ A   H ₂ such that upon application to this column the element 1 remains unchanged and the element 2 and all others are changed such that only element 2 remains non-zero, and the elements 3 through m become zero.

The procedure can be summarized in the following way. The matrix Q i is applied to the column vectors, it leaves the elements 1 through i − 1 unchanged and changes the elements i through m . It produces an orthogonal transformation in the subspace of the components i through m . At the time of applying Q i , however, these elements in the columns 1 through i − 1 are already zero, so that Q i must only be applied explicitly to columns i through  n . The corresponding considerations hold for the matrix H i . It acts on the row vectors, leaves elements 1 through i − 1 unchanged, and produces an orthogonal transformation in the subspace of the components i through n . When it is applied, these components in the rows 1 through i − 2 are all zero. The matrix H i must only be applied explicitly to the rows i − 1, … , m − 1. Since only n − 1 rows of A must be processed, the matrix H n in ( A.14.10 ) is the unit matrix; Q n is the unit matrix only if m  =  n .

In addition to the transformed matrix Q T A   H , the matrix $$H = H_{2}H_{3}\cdots H_{n-1}$$ is also stored. For this the information about each matrix H i is saved. As we determined at the end of Sect.  A.3 , it is sufficient to store the quantities defined there, u p and b , and the elements i + 1 through n of the ( i − 1)th row vectors defining the matrix H i . This can be done, however, in the array elements of these row vectors themselves, since it is these elements that will be transformed to zero and which therefore do not enter further into the calculation. One needs to declare additional variables only for the quantities u p and b of each of the matrices H i . If all of the transformations have been carried out, then the diagonal elements d i and the next-to-diagonal elements e i are transferred to the arrays e and d . Finally, the product matrix $$H = H_{2}H_{3}\cdots H_{n-1}I$$ is constructed in the first n rows of the array of the original matrix A , in the order H n−1 I , $$H_{n-2}(H_{n-1}I)$$ , … . Here as well, the procedure is as economical as possible, i.e., the Householder matrix is only applied to those columns of the matrix to the right for which it would actually produce a change. The unit matrix I is constructed to the extent that it is needed, row-by-row in the array of the original matrix A starting from below. For this there is exactly the right amount of space available, which up to this point was necessary for storing information about the Householder transformations that were just applied.

Diagonalization

This step is implemented in DatanMatrix .Sv2 . The bidiagonal matrix C , whose non-vanishing elements are stored in the arrays d and e , is now brought into diagonal form by appropriately chosen Givens transformations. The strategy is chosen such that the lowest non-diagonal element vanishes first and the non-diagonal elements always move up and to the left, until C is finally diagonal. All of the transformations applied to C from the left are also applied to the matrix stored in the array b , and all transformations that act from the right are also applied to the matrix in a . (We denote the matrix to be diagonalized during each step by C .)

Only the upper-left submatrix C k with k rows and columns is not yet diagonal and must still be considered. The index k is determined such that e k ≠ 0 and e j  = 0, j  >  k . This means that the program runs through the loop k  =  n , n − 1, … , 2. Before the lower non-diagonal element e k is systematically made zero by means of an iterative procedure, one checks for two special cases, which allow a shortening of the computation by means of special treatment.

Special case 1 , d k  = 0 (handled in DatanMatrix .s21 ): A Givens matrix W is applied from the right, which also causes e k to vanish. The matrix w is the product $$W_{k-1}W_{k-2}\cdots W_{1}$$ . Here W i acts on the columns i and k of C k , but of course only on those rows where at least one of these columns has a non-vanishing element. W k−1 acts on the row k − 1, annihilates the element $$e_{k} = C_{k-1,k}$$ , and changes $$C_{k-1,k-1}$$ . In addition, W k−1 acts on the row k − 2, changes the element $$C_{k-2,k-1}$$ , and produces a non-vanishing element $$H = C_{k-2,k}$$ in column k . Now the matrix W k−2 is applied, which annihilates exactly this element, but produces a new element in row k − 3 and column  k . When the additional element finally makes it to row 1, it can then be annihilated by the transformation W ₁ . As a result of this treatment of special case 1, C k decomposes into a $$(k - 1) \times (k - 1)$$ submatrix C k−1 and a 1 × 1 null matrix.

Special case 2 , C k decomposes into submatrices (handled in DatanMatrix .s22 ): If e ℓ  = 0 for any value ℓ , 2 ≤  ℓ  ≤  k , then the matrix

can be decomposed into two matrices C ℓ−1 and $$\bar{C}$$ . One can first diagonalize $$\bar{C}$$ and then C ℓ−1 . In particular, if ℓ  =  k , then one obtains

Here d k is the singular value, and the loop index can only be decreased by one, k  →  k − 1. First, however, the case d k  < 0 must be treated; see “ change of sign ” below. If $$d_{\ell-1} = 0$$ , but e ℓ ≠ 0, then C can still be decomposed. For this one uses the transformation matrix $$T = T_{k}T_{k-1},\cdots \,,T_{\ell+1}$$ acting on the left, where T i acts on the rows ℓ and i . In particular, T ℓ+1 annihilates the element $$e_{\ell} = C_{\ell,\ell+1}$$ and creates an element $$H = C_{\ell,\ell+2}$$ . T ℓ+2 annihilates this element and creates in its place $$H = C_{\ell,\ell+3}$$ . Finally T k annihilates the last created element H  =  C ℓ k by a transformation to C kk  =  d k .

After it has been checked whether one or both of the special cases were present and such an occurrence has been treated accordingly, it remains only to diagonalize the matrix $$\bar{C}$$ . This consists of the rows and columns ℓ through k of C . If special case 2 was not present, then one has ℓ  = 1. The problem is solved iteratively with the QR-algorithm.

QR-algorithm (carried out in DatanMatrix .s23 ): First we denote the square output matrix $$\bar{C}$$ by C ₁ . We then determine orthogonal matrices U i , V i and carry out transformations

which lead to a diagonal matrix S ,

The following prescription is used for determining U i and V i :

(A)

One determines the eigenvalues λ ₁ , λ ₂ of the lower-right 2 × 2 submatrix of $$C_{i}^{\mathrm{T}}C_{i}$$ . Here σ i is the eigenvalue closest to the lower-right element of $$C_{i}^{\mathrm{T}}C_{i}$$ .

(B)

The matrix V i is determined such that $$V _{i}^{\mathrm{T}}(C_{i}^{\mathrm{T}}C_{i} -\sigma _{i}I)$$ has upper triangular form.

(C)

The matrix U i is determined such that $$C_{i+1} = U_{i}^{\mathrm{T}}C_{i}V _{i}$$ is again bidiagonal.

The matrix V i from step (B) exists, according to a theorem by FRANCIS [ 22 ], if

(a)

$$C_{i}^{\mathrm{T}}C_{i}$$ is tridiagonal with non-vanishing subdiagonal elements,

(b)

V i is orthogonal,

(c)

σ i is an arbitrary scalar,

(d)

$$V _{i}^{\mathrm{T}}(C_{i}^{\mathrm{T}}C_{i})V _{i}$$ is tridiagonal, and

(e)

The first column of $$V _{i}^{\mathrm{T}}(C_{i}^{\mathrm{T}}C_{i} -\sigma _{i}I)$$ except the first element vanishes.

The requirement (a) is fulfilled, since C i is bidiagonal and the special case 2 has been treated if necessary; (b) is also fulfilled by constructing V i as the product of Givens matrices. This is done in such a way that simultaneously (d) and (e) are fulfilled. In particular one has

where

R j acts on the columns j and j + 1 of C ,

L j acts on the rows j and j + 1 of C ,

R ₁ is determined such that requirement (e) is fulfilled,

L ₁ , R ₂ , L ₂ , … are determined such that (e) is fulfilled without violating (d).

For σ i one obtains

with

and

The first column of the matrix $$(C_{i}^{\mathrm{T}}C_{i} -\sigma _{i}I)$$ is

One determines the matrix R ₁ , which defines a Givens transformation, such that all elements of the first column of $$R_{1}^{\mathrm{T}}(C_{i}^{\mathrm{T}}C_{i} -\sigma _{i}I)$$ except the first vanish. Application of R ₁ on C i produces, however, an additional element H  =  C ₂₁ , so that C i is no longer bidiagonal,

By application of L ₁ , this element is projected onto the diagonal. In its place a new element H  =  C ₁₃ is created,

By continuing the procedure, the additional element is moved further down and to the right, and can be completely eliminated in the last step:

If the lower non-diagonal element of C i+1 is already zero, then the lower diagonal element is already a singular value. Otherwise the procedure is repeated, whereby it is first checked whether now one of the two special cases is present. The procedure typically converges in about 2  k steps; ( k is the rank of the original matrix A ). If convergence has still not been reached after 10  k steps, then the algorithm is terminated without success.

Change of sign. If a singular value d k has been found, i.e., if e k  = 0, then it is checked whether it is negative. If this is the case, then a simple orthogonal transformation is applied that multiplies the element d k  =  C kk of C and all elements in the k th column of the matrix contained in a by − 1. The index k can then be reduced by one. Corresponding to ( A.12.6 ) the matrix B is multiplied on the left by U T . This is done successively with the individual factors making up U T .

Ordering of the Singular Values and Permutation

By the method DatanMatrix .sv3 the singular values are put into non-increasing order. This is done by a sequence of permutations of neighboring singular values, carried out if the singular value that follows is larger then the preceding. The matrices stored in a and b are multiplied by a corresponding sequence of permutation matrices; cf. ( A.14.4 ).

Singular Value Analysis

In the last step the singular value analysis is carried out as described in Sect.  A.13 by the method DatanMatrix .sv4 . For a given factor f  ≪ 1 a value ℓ  ≤  k is determined such that s i  <  fs ₁ for i  >  ℓ . The columns of the array B , which now contains the vectors g , are transformed in their first ℓ elements into $$\widetilde{\mathbf{p}}_{1}$$ according to ( A.12.11 ), and the elements ℓ + 1, … , n are set equal to zero. Then the solution vectors $$\widetilde{\mathbf{x}}$$ , which make up the columns of the solution matrix $$\widetilde{X}$$ , are computed according to ( A.12.12 ).

Least Squares with Weights

Instead of the problem ( A.5.3 ),

(A.15.1)

one often encounters a similar problem that in addition contains a positive-definite symmetric weight-matrix G m×m ,

(A.15.2)

In ( A.15.1 ) one simply has G  =  I . Using the Cholesky decomposition ( A.9.1 ) of G , i.e., G  =  U T U , one has

(A.15.3)

With the definitions

(A.15.4)

Eq. ( A.15.3 ) takes on the form

(A.15.5)

After the replacement ( A.15.4 ), the problem ( A.15.2 ) is thus equivalent to the original one ( A.15.1 ).

In Sect.  A.11 we called the n × n matrix

(A.15.6)

the unweighted covariance matrix of the unknowns x in the Problem ( A.15.1 ). In Problem ( A.15.2 ), the weighted covariance matrix

(A.15.7)

appears in its place.

Least Squares with Change of Scale

The goal in solving a problem of the type

(A.16.1)

is the most accurate numerical determination of the solution vector $$\widetilde{\mathbf{x}}$$ and the covariance matrix C . A change of scale in the elements of A , b , and x can lead to an improvement in the numerical precision.

Let us assume that the person performing the problem already has an approximate idea of $$\widetilde{\mathbf{x}}$$ and C , which we call z and K . The matrix K has the Cholesky decomposition K  =  L T L . By defining

(A.16.2)

Eq. ( A.16.1 ) becomes

The meaning of the new vector of unknowns $$\mathbf{x}^{\prime}$$ is easily recognizable for the case where K is a diagonal matrix. We write

where the quantities σ i ² are the estimated values of the variances of the unknowns x i . Thus the i th component of the vector $$\mathbf{x}^{\prime}$$ becomes

If in fact one has $$\widetilde{x}_{i} = z_{i}$$ and the corresponding variance σ i ² , then x i ′ = 0 and has a variance of one. If the estimates are at least of the correct order of magnitude, the x i ′ are close to zero and their variances are of order unity. In addition, in case the full matrix is, in fact, estimated with sufficient accuracy, the components of $$\mathbf{x}^{\prime}$$ are not strongly correlated with each other.

In practice, one carries out the transformation ( A.16.2 ) only in exceptional cases. One must take care, however, that “reasonable” variables are chosen for ( A.16.2 ). This technique is applied in the graphical representation of data. If it is known, for example, that a voltage U varies in the region σ  = 10 mV about the value U ₀  = 1 V, then instead of U one would plot the quantity $$U^{\prime} = (U - U_{0})/\sigma$$ , or some similar quantity.

Modification of Least Squares According to Marquardt

Instead of the problem

(A.17.1)

which we have also written in the shorter form

(A.17.2)

let us now consider the modified problem

(A.17.3)

Here I is the n × n unit matrix and λ is a non-negative number. The modified problem is of considerable importance for fitting nonlinear functions with the method of least squares (Sect.  9.​5 ) or for minimization (Sect.  10.​15 ). For λ  = 0 Eq. ( A.17.3 ) clearly becomes ( A.17.2 ). If, on the other hand, λ is very large, or more precisely, if it is large compared to the absolute values of the elements of A and b , then the last “row” of ( A.17.3 ) determines the solution $$\widetilde{\mathbf{x}}$$ , which is the null vector for λ  →  ∞ .

We first ask which direction the vector $$\widetilde{\mathbf{x}}$$ has for large values of λ . The normal equations corresponding to ( A.17.3 ), cf. ( A.5.16 ), are

with the solution

For large λ the second term in parentheses dominates, and one obtains simply

That is, for large values of λ , the solution vector tends toward the direction of the vector A T b .

We will now show that for a given λ the solution x (λ) to ( A.17.3 ) can easily be found with the singular value decomposition simultaneously with the determination of the solution $$\widetilde{\mathbf{x}}$$ of ( A.17.2 ). The singular value decomposition ( A.12.1 ) of A is

By substituting into ( A.17.3 ) and multiplying on the left we obtain

or

(A.17.4)

where, using the notation as in Sect.  A.12 ,

By means of Givens transformations, the matrix on the left-hand side of ( A.17.4 ) can be brought into diagonal form. One obtains

with

and thus

The solution $${\widetilde{\mathbf{x}}}^{(\lambda )}$$ of ( A.17.3 ) is then

The method DatanMatrix .marquardt computes these solution vectors for two values of λ . It proceeds mostly as DatanMatrix .singularValueDecomposition ; only in step 4 instead of DatanMatrix .sv4 the method DatanMatrix .svm is used which is adapted to the Marquardt problem.

Least Squares with Constraints

One often encounters the problem ( A.5.3 )

(A.18.1)

with the constraint

(A.18.2)

Here A is as before an m × n matrix and E is an ℓ × n matrix. We will restrict ourselves to the only case that occurs in practice,

(A.18.3)

The determination of an extreme value with constraints is usually treated in analysis textbooks with the method of Lagrange multipliers. Here as well we rely on orthogonal transformations. The following method is due to LAWSON and HANSON [ 18 ]. It uses a basis of the null space of E . First we carry out an orthogonal decomposition of E as in ( A.5.5 ),

(A.18.4)

Here we regard the orthogonal n × n matrix K as being constructed out of an n × ℓ matrix K ₁ and an n × ( n − ℓ ) matrix K ₂ ,

(A.18.5)

According to ( A.5.6 ) and ( A.5.7 ), all solutions of ( A.18.2 ) have the form

(A.18.6)

Here $$\widetilde{\mathbf{x}}$$ is the unique solution of minimum absolute value of ( A.18.2 ). For brevity we will write this in the form $$\widetilde{\mathbf{x}} = {E}^{+}\mathbf{d}$$ ; cf. ( A.10.3 ). p ₂ is an arbitrary ( n − ℓ )-vector, since the vectors K ₂ p ₂ form the null space of E ,

(A.18.7)

The constraint ( A.18.2 ) thus says that the vector x for which ( A.18.1 ) is a minimum must come from the set of all vectors $$\widehat{\mathbf{x}}$$ , i.e.,

(A.18.8)

This relation is a least-squares problem without constraints, from which the ( n − ℓ )-vector p ₂ can be determined. We write its solution using ( A.10.3 ) in the form

(A.18.9)

By substitution into ( A.18.6 ) we finally obtain

(A.18.10)

as the solution of ( A.18.1 ) with the constraint ( A.18.2 ).

The following prescription leads to the solution ( A.18.10 ). Its starting point is the fact that one can set H  =  I because of ( A.18.3 ).

Step 1: One determines an orthogonal matrix K  = ( K ₁ ,  K ₂ ) as in ( A.18.5 ) such that

and such that $$\widetilde{E}_{1}$$ is a lower triangular matrix. In addition one computes

Step 2: One determines the solution $$\widetilde{\mathbf{p}}_{1}$$ of

This is easy, since $$\widetilde{E}_{1}$$ is a lower triangular matrix of rank ℓ . Clearly one has $$\widetilde{\mathbf{x}} = K_{1}\widetilde{\mathbf{p}}_{1}$$ .

Step 3: One determines the vector

Step 4: One determines the solution $$\widetilde{\mathbf{p}}_{2}$$ to the least-squares problem ( A.18.8 ) (without constraints)

Step 5: From the results of steps 2 and 4 one finds the solution to ( A.18.1 ) with the constraint ( A.18.2 )

We will now consider a simple example that illustrates both the least-squares problem with constraints as well as the method of solution given above.

Example A.5:

Least squares with constraints

Suppose the relation ( A.18.1 ) has the simple form

for n  = 2. One then has $$m = n = 2$$ , A  =  I , and b  = 0. Suppose the constraint is

i.e., ℓ  = 1, E  = (1, 1), d  = 1.

The problem has been chosen such that it can be solved by inspection without mathematical complications. The function $$z ={ \mathbf{x}}^{2} = x_{1}^{2} + x_{2}^{2}$$ corresponds to a paraboloid in the ( x ₁ ,  x ₂ ,  z ) space, whose minimum is at $$x_{1} = x_{2} = 0$$ . We want, however, to find not the minimum in the entire ( x ₁ ,  x ₂ ) plane, but rather only on the line $$x_{1} + x_{2} = 1$$ , as shown in Fig.  A.7 . It clearly lies at the point where the line has its smallest distance from the origin, i.e., at $$x_{1} = x_{2} = 1/2$$ .

Fig. A.7:

The solution $$\widetilde{\mathbf{x}}$$ to Example A.5 lies on the line given by the constraint $$x_{1} + x_{2} = 1$$ .

Of course one obtains the same result with the algorithm. With

we obtain $$\widetilde{E}_{1} = \sqrt{2}$$ , $$\:\widetilde{\mathbf{p}}_{1} = 1/\sqrt{2}$$ ,

We solve the problem $${(\widetilde{A}_{2}\mathbf{p}_{2} -\bar{\mathbf{b}})}^{2} =\min$$ with the normal equations

The full solution is then

 ■ 

The method DatanMatrix .leastSquaresWithConstraints solves the problem of least squares $${(A\mathbf{x} -\mathbf{b})}^{2} =\min$$ with the linear constraint E x  =  d .

Java Classes and Example Programs

Java Classes for Vector and Matrix Operations

DatanVector contains methods for vector operations.

DatanMatrix contains methods for matrix operations.

Example Program A.1:

The class E1Mtx demonstrates simple operations of matrix and vector algebra At first the matrices

and the vectors

are defined. Then, with the appropriate methods, simple operations are performed with these quantities. Finally, the resulting matrices and vectors are displayed. The operations are

Example Program A.2:

The class E2Mtx demonstrates the handling of submatrices and subvectors.

The matrices

and the vectors

are defined.

Next a submatrix S is taken from A , and a submatrix of A is overwritten by D . A column vector and a row vector are taken from A and inserted into A . Finally some elements are taken according to a list from the vector u and assembled in a vector z , and elements of w are put into positions (defined by list) of the vector u .

Example Program A.3:

The class E3Mtx demonstrates the performance of Givens transformations Fist the two vectors

are defined. Next, by the use of DatanMatrix . defineGivensTransformation transformation parameters c and s for the vector u are computed and displayed. The Givens transformation of u with these parameters is performed with DatanMatrix . applyGivensTransformation yielding

Finally, by calling DatanMatrix .defineAndApplyGivensTransformation parameters c and s are computed for the vector w and the transformation is applied to this vector.

Example Program A.4:

The class E4Mtx demonstrates the performance of Householder transformations First the two vectors

are defined. Moreover, the indices n , p are set to ℓ n  = 6, p  = 3, ℓ  = 5. By calling DatanMatrix . defineHouseholderTransformation the Householder transformation defined by these indices and the vector v is initialized. The application of this transformation to the vectors v and c is performed by two calls of DatanMatrix . applyHouseholderTransformation . The results are displayed alphanumerically.

Example Program A.5:

The class E5Mtx demonstrates the Gaussian algorithm for the solution of matrix equationsThe matrix

and the 3 × 3 unit matrix B are defined. A call of DatanMatrix . matrixEquation solves the matrix equation AX  =  B , i.e., X is the inverse of A . The matrix A is identical to the one chosen in Example A.1. In that example the algorithm is shown step by step.

Example Program A.6:

The class E6Mtx demonstrates Cholesky decomposition and Cholesky inversion First, the matrices

are defined and the symmetric, positive-definite matrix A  =  B T B is constructed. By calling DatanMatrix .choleskyDecomposition the Cholesky decomposition A  =  U T U is performed and the triangular matrix U is displayed. Multiplication of U T by U yields in fact U T U  =  S . The method DatanMatrix .choleskyMultiply is then used to compute R  =  UC . Finally, by Cholesky inversion with DatanMatrix .choleskyInversion, the inverse S ⁻¹ of S is computed. Multiplication with the original matrix yields $$S{S}^{-1} = I$$ .

Example Program A.7:

The class E7Mtx demonstrates the singular value decomposition The program first operates on the same matrix as E5Mtx . However, the matrix inversion is now performed with DatanMatrix .pseudoInverse.

Next, the matrix is replaced by

This matrix, having two identical columns, is singular. With another call of DatanMatrix .pseudoInverse not only the psudoinverse matrix but also the residuals, the diagonal matrix D , and the two orthogonal matrices U and V are determined.

Example Program A.8:

The class E8Mtx demonstrates the solution of matrix equations by singular value decomposition for 9 different cases In the framework of our programs, in particular that of least-squares and minimization problems, matrix equations are solved nearly exclusively by singular value decomposition. In Sect.  A.5 we have listed the different cases of the matrix equation A x  ≈  b . If A is an m × n matrix then first we must distinguish between the cases m  =  n (case 1), m  >  n (case 2), and m  <  n (case 3). A further subdivision is brought about by the rank of A . If the rank k of A is k  = min( m ,  n ) then we have the case 1a (or 2a or 3a). If, however, k  < min( m ,  n ) then we are dealing with case 1b (or 2b or 3b). The rank of a matrix is equal to its number of non-vanishing singular values. In numerical calculations, which are always performed with finite accuracy, one obviously has to define more precisely the meaning of “non-vanishing”. For this definition we use the method of singular value analysis (Sect.  A.13 ) and set a singular value equal to zero if it is smaller than a fraction f of the largest singular value. The number of finite singular values remaining in this analysis is called the pseudorank . In addition to the cases mentioned above we consider as well the cases 1c, 2c, and 3c, in which the matrix A has full rank but not full pseudorank.

The program consists of two nested loops. The outer loop runs through the cases 1, 2, 3, the inner loop through the subcases a, b, c. For each case the matrix A is composed of individual vectors. In the subcase b two of these vectors are chosen to be identical. In the subcase c they are identical except for one element, which in one vector differs by $$\varepsilon = 1{0}^{-12}$$ compared to the value in the other vector. In case 3 the system of linear equations symbolized by the matrix equation Ax  =  b has less equations ( m ) than unknowns ( n ). This case does not appear in practice and, therefore, is not included in the programs. It is simulated here in the following way. In case 3 with m  = 2 and n  = 3, the matrix A is extended to become a 3 × 3 matrix by addition of another row the elements, of which are all set to zero, and correspondingly m is set to 3.

If the singular value analysis shows that one or several singular values are smaller than the fraction f of the largest singular value, then they are set to zero. In our example program for each of the 9 cases the analysis is performed twice, first for $$f = 1{0}^{-15}$$ and then for $$f = 1{0}^{-10}$$ . For $$f = 1{0}^{-15}$$ in our example cases 1c, 2c, 3c the matrix A has full rank, in spite of the small value of $$\varepsilon = 1{0}^{-12}$$ . the singular value analysis with $$f = 1{0}^{-10}$$ reduces the number of singular values. Note that the elements of the solution matrix differ as the choice of f changes for cases 1c, 2c, 3c. The unwieldly numerical values in the case of $$f = 1{0}^{-15}$$ show that we are near the limits of numerical stability.

Example Program A.9:

The class E9Mtx demonstrates the use of the method DatanMatrix .marquardt

The method  DatanMatrix .marquardt will be rarely called directly. It was written to be used in LsqMar and MinMar . For completeness we demonstrate it with a short class. It solves the problem A x  ≈  b  – modified according to ( A.17.3 ) – for given A , b , and λ and displays the results x ₁ and x ₂ .

Example Program A.10:

The class E10Mtx demonstrates solution of the least squares problem with constraints by the method DatanGraphics.leastSquaresWithConstraints The problem of Examples 9.9 and 9.10 is solved, i.e., the measurement x ₁  = 89, x ₂  = 31, x ₃  = 62 of the three angles of a triangle and the evaluation of these measurements using the constraint $$x_{1} + x_{2} + x_{3} = 180$$ . The evaluation requires the solution of $${(A\mathbf{x} -\mathbf{b})}^{2} =\min$$ with the constraint E x  =  d with

In the program the matrices and vectors A , b , E , d are provided. The solution is computed by calling DatanGraphics.leastSquaresWithConstraints . It is, of course, identical to the results found in the previously mentioned examples.

Combinatorics

Consider n distinguishable objects a ₁ , a ₂ , … , a n . We ask for the number of possible ways P n k , in which one can place k of them in a given order. Such orderings are called permutations . For the example n  = 4, k  = 2 these permutations are

i.e., P k n  = 12. The answer for the general problem can be derived from the following scheme. There are n different possible ways to occupy the first place in a sequence. When one of these ways has been chosen, however, there are only n − 1 objects left, i.e., there remain n − 1 ways to occupy the second place, and so forth. One therefore has

(B.1)

The result can also be written in the form

(B.2)

where

(B.3)

Often one is not interested in the order of the k objects within a permutation (the same k objects can be arranged in k ! different ways within the sequence), but rather one only considers the number of different ways of choosing of k objects out of a total of n . Such a choice is called a combination . The number of possible combinations of k elements out of n is then

(B.4)

For the binomial coefficients $${n\choose k}$$ one has the simple recursion relation

(B.5)

which can easily be proven by computation:

Fig. B.1:

Pascal’s triangle.

The recursion formula is the basis for the famous Pascal’s triangle , which, because of its beauty, is shown in Fig.  B.1 . The name “binomial coefficient” comes from the well-known binomial theorem ,

(B.6)

the proof of which (by induction) is left to the reader. We use the theorem in order to derive a very important property of the coefficient $${n\choose k}$$ . For this we write it in the simple form for b  = 1, i.e.,

and we then apply it a second time,

If we consider only the term with a ℓ , then by comparing coefficients we find

(B.7)

Formulas and Methods for the Computation of Statistical Functions

Binomial Distribution

We present here two function subprograms for computing the binomial distribution ( 5.​1.​3 )

(C.1.1)

and the distribution function

(C.1.2)

are computed by the methods StatFunct.binomial and StatFunct.cumulativeBinomial , respectively. For reasons of numerical stability the logarithm of Euler’s gamma function is used in the computation.

Hypergeometric Distribution

The hypergeometric distribution ( 5.​3.​1 )

(C.2.1)

and the corresponding distribution function

(C.2.2)

are computed by StatFunct.hypergeometric and StatFunct.cumulativeHypergeometric , respectively.

Poisson Distribution

The Poisson distribution ( 5.​4.​1 )

(C.3.1)

and the corresponding distribution function

(C.3.2)

are computed with the methods StatFunct.poisson and StatFunct.cumulativePoisson respectively.

The quantities f ( k ;  λ ) and F ( K ;  λ ) depend not only on the values of the discrete variables k and K , but also on the continuous parameter λ . For a given P there is a certain parameter value λ P that fulfills Eq. ( C.3.2 ). We can denote this as the quantile

(C.3.3)

of the Poisson distribution. It is computed by the method StatFunct.quantilePoisson.

Normal Distribution

The probability density of the standard normal distribution is

(C.4.1)

It is computed by the method StatFunct.standardNormal.

The normal distribution with mean x ₀ and variance σ ² ,

(C.4.2)

can easily be expressed in terms of the standardized variable

(C.4.3)

using ( C.4.1 ) to be

(C.4.4)

It is computed by the method StatFunct.normal.

The distribution function of the standard normal distribution

(C.4.5)

is an integral that cannot be computed in closed form. We can relate it, however, to the incomplete gamma function described in Sect.  D.5 .

The distribution function of a normal distribution with mean x ₀ and variance σ ² is found from ( C.4.5 ) to be

(C.4.6)

We now introduce the error function

(C.4.7)

Comparing with the definition of the incomplete gamma function ( D.5.1 ) gives

(C.4.8)

On the other hand there is a more direct connection between ( C.4.6 ) and ( C.4.7 ),

or

i.e.,

(C.4.9)

The methods StatFunct.cumulativeStandardNormal and StatFunct.cumulativeNormal yield the distribution functions ( C.4.4 ) and ( C.4.5 ), respectively.

Finally we compute the quantiles of the standard normal distribution (using the method StatFunct.quantileStandardNormal ). For a given probability P , the quantile x p is defined by the relation

(C.4.10)

We determine this by finding the zero of the function

(C.4.11)

using the procedure of Sect.  E.2 .

For the quantile x P for a probability P of the normal distribution with mean x ₀ and standard deviation σ (computed with StatFunct.quantileNormal) one has

(C.4.12)

χ ² -Distribution

The probability density ( 6.​6.​10 ) of the χ ² -distribution for n degrees of freedom,

(C.5.1)

is computed by the method StatFunct.chiSquared .

The distribution function

(C.5.2)

is seen from ( D.5.1 ) to be an incomplete gamma function

(C.5.3)

and computed by StatFunct.cumulativeChiSquared .

The quantile χ P ² of the χ ² -distribution for a given probability P , which is given by

(C.5.4)

is computed as the zero of the function h ( χ ² ) with StatFunct.quantileChiSquared .

F -Distribution

The probability density ( 8.​2.​3 ) of the F -distribution with f ₁ and f ₂ degrees of freedom,

(C.6.1)

is computed with StatFunct.fDistribution .

The distribution function

(C.6.2)

can be rearranged using

to be

(C.6.3)

i.e., it is related to the incomplete beta function; cf. ( D.6.1 ). We compute it with the method StatFunct.cumulativeFDistribution .

The quantile F P of the F -distribution for a given probability P is given by the zero of the function

(C.6.4)

It is computed by StatFunct.quantileFDistribution .

t -Distribution

The probability density ( 8.​3.​7 ) of Student’s t -distribution with f degrees of freedom,

(C.7.1)

is computed with by StatFunct.student .

By using the substitution

the distribution function of the t -distribution can be expressed in terms of the incomplete beta function ( D.6.1 ),

(C.7.2)

(C.7.3)

It is computed by the method StatFunct.cumulativeStudent .

The quantile t P of the t -distribution for a given probability P is computed by finding the zero of the function

(C.7.4)

with the method StatFunct.quantileStudent .

Java Class and Example Program

Java Class for the Computation of Statistical Functions

StatFunct contains all methods mentioned in this Appendix.

Example Program C.1:

The class FunctionsDemo demonstrates all methods mentioned in this AppendixThe user first selects a family of functions and then a function from that family. Next the parameters, needed in the chosen case, are entered. After the Go button is clicked, the function value is computed and displayed.

The Gamma Function and Related Functions: Methods and Programs for Their Computation

The Euler Gamma Function

Consider a real number x with x + 1 > 0. We define the Euler gamma function by

(D.1.1)

Integrating by parts gives

Thus one has the relation

(D.1.2)

This is the so-called recurrence relation of the gamma function. From ( D.1.1 ) it follows immediately that

With ( D.1.2 ) one then has generally that

(D.1.3)

We now substitute t by $$\frac{1} {2}{u}^{2}$$ (and d t by u  d u ) and get

If we now choose in particular $$x = -\frac{1} {2}$$ , we obtain

(D.1.4)

The integral can be evaluated in the following way. We consider

The integral A can transformed into polar coordinates:

Setting the two expressions for A equal gives

(D.1.5)

Using ( D.1.2 ) we can thus determine the value of the gamma function for half-integral arguments.

For arguments that are not positive integers or half-integers, the integral ( D.1.1 ) cannot be evaluated in closed form. In such cases one must rely on approximations. We discuss here the approximation of LANCZOS [ 17 ], which is based on the analytic properties of the gamma function. We first extend the definition of the gamma function to negative arguments by means of the reflection formula

(D.1.6)

(By relations ( D.1.1 ) and ( D.1.6 ) the gamma function is also defined for an arbitrary complex argument if x is complex.) One sees immediately that the gamma function has poles at zero and at all negative integer values. The approximation of LANCZOS [ 17 ],

(D.1.7)

takes into account the first few of these poles by the form of the function A γ ,

(D.1.8)

For γ  = 5 one has for the error $$\varepsilon$$ the approximation $$\vert \varepsilon \vert < 2 \cdot 1{0}^{-10}$$ for all points x in the right half of the complex plane. The method Gamma.gamma yields Euler’s gamma function.

Fig. D.2:

The functions Γ ( x ) and 1∕ Γ ( x ).

The gamma function is plotted in Fig.  D.2 . For large positive arguments the gamma function grows so quickly that it is difficult to represent its value in a computer. In many expressions, however, there appear ratios of gamma functions which have values in a an unproblematic region. In such cases it is better to use the logarithm of the gamma function which is computed by the method Gamma.logGamma .

Factorial and Binomial Coefficients

The expression

(D.2.1)

can either be directly computed as a product or as a gamma function using ( D.1.3 ).

When computing binomial coefficients,

(D.2.2)

the expression on the right-hand side is preferable for numerical reasons to the expression in the middle and used in the method Gamma.binomial .

Beta Function

The beta function has two arguments and is defined by

(D.3.1)

The integral can be written in a simple way in terms of gamma functions,

(D.3.2)

In this way the method Gamma.beta computes the beta function. Figure  D.3 shows it as a function of w for several fixed values of z .

Fig. D.3:

The beta function. For increasing z the curves B ( z ,  w ) are shifted to the left.

Computing Continued Fractions

In the next two sections there appear continued fractions , i.e., expressions of the type

(D.4.1)

that can also be written in the typographically simpler form

(D.4.2)

If we denote by f n the value of the fraction ( D.4.1 ) truncated after a finite number of terms up to the coefficients a n and b n , then one has

(D.4.3)

The quantities A n and B n can be obtained from the following recursion relation,

(D.4.4)

(D.4.5)

(D.4.6)

Since the relations ( D.4.5 ) and ( D.4.6 ) are linear in $$A_{j-1},A_{j-2}$$ and $$B_{j-1},B_{j-2}$$ , respectively, and since in ( D.4.3 ) only the ratio A n ∕ B n appears, one can always multiply the coefficients $$A_{j},A_{j-1},A_{j-2}$$ and $$B_{j},B_{j-1},B_{j-2}$$ by an arbitrary normalization factor. One usually chooses for this factor 1∕ B j and avoids in this way numerical difficulties from very large or very small numbers, which would otherwise occur in the course of the recursion. For steps in which B j  = 0, the normalization is not done.

Continued fractions, in a way similar to series expansions, appear in approximations of certain functions. In a region where the approximation for the continued fraction converges, the values of f n−1 and f n for sufficiently large n do not differ much. One can therefore use the following truncation criterion . If for a given $$\varepsilon \ll 1$$ the inequality

holds, then f n is a sufficiently good approximation of f .

Incomplete Gamma Function

The incomplete gamma function is defined for a  > 0 by the expression

(D.5.1)

It can be expressed as a series expansion

(D.5.2)

The sum converges quickly for x  <  a + 1. One uses the right-hand, not the middle form of ( D.5.2 ), since the ratio of the two gamma functions reduces to

In the region x  >  a + 1 we use the continued fraction

(D.5.3)

Fig. D.4:

The incomplete gamma function. With increasing a the graphs P ( a ,  x ) move to the right.

The method Gamma.incompleteGamma yields the incomplete gamma function. It is shown in Fig.  D.4 for several values of a . From the figure one sees immediately that

(D.5.4)

(D.5.5)

Incomplete Beta Function

The incomplete beta function is defined for a  > 0, b  > 0 by the relation

(D.6.1)

The function obeys the symmetry relation

(D.6.2)

The expression ( D.6.1 ) can be approximated by the following continued fraction:

(D.6.3)

with

The approximation converges quickly for

(D.6.4)

If this requirement is not fulfilled, then 1 − x is greater than the right-hand side of ( D.6.4 ). In this case one computes I 1−x as a continued fraction and then uses ( D.6.2 ).

The method Gamma.incompleteBeta computes the incomplete beta function. In Fig.  D.5 it is s displayed for various values of the parameters a and b . Regardless of these parameters one has

(D.6.5)

Java Class and Example Program

Java Class for for the Computation of the Gamma Function and Related Functions

Gamma contains all methods mentioned in this Appendix.

Fig. D.5:

The incomplete beta function. With increasing a the curves I x ( a ,  b ) move further to the right.

Example Program D.1:

The class FunctionsDemo demonstrates not only the methods of Appendix C but also all methods mentioned in the present AppendixThe user first selects a family of functions and the a function from that family. Next the parameters, needed in the chosen case, are entered. After the Go button is clicked, the function value is computed and displayed.

Utility Programs

Numerical Differentiation

The derivative d f ( x )∕d x of a function f ( x ) at the point x is given by the limit

Obviously one can approximate f ′( x ) by

for a small finite value of h . In fact, for this the symmetrical difference ratio

(E.1.1)

is more appropriate. This can be seen from the Taylor expansions at the point x for f ( x + h ) and f ( x − h ), which give

in which the leading additional term is already quadratic in h . Nevertheless, the choice of h is still critical, since for very small values of h there occur large rounding errors, and for larger values the approximation may not be valid.

One can compute δ ( h ) for a monotonic sequence of values $$h = h_{0},h_{1},h_{2},$$ … . If the sequence δ ( h ₀ ), δ ( h ₁ ), … is monotonic (rising or falling) then this is a sign of convergence of the series to f ′( x ). According to RUTISHAUSER [ 27 ], from the series δ ( h ₀ ), δ ( h ₁ ) one can also obtain others that converge more quickly. The method was modeled after the ROMBERG procedure [ 28 ] for numerical integration. Starting from h ₀  =  a one first chooses the sequence

sets

and computes the additional quantities T m (k)

Arranging the quantities $$T_{m}^{(k)}$$ in the form of a triangle,

the first column contains the sequence of our original difference ratios. Not only does $$T_{0}^{(k)}$$ converge to f ′( x ), but one has in general

The practical significance of the procedure is based on the fact that the columns on the right converge particularly quickly.

In the class AuxDer, the sequence $$T_{0}^{(0)}$$ , … , $$T_{0}^{(9)}$$ is computed starting from a  = 1. If it is not monotonic, then a is replaced by a ∕10 and a new sequence is computed. After 10 tries without success the procedure is terminated. If, however, a monotonic sequence is found, the triangle scheme is computed and T ₉ ⁽⁰⁾ is given as the best approximation for f ′( x ). The class AuxDer is similar to the program of KOELBIG [ 29 ] with the exception of minor changes in the termination criteria.

This program requires considerable computing time. For well behaved functions it is often sufficient to replace the differential ratio by the difference ratio ( E.1.1 ). To compute second derivatives the procedure of difference ratios is extended correspondingly. The classes AuxDri, AuxGrad and AuxHesse therefore operate on the basis of difference ratios.

Numerical Determination of Zeros

Computing the quantile x P of a distribution function F ( x ) for a given probability P is equivalent to determining the zero of the function

(E.2.1)

We treat the problem in two steps. In the first step we determine an interval ( x ₀ ,  x ₁ ) that contains the zero. In the second step we systematically reduce the interval such that its size becomes smaller than a given value $$\varepsilon$$ .

In the first step we make use of the fact that k ( x ) is monotonic, since f ( x ) is monotonic. We begin with initial values for x ₀ and x ₁ . If f ( x ₀ ) ⋅  f ( x ₁ ) < 0, i.e., if the function values have different signs, then the zero is contained within the interval. If this is not the case, then we enlarge the interval in the direction where the function has the smallest absolute value, and repeat the procedure with the new values of ( x ₀ ,  x ₁ ).

For localizing the zero within the initial interval ( x ₀ ,  x ₁ ) we use a comparatively slow but absolutely reliable procedure. The original interval is divided in half and replaced by the half for which the end points are of the opposite sign. The procedure is repeated until the interval width decreases below a given value.

This technique is implemented in the class AuxZero . It is also employed in several methods for the computation of quantiles in StatFunct in a direct way, i.e., without a call to AuxZero . An example for the application of AuxZero is the class E1MaxLike , see also Example Program 7.1.

Interactive Input and Output Under Java

Java programs usually are event driven , i.e., while running they react to actions by the user. Thus an interaction between user and program is enabled. Its detailed design depends on the problem at hand and also on the user’s taste. For our Example Programs four utility programs may suffice to establish simple interactions. They are explained in Fig.  E.1 .

It shows a screen window produced by the class DatanFrame . In its simplest form it consists only of a frame, a title line (here ‘Example for the creation of random numbers’) and an output region, into which the user’s output can be written. The method DatanFrame.add allows to add additional elements below the title line which will be arranged horizontally starting from the left. In Fig.  E.1 these are an input group , a radio-button group and a go button . The input group is created by the class AuxJInputGroup and the radio-button group by AuxJRButtonGroup . Both (as well as DatanFrame ) make use of the standard Java Swing classes. The go button is directly created by a standard class. The input group itself is composed of an arbitrary number of number-input regions , arranged vertically one below the other, which are created by the class AuxJNumberInput . The detailed usage of these classes is summarized in the online documentation. we also recommend to study the source code of some of our Example Programs.

Fig. E.1:

A window of the type DatanFrame with elements for interactive input, for starting the program, and for alphanumeric output of results.

Java Classes

AuxDer computes the derivative of a function using the Rutishauser method.

AuxDri computes the matrix A of derivatives required by LsqNon and LsqMar.

AuxGrad computes the gradient of a function at a given point.

AuxHesse computes the Hessian matrix of a function at a given point.

AuxZero finds the zero of a monotonic function.

DatanFrame creates a screen window with possibilities for interactive input and output.

AuxJInputGroup creates an input group within a screen window.

AuxJNumberInput creates a number-input region within an input group.

AuxJRButtonGroup creates a radio-button group within a screen window.

The Graphics Class DatanGraphics

Introductory Remarks

The graphical display of data and of curves of fitted functions has always been an important aid in data analysis. Here we present the class DatanGraphics comprising methods which produce graphics in screen windows and/or postscript files. We distinguish control, transformation, drawing, and auxilliary methods. All will be described in detail in this Appendix and there usage will be explained in a number of Example Programs. For many purposes, however, it is sufficient to use one of only five classes which, in turn, resort to DatanGraphics. These classes, by a single call, produce complete graphical structures; they are listed at the beginning of Sect.  F.8 .

Graphical Workstations: Control Routines

As mentioned, a plot can be output either as in the form of a screen window or as a file in postscript format. The latter is easily embedded in digital documents or directly printed on paper, if necessary after conversion to another format such as pdf, by the use of a freely available program. For historical reasons we call both the screen window and the postscript file a graphics workstation .

The method DatanGraphics.openWorkstation “opens” a screen window or a file or both, i.e., it initializes buffers into which information is written by methods mentioned later. Only after the method DatanGraphics.closeWorkstation has been called, is the window presented on the screen and/or is the postscript file made available. In this way several graphics can be produced one after another. There windows can coexist on the screen. They be changed in size using the computer mouse; but their contents is not alterable.

Coordinate Systems, Transformations and Transformation Methods

Coordinate Systems

World Coordinates (WC)

Figure  F.1 shows a plot made by DatanGraphics. Let us imagine for a moment that all of the graphical structures, including text, physically exist, e.g., that they are made out of bent wire. The coordinate system in which this wire structure is described is called the world coordinate system (WC). The coordinates of a point in the WC are denoted by ( X ,  Y ).

Fig. F.1:

Simple example of a plot produced with DatanGraphics.

Computing Coordinates (CC)

If we consider the axes in Fig.  F.1 , we note that the axes designated with x and y have about the same length in world coordinates, but have very different lengths in terms of the numbers shown. The figure shows a plot of a function

Each point ( x ,  y ) appears at the point ( X ,  Y ). We call the coordinate system of the ( x ,  y ) points the computing coordinate system (CC). The transformation between WC and CC is given by Eq. ( F.3.1 ).

Device Coordinates (DC)

From the (fictitious) world coordinate system, the plot must be brought onto the working surface of a graphics device (terminal screen or paper). We call the coordinates ( u ,  v ) on this surface the device coordinates (DC).

Linear Transformations: Window – Viewport

The concepts defined in this section and the individual transformations are illustrated in Fig.  F.2 .

Fig. F.2:

The various coordinate systems. Above : window in computing coordinates. Middle : viewport ( small rectangle ) and window ( large rectangle ) in world coordinates. Below : preliminary viewport ( dashed rectangle ), final adjusted viewport ( small rectangle ), and border of the display surface ( large rectangle ) in device coordinates. The mappings from computing coordinates to world coordinates and from world coordinates to device coordinates are indicated by dotted lines .

Let us assume that the computing coordinates in x cover the range

The corresponding range in world coordinates is

A linear transformation x  →  X is therefore defined by

(F.3.1)

The transformation for y  →  Y is defined in a corresponding way. One speaks of the mapping of the window in computing coordinates CC,

(F.3.2)

onto the viewport in world coordinates WC,

(F.3.3)

The mapping is in general distorted , i.e., a square in CC becomes a rectangle in WC. It is undistorted only if the aspect ratios of the window and viewport are equal,

(F.3.4)

The user of DatanGraphics computes values first in CC that are to be plotted. By specifying the window and viewport he or she defines the mapping into WC. A possible linear distortion is perfectly acceptable in this step, since it provides a simple means of changing the scale.

Next a mapping onto the physically implemented device coordinates DC must be done. It can of course be defined by again providing a window (in WC) and a viewport (in DC). One wants, however, to avoid an additional distortion. We define, therefore, a viewport

(F.3.5)

and a window

(F.3.6)

The mapping from the window ( F.3.6 ) onto the viewport ( F.3.5 ) is only carried out if both have the same aspect ratio. Otherwise the viewport ( F.3.6 ) is reduced symmetrically in width to the right and left or in height symmetrically above and below such that the viewport

(F.3.7)

has the same aspect ratio as the window ( F.3.6 ). In this way a distortion free mapping is defined between the two.

Transformation Methods

The user of DatanGraphics must define the transformations between the various coordinate systems by calling the appropriate routines. The transformations are then applied when drawing a graphical structure without any further intervention.

Transformation CC → WC

This transformation is defined by calling the following two methods. Datan Graphics.setWindowInComputingCoordinates sets the window in computing coodinates, DatanGraphics.setViewportInWorldCoordinates sets the viewport in world coordinates.

Transformation WC → DC

This transformation is defined by calling the following two methods. DatanGraphics.setWindowInWorldCoordinates sets the window in world coordinates. DatanGraphics.setFormat defines the temporary viewport in device coordinates. Into that the final view port is fitted so that the width-to-height ratio is the same as for the window in world coordinates. If DatanGraphics.setFormat is not called at all, the format is taken to be A5 landscape. If the workstation is a screen window, then only the width-to-height ratio is taken into account. In the case of a postscript file the absolute size in centimeters is valid only if a plot of that size will fit on the paper in the printer. Otherwise the plot is demagnified until it just fits. In both cases the plot is centered on the paper. A call to the method DatanGraphics.setStandardPaperSizeAndBorders informs the program about the paper size. If it is not called, then A4 is assumed with a margin of 5 mm on all 4 sides.

In most cases, having defined the transformations in this way, the user will be interested only in computing coordinates.

Clipping

The graphical structures are not completely drawn under certain circumstances. They are truncated if they extend past the boundary of the so-called clipping region . The structures are said to be clipped . For polylines, markers, data points, and contour lines (Sect.  F.5 ) the clipping region is the window in computing coordinates; for text and graphics utility structures the clipping region is the window in world coordinates. These regions can can be set explicitely using ; and DatanGraphics.setBigClippingWindow , respectively.

Drawing Methods

Colors and Line Widths

The methods mentioned up to this point carry out organizational tasks, but do not, however, produce any graphical structures on the workstation. All of the graphical structures created by DatanGraphics consist of lines. The lines possess a given color and width. The selection of these two attributes is done as follows. A pair of properties (color, linewidth) is assigned to each of set of 8 color indices . The set of 8 is different for screen window and postscript file. With the method DatanGraphics.chooseColor the user selects one particular color index. That then is valid until another color index is chosen. The user may assign his own choice of color and line width to a color index by the methods DatanGraphics.setScreenColor and/or DatanGraphics.setPSColor. For the background of the screen window (the standard is blue) another color can be chosen with DatanGraphics.setScreenBackground. For the postscript file the background is always transparent.

Polylines

The concepts of a polyline and polymarker have been introduced for particularly simple graphics structures. A polyline defines a sequence of line segments from the point ( x ₁ ,  y ₁ ) through the points ( x ₂ ,  y ₂ ), ( x ₃ ,  y ₃ ), … to the point ( x n ,  y n ). A polyline is drwan with the method DatanGraphics.drawPolyline.

A polymarker marks a plotting point with a graphical symbol. The polymarkers available in DatanGraphics are shown in Fig.  F.3 .

Fig. F.3:

Polymarkers.

One can clearly achieve an arbitrarily good approximation of any graphical structure by means of polylines. For example, graphs of functions can be displayed with polylines as long as the individual points are sufficiently close to each other.

Sometimes one wants to draw a polyline not as a solid line but rather dashed , dotted , or dot-dashed . That is achieved by the method DatanGraphics.drawBrokenPolyline.

Polymarkers are especially suitable for marking data points. If a data point has error bars, then one would like to indicate these errors in one or both coordinates by means of error bars . In certain circumstances one would even like to show the complete covariance ellipse. This task is performed by the method DatanGraphics.drawDatapoint. Examples are shown in Fig.  F.4 .

Fig. F.4:

Example for plotting data points.

An error bar in the x direction is only drawn if σ x  > 0, and in the y direction only if σ y  > 0. The covariance ellipse is only draw if σ x  > 0, σ y  > 0, and cov( x ,  y ) ≠ 0. Error bars are not drawn if they would lie completely within the polymarker itself. If part of the structure falls outside of the CC window ( F.3.2 ), then this part is not drawn.

Histogram

The method DatanGraphics.drawHistogram displays data in the form of a histogram.

Contour Lines

A function f  =  f ( x ,  y ) defines a surface in a three dimensional ( x ,  y ,  f ) space. One can also get an idea of the function in the ( x ,  y ) plane by marking points for which f ( x ,  y ) is equal to a given constant c . The set of all such points forms the contour line f ( x ,  y ) =  c . By drawing a set of such contour lines f ( x ,  y ) =  c ₁ ,  c ₂ ,  … one obtains (as with a good topographical map) a rather good impression of the function.

Naturally it is impossible to compute the function for all points in the ( x ,  y ) plane. We restrict ourselves to a rectangular region in the ( x ,  y ) plane, usually the window in computing coordinates, and we break it into a total of N  =  n x n y smaller rectangles. The corner points of these smaller rectangles have x coordinates that are neighboring values in the sequence

The y coordinates of the corner points are adjacent points of the sequence

In each rectangle the contour line is approximated linearly. To do this one considers the function f ( x ,  y ) − c at the four corner points. If the function is of a different sign at two corner points that are the end points of an edge of the rectangle, then it is assumed that the contour lines intersect the edge. The intersection point is computed with linear interpolation. If the intersection points are on two edges of the rectangle, then they are joined by a line segment. If there are intersection points on more than two edges, then all pairs of such points are joined by line segments.

Clearly the approximation of the contour lines by line segments becomes better for a finer division into small rectangles. With a finer division, of course, the required computing time also becomes longer.

The method DatanGraphics.drawContour computes and draws a contour line. An example of a function represented by contour lines is shown in Fig.  F.5 .

Fig. F.5:

Contour lines $$f(x,y) = -0.9,-0.8,\ldots,0.8,0.9$$ of the function $$f(x) =\sin (x + y)\cos ([x - y]/2)$$ in the ( x ,  y ) plane.

Utility Methods

With the few methods described up to this point, a great variety of complicated plots can be produced. By using the methods of this section and the next, the tasks are made easier for the user, since they help to create graphical structures typically used in conjunction with the plots of data analysis, such as axes, coordinate crosses, and explanatory text.

Frames

The methode DatanGraphics.drawFrame draws a frame around the plotted part of world coordinate system, i.e., the outer frame of the plots reproduced here. The metod DatanGraphics.drawBoundary, on the other hand, draws a frame around the window of the cumputing coordinate system.

Scales

The method DatanGraphics.drawScaleX draws a scale in x direction. Ticks appear at the upper and lower edge of the CC window pointing to the inside of the window. Below the lower edge numbers appear, marking some of these ticks. It is recommended to first call the method DatanGraphics.drawBoundary, to mark the edges themselves by lines. In addition an arrow with text can be drawn, showing in the direction of increasing x values. The method DatanGraphics.drawScaleY performs analogous tasks for an axis in y direction.

The creation of axis divisions and labels with these methods is usually done automatically without intervention of the user. Sometimes, however, the user will want to influence these operations. This can be done by using the method DatanGraphics.setParametersForScale. A call to this method influences only that scale which is generated by the very next call of DatanGraphics.drawScaleX or DatanGraphics.drawScaleY, respectively.

Coordinate Cross

The method DatanGraphics.drawCoordinateCross draws a coordinate cross in the computing coordinate system. The axes of that system appear as broken lines inside the CC window.

Text Within the Plot

Explanatory text makes plots easier to understand. The methods in this section create text superimposed on a plot which can be placed at any location.

The text must be supplied by the user as a character string. Before this text is translated into graphics characters, however, it must first be encoded. The simple encoding system used here allows the user to display simple mathematical formulas. For this there are three character sets : Roman, Greek, and mathematics, as shown in Table  F.1 . The character set is selected by control characters . These are the special characters

@ for Roman,

& for Greek,

% for mathematics.

Table F.1:

The various character sets for producing text.

Control characters

Control characters

Roman

Greek

Math

Roman

Greek

Math

Input

@

&

%

Input

@

&

%

A

A

A(ALPHA)

Ä

a

a

α (alpha)

ä

B

B

B(BETA)

B

b

b

β (beta)

b

C

C

X(CHI)

c

c

χ (chi)

c

D

D

Δ (DELTA)

Δ

d

d

δ (delta)

d

E

E

E(EPSILON)

E

e

e

ε (epsilon)

e

F

F

Φ (PHI)

F

f

f

$$\varphi$$ (phi)

f

G

G

Γ (GAMMA)

≠

g

g

γ (gamma)

g

H

H

H(ETA)

H

h

h

η (eta)

h

I

I

I(IOTA)

∫

i

i

ι (iota)

i

J

J

I(IOTA)

J

j

j

ι (iota)

j

K

K

K(KAPPA)

K

k

k

κ (kappa)

k

L

L

Λ (LAMBDA)

 | 

l

l

λ (lambda)

l

M

M

M(MU)

±

m

m

μ (mu)

m

N

N

N(NU)

N

n

n

ν (nu)

n

O

O

Ω (OMEGA)

Ö

o

o

ω (omega)

ö

P

P

Π (PI)

Ö

p

p

π (pi)

p

Q

Q

Θ (THETA)

Q

q

q

$$\vartheta$$ (theta)

q

R

R

R(RHO)

∘

r

r

ρ (rho)

r

S

S

Σ (SIGMA)

ß 

s

s

σ (sigma)

s

T

T

T(TAU)

t

t

τ (tau)

t

U

U

O(OMICRON)

Ü

u

u

o(omicron)

ü

V

V

Ü

v

v

v

W

W

Ψ (PSI)

√

w

w

ψ (psi)

w

X

X

Ξ (XI)

X

x

x

ξ (xi)

x

Y

Y

Υ (UPSILON)

Å

y

y

υ (upsilon)

y

Z

Z

Z(ZETA)

Z

z

z

ζ (zeta)

z

 ∼ 

 ∼ 

 ∼ 

-

−

−

−

!

!

!

!

=

 = 

 = 

$

$

$

$

{

{

{

*

*

#

×

}

}

}

(

(

↑

 ← 

 | 

 | 

 | 

)

)

↓

 → 

[

[

&

[

+

+

+

+

]

]

@

]

‘

‘

‘

‘

1

1

1

1

:

:

:

:

2

2

2

2

;

;

;

;

3

3

3

3

’

’

‘

’

4

4

4

4

<

 < 

 ⊂ 

 ≤ 

5

5

5

5

>

 > 

 ⊃ 

 ≥ 

6

6

6

6

?

?

§

 ∼ 

7

7

7

7

,

,

,

,

8

8

8

8

.

.

.

.

9

9

9

9

/

/

∖

%

0

0

0

0

A control character in the text string causes all of the following characters to be produced with the corresponding character set, until another control symbol appears. The default character set is Roman.

In addition there exist the following positioning symbols:

$$\bigwedge$$ for superscript (exponent),

_​_ for subscript (index),

# for normal height,

" for backspace.

All characters appear at normal height as long as no positioning symbol has appeared. One can move a maximum of two steps from normal height, e.g., $$A_{\alpha _{\beta }},A_{{\alpha }^{\beta }}$$ . The positioning symbols ˆ and remain in effect until the appearance of a # . The symbol " acts only on the character following it. This then appears over the previous character instead of after it. In this way one obtains, e.g., $$A_{\alpha }^{\beta }$$ instead of $$A_{\alpha }^{\:\:\beta }$$ .

The method DatanGraphics.drawCaption draws a caption, centered slightly below the upper edge of the plotted section of the world coordinate system.

Sometimes the user wants to write text at a certain place in the plot, e.g., next to an individual curve or data point, and also to choose the text size. This is made possible by the method DatanGraphics.drawText.

Java Classes and Example Programs

Java Classes Poducing Graphics

DatanGraphics contains the methods mentioned in this Appendix.

GraphicsWithHistogram produces a complete plot with a histogram (an Example Program is E2Sample).

GraphicsWith2DScatterDiagram produces a complete plot with a two-dimensional scatter diagram (an Example Program is E3Sample).

GraphicsWithHistogramAndPolyline produces a complete graphics with a histogram and a polyline (an Example Program isE6Gr).

GraphicsWithDataPointsAndPolyline produces a complete plot with data points and one polyline (an Example Program is E7Gr).

GraphicsWithDataPointsAndMultiplePolylines produces a complete plot with data points and several polylines (an Example Program is E8Gr).

Example Program F.1:

The class E1Gr demonstrates the use of the following methods of the class DatanGraphics: openWorkstation, closeWorkstation, setWindowInComputingCoordinates, setViewportInWorldCoordinates, setWindowInWorldCoordinates, setFormat, drawFrame, drawBoundary, chooseColor, drawPolyline, drawBrokenPolyline, drawScaleX, drawScaleY, drawCaption, drawTextThe program generates the simple plot of Fig.  F.1 . It opens the workstation and defines the different coordinate systems. The outer frame is drawn (enclosing the section of the world coordinate system to be displayed) and the inner frame (the boudary of the computing coordinate system). Next, the lettered scales for abswcissa and ordinate are produced as is a caption for the plot. Now, the color index is changed. In a short loop a total of 201 coordinate pairs ( x i ,  y i ) are computed with $$x_{i} = -10,\,-9.9,\,-9.8,\,\ldots,\,10$$ and y i  =  f ( x i ). The function f ( x ) is the probability density of the standardized normal distribution. A polyline, defined by these pairs is drawn. In a second loop the points for a polyline are computed which correspond to a normal distribution with with mean a  = 2 and standard deviation σ  = 3. That polyline is represented as a broken line. Finally, two short straight lines are displayed in the upper left corner of the plot (one as a solid line and one as a dashed line). To the right each of these polylines a short text is displayed, indicating the parameters of the Gaussians displayed as solid and dashed curves, respectively. Before termination of the program the workstation is closed.

Example Program F.2:

The class E2Gr demonstrates the use of the method DatanGraphics.drawMarkThe short program generates the plot of Fig.  F.3 , showing the different polymarkers, which can be drawn with DatanGraphics.drawMark.

Example Program F.3:

The class E3Gr demonstrates the use of the method DatanGraphics.drawDatapointThe program produces the plot of Fig.  F.4 , which contains examples for the different ways to present data points with errors.

Fig. F.6:

Four versions of the same plot with different types of scales.

Fig. F.7:

A plot generated with GraphicsWithHistogramAndPolyline containing a histogram and a polyline.

Example Program F.4:

The class E4Gr demonstrates the use of the method DatanGraphics.drawContourA window of computing coordinates − π  ≤  x  ≤  π , − π  ≤  y  ≤  π and a square viewport in world coordinates are selected. After creating scales and the caption, input parameters for DatanGraphics.drawContour are prepared. Next by successive calls of of this method in a loop, contours of the function $$f(x,y) =\sin (x + y)\cos ((x - y)/2)$$ are drawn. The result is a plot corresponding to Fig.  F.5 .

Suggestions: Extend the program such that the parameters ncont and nstep defining the number of contours and the number of intervals in x and y can be set interactively by the user. Study the changes in the plot resulting from very small values of nstep .

Example Program F.5:

The class E5Gr demonstrates the methods DatanGraphics.setParametersForScale and DatanGraphics.drawCoordinateCrossThe program generates the plots shown in Fig.  F.6 . It contains four plots which differ only by the design of their scales. The plots are generated in a loop where different viewports in world coordinates are chosen in each step, so that the plots correspond to the upper-left, upper-right, lower-left, and lower-right quadrant of the window in world coordinates. For the upper-left plot the default values for the scale design are used. In the upper-right plot the number of ticks and the lettering of the scale is predefined. In the lower-left plot the size of the symbols used in the lettering of the scales is changed. In the lower-right plot the numbers are written in exponential notation. All plots contain a coordinate cross, which is generated by calling DatanGraphics.drawCoordinateCross and a curve corresponding to a Gaussian.

Fig. F.8:

A plot with data points and a polyline generated by calling GraphicsWith DataPointsAndPolyline.

Example Program F.6:

The class E6Gr demonstrates the use of the class GraphicsWithHistogramAndPolylineThe program first sets up a histogram which for each bin k contains the Poisson probability f ( k ;  λ ) for the parameter λ  = 10. Next, points on a polyline are computed corresponding to the probability density of a normal distribution with mean λ and variance λ . Finally the text strings for the plot are defined and the complete plot is displayed by a call of GraphicsWithHistogramAndPolyline (Fig.  F.7 ).

Example Program F.7:

The class E7Gr demonstrates the use of the class GraphicsWithDataPointsAndPolylineFirst, by calling DatanRandom.line, data points are generated which lie on a straight line $$y = at + b$$ within the simulated errors. Next, the errors to be presented in the directions of the horizontal and vertical axes and their covariance are defined. The latter two quantities in our example are equal to zero. The polyline defining the straight line consists of only two points. Their computation is trivial. After the definition of the axis labels and the caption, the plot is displayed by calling GraphicsWithDataPointsAndPolyline (Fig.  F.8 ).

Example Program F.8:

The class E8Gr demonstrates the use of the class GraphicsWithDataPointsAndMultiplePolylines

The program generates 21 data points which lie within the simulated errors on a Gaussian curve with zero mean and standard deviation σ  = 1, and which span the abscissa region − 3 ≤  x  ≤ 3. Next, points on three polylines are computed corresponding to Gaussian curves with means of zero and standard deviations σ  = 0. 5, σ  = 1, and σ  = 1. 5. The polylines span the abscissa region − 10 ≤  x  ≤ 10. They are displayed in different colors. One polyline is shown as a continuous line, the other two as dashed lines. Three plots are produced: The first displays only the data points, the second only the polylines, and the third shows the data points together with the polylines. In this way the automatic choice of the scales in the different cases is demonstrated.

Problems, Hints and Solutions, and Programming Problems

Problems

Problem 2.1:

Determination of Probabilities through Symmetry Considerations

There are n students in a classroom. What is the probability for the fact that at least two of them have their birthday on the same day? Solve the problem by working through the following questions:

(a)

What is the number N of possibilities to distribute the n birthdays over the year (365 days)?

(b)

How large is the number N ′ of possibilities for which all n birthdays are different?

(c)

How large then is the probability P diff that the birthdays are different?

(d)

How large finally is the probability P that at least two birthdays are not different?

Problem 2.2:

Probability for Non-exclusive Events

The probabilities P ( A ), P ( B ), and P ( AB ) ≠ 0 for non-exclusive events A and B are given. How large is the probability P ( A + B ) for the observation of A or B ? As an example compute the probability that a playing card which was drawn at random out of a deck of 52 cards is either an ace or a diamond.

Problem 2.3:

Dependent and Independent Events

Are the events A and B that a playing card out of a deck is an ace or a diamond independent

(a)

If an ordinary deck of 52 cards is used,

(b)

If a joker is added to the deck?

Problem 2.4:

Complementary Events

Show that $$\bar{A}$$ and $$\bar{B}$$ are independent if A and B are independent. Use the result of Problem 2.2 to express $$P(\overline{AB})$$ by P ( A ), P ( B ), and P ( AB ).

Problem 2.5:

Probabilities Drawn from Large and Small Populations

A container holds a large number ( > 1000) of coins. They are divided into three types A , B , and C , which make up 20, 30, and 50  % of the total.

(a)

What are the probabilities P ( A ), P ( B ), P ( C ) of picking a coin of type A , B , or C if one coin is taken at random? What are the probabilities P ( AB ), P ( AC ), P ( BC ), P ( AA ), P ( BB ), P ( CC ), P (2 identical coins), P (2 different coins) for picking 2 coins?

(b)

What are the probabilities if 10 coins (2 of type A , 3 of type B , and 5 of type C ) are in the container?

Problem 3.1:

Mean, Variance, and Skewness of a Discrete Distribution

The throwing of a die yields as possible results x i  = 1, 2,  … , 6. For an ideally symmetric die one has $$p_{i} = P(x_{i}) = 1/6$$ , i  = 1, 2,  … , 6. Determine the expectation value $$\widehat{x}$$ , the variance σ ² ( x ) =  μ ₂ , and the skewness γ of the distribution,

(a)

For an ideally symmetric die,

(b)

For a die with

Problem 3.2:

Mean, Mode, Median, and Variance of a Continuous Distribution

Consider the probability density f ( x ) of a triangular distribution of the form shown in Fig.  G.1 , given by

Fig. G.1:

Triangular probability density.

Determine the mean $$\widehat{x}$$ , the mode x m , the median x 0. 5 , and the variance σ ² of the distribution. For simplicity choose c  = 0 (which corresponds to the substitution of x by $$x^{\prime} = x - c$$ ). Give explicit results for the symmetric case $$a = -b$$ and for the case $$a = -2b$$ .

Problem 3.3:

Transformation of a Single Variable

In Appendix D it is shown that

Use the transformation $$y = x/\sigma$$ to show that

Problem 3.4:

Transformation of Several Variables

A “normal distribution of two variables” (see Sect.  5.​10 ) can take the form

(a)

Determine the marginal probability densities f ( x ), f ( y ) by using the results of Problem 3.3.

(b)

Are x and y independent?

(c)

Transform the distribution f ( x ,  y ) to the variables

(The u , v coordinate system has the same origin as the x , y coordinate system, but it is rotated with respect to the latter by an angle ϕ .)

Hint: Show that the transformation is orthogonal and use ( 3.​8.​12 ).

(d)

Show that u and v are independent variables only if ϕ  = 0 ∘ , 90 ∘ , 180 ∘ , 270 ∘ or if $$\sigma _{x} =\sigma _{y} =\sigma$$ .

(e)

Consider the case $$\sigma _{x} =\sigma _{y} =\sigma$$ , i.e.,

Transform the distribution to polar coordinates r , ϕ , determine the marginal probability densities g ( r ) and g ( ϕ ) and show that r and ϕ are independent.

Problem 3.5:

Error Propagation

The period T of a pendulum is given by $$T = 2\pi \sqrt{\ell/g}$$ . Here ℓ is the length of the pendulum and g is the gravitational acceleration. Compute g and Δ g using the measured values ℓ  = 99. 8 cm, Δ ℓ  = 0. 3 cm, T  = 2. 03 s, Δ T  = 0. 05 s and assuming that the measurements of ℓ and T are uncorrelated.

Problem 3.6:

Covariance and Correlation

We denote the mass and the velocity of an object by m and v and their measurement errors by $$\varDelta m = \sqrt{{\sigma }^{2 } (m)}$$ and $$\varDelta v = \sqrt{{\sigma }^{2 } (v)}$$ . The measurements are assumed to be independent, i.e., cov( m ,  v ) = 0. Furthermore, the relative errors of measurement are known, i.e.,

(a)

Consider the momentum p  =  mv and the kinetic energy $$E = \frac{1} {2}m{v}^{2}$$ of the object and compute σ ² ( p ), σ ² ( E ), cov( p ,  E ), and the correlation ρ ( p ,  E ). Discuss ρ ( p ,  E ) for the special cases a  = 0 and b  = 0. Hint: Form the vectors x  = ( m ,  v ) and y  = ( p ,  E ). Then approximate y  =  y ( x ) by a linear transformation and finally compute the covariance matrix.

(b)

For the case where the measured values of E , p and the covariance matrix are known, compute the mass m and its error by error propagation. Use the results from (a) to verify your result. Note that you will obtain the correct result only if cov( p ,  E ) is taken into account in the error propagation.

Problem 5.1:

Binomial Distribution

(a)

Prove the recursion formula

(b)

It may be known for a certain production process that a fraction q  = 0. 2 of all pieces produced are defective. This means that in 5 pieces produced the expected number of non-defective pieces is $$np = n(1 - q) = 5 \cdot 0.8 = 4$$ . What is the probability P ₂ and the probability P ₃ that at most 2 or at most 3 pieces are free from defects? Use relation (a) to simplify the calculation.

(c)

Determine the value k m for which the binomial distribution is maximum, i.e., k m is the most probable value of the distribution. Hint: Since W k n is not a function of a continuous variable k , the maximum cannot be found by looking for a zero in the derivative. Therefore, one has to study finite differences $$W_{k}^{n} - W_{k-1}^{n}$$ .

(d)

In Sect.  5.​1 the binomial distribution was constructed by considering the random variable $$\mathsf{x} =\sum _{ i=1}^{n}\mathsf{x}_{i}$$ . Here x i was a random variable that took only the values 0 and 1 with the probabilities $$P(\mathsf{x}_{i} = 1) = p$$ and $$P(\mathsf{x}_{i} = 0) = q$$ .

The binomial distribution

was then obtained by considering in detail the probability to have k cases of x i  = 1 in a total of n observations of the variable x i . Obtain the binomial distribution in a more formal way by constructing the characteristic function $$\varphi _{\mathsf{x}_{i}}$$ of the variable x i . From the n th power of $$\varphi _{\mathsf{x}_{i}}$$ you obtain the characteristic function of x . Hint: Use the binomial theorem ( B.6 ).

Problem 5.2:

Poisson Distribution

In a certain hospital the doctor on duty is called on the average three times per night. The number of calls may be considered to be Poisson distributed. What is the probability for the doctor to have a completely quiet night?

Problem 5.3:

Normal Distribution

The resistance R of electrical resistors produced by a particular machine may be described by a normal distribution with mean R m and standard deviation σ .

The production cost for one resistor is C , and the price is 5 C if R  =  R ₀ ± Δ ₁ , and 2 C if $$R_{0} -\varDelta _{2} < R < R_{0} -\varDelta _{1}$$ or $$R_{0} +\varDelta _{1} < R < R_{0} +\varDelta _{2}$$ . Resistors outside these limits cannot be sold.

(a)

Determine the profit P per resistor produced for R m  =  R ₀ , $$\varDelta _{1} = a_{1}R_{0}$$ , $$\varDelta _{2} = a_{2}R_{0}$$ , σ  =  bR ₀ . Use the distribution function ψ ₀ .

(b)

Use Table  I.2 to compute the numerical values of P for a ₁  = 0. 01, a ₂  = 0. 05, b  = 0. 05.

(c)

Show that the probability density ( 5.​7.​1 ) has points of inflection (i.e., second derivative equal to zero) at x  =  a ± b .

Problem 5.4:

Multivariate Normal Distribution

A planar xy coordinate system is used as target. The probability to observe a hit in the plane may be given by the normal distribution of Problem 3.4 (e). Use the result of that problem to determine

(a)

The probability P ( R ), to observe a hit within a given radius R around the origin,

(b)

The radius R , within which a hit is observed with a given probability. Compute as a numerical example the value of R for P  = 90  % and σ  = 1.

Problem 5.5:

Convolution

(a)

Prove the relation ( 5.​11.​11 ). Begin with ( 5.​11.​9 ) and use the expression ( 5.​11.​10 ) for f y ( y ). In the intervals 0 ≤  u  < 1 and 2 ≤  u  < 3 relations (5.11.10a) and (5.11.10b) hold, since in these intervals one always has y  < 1 and y  > 1, respectively. In the interval 1 ≤  u  < 2 the resulting distribution f ( u ) must be constructed as sum of two integrals of the type ( 5.​11.​9 ) of which each contains one of the possible expression for f y ( y ). In this case particular care is necessary in the determination of the limits of integration. They are given by the limits of the intervals in which u and f y ( y ) are defined.

(b)

Prove the relation ( 5.​11.​15 ) by performing the integration ( 5.​11.​5 ) for the case that f x and f y are normal distributions with means 0 and standard deviations σ x and σ y .

Problem 6.1:

Efficiency of Estimators

Let $$\mathsf{x}_{1},\mathsf{x}_{2},\mathsf{x}_{3}$$ be the elements of a sample from a continuous population with unknown mean $$\widehat{x}$$ , but known variance σ ² .

(a)

Show that the following quantities are unbiased estimators of $$\widehat{x}$$ ,

Hint: It is simple to show that $$\mathsf{S} =\sum _{ i=1}^{n}a_{i}\mathsf{x}_{i}$$ is unbiased if $$\sum _{i=1}^{n}a_{i} = 1$$ holds.

(b)

Determine the variances σ ² ( S ₁ ), σ ² ( S ₂ ), σ ² ( S ₃ ) using ( 3.​8.​7 ) and the assumption that the elements x ₁ , x ₂ , x ₃ are independent.

(c)

Show that the arithmetic mean $$\bar{\mathsf{x}} = \frac{1} {3}\mathsf{x}_{1} + \frac{1} {3}\mathsf{x}_{2} + \frac{1} {3}\mathsf{x}_{3}$$ has the smallest variance of all estimators of the type $$\mathsf{S} =\sum _{ i=1}^{3}a_{i}\mathsf{x}_{i}$$ fulfilling the requirement $$\sum _{i=1}^{3}a_{i} = 1$$ .

Hint: Minimize the variance of $$\mathsf{S} = a_{1}\mathsf{x}_{1} + a_{2}\mathsf{x}_{2} + (1 - a_{1} - a_{2})\mathsf{x}_{3}$$ with respect to a ₁ and a ₂ . Compute this variance and compare it with the variances which you found (b).

Problem 6.2:

Sample Mean and Sample Variance

Compute the sample mean $$\bar{\mathsf{x}}$$ , the sample variance s ² , and an estimate for the variance of the sample mean $$\mathsf{s}_{\bar{\mathsf{x}}}^{2} = (1/n){\mathsf{s}}^{2}$$ for the following sample:

Use the method of Example 6.1.

Problem 6.3:

Samples from a Partitioned Population

An opinion poll is performed on an upcoming election. In our (artificially constructed) example the population is partitioned into three subpopulations, and from each subpopulation a preliminary sample of size 10 is drawn. Each element of the sample can have the values 0 (vote for party A) or 1 (vote for party B). The samples are

(a)

Use these samples to form an estimator for the result of the election and its variance $$\mathsf{s}_{\bar{\mathsf{x}}}^{2}$$ . Does this result show a clear advantage for one party?

(b)

Use these samples to determine for a much larger sample of size n the sizes n i of the subsamples in such a way that $$\widetilde{x}$$ has the smallest variance (cf. Example 6.6).

Problem 6.4:

χ ² -distribution

(a)

Determine the skewness $$\gamma =\mu _{3}{/\sigma }^{3}$$ of the χ ² -distribution using ( 5.​5.​7 ). Begin by expressing μ ₃ by λ ₃ , $$\widehat{x}$$ , and E ( x ² ).

(b)

Show that γ  → 0 for n  →  ∞ .

Problem 6.5:

Histogram

Construct a histogram from the following measured values:

Use a bin size of Δ x  = 1.

Hint: For each value draw a cross of width Δ x and height 1. In this way you do not need to order the measured values, since each cross is drawn within a bin on top of a preceding cross. In this way the bars of the histogram grow while drawing.

Problem 7.1:

Maximum-Likelihood Estimates

(a)

Suppose it is known that a random variable x follows the uniform distribution $$f(x) = 1/b$$ for 0 <  x  <  b . The parameter b is to be estimated from a sample. Show that $$\mathsf{S} =\widetilde{ b} = \mathsf{x}_{\max }$$ is the maximum-likelihood estimator of b . (Hint: This result cannot be obtained by differentiation, but from a simple consideration about the likelihood function).

(b)

Write down the likelihood equations for the two parameters a and $$\Gamma $$ of the Lorentz distribution (see Example 3.5). Show that these do not necessarily have unique solutions. You can, however, easily convince yourself that for $$\vert {\mathsf{x}}^{(j)} - a\vert \ll \Gamma $$ the arithmetic mean $$\bar{\mathsf{x}}$$ is an estimator of a .

Problem 7.2:

Information

(a)

Determine the information I ( λ ) of a sample of size N that was obtained from a normal distribution of known variance σ ² but unknown mean λ  =  a .

(b)

Determine the information I ( λ ) of a sample of size N which was drawn from a normal distribution of known mean a but unknown variance λ  =  σ ² . Show that the maximum-likelihood estimator of σ ² is given by

and that the estimator is unbiased, i.e., $$B(\mathsf{S}) = E(\mathsf{S})-\lambda = 0$$ .

Problem 7.3:

Variance of an Estimator

(a)

Use the information inequality to obtain a lower limit on the variance of the estimator of the mean in Problem 7.2 (a). Show that this limit is equal to the minimum variance that was determined in Problem 6.1 (c).

(b)

Use the information inequality to obtain a lower limit on the variance of S in Problem 7.2 (b).

(c)

Show using Eq. ( 7.​3.​12 ) that S is a minimum variance estimator with the same variance the lower limit found in (b).

Problem 8.1:

F -Test

Two samples are given:

Does sample (2) have a smaller variance than sample (1) at a significance level of α  = 5  % ?

Problem 8.2:

Student’s Test

Test the hypothesis that the 30 measurements of Problem 6.5 were drawn from a population with mean 25.5. Use a level of significance of α  = 10  % . Assume that the population is normally distributed.

Problem 8.3:

χ ² -Test for Variance

Use the likelihood-ratio method to construct a test of the hypothesis $$H_{0}{(\sigma }^{2} =\sigma _{ 0}^{2})$$ that a sample stems from a normal distribution with unknown mean a and unknown variance σ ₀ ² . The parameters are $$\boldsymbol{\uplambda } = (a,\sigma )$$ . In ω one has $${\widetilde{\boldsymbol{\uplambda }}}^{(\omega )} = (\bar{\mathsf{x}},\sigma _{0})$$ , in Ω : $${\bar{\boldsymbol{\uplambda }}}^{(\varOmega )} = (\bar{\mathsf{x}},\mathsf{s})$$ .

(a)

Form the likelihood ratio T .

(b)

Show that instead of T , the test statistic $$T^{\prime} = N{\mathsf{s}^{\prime}}^{2}/\sigma _{0}^{2}$$ can be used as well.

(c)

Show that T ′ follows a χ ² -distribution with N − 1 degrees of freedom so that the test can be performed with the help of Table  I.7 .

Problem 8.4:

χ ² -Test of Goodness-of-Fit

(a)

Determine the mean $$\widetilde{a}$$ and the variance $$\widetilde{{\sigma }}^{2}$$ for the histogram of Fig.​ 6.​1 b, i.e.,

Use the result of Example 7.8 to construct the estimates. Give the estimators explicitly as functions of n k and x k .

(b)

Perform (at a significance level of α  = 10  % ) a χ ² -test on the goodness-of-fit of a normal distribution with mean $$\widetilde{a}$$ and variance $$\widetilde{{\sigma }}^{2}$$ to the histogram. Use only those bins of the histogram for which np k  ≥ 4. Determine p k from the difference of two entries in Table  I.2 . Give a formula for p k as function of x k , Δ x , $$\widetilde{a}$$ , $$\widetilde{{\sigma }}^{2}$$ , and ψ ₀ . Construct a table for the computation of χ ² containing columns for x k , n k , np k , and $${(n_{k} - np_{k})}^{2}/np_{k}$$ .

Problem 8.5:

Contingency Table

(a)

In an immunology experiment [taken from SOKAL and ROHLF , Biometry (Freeman, San Francisco, 1969)] the effect of an antiserum on a particular type of bacteria is studied. 57 mice received a certain dose of bacteria and antiserum, whereas 54 mice received bacteria only. After some time the mice of both groups were counted and the following contingency table constructed.

Dead

Alive

Sum

Bacteria and

antiserum

13

44

57

Only bacteria

25

29

54

Sum

38

73

Total

111

Test (at a significance level of α  = 10  % ) the hypothesis that the antiserum has no influence on the survival probability.

(b)

In the computation of χ ² in (a) you will have noticed that the numerators in ( 8.​8.​1 ) all have the same value. Show that generally for 2 × 2 contingency tables the following relation holds,

Hints and Solutions

Problem 2.1

(a)

N  = 365 n  . 

(b)

(c)

(d)

Putting in numbers one obtains P  ≈ 0. 5 for n  = 23 and P  ≈ 0. 99 for n  = 57.

Problem 2.2

Problem 2.3

(a)

$$P(A) = \frac{4} {52}\,,\:P(B) = \frac{13} {52}\,,\:P(AB) = \frac{1} {52}$$ , i.e., P ( AB ) =  P ( A ) P ( B ) . 

(b)

$$P(A) = \frac{4} {53}\,,\:P(B) = \frac{13} {53}\,,\:P(AB) = \frac{1} {53}$$ , i.e., P ( AB ) ≠ P ( A ) P ( B ) . 

Problem 2.4

For A and B independent one has P ( AB ) =  P ( A ) P ( B ). Therefore,

Problem 2.5

(a)

P ( CC ) = 0. 5 ²  . 

(b)

For (a) and (b) it holds that

Problem 3.1

(a)

(b)

Problem 3.2

For $$a = -b$$ : $$\widehat{x} = 0\quad,\qquad x_{0.5} = 0\quad,{\qquad \sigma }^{2}(x) = \frac{{b}^{2}} {6} \quad.$$

For $$a=-2b$$ : $$\widehat{x}=-\frac{b} {3}=-0.33b,\quad x_{0.5}=b(\sqrt{3} - 2)=-0.27b,{\quad \sigma }^{2}(x)= \frac{7} {18}{b}^{2}.$$

Problem 3.3

Problem 3.4

(a)

(b)

Yes, since ( 3.​4.​6 ) is fulfilled.

(c)

The transformation can be written in the form

It is orthogonal, since R T R  =  I . Therefore,

(d)

For $$\phi = 9{0}^{\circ }:\cos \phi = 0$$ , sin ϕ  = 1 etc. the expression g ( u ,  v ) factorizes, i.e., $$g(u,v) = g_{u}(u)g_{v}(v)$$ .

(e)

therefore, r and ϕ are independent.

Problem 3.5

Problem 3.6

(a)

y  =  Tx , 

For a  = 0 or b  = 0: ρ  = 1. (In this case either m or v are completely determined. Therefore, there is a strict relation between E and p .) If, however, a , b ≠ 0, one has ρ ≠ 1, e.g., for a  =  b one obtains $$\rho = 3/\sqrt{10}$$ .

(b)

Problem 5.1

(a)

(b)

(c)

Using the result of (a) we obtain

Thus the probability W k n increases as long as the expression in brackets is positive, i.e.,

Since k and q are positive we have

The most probable value k m is the largest value of k for which this inequality holds.

(d)

Problem 5.2

For λ  = 3 one has $$f(0) = \frac{{\lambda }^{0}} {0!}{\mathrm{e}}^{-\lambda } ={ \mathrm{e}}^{-\lambda } = 0.0498 \approx 0.05 = 5\,\%$$ .

Problem 5.3

(a)

The fraction

is rejected, since R  <  R ₀ − Δ ₂ or R  >  R ₀ + Δ ₂ . Correspondingly the fractions

give prices 2 C and 5 C , respectively. Therefore,

(b)

i.e., $$P = C\{4 - 2.526 - 0.636\} = 0.838C\quad.$$

(c)

is fulfilled if the expression in the last set of brackets vanishes.

Problem 5.4

(a)

(b)

For σ  = 1, P  = 0. 9 one obtains

Problem 5.5

(a)

0 ≤  u  < 1: 

1 ≤  u  < 2: 

2 ≤  u  < 3: 

(b)

Completing the square yields for the exponent

With the change of variables $$v = (\sigma /\sigma _{x}\sigma _{y})(x -\sigma _{x}^{2}u{/\sigma }^{2})$$ we obtain

since $$\sigma _{x}^{4} =\sigma _{ x}^{2}{(\sigma }^{2} -\sigma _{y}^{2})\:.$$

Problem 6.1

(a)

if and only if

(b)

(c)

Problem 6.2

Problem 6.3

(a)

The result $$\widetilde{x} = 0.51 \pm 0.12$$ does not favor any one party significantly.

(b)

Problem 6.4

(a)

For simplicity we write χ ²  =  u .

(b)

Obviously $$\gamma = 2/\sqrt{\lambda }\rightarrow 0$$ holds for $$\lambda = \frac{1} {2}n \rightarrow \infty $$ .

Problem 6.5

See Fig.  G.2 .

Fig. G.2:

Histogram of the data of Problem 6.5.

Problem 7.1

(a)

$$L =\prod _{ j=1}^{N}\frac{1} {b} = \frac{1} {{b}^{N}}\quad; \qquad$$ obviously one has L  = max for b  =  x max . 

(b)

λ ₁  =  a , $$\lambda _{2} = \Gamma $$ ,

There is not necessarily a unique solution since the equations are not linear in λ ₁ and λ ₂ . For $$\vert {\mathsf{x}}^{(j)} -\lambda _{1}\vert \ll \lambda _{2}$$ , however, we may write

Problem 7.2

(a)

(b)

Problem 7.3

(a)

In Problem 6.1 (c) we had $${\sigma }^{2}(\bar{\mathsf{x}}) = 1/{3\sigma }^{2}$$ for N  = 3.

(b)

(c)

From Problem 7.2 we take

Thus, $${\sigma }^{2}(\mathsf{S}) = \frac{{2\lambda }^{2}} {N}\:.$$

Problem 8.1

The variance of sample (2) is not significantly smaller.

Problem 8.2

Therefore the hypothesis cannot be rejected.

Problem 8.3

(a)

(b)

$$T^{\prime} = N{\mathsf{s}^{\prime}}^{2}/\sigma _{0}^{2}$$ is a monotonically increasing function of s ′ if s ′ >  σ ₀ , otherwise it is monotonically decreasing. Therefore the test takes on the following form:

(c)

The answer follows from ( 6.​7.​2 ) and $${\mathsf{s}^{\prime}}^{2} = (N - 1){\mathsf{s}}^{2}/N$$ .

Problem 8.4

(a)

(b)

x k

n k

ψ ₀₊

ψ ₀₋

np k

193

1

0.011

0.002

(0.9)

–

195

2

0.041

0.011

(3.0)

–

197

9

0.117

0.041

7.6

0.271

199

12

0.260

0.117

14.3

0.362

201

23

0.461

0.260

20.1

0.411

203

25

0.673

0.463

21.2

0.679

205

11

0.840

0.673

16.7

1.948

207

9

0.938

0.840

9.8

0.071

209

6

0.982

0.938

4.3

0.647

211

2

0.996

0.982

(1.4)

–

X ²  = 4. 389

The number of degrees of freedom is $$7 - 2 = 5$$ . Since $$\chi _{0.90}^{2}(5) = 9.24$$ , the test does not reject the hypothesis.

Problem 8.5

(a)

Since $$\chi _{0.90}^{2} = 2.71$$ for f  = 1, the hypothesis of independence is rejected.

(b)

One can easily show that the expression in square brackets takes the form $${(-1)}^{i+j}(n_{11}n_{22} - n_{12}n_{21})$$ for all i , j .

Programming Problems

Programming Problem 4.1:

Program to Generate Breit–Wigner-Distributed Random Numbers

Write a method with the following declaration

double[] breitWignerNumbers(double a, double gamma, int n).

It is to yield n random numbers, which follow a Breit–Wigner distribution having a mean of a and a FWHM of Γ . Make the method part of a class which allows for interactive input of n ,  a ,  Γ and numerical as well as graphical output of the random numbers in the form a histogram, Fig.  G.3 . (Example solution: S1Random)

Fig. G.3:

Histogram of 1000 random numbers following a Breit–Wigner distribution with a  = 10 and $$\Gamma = 3$$ .

Programming Problem 4.2:

Program to Generate Random Numbers from a Triangular Distribution

Write a method with the declaration

double[] triangularNumbersTrans(double a, double b, double c,

int n).

It is to yield n random numbers, following a triangular distribution with the parameters a , b , c generated by the transformation procedure of Example 4.3.

Write a second method with the declaration

double[] triangularNumbersRej(double a, double b, double c,

int n),

which solves the same problem, but uses von Neumann’s acceptance–rejection method. Which of the two programs is faster?

Write a class which allows to interactively choose either method. It should also allow for numerical and graphical output (as histogram) of the generated numbers, Fig.  G.4 . (Example solution: S2Random)

Fig. G.4:

Histogram of 1000 random numbers following a triangular distribution with a  = 1, b  = 4, c  = 3.

Programming Problem 4.3:

Program to Generate Data Points with Errors of Different Size

Write a method similar to DatanRandom.line which generates data points $$y = at + b +\varDelta y$$ . The errors Δ y , however, are not to be taken for all data points y i from the same uniform distribution with the width σ , but Δ y i is to be sampled from a normal distribution with the width σ i . The widths σ i are to be taken from a uniform distribution within the region $$\sigma _{\min } <\sigma _{i} <\sigma _{\max }$$ .

Write a class which calls this method and which displays graphically the straight line $$y = at + b$$ as well as the simulated data points with error bars y i ± Δ y i , Fig.  G.5 . (Example solution: S3Random)

Fig. G.5:

Data points with errors of different size.

Programming Problem 5.1:

Convolution of Uniform Distributions

Fig. G.6:

Histogram of 90 000 random numbers x , each of which is a sum of three uniformly distributed random numbers. The curve corresponds to the standard normal distribution. Significant differences between curve and histogram are visible only because of the very large number of random numbers used.

Because of the Central Limit Theorem the quantity $$\mathsf{x} =\sum _{ i=1}^{N}\mathsf{x}_{i}$$ follows in the limit N  →  ∞ the standard normal distribution if the x i come from an arbitrary distribution with mean zero and standard deviation $$1/\sqrt{N}$$ . Choose for the x i the uniform distribution with the limits

Perform a large number n exp of Monte Carlo experiments, each giving a random value x . Produce a histogram of the quantity  x and show in addition the distribution of x as a continuous curve which you would expect from the standard normal distribution (Fig.  G.6 ). (Use the class GraphicsWithHistogramAndPolyline for the simultaneous representation of histograms and curves.) Allow for interactive input of the quantities n exp and N . (Example solution: S1Distrib)

Programming Problem 5.2:

Convolution of Uniform Distribution and Normal Distribution

If x is taken from a uniform distribution between a and b and if y is taken from a normal distribution with mean zero and width σ , then the quantity $$\mathsf{u} = \mathsf{x} + \mathsf{y}$$ follows the distribution ( 5.​11.​14 ). Perform a large number n exp of Monte Carlo experiments, each resulting in a random number u . Display a histogram of the quantity u and show in addition a curve of the distribution you would expect from ( 5.​11.​14 ), Fig.  G.7 . Allow for the interactive input of the quantities n exp , a , b , and σ .

Fig. G.7:

A histogram of 10 000 random numbers, each of which is the sum of a uniformly distributed random number and a normally distributed random number. The curve corresponds to the convolution of a uniform and a normal distribution.

Programming Problem 7.1:

Distribution of Lifetimes Determined from a Small Number of Radioactive Decays

In Example Program 7.1, an estimate $$\bar{t}$$ of the mean lifetime τ and its asymmetric errors Δ ₋ and Δ ₊ are found from a single small sample. In all cases the program yields $$\varDelta _{-} <\varDelta _{+}$$ . Write a program that simulates a large number n exp of experiments, in each of which N radioactive decays of mean lifetime τ  = 1 are measured. Compute for each experiment the estimate $$\bar{t}$$ and construct a histogram of the quantity $$\bar{t}$$ for all experiments. Present this histogram $$N_{i}(\bar{t}_{i} \leq \bar{ t} <\bar{ t}_{i} +\varDelta \bar{ t})$$ and also the cumulative frequency distribution $$h_{i} = (1/n_{\exp })\sum _{\bar{t}<\bar{t}_{i}}N_{i}$$ . Allow for interactive input of n exp and N . Demonstrate that the distributions are asymmetric for small N and that they become symmetric for large N . Show that for small N the value $$\bar{t} = 1$$ is not the most probable value, but that it is the expectation value of $$\bar{t}$$ . Determine for a fixed value of N , e.g., N  = 4, limits Δ ₋ and Δ ₊ in such a way that $$\bar{t} < 1 -\varDelta _{-}$$ holds with the probability 0. 683∕2 and that with the same probability one has $$\bar{t} > 1 +\varDelta _{+}$$ . Compare the results found with a series of simulated experiments from the program E1MaxLike , Example Program 7.1. (Example solution: S1MaxLike)

Programming Problem 7.2:

Distribution of the Sample Correlation Coefficient

Modify the class E2MaxLike so that instead of numerical output, a histogram of the correlation coefficient r is presented (Fig.  G.8 ). Produce histograms for ρ  = 0 and ρ  = 0. 95, each for n pt  = 5, 50, 500. Under what circumstances is the distribution asymmetric and why? Is this asymmetry in contradiction to the Central Limit theorem? (Example solution: S2MaxLike)

Fig. G.8:

Histogram of the sample correlation coefficient computed for 1000 samples of size 10 from a bivariate Gaussian distribution with the correlation coefficient $$\rho = -0.8$$ .

Programming Problem 9.1:

Fit of a First-Degree Polynomial to Data that Correspond to a Second-Degree Polynomial

In experimental or empirical studies one is often confronted with a large number of measurements or objects of the same kind (animals, elementary particle collisions, industrial products from a given production process, … ). The outcomes of the measurements performed on each object are described by some law. Certain assumptions are made about that law, which are to be checked by experiment.

Consider the following example. A series of measurements may contain n exp experiments. Each experiment yields the measurements $$y_{i} = x_{1} + x_{2}t_{i} + x_{3}t_{i}^{2} +\varepsilon _{i}$$ for 10 values t i  = 1, 2 … , 10 of the controlled variable t . The $$\varepsilon _{i}$$ are taken from a normal distribution with mean zero and width σ . In the analysis of the experiments it is assumed, however, that the true values η i underlying the measurements y i can be described by a first-degree polynomial $$\eta _{i} = x_{1} + x_{2}t$$ . As result of the fit we obtain a minimum function M from which we can compute the “ χ ² -probability” $$P = 1 - F(M,n - r)$$ . Here F ( M ,  f ) is the distribution function of a χ ² distribution with f degrees of freedom, n is the number of data points, and r the number of parameters determined in the fit. If P  <  α , then the fit of a first-degree polynomial to the data is rejected at a confidence level of $$\beta = 1-\alpha$$ .

Write a class performing the following steps:

(i)

Interactive input of n exp , x ₁ , x ₂ , x ₃ , σ , Δ y .

(ii)

Generation of n exp sets of data ( t i ,  y i ,  Δ y ), fit of a first-degree polynomial to each set of data and computation of P . Entry of P into a histogram.

(iii)

Graphical representation of the histogram.

Suggestions: (a) Choose n exp  = 1000, $$x_{1} = x_{2} = 1$$ , x ₃  = 0, $$\sigma =\varDelta y = 1$$ . As expected you will obtain a flat distribution for P . (b) Choose (keeping the other input quantities as above) different values x ₃ ≠ 0. You will observe a shift of the distribution towards small P values, cf. Fig.  G.9 . Determine approximately the smallest positive value of x ₃ such that the hypothesis of a first-degree polynomial is rejected at 90  % confidence level in 95  % of all experiments. (c) Choose x ₃  = 0, but σ ≠ Δ y . You will again observe a shift in the distribution, e.g., towards larger P values for Δ y  >  σ . (d) From the experience gained in (a), (b), and (c), one might conclude that if erroneously too large measurement errors are assumed ( Δ y  >  σ ) then a flat P distribution would result. In this way one would get the impression that a first-degree polynomial could describe the data. Begin with n exp  = 1000, $$x_{1} = x_{2} = 1$$ , x ₃  = 0. 2, σ  = 1, Δ y  = 1 and increase Δ y in steps of 0.1 up to Δ y  = 2. (Example solution: S1Lsq)

Fig. G.9:

Histogram of the χ ² -probability for fits of a first-degree polynomial to 1000 data sets generated according to a second-degree polynomial.

Programming Problem 9.2:

Fit of a Power Law (Linear Case)

A power law

is linear in the parameter x if w is a constant. This function is to be fitted to measurements ( t i ,  y i ) given by

Here the $$\varepsilon _{i}$$ follow a normal distribution centered about zero with width σ .

Write a class performing the following steps:

(i)

Interactive input of n , t ₀ , Δ t , x , w , σ .

(ii)

Generation of measured points.

(iii)

Fit of the power law.

(iv)

Graphical display of the data and the fitted function, cf. Fig.  G.10 .

Fig. G.10:

Result of the fit of a parabola y  =  xt ² to 20 measured points.

(Example solution: S2Lsq)

Programming Problem 9.3:

Fit of a Power Law (Nonlinear Case)

If the power law has the form

i.e., if the power itself is an unknown parameter, the problem becomes nonlinear. For the fit of a nonlinear function we have to start from a first approximation of the parameters. We limit ourselves to the case t i  > 0 for all i which occurs frequently in practice. Then one has $$\ln \eta =\ln x_{1} + x_{2}\ln t$$ . If instead of ( t i ,  y i ) we now use (ln t i , ln y i ) as measured variables, we obtain a linear function in the parameters ln x ₁ and x ₂ . However, in this transformation the errors are distorted so that they are no longer Gaussian. We simply choose all errors to be of equal size and use the result of the linear fit as the first approximation of a nonlinear fit to the ( t i ,  y i ). We still have to keep in mind (in any case for x ₁  > 0) that one always has η  > 0 for t  > 0. Because of measurement errors, however, measured values y i  < 0 can occur. Such points of course must not be used for the computation of the first approximation.

Write a class with the following steps:

(i)

Interactive input of n , t ₀ , Δ t , x ₁ , x ₂ , σ .

(ii)

Generation of the n measured points

where $$\varepsilon _{i}$$ comes from a normal distribution centered about zero with width σ .

(iii)

Computation of first approximations x ₁ , x ₂ by fitting a linear function to (ln t i , ln y i ) with LsqLin .

(iv)

Fit of a power law to ( t i ,  y i ) with LsqNon .

(v)

Graphical display of the results, cf. Fig.  G.11 .

Fig. G.11:

Fit of a function $$y = x_{1}{t}^{x_{2}}$$ to 20 data points. The data points are identical to those in Fig.  G.10 .

(Example solution: S3Lsq)

Programming Problem 9.4:

Fit of a Breit–Wigner Function to Data Points with Errors

For the N  = 21 values $$t_{i} = -3,\,-2.7,\,\ldots,\,3$$ of the controlled variable the measured values

(G.3.1)

are to be simulated. Here,

(G.3.2)

is the Breit–Wigner function ( 3.​3.​32 ) with a  =  x ₁ and $$\Gamma = x_{2}$$ . The measurement errors $$\varepsilon _{i}$$ are to be taken from a normal distribution around zero with width σ . Choose a  = 0 and $$\Gamma = 1$$ . The points ( t i ,  y i ) scatter within the measurement errors around a bell-shaped curve with a maximum at t  =  a . A bell-shaped curve with a maximum at the same position, however, could be given by the Gaussian function,

(G.3.3)

Write a class with the following properties:

(i)

Interactive input of σ and possibility to choose whether a Breit–Wigner function or a Gaussianis to be fitted.

(ii)

Generation of the data, i.e., of the triplets of numbers ( $$t_{i},y_{i},\varDelta y_{i} =\varepsilon _{i}$$ ).

(iii)

Fit of the Breit–Wigner function ( G.3.2 ) or of the Gaussian ( G.3.3 ) to the data and computation of the minimum function M .

(iv)

Graphical representation of the measured points with measurement errors and of the fitted function, cf. Fig.  G.12 .

Fig. G.12:

Fit of a Gaussian to data points that correspond to a Breit–Wigner function. The goodness-of-fit is poor.

Run the program using different values of σ and find out for which range of σ the data allow a clear discrimination between the Breit–Wigner and Gaussian functions. (Example solution: S4Lsq)

Programming Problem 9.5:

Asymmetric Errors and Confidence Region for the Fit of a Breit–Wigner Function

Supplement the solution of Programming Problem 9.4 such that it yields a graphical representation for the parameters, their errors. covariance ellipse, asymmetric errors, and confidence region similar to Fig.​ 9.​11 . Discuss the differences obtained when fitting a Breit–Wigner or a Gaussian, respectively. For each case try σ  = 0. 1 and σ  = 1. (Example solution: S5Lsq)

Programming Problem 9.6:

Fit of a Breit–Wigner Function to a Histogram

In the Programming Problem 9.4 we started from measurements $$y_{i} = f(t_{i}) +\varepsilon _{i}$$ . Here f ( t ) was a Breit–Wigner function ( 3.​3.​32 ), and the measurement errors $$\varepsilon _{i}$$ corresponded to a Gaussian distribution centered about zero with width σ .

We now generate a sample of size n ev from a Breit–Wigner distribution with mean a  = 0 and full width at half maximum $$\Gamma = 1$$ . We represent the sample by a histogram that is again characterized by triplets of numbers $$(t_{i},y_{i},\varDelta y_{i})$$ . Now t i is the center of the i th bin $$t_{i} -\varDelta t/2 \leq t < t_{i} +\varDelta t/2$$ , and y i is the number of sample elements falling into this bin. For not too small y i the corresponding statistical error is $$\varDelta y_{i} = \sqrt{y_{i}}$$ . For small values y i this simple statement is problematic. It is completely wrong for y i  = 0. In the fit of a function to a histogram, care is therefore to be taken that empty bins (possibly also bins with few entries) are not to be considered as data points. The function to be fitted is

(G.3.4)

The function is similar to ( G.3.2 ) but there is an additional parameter x ₃ .

Fig. G.13:

Fit of a Breit–Wigner function to a histogram.

Write a class with the following steps:

(i)

Interactive input of n ev (sample size) and n t (number of histogram bins). The lower limit of the histogram is to be fixed at $$t = -3$$ , the upper limit at t  = 3.

(ii)

Generation of the sample, cf. Programming Problem 4.1.

(iii)

Construction of the histogram.

(iv)

Construction of the triplets $$(t_{i},y_{i},\varDelta y_{i})$$ to be used for the fit.

(v)

Fit of the function ( G.3.4 ).

(vi)

Output of the results in numerical and graphical form, cf. Fig.  G.13 .

Suggestions: Perform consecutive fits for the same sample but different numbers of bins. Try to find an optimum for the number of bins. (Example solution: S6Lsq)

Programming Problem 9.7:

Fit of a Circle to Points with Measurement Errors in Abscissa and Ordinate

A total of m data points ( s i ,  t i ) are given in the ( s ,  t ) plane. The measurement errors are defined by 2 × 2 covariance matrices of the form

(G.3.5)

as in Example 9.11. Here $$c_{i} =\varDelta s_{i}\varDelta t_{i}\rho _{i}$$ and ρ i is the correlation coefficient between the measurement errors Δ s i , Δ t i . As in Example 9.11, construct the vector y of measurements from the s i and t i and construct the covariance matrix C y . Set up the equations $$f_{k}(x,\boldsymbol{\upeta }) = 0$$ assuming that the true positions underlying the measured points lie on a circle with center ( x ₁ ,  x ₂ ) and radius x ₃ . Write a program with the following steps:

(i)

Input of m , Δ t , Δ s , ρ .

(ii)

Generation of m measured points ( s i ,  t i ) using bivariate normal distributions, the means of which are positioned at regular intervals on the unit circle ( $$x_{1} = x_{2} = 0$$ , x ₃  = 1) and the covariance matrix of which is given by ( G.3.5 ) with Δ s i  =  Δ s , Δ t i  =  Δ t , c  =  Δ s Δ t ρ .

(iii)

Determination of a first approximation for x ₁ , x ₂ , x ₃ by computation of the parameters of a circle through the first three measured points.

(iv)

Fit to all measured points using LsqGen and a user function specially written for this problem.

(v)

Graphical representation of the measured points and the fitted circle as in Fig.  G.14 .

(Example solution: S6Lsq)

Fig. G.14:

Measured points with covariance ellipses, circle of the first approximation which is given by 3 measured points ( broken line ), and circle fitted to all measured points.

Programming Problem 10.1:

Monte Carlo Minimization to Choose a Good First Approximation

For some functions in Example Program 10.1 the choice of the point x ₀ defining the first approximation was decisive for success or failure of the minimization. If a function has several minima and if its value is smallest at one of them, that is if an “absolute minimum” exists, the following procedure will work. One uses the Monte Carlo method to determine a first approximation x ₀ of the absolute minimum in a larger region of the parameter space by generating points $$\mathbf{x} = (x_{1},x_{2},\ldots,x_{n})$$ according to a uniform distribution in that region and by choosing that point at which the function has the smallest value.

On the basis of E1Min write class that determines the absolute minimum of the function f ₇ ( x ) described in Sect.  10.​1 . A first approximation x ₀ within the region

is to be determined by the Monte Carlo method. Perform the search for the first approximation with N points generated at random and allow for an interactive input of N . (Example solution: S1Min)

Programming Problem 10.2:

Determination of the Parameters of a Breit–Wigner Distribution from the Elements of a Sample

By modifying a copy of E2Min produce a class that simulates a sample from a Breit–Wigner distribution with mean a and full width at half maximum $$\Gamma $$ , and that subsequently determines the numerical values of these parameters by minimization of the negative log-likelihood function of the sample. Allow for interactive input of the sample size and the parameters a and $$\Gamma $$ used in the simulation. (Example solution: S2Min)

Programming Problem 11.1:

Two-Way Analysis of Variance with Crossed Classification

The model ( 11.​2.​16 ) for the data in an analysis of variance with crossed classification is

The analysis of variance tests the null hypothesis

Data corresponding to this null hypothesis are generated by the program E2Anova , Sect.  11.​2 , and are subsequently analyzed.

Write a program similar to E2Anova which generates data x ijk according to the above formula with

These relations fulfill the requirements ( 11.​2.​7 ). The $$\varepsilon _{ijk}$$ as in E2Anova are to be drawn from a normal distribution with mean zero and standard deviation σ . Allow for interactive input of the quantities a , b , ab , σ , μ . Perform an analysis of variance on the simulated data. Study different cases, e.g., a  = 0, b ≠ 0, ab  = 0; a ≠ 0, b  = 0, ab  = 0; a  = 0, b  = 0, ab ≠ 0; etc. (Example solution: S1Anova)

Programming Problem 11.2:

Two-Way Analysis of Variance with Nested Classification

Modify Programming Problem 11.1 for the treatment of a nested classification with data of the form ( 11.​2.​22 ), e.g.,

and use the relations

(Example solution: S2Anova)

Programming Problem 12.1:

Simulation of Data and Plotting Regression Polynomials of Different Degrees

Fig. G.15:

Data points with errors and regression polynomials of different degrees.

Write a class which generates n data points ( t i ,  y i ). The t i are to be spread equidistantly over the interval − 1 ≤  t  ≤ 1; the y i are to correspond to a polynomial with r terms and to have errors with standard deviation σ . A regression analysis is to be performed and a plot as in Fig.  G.15 of the data and the regression polynomials to be produced. Your program may be largely based on the classes E4Reg and E2Reg . (Example solution: S1Reg)

Programming Problem 12.2:

Simulation of Data and Plotting the Regression Line with Confidence Limits

Extend the solution of Programming Problem 12.1 so that a regression polynomial of the desired degree together with confidence limits, corresponding to Sect.  12.​2 is shown. (Example solution: S2Reg)

Programming Problem 13.1:

Extrapolation in a Time Series Analysis

In Sect.​ 13.​3 we have stressed that one must be very cautious with the interpretation of the results of a time series analysis at the edge of a time series, and of the extrapolation in regions outside the time series. In particular, we found that the extrapolation yields meaningless results if the data are not at least approximately described by a polynomial. The degree of this polynomial must be smaller than or equal to the degree of the polynomial used for the time series analysis.

Study these statements by simulating a number of time series and analyzing them. Write a program – starting from E2TimSer  – that for n  = 200, i  = 1, 2,  … ,  n , generates data of the form

Here the $$\varepsilon _{i}$$ are to be generated according to a normal distribution with mean zero and standard deviation σ . Allow for the interactive input of m , σ , k , ℓ , and P . After generating the data perform a time series analysis and produce a plot as in E2TimSer . Study different combinations of m , k , and ℓ , and for each combination use small values of σ (e.g., σ  = 0. 001) and large values of σ (e.g., σ  = 0. 1). (Example solution: S1TimSer)

Programming Problem 13.2:

Discontinuities in Time Series

In the development of time series analysis it was assumed that the measurements, apart from their statistical fluctuations, are continuous functions of time. We therefore expect unreliable results in regions where the measurements are discontinuous. Write a program that generates the following three types of time series, analyzes them, and displays the results graphically. One of them is continuous; the other two contain discontinuities.

Sine function:

Step function:

Sawtooth function:

The $$\varepsilon _{i}$$ are again to be generated according to normal distribution with mean zero and standard deviation σ . Allow for the choice of one of the functions and for the interactive input of n , σ , k , ℓ , and P . Study the time series using different values for the parameters and discuss the results. Figure  G.16 shows an example. (Example solution: S2TimSer)

Fig. G.16:

Time series corresponding to a step function with moving averages and confidence limits.

Collection of Formulas

Probability

A , B , … are events ; $$\bar{A}$$ is the event “ not A ”.

( A + B ) and ( AB ) combine events with logical “or” or logical “and” .

P ( A ) is the probability of the event A .

$$P(B\vert A) = P(AB)/P(A)$$ is the probability for B given the condition A ( conditional probability) .

The following rules hold:

For every event A

for mutually exclusive events A , B , … , Z

for independent events A , B , … , Z

Single Random Variable

Distribution function : F ( x ) =  P ( x  <  x )

Probability density (for F ( x ) differentiable):

Moments of order ℓ :

(a)

About the point c : $$\alpha _{\ell} = E\{{(\mathsf{x} - c)}^{\ell}\}$$

(b)

About the origin (central moments): $$\lambda _{\ell} = E\{{\mathsf{x}}^{\ell}\}$$

(c)

About the mean: $$\mu _{\ell} = E\{{(\mathsf{x} -\widehat{ x})}^{\ell}\}$$

Table H.1:

Expectation values for discrete and continuous distributions.

x discrete

x continuous; F ( x ) differentiable

Probability density

–

Mean of x (expectation value)

Mean of the function H ( x )

E { H ( x )} $$=\sum _{i}H(x_{i})P(\mathsf{x} = x_{i})$$

E { H ( x )} $$=\int _{ -\infty }^{\infty }H(x)f(x)\,\mathrm{d}x$$

Variance : $${\sigma }^{2}(\mathsf{x}) = \mbox{ var}(\mathsf{x}) =\mu _{2} = E\{{(\mathsf{x} -\widehat{ x})}^{2}\}$$

Standard deviation or error of x : $$\varDelta \mathsf{x} =\sigma (\mathsf{x}) = +\sqrt{{\sigma }^{2 } (\mathsf{x} )}$$

Skewness : $$\gamma =\mu _{3}{/\sigma }^{3}$$

Reduced variable: $$\mathsf{u} = (\mathsf{x} -\widehat{ x})/\sigma (\mathsf{x})\quad;\qquad E(\mathsf{u}) = 0\quad,{\qquad \sigma }^{2}(\mathsf{u}) = 1$$

Mode (most probable value) x m defined by: $$P(\mathsf{x} = x_{\mathrm{m}}) =\max$$

Median x 0. 5 defined by: $$F(x_{0.5}) = P(\mathsf{x} < x_{0.5}) = 0.5$$

Quantile x q defined by: $$F(x_{q}) = P(\mathsf{x} < x_{q}) = q\quad;\qquad 0 \leq q \leq 1$$

Several Random Variables

Distribution function:

Joint probability density (only for F ( x ) differentiable with respect to all variables):

Marginal probability density of the variable x i :

Expectation value of a function H ( x ):

Expectation value of the variables x i :

The variables $$\mathsf{x}_{1},\mathsf{x}_{2},\ldots,\mathsf{x}_{n}$$ are independent , if

Moments of order $$\ell_{1},\ell_{2},\ldots,\ell_{n}$$ :

(a)

About $$\mathbf{c} = (c_{1},c_{2},\ldots,c_{n})$$ :

(b)

About the origin: $$\quad \lambda _{\ell_{1}\ell_{2}\ldots \ell_{n}} = E\{\mathsf{x}_{1}^{\ell_{1}}\mathsf{x}_{2}^{\ell_{2}}\cdots \mathsf{x}_{n}^{\ell_{n}}\}$$

(c)

About $$\widehat{\mathbf{x}}$$ :

Variance of x i : $${\sigma }^{2}(\mathsf{x}_{i}) = E\{{(\mathsf{x}_{i} -\widehat{ x}_{i})}^{2}\} = c_{ii}$$

Covariance between x i and x j : $$\mathop{\mathrm{cov}}\nolimits (\mathsf{x}_{i},\mathsf{x}_{j}) = E\{(\mathsf{x}_{i} -\widehat{ x}_{i})(\mathsf{x}_{j} -\widehat{ x}_{j})\} = c_{ij}$$

For x i , x j independent : $$\mathop{\mathrm{cov}}\nolimits (\mathsf{x}_{i},\mathsf{x}_{j}) = 0$$

Covariance matrix: $$C = E\{(\mathsf{\mathbf{x}} -\widehat{\mathbf{x}}){(\mathsf{\mathbf{x}} -\widehat{\mathbf{x}})}^{\mathrm{T}}\}$$

Correlation coefficient:

Rules of computation:

Transformation of Variables

Original variables: $$\mathsf{\mathbf{x}} = (\mathsf{x}_{1},\mathsf{x}_{2},\ldots,\mathsf{x}_{n})$$

Probability density: f ( x )

Transformed variables: $$\mathsf{\mathbf{y}} = (\mathsf{y}_{1},\mathsf{y}_{2},\ldots,\mathsf{y}_{n})$$

Mapping: $$\mathsf{y}_{1} = \mathsf{y}_{1}(\mathsf{\mathbf{x}}),\:\mathsf{y}_{2} = \mathsf{y}_{2}(\mathsf{\mathbf{x}}),\:\ldots,\:\mathsf{y}_{n} = \mathsf{y}_{n}(\mathsf{\mathbf{x}})$$

Probability density: g ( y ) =  |  J  |  f ( x )

with the Jacobian determinant

Error Propagation

The original variables x have the covariance matrix C x . The covariance matrix of the transformed variables y is

The formula is only exact for a linear relation between y and x , but is a good approximation for small deviations from linearity in a region around $$\widehat{\mathbf{x}}$$ of the magnitude of the standard deviation. Only for vanishing covariances in C x does one have

The Law of Large Numbers

A total of n observations are carried out, which are characterized by the random variable x i ( = 1, if on the i th observation the event A  occurs, otherwise  = 0). The frequency of A is

For n  →  ∞ this frequency is equal to the probability p for the occurrence of  A ,

Central Limit Theorem

If x i are independent variables with mean a and variance σ ² , then (1∕ n ) $$\sum _{i=1}^{n}\mathsf{x}_{i}$$  for n  →  ∞ follows a normal distribution with mean a and variance σ ² ∕ n .

Table H.2:

Distributions of discrete variables.

Distribution

Probability for observing x  =  k ( $$\mathsf{x}_{1} = k_{1},\ldots,\mathsf{x}_{l} = k_{l}$$ )

Mean

Variance (elements of covariance matrix)

Binomial

Multinomial $$\sum _{j=1}^{l}p_{j} = 1$$

Hypergeometric $$L = N - K,$$ $$l = n - k$$

Poisson

σ ² ( x ) =  λ

Table H.3:

Distributions of continuous variables.

Distribution

Probability density

Mean

Variance (covariance matrix)

Uniform

Gaussian

a

b ²

Stand. Gaussian

0

1

Gaussian of several variables

a

χ ²

f

2 f

Fisher’s F

$$\frac{2f_{2}^{2}(f_{ 1}+f_{2}-2)} {f_{1}{(f_{2}-2)}^{2}(f_{2}-4)}\,,$$ f ₂  > 4

Student’s t

0

Convolutions of Distributions

The probability density of the sum $$\mathsf{u} = \mathsf{x} + \mathsf{y}$$ of two independent random variables x and y is

The convolution of two Poisson distributions with the parameters λ ₁ and λ ₂ is a Poisson distribution with the parameter λ ₁ + λ ₂ .

The convolution of two Gaussian distributions with means a ₁ , a ₂ and variances $$\sigma _{1}^{2}$$ , $$\sigma _{2}^{2}$$ is a Gaussian distribution with mean $$a = a_{1} + a_{2}$$ and variance $${\sigma }^{2} =\sigma _{ 1}^{2} +\sigma _{ 2}^{2}$$ .

The convolution of two χ ² -distributions with f ₁ and f ₂ degrees of freedom is a χ ² -distribution with $$f = f_{1} + f_{2}$$ degrees of freedom.

Samples

Population: An infinite (in some cases finite) set of elements described by a discrete or continuously distributed random variable x .

(Random) sample of size N : A selection of N elements ( x ⁽¹⁾ , x ⁽²⁾ , … , x (N) ) from the population. (For the requirements for a sample to be random see Sect.  6.​1.​ )

Distribution function of the sample: $$W_{n}(x) = n_{x}/N$$ ,

where n x is the number of elements in the sample for which x  <  x .

Statistic: An arbitrary function of the elements of a sample

Table H.4:

Samples from different populations.

Sample of size n from a continuously distributed population

Sample of size n from a discrete population of size N . Variable of the population is y , variable of the sample is x

Population mean

Populationvariance

σ ²(x)

Samplemean

Samplevariancemean

Variance ofthe sample

Estimator: A statistic used to estimate a parameter λ of the population. An estimator is unbiased , if E ( S ) =  λ and consistent , if

Maximum-Likelihood Method

Consider a population described by the probability density $$f(\mathbf{x},\boldsymbol{\uplambda })$$ , where $$\boldsymbol{\uplambda } = (\lambda _{1},\lambda _{2},\ldots,\lambda _{p})$$ is a set of parameters. If a sample x ⁽¹⁾ , x ⁽²⁾ , … , x (N) is obtained, then the likelihood function is $$L =\prod _{ j=1}^{N}f({\mathsf{\mathbf{x}}}^{(j)},\boldsymbol{\uplambda })$$ and the log-likelihood function is ℓ  = ln L . In order to determine the unknown parameters $$\boldsymbol{\uplambda }$$ from the sample the maximum-likelihood method prescribes the value of $$\boldsymbol{\uplambda }$$ for which L (or ℓ ) is a maximum. That is, one must solve the likelihood equation $$\partial \ell/\partial \lambda _{i} = 0$$ ; i  = 1, 2,  … ,  p or (for only one parameter) $$\mathrm{d}\ell/\mathrm{d}\lambda =\ell ^{\prime} = 0$$ .

Information of a sample: $$I(\lambda ) = E{(\ell^{\prime}}^{2}) = -E(\ell^{\prime\prime})$$

Information inequality: $${\sigma }^{2}(\mathsf{S}) \geq \{ 1 - B^{\prime}{(\lambda )\}}^{2}/I(\lambda )$$

Here S is an estimator for λ , and $$B(\lambda ) = E(\mathsf{S})-\lambda$$ is its bias .

An estimator has minimum variance if $$\ell^{\prime} = A(\lambda )(\mathsf{S} - E(\mathsf{S}))$$ , where A ( λ ) does not depend on the sample.

The maximum-likelihood estimator $$\widetilde{\lambda }$$ , i.e., the solution of the likelihood equation, is unique , asymptotically unbiased (i.e., for N  →  ∞ ), and has minimum variance .

The asymptotic form of the likelihood function for one parameter λ is

and for several parameters

with the covariance matrix $$C = {B}^{-1}$$ and

Testing Hypotheses

Null hypothesis $$H_{0}(\boldsymbol{\uplambda } = \boldsymbol{\uplambda }_{0})$$ : Assumption of values for the parameters $$\boldsymbol{\uplambda }$$ that determine the probability distribution $$f(\mathbf{x},\boldsymbol{\uplambda })$$ of a population.

Alternative hypotheses $$H_{1}(\boldsymbol{\uplambda } = \boldsymbol{\uplambda }_{1})$$ , $$H_{2}(\boldsymbol{\uplambda } = \boldsymbol{\uplambda }_{2})$$ , … : Other possibilities for $$\boldsymbol{\uplambda }$$ , against which the null hypothesis is to be tested by consideration of a sample $$X = ({\mathsf{\mathbf{x}}}^{(1)},\,{\mathsf{\mathbf{x}}}^{(2)},\,\ldots,\,{\mathsf{\mathbf{x}}}^{(N)})$$ from the population.

A hypothesis is simple if the parameters are completely determined, e.g., $$H_{0}(\lambda _{1} = 1,\:\lambda _{2} = 5)$$ , otherwise it is composite , e.g., $$H_{1}(\lambda _{1} = 2,\:\lambda _{2} < 7)$$ .

Test of a hypothesis H ₀ with a significance level α or confidence level 1 − α : H ₀ is rejected if X  ∈  S c , where S c is the critical region in the sample space and

Error of the first kind: Rejection of H ₀ , although H ₀ is true. The probability of this error is α .

Error of the second kind: H ₀ is not rejected although H ₁ is true. The probability of this error is $$P(X\not\in S_{c}\vert H_{1}) =\beta$$ .

Power function:

Operating characteristic function:

Most powerful test of H ₀ with respect to H ₁ has $$M(S_{c},\,\boldsymbol{\uplambda }_{1}) = 1-\beta =\max$$ . A uniformly most powerful test is a most powerful test with respect to all possible H ₁ .

An unbiased test has $$M(S_{c},\,\boldsymbol{\uplambda }_{1}) \geq \alpha$$ for all possible H ₁ .

NEYMAN–PEARSON LEMMA : A test of H ₀ with respect to H ₁ (both simple hypotheses) with the critical region S c is a most powerful test if f ( X  |  H ₀ )∕ f ( X  |  H ₁ ) ≤  c for X  ∈  S c and ≥  c for $$X\not\in S_{c}$$ , where c is a constant only depending on α .

Test statistic T ( X ): Scalar function of the sample X . By means of a mapping X  →  T ( X ), S c ( X ) →  U the question as to whether X  ∈  S c can be reformulated as T  ∈  U .

Likelihood-ratio test: If ω denotes the region in the parameter space corresponding to the null hypothesis and Ω denotes the entire possible parameter region, then the test statistic

is used. Here $${\widetilde{\boldsymbol{\uplambda }}}^{(\varOmega )}$$ and $${\widetilde{\boldsymbol{\uplambda }}}^{(\omega )}$$ are the maximum-likelihood estimators in the regions Ω and ω . H ₀ is rejected if T  >  T 1−α with $$P(T > T_{1-\alpha }\vert H_{0}) =\int _{ T_{1-\alpha }}^{\infty }g(T)\,\mathrm{d}T =\alpha$$ ; g ( T ) is the conditional probability density of T for a given  H ₀ .

WILKS theorem (holds for weak requirements concerning the probability density of the population): If H ₀ specifies p − r out of the p parameters, then − 2ln T (where T is the likelihood-ratio test statistic) follows a χ ² -distribution with $$f = p - r$$ degrees of freedom in the limit n  →  ∞ .

Table H.5:

Frequently used statistical tests for a sample from a normal distribution with mean λ and variance σ ² . (Case 1: σ known; case 2: σ unknown (Student’s test); case 3: χ ² -test of the variance; case 4: Student’s difference test of two samples of sizes N ₁ and N ₂ on the significance of $$s_{\varDelta }^{2}$$ , cf. ( 8.​3.​19 ); case 5: F -test of two samples).

Case

Test statistic

Null hy-pothesis

Critical regionfor test statistic

Number ofdegrees offreedom

1

λ  =  λ ₀ λ  ≤  λ ₀ λ  ≥  λ ₀

–

2

λ  =  λ ₀ λ  ≤  λ ₀ λ  ≥  λ ₀

N − 1

3

N − 1

4

λ ₁  =  λ ₂

5

χ ² -Test for Goodness-of-Fit

Hypothesis: The N measured values y i with normally distributed errors σ i are described by given quantities f i .

Test function: $$T =\sum _{ i=1}^{N}{(y_{i} - f_{i})}^{2}/\sigma _{i}^{2}$$ .

Critical region: $$T >\chi _{ 1-\alpha }^{2}$$ .

Number of degrees of freedom: N (or N − p , if p parameters are determined from the measurements).

The Method of Least Squares

One considers a set of m equations $$f_{k}(\mathbf{x},\boldsymbol{\upeta }) = 0$$ ; k  = 1,  … ,  m relating the r -vector of the unknowns x  = ( x ₁ ,  x ₂ ,  … ,  x r ) to the n -vector of measurable quantities $$\boldsymbol{\upeta } = (\eta _{1},\,\eta _{2},\,\ldots,\,\eta _{n})$$ . Instead of $$\boldsymbol{\upeta }$$ , the quantities y are measured, which deviate from them by the measurement errors $$\boldsymbol{\upvarepsilon }$$ , i.e., $$\mathbf{y} = \boldsymbol{\upeta } + \boldsymbol{\upvarepsilon }$$ . The quantities $$\boldsymbol{\upvarepsilon }$$ are assumed to be normally distributed about zero. This is expressed by the covariance matrix $$C_{y} = G_{y}^{-1}$$ . In order to obtain the solution $$\widetilde{\mathbf{x}}$$ , $$\widetilde{\boldsymbol{\upeta }}$$ , one expands the f k for the first approximations x ₀ , $$\boldsymbol{\upeta }_{0} = \mathbf{y}$$ . Only the linear terms of the expansion are kept and the second approximation $$\mathbf{x}_{1} = \mathbf{x}_{0} + \boldsymbol{\upxi }$$ , $$\boldsymbol{\upeta }_{1} = \boldsymbol{\upeta } + \boldsymbol{\updelta }$$ is computed. The procedure is repeated iteratively until certain convergence criteria are met, for example, until a scalar function M no longer decreases. If the f k are linear in x and $$\boldsymbol{\upeta }$$ , then only one step is necessary. The method can be interpreted as a procedure to minimize M . The function M corresponding to the solution depends on the measurement errors. It is a random variable and follows a χ ² -distribution with $$f = m - r$$ degrees of freedom. It can therefore be used for a χ ² - test of the goodness-of-fit or of other assumptions, in particular the assumption $$C_{y} = G_{y}^{-1}$$ . If the errors $$\boldsymbol{\upvarepsilon }$$ are not normally distribution, but still distributed symmetrically about zero, then the least-squares solution $$\widetilde{\mathbf{x}}$$ still has the smallest possible variance, and one has $$E(M) = m - r$$ (Gauss–Markov theorem) .

Table H.6:

Least squares in the general case and in the case of constrained measurements.

General case

Constrained measurements

Equations

First approx-imations

Equationsexpanded

Covariancematrix of themeasurements

Corrections

Nextstep

Solution (after s steps)

Minimumfunction

Covariancematrices

Table H.7:

Least squares for indirect and direct measurements.

Indirect measurements

Direct measurementsof differentaccuracy

Equations

First approx-imations

Equationsexpanded

Covariancematrix of themeasurements

Corrections

–

Nextstep

–

Solution(after s steps)

Minimumfunction

Covariancematrices

–

Analysis of Variance

The influence of external variables on a measured random variable x is investigated. One tries to decide by means of appropriately constructed F -tests whether x is independent of the external variables . Various models are constructed depending on the number of external variables and the assumptions concerning their influence.

A simple model is the crossed two-way classification with multiple observations. Two external variables are used to classify the observations into the classes A i , B j ( i  = 1, 2,  … ,  I ; j  = 1, 2,  … ,  J ). Each class A i , B j contains K observations x ijk ( k  = 1, 2,  … ,  K ). One assumes the model

where the error of the observation $$\varepsilon _{ijk}$$ is assumed to be normally distributed about zero and a i , b j , and ( ab ) ij are the influences of the classifications in A , B and their interactions . Three null hypotheses

can be tested with the ratios F (A) , F (B) , F (AB) . They are summarized in an analysis of variance (ANOVA) table. For other models see Chap.  11.​

Polynomial Regression

Problem: The true values η ( t ), for which one has N measurements y i ( t i ) with normally distributed measurement errors σ i , are to be described by a polynomial of order r − 1 in the controlled variable t . Instead of $$\eta (t) = x_{1} + x_{2}t + \cdots + x_{r}{t}^{r-1}$$ one writes

Here the f j are orthogonal polynomials of order j − 1,

whose coefficients b jk are determined by the orthogonality conditions

The unknowns x j are obtained by least squares from

The covariance matrix of the x j is the r -dimensional unit matrix.

Time Series Analysis

One is given a series of measured values y i ( t i ), i  = 1,  … ,  n , which (in an unknown way) depend on a controlled variable t (usually time). One treats the y i as the sum of a trend η i and an error $$\varepsilon _{i}$$ , $$y_{i} =\eta _{i} +\varepsilon _{i}$$ . The measurements are carried out at regular time intervals, i.e., $$t_{i} - t_{i-1} =$$ const. In order to minimize the errors $$\varepsilon _{i}$$ , a moving average is constructed for every t i ( i  >  k , i  ≤  n − k ), by fitting a polynomial of order ℓ to the 2 k + 1 measurements situated symmetrically about measurement i . The result of the fit at the point t i is the moving average

The coefficients a −k , … , a k are given in Table 13.​1 for low values of k and ℓ . For the beginning and end points t i ( i  <  k , i  >  n − k ), the results of the fit can be used with caution also for points other than at the center of the interval of the 2 k + 1 measurements.

Statistical Tables

Table I.1:

Quantiles λ P ( k ) of the Poisson distribution.

P

k

0.0005

0.0010

0.0050

0.0100

0.0250

0.0500

0.1000

1

7.601

6.908

5.298

4.605

3.689

2.996

2.303

2

9.999

9.233

7.430

6.638

5.572

4.744

3.890

3

12.051

11.229

9.274

8.406

7.225

6.296

5.322

4

13.934

13.062

10.977

10.045

8.767

7.754

6.681

5

15.710

14.794

12.594

11.605

10.242

9.154

7.994

6

17.411

16.455

14.150

13.108

11.668

10.513

9.275

7

19.055

18.062

15.660

14.571

13.059

11.842

10.532

8

20.654

19.626

17.134

16.000

14.423

13.148

11.771

9

22.217

21.156

18.578

17.403

15.763

14.435

12.995

10

23.749

22.657

19.998

18.783

17.085

15.705

14.206

11

25.256

24.134

21.398

20.145

18.390

16.962

15.407

12

26.739

25.589

22.779

21.490

19.682

18.208

16.598

13

28.203

27.026

24.145

22.821

20.962

19.443

17.782

14

29.650

28.446

25.497

24.139

22.230

20.669

18.958

15

31.081

29.852

26.836

25.446

23.490

21.886

20.128

16

32.498

31.244

28.164

26.743

24.740

23.097

21.292

17

33.902

32.624

29.482

28.030

25.983

24.301

22.452

18

35.294

33.993

30.791

29.310

27.219

25.499

23.606

19

36.676

35.351

32.091

30.581

28.448

26.692

24.756

20

38.047

36.701

33.383

31.845

29.671

27.879

25.903

21

39.410

38.042

34.668

33.103

30.888

29.062

27.045

22

40.764

39.375

35.946

34.355

32.101

30.240

28.184

23

42.110

40.700

37.218

35.601

33.308

31.415

29.320

24

43.449

42.019

38.484

36.841

34.511

32.585

30.453

25

44.780

43.330

39.745

38.077

35.710

33.752

31.584

k

0.9000

0.9500

0.9750

0.9900

0.9950

0.9990

0.9995

1

0.105

0.051

0.025

0.010

0.005

0.001

0.001

2

0.532

0.355

0.242

0.149

0.103

0.045

0.032

3

1.102

0.818

0.619

0.436

0.338

0.191

0.150

4

1.745

1.366

1.090

0.823

0.672

0.429

0.355

5

2.433

1.970

1.623

1.279

1.078

0.739

0.632

6

3.152

2.613

2.202

1.785

1.537

1.107

0.967

7

3.895

3.285

2.814

2.330

2.037

1.520

1.348

8

4.656

3.981

3.454

2.906

2.571

1.971

1.768

9

5.432

4.695

4.115

3.507

3.132

2.452

2.220

10

6.221

5.425

4.795

4.130

3.717

2.961

2.699

11

7.021

6.169

5.491

4.771

4.321

3.491

3.202

12

7.829

6.924

6.201

5.428

4.943

4.042

3.726

13

8.646

7.690

6.922

6.099

5.580

4.611

4.269

14

9.470

8.464

7.654

6.782

6.231

5.195

4.828

15

10.300

9.246

8.395

7.477

6.893

5.794

5.402

16

11.135

10.036

9.145

8.181

7.567

6.405

5.990

17

11.976

10.832

9.903

8.895

8.251

7.028

6.590

18

12.822

11.634

10.668

9.616

8.943

7.662

7.201

19

13.671

12.442

11.439

10.346

9.644

8.306

7.822

20

14.525

13.255

12.217

11.082

10.353

8.958

8.453

21

15.383

14.072

12.999

11.825

11.069

9.619

9.093

22

16.244

14.894

13.787

12.574

11.792

10.288

9.741

23

17.108

15.719

14.580

13.329

12.521

10.964

10.397

24

17.975

16.549

15.377

14.089

13.255

11.647

11.060

25

18.844

17.382

16.179

14.853

13.995

12.337

11.730

Table I.2:

Normal distribution ψ ₀ ( x ).

x

0

1

2

3

4

5

6

7

8

9

− 3. 0

0.001

0.001

0.001

0.001

0.001

0.001

0.001

0.001

0.001

0.001

− 2. 9

0.002

0.002

0.002

0.002

0.002

0.002

0.002

0.001

0.001

0.001

− 2. 8

0.003

0.002

0.002

0.002

0.002

0.002

0.002

0.002

0.002

0.002

− 2. 7

0.003

0.003

0.003

0.003

0.003

0.003

0.003

0.003

0.003

0.003

− 2. 6

0.005

0.005

0.004

0.004

0.004

0.004

0.004

0.004

0.004

0.004

− 2. 5

0.006

0.006

0.006

0.006

0.006

0.005

0.005

0.005

0.005

0.005

− 2. 4

0.008

0.008

0.008

0.008

0.007

0.007

0.007

0.007

0.007

0.006

− 2. 3

0.011

0.010

0.010

0.010

0.010

0.009

0.009

0.009

0.009

0.008

− 2. 2

0.014

0.014

0.013

0.013

0.013

0.012

0.012

0.012

0.011

0.011

− 2. 1

0.018

0.017

0.017

0.017

0.016

0.016

0.015

0.015

0.015

0.014

− 2. 0

0.023

0.022

0.022

0.021

0.021

0.020

0.020

0.019

0.019

0.018

− 1. 9

0.029

0.028

0.027

0.027

0.026

0.026

0.025

0.024

0.024

0.023

− 1. 8

0.036

0.035

0.034

0.034

0.033

0.032

0.031

0.031

0.030

0.029

− 1. 7

0.045

0.044

0.043

0.042

0.041

0.040

0.039

0.038

0.038

0.037

− 1. 6

0.055

0.054

0.053

0.052

0.051

0.049

0.048

0.047

0.046

0.046

− 1. 5

0.067

0.066

0.064

0.063

0.062

0.061

0.059

0.058

0.057

0.056

− 1. 4

0.081

0.079

0.078

0.076

0.075

0.074

0.072

0.071

0.069

0.068

− 1. 3

0.097

0.095

0.093

0.092

0.090

0.089

0.087

0.085

0.084

0.082

− 1. 2

0.115

0.113

0.111

0.109

0.107

0.106

0.104

0.102

0.100

0.099

− 1. 1

0.136

0.133

0.131

0.129

0.127

0.125

0.123

0.121

0.119

0.117

− 1. 0

0.159

0.156

0.154

0.152

0.149

0.147

0.145

0.142

0.140

0.138

− 0. 9

0.184

0.181

0.179

0.176

0.174

0.171

0.169

0.166

0.164

0.161

− 0. 8

0.212

0.209

0.206

0.203

0.200

0.198

0.195

0.192

0.189

0.187

− 0. 7

0.242

0.239

0.236

0.233

0.230

0.227

0.224

0.221

0.218

0.215

− 0. 6

0.274

0.271

0.268

0.264

0.261

0.258

0.255

0.251

0.248

0.245

− 0. 5

0.309

0.305

0.302

0.298

0.295

0.291

0.288

0.284

0.281

0.278

− 0. 4

0.345

0.341

0.337

0.334

0.330

0.326

0.323

0.319

0.316

0.312

− 0. 3

0.382

0.378

0.374

0.371

0.367

0.363

0.359

0.356

0.352

0.348

− 0. 2

0.421

0.417

0.413

0.409

0.405

0.401

0.397

0.394

0.390

0.386

− 0. 1

0.460

0.456

0.452

0.448

0.444

0.440

0.436

0.433

0.429

0.425

0.0

0.500

0.496

0.492

0.488

0.484

0.480

0.476

0.472

0.468

0.464

0.0

0.500

0.504

0.508

0.512

0.516

0.520

0.524

0.528

0.532

0.536

0.1

0.540

0.544

0.548

0.552

0.556

0.560

0.564

0.567

0.571

0.575

0.2

0.579

0.583

0.587

0.591

0.595

0.599

0.603

0.606

0.610

0.614

0.3

0.618

0.622

0.626

0.629

0.633

0.637

0.641

0.644

0.648

0.652

0.4

0.655

0.659

0.663

0.666

0.670

0.674

0.677

0.681

0.684

0.688

0.5

0.691

0.695

0.698

0.702

0.705

0.709

0.712

0.716

0.719

0.722

0.6

0.726

0.729

0.732

0.736

0.739

0.742

0.745

0.749

0.752

0.755

0.7

0.758

0.761

0.764

0.767

0.770

0.773

0.776

0.779

0.782

0.785

0.8

0.788

0.791

0.794

0.797

0.800

0.802

0.805

0.808

0.811

0.813

0.9

0.816

0.819

0.821

0.824

0.826

0.829

0.831

0.834

0.836

0.839

1.0

0.841

0.844

0.846

0.848

0.851

0.853

0.855

0.858

0.860

0.862

1.1

0.864

0.867

0.869

0.871

0.873

0.875

0.877

0.879

0.881

0.883

1.2

0.885

0.887

0.889

0.891

0.893

0.894

0.896

0.898

0.900

0.901

1.3

0.903

0.905

0.907

0.908

0.910

0.911

0.913

0.915

0.916

0.918

1.4

0.919

0.921

0.922

0.924

0.925

0.926

0.928

0.929

0.931

0.932

1.5

0.933

0.934

0.936

0.937

0.938

0.939

0.941

0.942

0.943

0.944

1.6

0.945

0.946

0.947

0.948

0.949

0.951

0.952

0.953

0.954

0.954

1.7

0.955

0.956

0.957

0.958

0.959

0.960

0.961

0.962

0.962

0.963

1.8

0.964

0.965

0.966

0.966

0.967

0.968

0.969

0.969

0.970

0.971

1.9

0.971

0.972

0.973

0.973

0.974

0.974

0.975

0.976

0.976

0.977

2.0

0.977

0.978

0.978

0.979

0.979

0.980

0.980

0.981

0.981

0.982

2.1

0.982

0.983

0.983

0.983

0.984

0.984

0.985

0.985

0.985

0.986

2.2

0.986

0.986

0.987

0.987

0.987

0.988

0.988

0.988

0.989

0.989

2.3

0.989

0.990

0.990

0.990

0.990

0.991

0.991

0.991

0.991

0.992

2.4

0.992

0.992

0.992

0.992

0.993

0.993

0.993

0.993

0.993

0.994

2.5

0.994

0.994

0.994

0.994

0.994

0.995

0.995

0.995

0.995

0.995

2.6

0.995

0.995

0.996

0.996

0.996

0.996

0.996

0.996

0.996

0.996

2.7

0.997

0.997

0.997

0.997

0.997

0.997

0.997

0.997

0.997

0.997

2.8

0.997

0.998

0.998

0.998

0.998

0.998

0.998

0.998

0.998

0.998

2.9

0.998

0.998

0.998

0.998

0.998

0.998

0.998

0.999

0.999

0.999

3.0

0.999

0.999

0.999

0.999

0.999

0.999

0.999

0.999

0.999

0.999

Table I.3:

Normal distribution 2 ψ ₀ ( x ) − 1.

x

0

1

2

3

4

5

6

7

8

9

0.0

0.000

0.008

0.016

0.024

0.032

0.040

0.048

0.056

0.064

0.072

0.1

0.080

0.088

0.096

0.103

0.111

0.119

0.127

0.135

0.143

0.151

0.2

0.159

0.166

0.174

0.182

0.190

0.197

0.205

0.213

0.221

0.228

0.3

0.236

0.243

0.251

0.259

0.266

0.274

0.281

0.289

0.296

0.303

0.4

0.311

0.318

0.326

0.333

0.340

0.347

0.354

0.362

0.369

0.376

0.5

0.383

0.390

0.397

0.404

0.411

0.418

0.425

0.431

0.438

0.445

0.6

0.451

0.458

0.465

0.471

0.478

0.484

0.491

0.497

0.503

0.510

0.7

0.516

0.522

0.528

0.535

0.541

0.547

0.553

0.559

0.565

0.570

0.8

0.576

0.582

0.588

0.593

0.599

0.605

0.610

0.616

0.621

0.627

0.9

0.632

0.637

0.642

0.648

0.653

0.658

0.663

0.668

0.673

0.678

1.0

0.683

0.688

0.692

0.697

0.702

0.706

0.711

0.715

0.720

0.724

1.1

0.729

0.733

0.737

0.742

0.746

0.750

0.754

0.758

0.762

0.766

1.2

0.770

0.774

0.778

0.781

0.785

0.789

0.792

0.796

0.799

0.803

1.3

0.806

0.810

0.813

0.816

0.820

0.823

0.826

0.829

0.832

0.835

1.4

0.838

0.841

0.844

0.847

0.850

0.853

0.856

0.858

0.861

0.864

1.5

0.866

0.869

0.871

0.874

0.876

0.879

0.881

0.884

0.886

0.888

1.6

0.890

0.893

0.895

0.897

0.899

0.901

0.903

0.905

0.907

0.909

1.7

0.911

0.913

0.915

0.916

0.918

0.920

0.922

0.923

0.925

0.927

1.8

0.928

0.930

0.931

0.933

0.934

0.936

0.937

0.939

0.940

0.941

1.9

0.943

0.944

0.945

0.946

0.948

0.949

0.950

0.951

0.952

0.953

2.0

0.954

0.956

0.957

0.958

0.959

0.960

0.961

0.962

0.962

0.963

2.1

0.964

0.965

0.966

0.967

0.968

0.968

0.969

0.970

0.971

0.971

2.2

0.972

0.973

0.974

0.974

0.975

0.976

0.976

0.977

0.977

0.978

2.3

0.979

0.979

0.980

0.980

0.981

0.981

0.982

0.982

0.983

0.983

2.4

0.984

0.984

0.984

0.985

0.985

0.986

0.986

0.986

0.987

0.987

2.5

0.988

0.988

0.988

0.989

0.989

0.989

0.990

0.990

0.990

0.990

2.6

0.991

0.991

0.991

0.991

0.992

0.992

0.992

0.992

0.993

0.993

2.7

0.993

0.993

0.993

0.994

0.994

0.994

0.994

0.994

0.995

0.995

2.8

0.995

0.995

0.995

0.995

0.995

0.996

0.996

0.996

0.996

0.996

2.9

0.996

0.996

0.996

0.997

0.997

0.997

0.997

0.997

0.997

0.997

3.0

0.997

0.997

0.997

0.998

0.998

0.998

0.998

0.998

0.998

0.998

Table I.4:

Quantiles x P  =  Ω ( P ) of the normal distribution.

P

0

1

2

3

4

5

6

7

8

9

0. 0

− ∞

−2. 33

−2. 05

−1. 88

−1. 75

−1. 64

−1. 55

−1. 48

−1. 41

−1. 34

0. 1

−1. 28

−1. 23

−1. 17

−1. 13

−1. 08

−1. 04

−0. 99

−0. 95

−0. 92

−0. 88

0. 2

−0. 84

−0. 81

−0. 77

−0. 74

−0. 71

−0. 67

−0. 64

−0. 61

−0. 58

−0. 55

0. 3

−0. 52

−0. 50

−0. 47

−0. 44

−0. 41

−0. 39

−0. 36

−0. 33

−0. 31

−0. 28

0. 4

−0. 25

−0. 23

−0. 20

−0. 18

−0. 15

−0. 13

−0. 10

−0. 08

−0. 05

−0. 03

0. 5

0. 00

0. 03

0. 05

0. 08

0. 10

0. 13

0. 15

0. 18

0. 20

0. 23

0. 6

0. 25

0. 28

0. 31

0. 33

0. 36

0. 39

0. 41

0. 44

0. 47

0. 50

0. 7

0. 52

0. 55

0. 58

0. 61

0. 64

0. 67

0. 71

0. 74

0. 77

0. 81

0. 8

0. 84

0. 88

0. 92

0. 95

0. 99

1. 04

1. 08

1. 13

1. 17

1. 23

0. 9

1. 28

1. 34

1. 41

1. 48

1. 55

1. 64

1. 75

1. 88

2. 05

2. 33

Table I.5:

Quantiles $$x_{P}^{{\prime}} {=\varOmega }^{{\prime}}(P)$$ of the normal distribution.

P

0

1

2

3

4

5

6

7

8

9

0. 0

0. 000

0. 013

0. 025

0. 038

0. 050

0. 063

0. 075

0. 088

0. 100

0. 113

0. 1

0. 126

0. 138

0. 151

0. 164

0. 176

0. 189

0. 202

0. 215

0. 228

0. 240

0. 2

0. 253

0. 266

0. 279

0. 292

0. 305

0. 319

0. 332

0. 345

0. 358

0. 372

0. 3

0. 385

0. 399

0. 412

0. 426

0. 440

0. 454

0. 468

0. 482

0. 496

0. 510

0. 4

0. 524

0. 539

0. 553

0. 568

0. 583

0. 598

0. 613

0. 628

0. 643

0. 659

0. 5

0. 674

0. 690

0. 706

0. 722

0. 739

0. 755

0. 772

0. 789

0. 806

0. 824

0. 6

0. 842

0. 860

0. 878

0. 896

0. 915

0. 935

0. 954

0. 974

0. 994

1. 015

0. 7

1. 036

1. 058

1. 080

1. 103

1. 126

1. 150

1. 175

1. 200

1. 227

1. 254

0. 8

1. 282

1. 311

1. 341

1. 372

1. 405

1. 440

1. 476

1. 514

1. 555

1. 598

0. 9

1. 645

1. 695

1. 751

1. 812

1. 881

1. 960

2. 054

2. 170

2. 326

2. 576

P

0

1

2

3

4

5

6

7

8

9

0. 90

1. 645

1. 650

1. 655

1. 660

1. 665

1. 670

1. 675

1. 680

1. 685

1. 690

0. 91

1. 695

1. 701

1. 706

1. 711

1. 717

1. 722

1. 728

1. 734

1. 739

1. 745

0. 92

1. 751

1. 757

1. 762

1. 768

1. 774

1. 780

1. 787

1. 793

1. 799

1. 805

0. 93

1. 812

1. 818

1. 825

1. 832

1. 838

1. 845

1. 852

1. 859

1. 866

1. 873

0. 94

1. 881

1. 888

1. 896

1. 903

1. 911

1. 919

1. 927

1. 935

1. 943

1. 951

0. 95

1. 960

1. 969

1. 977

1. 986

1. 995

2. 005

2. 014

2. 024

2. 034

2. 044

0. 96

2. 054

2. 064

2. 075

2. 086

2. 097

2. 108

2. 120

2. 132

2. 144

2. 157

0. 97

2. 170

2. 183

2. 197

2. 212

2. 226

2. 241

2. 257

2. 273

2. 290

2. 308

0. 98

2. 326

2. 346

2. 366

2. 387

2. 409

2. 432

2. 457

2. 484

2. 512

2. 543

0. 99

2. 576

2. 612

2. 652

2. 697

2. 748

2. 807

2. 878

2. 968

3. 090

3. 291

P

0

1

2

3

4

5

6

7

8

9

0. 990

2. 576

2. 579

2. 583

2. 586

2. 590

2. 594

2. 597

2. 601

2. 605

2. 608

0. 991

2. 612

2. 616

2. 620

2. 624

2. 628

2. 632

2. 636

2. 640

2. 644

2. 648

0. 992

2. 652

2. 656

2. 661

2. 665

2. 669

2. 674

2. 678

2. 683

2. 687

2. 692

0. 993

2. 697

2. 702

2. 706

2. 711

2. 716

2. 721

2. 727

2. 732

2. 737

2. 742

0. 994

2. 748

2. 753

2. 759

2. 765

2. 770

2. 776

2. 782

2. 788

2. 794

2. 801

0. 995

2. 807

2. 814

2. 820

2. 827

2. 834

2. 841

2. 848

2. 855

2. 863

2. 870

0. 996

2. 878

2. 886

2. 894

2. 903

2. 911

2. 920

2. 929

2. 938

2. 948

2. 958

0. 997

2. 968

2. 978

2. 989

3. 000

3. 011

3. 023

3. 036

3. 048

3. 062

3. 076

0. 998

3. 090

3. 105

3. 121

3. 138

3. 156

3. 175

3. 195

3. 216

3. 239

3. 264

0. 999

3. 291

3. 320

3. 353

3. 390

3. 432

3. 481

3. 540

3. 615

3. 719

3. 891

Table I.6:

χ ² -distribution F ( χ ² ).

f

χ ²

1

2

3

4

5

6

7

8

9

10

0. 1

0. 248

0. 049

0. 008

0. 001

0. 000

0. 000

0. 000

0. 000

0. 000

0. 000

0. 2

0. 345

0. 095

0. 022

0. 005

0. 001

0. 000

0. 000

0. 000

0. 000

0. 000

0. 3

0. 416

0. 139

0. 040

0. 010

0. 002

0. 001

0. 000

0. 000

0. 000

0. 000

0. 4

0. 473

0. 181

0. 060

0. 018

0. 005

0. 001

0. 000

0. 000

0. 000

0. 000

0. 5

0. 520

0. 221

0. 081

0. 026

0. 008

0. 002

0. 001

0. 000

0. 000

0. 000

0. 6

0. 561

0. 259

0. 104

0. 037

0. 012

0. 004

0. 001

0. 000

0. 000

0. 000

0. 7

0. 597

0. 295

0. 127

0. 049

0. 017

0. 006

0. 002

0. 000

0. 000

0. 000

0. 8

0. 629

0. 330

0. 151

0. 062

0. 023

0. 008

0. 003

0. 001

0. 000

0. 000

0. 9

0. 657

0. 362

0. 175

0. 075

0. 030

0. 011

0. 004

0. 001

0. 000

0. 000

1. 0

0. 683

0. 393

0. 199

0. 090

0. 037

0. 014

0. 005

0. 002

0. 001

0. 000

2. 0

0. 843

0. 632

0. 428

0. 264

0. 151

0. 080

0. 040

0. 019

0. 009

0. 004

3. 0

0. 917

0. 777

0. 608

0. 442

0. 300

0. 191

0. 115

0. 066

0. 036

0. 019

4. 0

0. 954

0. 865

0. 739

0. 594

0. 451

0. 323

0. 220

0. 143

0. 089

0. 053

5. 0

0. 975

0. 918

0. 828

0. 713

0. 584

0. 456

0. 340

0. 242

0. 166

0. 109

6. 0

0. 986

0. 950

0. 888

0. 801

0. 694

0. 577

0. 460

0. 353

0. 260

0. 185

7. 0

0. 992

0. 970

0. 928

0. 864

0. 779

0. 679

0. 571

0. 463

0. 363

0. 275

8. 0

0. 995

0. 982

0. 954

0. 908

0. 844

0. 762

0. 667

0. 567

0. 466

0. 371

9. 0

0. 997

0. 989

0. 971

0. 939

0. 891

0. 826

0. 747

0. 658

0. 563

0. 468

10. 0

0. 998

0. 993

0. 981

0. 960

0. 925

0. 875

0. 811

0. 735

0. 650

0. 560

11. 0

0. 999

0. 996

0. 988

0. 973

0. 949

0. 912

0. 861

0. 798

0. 724

0. 642

12. 0

0. 999

0. 998

0. 993

0. 983

0. 965

0. 938

0. 899

0. 849

0. 787

0. 715

13. 0

1. 000

0. 998

0. 995

0. 989

0. 977

0. 957

0. 928

0. 888

0. 837

0. 776

14. 0

1. 000

0. 999

0. 997

0. 993

0. 984

0. 970

0. 949

0. 918

0. 878

0. 827

15. 0

1. 000

0. 999

0. 998

0. 995

0. 990

0. 980

0. 964

0. 941

0. 909

0. 868

16. 0

1. 000

1. 000

0. 999

0. 997

0. 993

0. 986

0. 975

0. 958

0. 933

0. 900

17. 0

1. 000

1. 000

0. 999

0. 998

0. 996

0. 991

0. 983

0. 970

0. 951

0. 926

18. 0

1. 000

1. 000

1. 000

0. 999

0. 997

0. 994

0. 988

0. 979

0. 965

0. 945

19. 0

1. 000

1. 000

1. 000

0. 999

0. 998

0. 996

0. 992

0. 985

0. 975

0. 960

20. 0

1. 000

1. 000

1. 000

1. 000

0. 999

0. 997

0. 994

0. 990

0. 982

0. 971

Table I.7:

Quantiles χ P ² of the χ ² -distribution.

P

f

0.900

0.950

0.990

0.995

0.999

1

2.706

3.841

6.635

7.879

10.828

2

4.605

5.991

9.210

10.597

13.816

3

6.251

7.815

11.345

12.838

16.266

4

7.779

9.488

13.277

14.860

18.467

5

9.236

11.070

15.086

16.750

20.515

6

10.645

12.592

16.812

18.548

22.458

7

12.017

14.067

18.475

20.278

24.322

8

13.362

15.507

20.090

21.955

26.124

9

14.684

16.919

21.666

23.589

27.877

10

15.987

18.307

23.209

25.188

29.588

11

17.275

19.675

24.725

26.757

31.264

12

18.549

21.026

26.217

28.300

32.909

13

19.812

22.362

27.688

29.819

34.528

14

21.064

23.685

29.141

31.319

36.123

15

22.307

24.996

30.578

32.801

37.697

16

23.542

26.296

32.000

34.267

39.252

17

24.769

27.587

33.409

35.718

40.790

18

25.989

28.869

34.805

37.156

42.312

19

27.204

30.144

36.191

38.582

43.820

20

28.412

31.410

37.566

39.997

45.315

30

40.256

43.773

50.892

53.672

59.703

40

51.805

55.758

63.691

66.766

73.402

50

63.167

67.505

76.154

79.490

86.661

60

74.397

79.082

88.379

91.952

99.607

70

85.527

90.531

100.425

104.215

112.317

80

80.000

101.879

112.329

116.321

124.839

90

107.565

113.145

124.116

128.299

137.208

100

118.498

124.342

135.807

140.169

149.449

Table I.8:

Quantiles F P of the F -distribution.

f ₁

f ₂

1

2

3

4

5

6

7

8

9

10

1

39. 86

49. 50

53. 59

55. 83

57. 24

58. 20

58. 91

59. 44

59. 86

60. 19

2

8. 526

9. 000

9. 162

9. 243

9. 293

9. 326

9. 349

9. 367

9. 381

9. 392

3

5. 538

5. 462

5. 391

5. 343

5. 309

5. 285

5. 266

5. 252

5. 240

5. 230

4

4. 545

4. 325

4. 191

4. 107

4. 051

4. 010

3. 979

3. 955

3. 936

3. 920

5

4. 060

3. 780

3. 619

3. 520

3. 453

3. 405

3. 368

3. 339

3. 316

3. 297

6

3. 776

3. 463

3. 289

3. 181

3. 108

3. 055

3. 014

2. 983

2. 958

2. 937

7

3. 589

3. 257

3. 074

2. 961

2. 883

2. 827

2. 785

2. 752

2. 725

2. 703

8

3. 458

3. 113

2. 924

2. 806

2. 726

2. 668

2. 624

2. 589

2. 561

2. 538

9

3. 360

3. 006

2. 813

2. 693

2. 611

2. 551

2. 505

2. 469

2. 440

2. 416

10

3. 285

2. 924

2. 728

2. 605

2. 522

2. 461

2. 414

2. 377

2. 347

2. 323

f ₁

f ₂

1

2

3

4

5

6

7

8

9

10

1

161. 4

199. 5

215. 7

224. 6

230. 2

234. 0

236. 8

238. 9

240. 5

241. 9

2

18. 51

19. 00

19. 16

19. 25

19. 30

19. 33

19. 35

19. 37

19. 38

19. 40

3

10. 13

9. 552

9. 277

9. 117

9. 013

8. 941

8. 887

8. 845

8. 812

8. 786

4

7. 709

6. 944

6. 591

6. 388

6. 256

6. 163

6. 094

6. 041

5. 999

5. 964

5

6. 608

5. 786

5. 409

5. 192

5. 050

4. 950

4. 876

4. 818

4. 772

4. 735

6

5. 987

5. 143

4. 757

4. 534

4. 387

4. 284

4. 207

4. 147

4. 099

4. 060

7

5. 591

4. 737

4. 347

4. 120

3. 972

3. 866

3. 787

3. 726

3. 677

3. 637

8

5. 318

4. 459

4. 066

3. 838

3. 687

3. 581

3. 500

3. 438

3. 388

3. 347

9

5. 117

4. 256

3. 863

3. 633

3. 482

3. 374

3. 293

3. 230

3. 179

3. 137

10

4. 965

4. 103

3. 708

3. 478

3. 326

3. 217

3. 135

3. 072

3. 020

2. 978

f ₁

f ₂

1

2

3

4

5

6

7

8

9

10

1

647. 8

799. 5

864. 2

899. 6

921. 8

937. 1

948. 2

956. 7

963. 3

968. 6

2

38. 51

39. 00

39. 17

39. 25

39. 30

39. 33

39. 36

39. 37

39. 39

39. 40

3

17. 44

16. 04

15. 44

15. 10

14. 88

14. 73

14. 62

14. 54

14. 47

14. 42

4

12. 22

10. 65

9. 979

9. 605

9. 364

9. 197

9. 074

8. 980

8. 905

8. 844

5

10. 01

8. 434

7. 764

7. 388

7. 146

6. 978

6. 853

6. 757

6. 681

6. 619

6

8. 813

7. 260

6. 599

6. 227

5. 988

5. 820

5. 695

5. 600

5. 523

5. 461

7

8. 073

6. 542

5. 890

5. 523

5. 285

5. 119

4. 995

4. 899

4. 823

4. 761

8

7. 571

6. 059

5. 416

5. 053

4. 817

4. 652

4. 529

4. 433

4. 357

4. 295

9

7. 209

5. 715

5. 078

4. 718

4. 484

4. 320

4. 197

4. 102

4. 026

3. 964

10

6. 937

5. 456

4. 826

4. 468

4. 236

4. 072

3. 950

3. 855

3. 779

3. 717

f ₁

f ₂

1

2

3

4

5

6

7

8

9

10

1

4052

5000

5403

5625

5764

5859

5928

5981

6022

6056

2

98. 50

99. 00

99. 17

99. 25

99. 30

99. 33

99. 36

99. 37

99. 39

99. 40

3

34. 12

30. 82

29. 46

28. 71

28. 24

27. 91

27. 67

27. 49

27. 35

27. 23

4

21. 20

18. 00

16. 69

15. 98

15. 52

15. 21

14. 98

14. 80

14. 66

14. 55

5

16. 26

13. 27

12. 06

11. 39

10. 97

10. 67

10. 46

10. 29

10. 16

10. 05

6

13. 75

10. 92

9. 780

9. 148

8. 746

8. 466

8. 260

8. 102

7. 976

7. 874

7

12. 25

9. 547

8. 451

7. 847

7. 460

7. 191

6. 993

6. 840

6. 719

6. 620

8

11. 26

8. 649

7. 591

7. 006

6. 632

6. 371

6. 178

6. 029

5. 911

5. 814

9

10. 56

8. 022

6. 992

6. 422

6. 057

5. 802

5. 613

5. 467

5. 351

5. 257

10

10. 04

7. 559

6. 552

5. 994

5. 636

5. 386

5. 200

5. 057

4. 942

4. 849

Table I.9:

Quantiles t P of Student’s distribution.

P

f

0.9000

0.9500

0.9750

0.9900

0.9950

0.9990

0.9995

1

3.078

6.314

12.706

31.821

63.657

318.309

636.619

2

1.886

2.920

4.303

6.965

9.925

22.327

31.599

3

1.638

2.353

3.182

4.541

5.841

10.215

12.924

4

1.533

2.132

2.776

3.747

4.604

7.173

8.610

5

1.476

2.015

2.571

3.365

4.032

5.893

6.869

6

1.440

1.943

2.447

3.143

3.707

5.208

5.959

7

1.415

1.895

2.365

2.998

3.499

4.785

5.408

8

1.397

1.860

2.306

2.896

3.355

4.501

5.041

9

1.383

1.833

2.262

2.821

3.250

4.297

4.781

10

1.372

1.812

2.228

2.764

3.169

4.144

4.587

11

1.363

1.796

2.201

2.718

3.106

4.025

4.437

12

1.356

1.782

2.179

2.681

3.055

3.930

4.318

13

1.350

1.771

2.160

2.650

3.012

3.852

4.221

14

1.345

1.761

2.145

2.624

2.977

3.787

4.140

15

1.341

1.753

2.131

2.602

2.947

3.733

4.073

16

1.337

1.746

2.120

2.583

2.921

3.686

4.015

17

1.333

1.740

2.110

2.567

2.898

3.646

3.965

18

1.330

1.734

2.101

2.552

2.878

3.610

3.922

19

1.328

1.729

2.093

2.539

2.861

3.579

3.883

20

1.325

1.725

2.086

2.528

2.845

3.552

3.850

30

1.310

1.697

2.042

2.457

2.750

3.385

3.646

40

1.303

1.684

2.021

2.423

2.704

3.307

3.551

50

1.299

1.676

2.009

2.403

2.678

3.261

3.496

60

1.296

1.671

2.000

2.390

2.660

3.232

3.460

70

1.294

1.667

1.994

2.381

2.648

3.211

3.435

80

1.292

1.664

1.990

2.374

2.639

3.195

3.416

90

1.291

1.662

1.987

2.368

2.632

3.183

3.402

100

1.290

1.660

1.984

2.364

2.626

3.174

3.390

200

1.286

1.653

1.972

2.345

2.601

3.131

3.340

500

1.283

1.648

1.965

2.334

2.586

3.107

3.310

1000

1.282

1.646

1.962

2.330

2.581

3.098

3.300

Literature

Literature Cited in the Text

[1]

A. Kolmogorov, Ergebn. Math. 2 (1933) 3

[2]

D.E. Knuth, The Art of Computer Programming , vol. 2, Addison-Wesley, Reading MA 1981 MATH

[3]

P. L’Ecuyer, Comm. ACM 31 (1988) 742 MathSciNetCrossref

[4]

B.A. Wichmann and I.D. Hill, Appl. Stat. 31 (1982) 188

[5]

G.E.P. Box and M.E. Muller, Ann. Math. Stat. 29 (1958) 611 Crossref

[6]

L. von Bortkiewicz, Das Gesetz der kleinen Zahlen , Teubner, Leipzig 1898

[7]

D.J. De Solla Price, Little Science, Big Science , Columbia University Press, New York 1965

[8]

M.G. Kendall and A. Stuart, The Advanced Theory of Statistics , vol. 2, Charles Griffin, London 1968 MATH

[9]

S.S. Wilks, Ann. Math. Stat. , 9 (1938) 60 Crossref

[10]

A.H. Rosenfeld, H.A. Barbero-Galtieri, W.J. Podolski, L.R. Price, P. Söding, Ch.G. Wohl, M. Roos, and W.J. Willis, Rev. Mod. Phys. 33 (1967) 1 ADSCrossref

[11]

Particle Data Group, Physics Letters B204 (1988) 1

[12]

W.H. Press, B.P. Flannery, S.A. Teukolsky and W.T. Vetterling, Numerical Recipes , Cambridge University Press, Cambridge 1986 MATH

[13]

R.P. Brent, Algorithms for Minimization without Derivatives , Prentice-Hall, Englewood Cliffs NJ 1973 MATH

[14]

J.A. Nelder and R. Mead, Computer Journal 7 (1965) 308

[15]

M.J.D. Powell, Computer Journal 7 (1965) 155 Crossref

[16]

D.W. Marquardt, J. Soc. Ind. Appl. Math. 11 (1963) 431 Crossref

[17]

C. Lanczos, SIAM J. Numerical Analysis 1 (1964) 86 ADS

[18]

C.L. Lawson and R.J. Hanson, Solving Least Squares Problems , Prentice-Hall, Englewood Cliffs NJ 1974 MATH

[19]

G.H. Golub and W. Kahn, SIAM J. Numerical Analysis 2 (1965) 205 ADS

[20]

P.A. Businger and G.H. Golub, Comm. ACM 12 (1969) 564

[21]

G.H. Golub and C. Reinsch, in Linear Algebra ( J.H. Wilkinson and C. Reinsch, eds.), p. 134, Springer, Berlin 1971

[22]

J.G.F. Francis, Computer Journal 4 (1960) 265, 332

[23]

F. James and M. Roos, Nuclear Physics B172 (1980) 475

[24]

M.L. Swartz, Nuclear Instruments and Methods A294 (1966) 278

[25]

V.H. Regener, Physical Review 84 (1951) 161 ADSCrossref

[26]

G. Zech, Nuclear Instruments and Methods A277 (1989) 608

[27]

H. Rutishauser, Numerische Mathematik 5 (1963) 48 MathSciNetCrossref

[28]

W. Romberg, Det. Kong. Norske Videnskapers Selskap Forhandlinger 28 (1955) Nr. 7

[29]

K.S. Koelbig, in CERN Computer Centre Program Library , Program D401, CERN, Geneva 1990

Bibliography

The following short list contains several books which can serve to extend the breadth and depth of the present material. It is of course in no way complete, but instead merely indicates a number of books of varied levels of difficulty.

Probability

L. Breiman,  Probability , Addison-Wesley, Reading MA 1968 MATH

H. Cramèr,  The Elements of Probability Theory , Wiley, New York 1955

B.V. Gnedenko,  The Theory of Probability, 4th ed. , Chelsea, New York 1967

Kai Lai Chung,  A Course in Probability Theory , Harcourt, Brace and World, New York 1968

K. Krickeberg,  Wahrscheinlichkeitsrechnung , Teubner, Stuttgart 1963

Mathematical Statistics

P.R. Bevington,  Data Reduction and Error Analysis for the Physical Sciences , McGraw-Hill, New York 1969

B.E. Cooper,  Statistics for Experimentalists , Pergamon, Oxford 1969

G. Cowan,  Statistical Data Analysis , Clarendon Press, Oxford 1998

H. Cramèr,  Mathematical Methods of Statistics , University Press, Princeton 1946 MATH

W.J. Dixon and F.J. Massey , Introduction to Statistical Analysis , McGraw-Hill, New York 1969

D. Dumas de Rauly,  L’Estimation Statistique , Gauthier-Villars, Paris 1968 MATH

W.T. Eadie,  D. Drijard, F.E. James, M. Roos and B.Sadoulet , Statistical Methods in Experimental Physics , North-Holland, Amsterdam 1971

W. Feller,  An Introduction to Probability Theory and its Allpications , 2 Vols., Wiley, New York 1968 MATH

M. Fisz,  Probability Theory and Mathematical Statistics , Wiley, New York 1963 MATH

D.A.S. Fraser,  Statistics; An Introduction , Wiley, New York 1958 CrossrefMATH

H. Freeman,  Introduction to Statistical Inference , Addison-Wesley, Reading, MA 1963 MATH

A.G. Frodensen and O. Skjeggestad , Probability and Statistics in Particle Physics , Universitetsforlaget, Bergen 1979

P.G. Hoel,  Introduction to Mathematical Statistics, 4th ed. , Wiley, New York 1971

M.G. Kendall and A. Stuart   The Advanced Theory of Statistics, 4th ed., 3 vols. , Charles Griffin, London 1977

L. Lyons,  Statistics for Nuclear and Particle Physicists , Cambridge University Press, Cambridge 1986

J. Mandel,  The Statistical Analysis of Experimental Data , Interscience, New York 1964

S.L. Meyer,  Data Analysis for Scientists and Engineers , Wiley, New York 1975

L. Sachs,  Statistische Auswertungsmethoden, 5. Aufl. , Springer, Berlin 1978

E. Sverdrup,  Laws and Chance Variations , 2 Bde., North-Holland, Amsterdam 1967

B.L. van der Waerden,  Mathematische Statistik, 3. Aufl. , Springer, Berlin 1971

S.S. Wilks,  Mathematical Statistics , Wiley, New York 1962 MATH

T. Yamane,  Elementary Sampling Theory , Prentice-Hall, EnglewoodCliffs, NJ 1967 MATH

T. Yamane,  Statistics, An Introductory Analysis , Harper and Row, New York 1967

Numerical Methods: Matrix Programs

A. Björk and G. Dahlquist ,  Numerische Methoden , R. OldenbourgVerlag, München 1972

R.P. Brent,  Algorithms for Minimization without Derivatives , Prentice-Hall, Englewood Cliffs, NJ 1973 MATH

C.T. Fike,  Computer Evaluation of Mathematical Functions , Prentice-Hall, Englewood Cliffs, NJ 1968 MATH

G.H. Golub and C.F. Van Loan , Matrix Computations , Johns Hopkins University Press, Baltimore 1983

R.W. Hamming,  Numerical Methods for Engineers and Scientists, 2nd ed. , McGraw-Hill, New York 1973 MATH

D.E. Knuth,  The Art of Computer Programming , 3 vols., Addison-Wesley, Reading MA 1968 MATH

C.L. Lawson and R.J. Hanson , Solving Least Squares Problems , Prentice-Hall, Englewood Cliffs NJ 1974

W.H. Press,  B.P. Flannery, S.A. Teukolsky , and W.T. Vetterling , Numerical Recipes , Cambridge University Press, Cambridge 1986

J.R. Rice,  Numerical Methods, Software and Analysis , McGraw-Hill, New York 1983

J. Stoer and R. Burlisch ,  Einführung in die Numerische Mathematik , 2 Bde., Springer, Berlin 1983

J.H. Wilkinson and C. Reinsch , Linear Algebra , Springer, Berlin 1971

Collections of Formulas: Statistical Tables

M. Abramowitz and A. Stegun , Handbook of Mathematical Functions , Dover, New York, 1965

R.A. Fisher and F. Yates , Statistical Tables for Biological, Agricultural and Medical Research , Oliver and Boyd, London 1957

U. Graf and H.J. Henning , Formeln und Tabellen zur Mathematischen Statistik , Springer, Berlin 1953

A. Hald,  Statistical Tables and Formulas , Wiley, New York 1960 MATH

G.A. Korn and T.M. Korn , Mathematical Handbook for Scientists and Engineers, 2nd ed. , McGraw-Hill, New York 1968

D.V. Lindley and J.C.P. Miller , Cambridge Elementary Statistical Tables , University Press, Cambridge 1961

D.B. Owen,  Handbook of Statistical Tables , Addison-Wesley, Reading MA 1962 MATH

E.S. Pearson and H.O. Hartley , Biometrica Tables for Statisticians , University Press, Cambridge 1958

W. Wetzel, M.D. Jöhnk, and P. Naeve , Statistische Tabellen , Walter de Gruyter, Berlin 1967

Index

A

a posteriori probability

acceptance probability

addition of errors quadratic

alternative hypothesis

analysis of variance

model

one-way

table

two-way

crossed classification

nested classification

and

ANOVA

table

asymmetric error

asymptotically unbiased estimator

average moving

B

background

backward substitution

basis

beta function

incomplete

bias

bidiagonal matrix

binomial coefficient

binomial distribution

parameter estimation

binomial theorem

bit

bracketing the minimum

Breit–Wigner distribution

byte

C

Cauchy distribution

central limit theorem

characteristic equation

χ 2-distribution

quantiles

χ2-test

Cholesky decomposition

Cholesky inversion

classification

crossed

nested

one-way

two-way

clipping region

cofactor

color index

column space

column vector

combination

combinatorics

computing coordinates

confidence

ellipsoid

interval

level

limits

region

conjugate directions

consistent estimator

constrained measurements

constraint

equations

contingency table

continued fraction

contour line

controlled variable

convolution

of uniform distribution and normal distribution

of uniform distributions

with the normal distribution

coordinate cross

in graphics

correlation coefficient

correlation coefficient of a sample

counted number

covariance

ellipse

ellipsoid

matrix

weighted

critical region

D

decile

degrees of freedom

derivative

logarithmic

determinant

device coordinates

diagonal matrix

dice

direct measurements

of different accuracy

of equal accuracy

direct sum

dispersion

distribution

binomial

Breit–Wigner

Breit–Wigner

Cauchy

χ2

quantiles

F

quantiles

frequency

function

of sample

of several variables

of several variables

of two variables

Gaussian

hypergeometric

Lorentz

multivariate normal

multivariate normal

normal

quantiles

of a continuous variable

of a discrete variable

Poisson

Polya

standard normal

quantiles

Student’s

quantiles

t

quantiles

triangular

uniform

unimodal

E

efficient estimator

eigenvalue

equation

eigenvector

elements

equations of constraint

error

asymmetric

combination

of mean

of sample variance

of the first kind

of the second kind

standard

statistical

symmetric

error bars

error function

error model of Laplace

error propagation

E space

estimation

estimator

asymptotically unbiased

consistent

efficiency of

efficient

minimum variance

unbiased

unique

event

expectation value

experiment

F

factorial

F-distribution

quantiles

fit

of a Breit–Wigner-function

of a polynomial

of a power law

of an arbitrary linear function

of an exponential function

of a Breit–Wigner function

of a circle to points with measurement errors in abscissa and ordinate

of a Gaussian

of a nonlinear function

of a polynomial

of a proportionality

of a straight line

of a straight line to points with measurement errors in abscissa and ordinate

of a sum of exponential functions,

of a sum of two Gaussians and a polynomial

fluctuation

statistical

forward substitution

frequency

frequency distribution

F-test

full width at half maximum (FWHM)

functional equation

FWHM

G

Galton’s board

gamma function

incomplete

Gauss–Markov theorem

Gaussian distribution

multivariate

Gaussian elimination

with pivoting

Givens transformation

go button

golden section

graphics

class

workstation

H

Hessian matrix

histogram

bin width

determination of parameters from,

Householder transformation

hypergeometric distribution

hyperplane

hypothesis

alternative

composite

null

simple

test of

I

implementation

incomplete beta function

incomplete gamma function

independence

of events

of random variables

indirect measurements

linear case

nonlinear case

information

inequality

of a sample

input group

interaction

inverse matrix

J

Jacobian determinant

of an orthogonal transformation,

K

kernel

Kronecker symbol

L

Lagrange function

Lagrange multipliers

Laplace error model

law of large numbers

LCG

least squares

according to Marquardt

constrained measurements

direct measurements

general case

indirect measurements

properties of the solution

with constraints

with change of scale

with weights

likelihood

equation

function

logarithmic

ratio

likelihood-ratio test

linear combination

linear system of equations

Lorentz distribution

lotto

LR-decomposition

M

main diagonal

mantissa

mapping

marginal probability density

Marquardt minimization

matrix

addition

adjoint

antisymmetric

bidiagonal

diagonal

equations

inverse

main diagonal

multiplication

by a constant

by a matrix

norm

null

orthogonal

product

pseudo-inverse

rank

singular

subtraction

symmetric

transposition

triangular

tridiagonal

unit

maximum likelihood

maximum-likelihood estimates

mean

error

of sample

mean square

mean square deviation

median

minimization

combined method by BRENT

along a direction

along chosen directions

along the coordinate directions

bracketing the minimum

by the method of POWELL

choice of method

errors in

examples

golden section

in the direction of steepest descent,

Marquardt procedure

quadratic interpolation

simplex method

with the quadratic form

minimum variance

bound

estimator

MLCG

mode

moments

about the mean

Monte Carlo method

for simulation

minimization

of integration

moving average

N

Neyman–Pearson lemma

norm Euclidian

normal distribution

multivariate

quantiles

standard

normal equations

null hypothesis

null matrix

null space

null vector

number-input region

O

one-sided test

operating characteristic function

orthogonal

orthogonal complements

orthogonal matrix

orthogonal polynomials

orthogonal transformation

P

parabola through three points

Pascal’s triangle

permutation

permutation transformation

pivot

Poisson distribution

parameter estimation

quantiles

Polya distribution

polyline

polymarker

polynomial orthogonal

polynomial regression

population

portability

power function

precision

absolute

relative

primitive element

principal axes

principal axis transformation

probability

a posteriori

conditional

density

conditional

joint

marginal

of several variables

of two variables

frequency definition

total

pseudo-inverse matrix

Q

quadratic average of individual errors

quantile

quartile

R

radio-button group

random component

random number generator

linear congruential (LCG)

multiplicative linear congruential (MLCG)

random numbers

arbitrarily distributed

acceptance-rejection technique,

transformation procedures

normally distributed

random variable

continuous

discrete

rank

ratio of small numbers of events

with background

reduced variable

regression

curve

polynomial

representation of numbers in a computer

row space

row vector

S

sample

correlation coefficient

distribution function

error

of variance

of mean

from a bivariate normal distribution,

from a continuous population

from finite population

from Gaussian distribution

from partitioned population

from subpopulation

graphical representation

information

mean

random

size

small

with background

space

variance

scalar

scalar product

scale factor

scale in graphics

scatter diagram

one-dimensional

two-dimensional

seed

sign inversion

signal

significance level

simplex

singular matrix

singular value

singular value analysis

singular value decomposition

skewness

small numbers of events

ratio

with background

small sample

span

standard deviation

standard normal distribution

statistic

test

statistical error

statistical test

steepest descent

step diagram

Student’s difference test

Student’s distribution

quantiles

Student’s test

subpopulation

subspace

sum of squares

symmetric errors

system of equations

linear

triangular

underdetermined

T

t-distribution

quantiles

test

χ2

likelihood-ratio

most powerful

one-sided

statistic

Student’s

two-sided

unbiased

uniformly most powerful

text in plot

three-door game

time series

analysis

extrapolation

discontinuities

transformation

Givens

Householder

linear

of a vector

of variables

orthogonal

permutation

principal axis

sign inversion

transposition

trend

triangular distribution

triangular matrix

triangular system of equations

tridiagonal matrix

two-sided test

2×2 table

2×2 table test

U

underdetermined system of equations

uniform distribution

unit matrix

unit vector

V

variance

of sample

of an estimator

of a random variable

vector

absolute value

components

norm

null

row

space

basis

closed

dimension

transformation

unit

vectors

linearly dependent

linearly independent

orthonormal

viewport

W

weight matrix

weighted covariance matrix

width

full at half maximum

Wilks theorem

window

world coordinates