Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. Granger

Fundamentals of Clinical Trials5th ed. 2015

Lawrence M. Friedman

North Bethesda, MD, USA

Curt D. Furberg

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

David L. DeMets

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

David M. Reboussin

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

Christopher B. Granger

Department of Medicine, Duke University, Durham, NC, USA

ISBN 978-3-319-18538-5e-ISBN 978-3-319-18539-2

DOI 10.1007/978-3-319-18539-2

Springer Cham Heidelberg New York Dordrecht London

Library of Congress Control Number: 2015942127

© Springer International Publishing Switzerland 2015

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation, broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed.

The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use.

The publisher, the authors and the editors are safe to assume that the advice and information in this book are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or the editors give a warranty, express or implied, with respect to the material contained herein or for any errors or omissions that may have been made.

Printed on acid-free paper

Springer International Publishing AG Switzerland is part of Springer Science+Business Media (www.springer.com)

Preface

The clinical trial is “the most definitive tool for evaluation of the applicability of clinical research.” It represents “a key research activity with the potential to improve the quality of health care and control costs through careful comparison of alternative treatments” [1]. It has been called on many occasions, “the gold standard” against which all other clinical research is measured.

Although many clinical trials are of high quality, a careful reader of the medical literature will notice that a large number have deficiencies in design, conduct, analysis, presentation, and/or interpretation of results. Improvements have occurred over the past few decades, but too many trials are still conducted without adequate attention to the fundamental principles. Certainly, numerous studies could have been improved if the authors had had a better understanding of the fundamentals.

Since the publication of the first edition of this book in 1981, a large number of other texts on clinical trials have appeared, most of which are indicated here [2–21]. Several of them, however, discuss only specific issues involved in clinical trials. Additionally, many are no longer current. The purpose of this fifth edition is to update areas in which major progress has been made since the publication of the fourth edition. We have revised most chapters considerably. Because it was becoming unwieldy, we divided the chapter on monitoring response variables into two chapters, one on monitoring committees and the other on monitoring approaches. We also added a chapter on regulatory issues.

Importantly, two new authors are now involved. This brings fresh perspectives to a book originally published over three decades ago.

In this book, we hope to assist investigators in improving the quality of their clinical trials by discussing fundamental concepts with examples from our experience and the literature. The book is intended both for investigators with some clinical trial experience and for those who plan to conduct a trial for the first time. It is also intended to be used in the teaching of clinical trial methodology and to assist members of the scientific and medical community who wish to evaluate and interpret published reports of trials. Although not a technically oriented book, it may be used as a reference for graduate courses in clinical trials. Those readers who wish to consult more technical books and articles are provided with the relevant literature.

Because of the considerable differences in background and objectives of the intended readership, we have not attempted to provide exercises at the end of each chapter. We have, however, found two exercises to be quite useful and that apply most of the fundamental principles of this text. First, ask students to critique a clinical trial article from the current literature. Second, have each student develop a protocol on a clinically relevant research question that is of interest to the student. These draft protocols can often be turned into protocols that are implemented. Although there is a chapter on regulatory issues, this book is not meant to replace going to the actual agencies for guidance on regulations and policies. Those differ among countries and frequently change. Rather, as the title indicates, we hope to provide the fundamentals of clinical trials ethics, design, conduct, analysis, and reporting.

The first chapter describes the rationale and phases of clinical trials. Chapter 2 covers selected ethical issues. Chapter 3 describes the questions that clinical trials seek to answer and Chap. 4 discusses the populations from which the study samples are derived. The strengths and weaknesses of various kinds of study designs, including noninferiority trials, are reviewed in Chap. 5 . The process of randomization is covered in Chap. 6 . In Chap. 7 , we discuss the importance of and difficulties in maintaining blinding. How the sample size is estimated is covered in Chap. 8 . Chapter 9 describes what constitutes the baseline measures. Chapter 10 reviews recruitment techniques and may be of special interest to investigators not having ready access to trial participants. Methods for collecting high-quality data and some common problems in data collection are included in Chap. 11 . Chapters 12 and 13 focus on assessment of harm and health-related quality of life that are important clinical trial outcomes. Measures to enhance and monitor participant adherence are presented in Chap. 14 . Chapter 15 reviews techniques of survival analysis. Chapter 16 presents the functions of data monitoring committees and Chap. 17 reviews methods of data monitoring. Which data should be analyzed? The authors develop this question in Chap. 18 by discussing reasons for not withdrawing participants from analysis. Topics such as subgroup analysis and meta-analysis are also addressed. Chapter 19 deals with phasing out clinical trials and Chap. 20 with reporting and interpretation of results. In Chap. 21 , we present information about multicenter, including multinational, studies, which have features requiring special attention. Several points covered in Chap. 21 may also be of value to investigators conducting single center studies. Finally, selected regulatory issues, as they apply to clinical trials are reviewed in Chap. 22 .

This book is a collaborative effort and is based on knowledge gained in over four decades of developing, conducting, overseeing, and analyzing data from a number of clinical trials. This experience is chiefly, but not exclusively, in trials of heart and lung diseases, AIDS, and cancer. As a consequence, many of the examples cited are based on work done in these fields. However, the principles are applicable to clinical trials in general. The reader will note that although the book contains examples that are relatively recent, others are quite old. The fundamentals of clinical trials were developed in those older studies, and we cite them because, despite important advances, many of the basic features remain unchanged.

In the first edition, the authors had read or were familiar with much of the relevant literature on the design, conduct, and analysis of clinical trials. Today, that task would be nearly impossible as the literature over the past three and a half decades has expanded enormously. The references used in this text are not meant to be exhaustive but rather to include the literature that established the fundamentals and newer publications that support the basic concepts.

The views expressed in this book are those of the authors and do not necessarily represent the views of the institutions with which the authors have been or are affiliated.

References

1. NIH Inventory of Clinical Trials: Fiscal Year 1979. Volume 1. National Institutes of Health, Division of Research Grants, Research Analysis and Evaluation Branch, Bethesda, MD.

2. Bulpitt CJ. Randomised Controlled Clinical Trials . The Hague: Martinus Nijhoff, 1983.

3. Pocock SJ. Clinical Trials—A Practical Approach . New York: John Wiley and Sons, 1983.

4. Ingelfinger JA, Mosteller F, Thibodeau LA, et al. Biostatistics in Clinical Medicine . New York: Macmillan, 1983.

5. Iber FL Riley WA, Murray PJ. Conducting Clinical Trials . New York: Plenum, 1987.

6. Peace KE (ed.). Statistical Issues in Drug Research and Development . New York: Marcel Dekker, 1990.

7. Spilker B. Guide to Clinical Trials . New York: Raven Press, 1991.

8. Spriet A, Dupin-Spriet T, Simon P. Methodology of Clinical Drug Trials (2 nd edition). Basel: Karger, 1994.

9. Chow S-C, Shao J. Statistics in Drug Research : Methodologies and Recent Developments . New York: Marcel Dekker, 2002.

10. Rosenberger WF, Lachin JM. Randomization in Clinical Trials: Theory and Practice . New York: Wiley, 2002.

11. Geller NL (ed.). Advances in Clinical Trial Biostatistics . New York: Marcel Dekker, 2003.

12. Piantadosi S. Clinical Trials: A Methodologic Perspective (2nd edition). New York: John Wiley and Sons, 2005.

13. Matthews JNS. An Introduction to Randomised Controlled Clinical Trials (2 nd edition). Boca Raton: Chapman & Hall/CRC, 2006.

14. Machin D, Day S, Green S. Textbook of Clinical Trials (2 nd edition). West Sussex: John Wiley and Sons, 2006.

15. Keech A, Gebski V, Pike R (eds.). Interpreting and Reporting Clinical Trials . Sidney: Australasian Medical Publishing Company, 2007.

16. Cook TD, DeMets DL (eds.). Introduction to Statistical Methods for Clinical Trials . Boca Raton: Chapman & Hall/CRC, Taylor & Francis Group, LLC, 2008.

17. Machin D, Fayers P. Randomized Clinical Trials: Design , Practice and Reporting . Chichester, West Sussex: Wiley-Blackwell, 2010.

18. Green S, Benedetti J, Crowley J. Clinical Trials in Oncology (3rd edition). Boca Raton: Chapman & Hall/CRC Press, 2012.

19. Meinert CL. Clinical Trials: Design, Conduct, and Analysis (2 nd edition). New York: Oxford University Press, 2012.

20. Hulley SB, Cummings SR, Browner WS, et al. Designing Clinical Research (4 rd edition). New York: Wolters Kluwer/Lippincott Williams & Wilkins, 2013.

21. Chow S-C, Liu J-P. Design and Analysis of Clinical Trials: Concepts and Methodologies (3 rd edition). Hoboken, NJ: Wiley, 2014.

Lawrence M. Friedman

Curt D. Furberg

David L. DeMets

David M. Reboussin

Christopher B. Granger

Acknowledgments

Most of the ideas and concepts discussed in this book represent what we first learned during our years at the National Heart, Lung, and Blood Institute and in academia. We are indebted to many colleagues, and particularly in the case of the original three authors, to the late Dr. Max Halperin, with whom we had numerous hours of discussion for the earlier editions on theoretical and operational aspects of the design, conduct, and analysis of clinical trials.

Many have contributed to this fifth edition of the book. We appreciate the efforts of Drs. Michelle Naughton and Sally Shumaker in revising the chapter on health-related quality of life and Mr. Thomas Moore for his contributions to the chapter on assessment and reporting of harm. Also appreciated are the constructive comments of Drs. Bengt Furberg and Bradi Granger. We want to particularly acknowledge the outstanding administrative support of Donna Ashford and Catherine Dalsing. Finally, nobody deserves more credit than our families and colleagues who have unfailingly encouraged us in this effort. This is especially true of Gene, Birgitta, Kathy, Beth, and Bradi, who gave up many evenings and weekends with their spouses.

Contents

1 Introduction to Clinical Trials 1

Fundamental Point 2

What Is a Clinical Trial?​ 2

Clinical Trial Phases 4

Phase I Studies 5

Phase II Studies 7

Phase III/​IV Trials 9

Why Are Clinical Trials Needed?​ 10

Problems in the Timing of a Trial 14

Study Protocol 15

Appendices 17

References 18

2 Ethical Issues 25

Fundamental Point 27

Planning and Design 27

Ethics Training 27

Does the Question Require a Clinical Trial?​ 27

Randomization 30

Control Group 31

Protection from Conflicts of Interest 33

Informed Consent 34

Conduct 37

Trials in Low- and Middle-Income Countries 37

Recruitment 38

Safety and Efficacy Monitoring 39

Early Termination for Other Than Scientific or Safety Reasons 40

Privacy and Confidentiality 41

Data Falsification 41

Reporting 42

Publication Bias, Suppression, and Delays 42

Conflicts of Interest and Publication 43

References 43

3 What Is the Question?​ 49

Fundamental Point 50

Selection of the Questions 50

Primary Question 50

Secondary Questions Regarding Benefit 51

Questions Regarding Harm 52

Ancillary Questions 53

Kinds of Trials 54

Trials with Extensive Data Collection vs.​ Large, Simple 54

Superiority vs.​ Noninferiority Trials 55

Comparative Effectiveness Trials 55

Intervention 56

Response Variables 57

Kinds of Response Variables 57

Specifying the Question 60

Biomarkers and Surrogate Response Variables 62

Changing the Question 65

General Comments 66

References 67

4 Study Population 73

Fundamental Point 73

Definition of Study Population 74

Considerations in Defining the Study Population 75

Potential for Benefit 76

High Likelihood of Showing Benefit 78

Avoiding Adverse Effects 80

Competing Risk 81

Avoiding Poor Adherers 81

Pharmacogenetics​ 82

Generalization 83

Recruitment 85

References 86

5 Basic Study Design 89

Fundamental Point 90

Overview 90

Randomized Control Trials 92

Nonrandomized Concurrent Control Studies 94

Historical Controls and Databases 95

Strengths of Historical Control Studies 95

Limitations of Historical Control Studies 96

Role of Historical Controls 100

Cross-Over Designs 102

Withdrawal Studies 103

Factorial Design 104

Group Allocation Designs 106

Hybrid Designs 107

Large, Simple and Pragmatic Clinical Trials 107

Studies of Equivalency and Noninferiority 109

Adaptive Designs 114

References 115

6 The Randomization Process 123

Fundamental Point 123

Fixed Allocation Randomization 124

Simple Randomization 125

Blocked Randomization 126

Stratified Randomization 128

Adaptive Randomization Procedures 131

Baseline Adaptive Randomization Procedures 132

Minimization 133

Response Adaptive Randomization 135

Mechanics of Randomization 137

Recommendations 139

Appendix:​ Adaptive Randomization Algorithm 139

References 141

7 Blinding 147

Fundamental Point 148

Who Is Blinded?​ 148

Types of Blinding 148

Unblinded 148

Single-Blind 150

Double-Blind 151

Triple-Blind 153

Protecting the Double-Blind Design 154

Matching of Drugs 155

Coding of Drugs 157

Official Unblinding 158

Inadvertent Unblinding 159

Assessment and Reporting of Blinding 160

Debriefing of Participants 161

References 162

8 Sample Size 165

Fundamental Point 166

Statistical Concepts 167

Dichotomous Response Variables 171

Two Independent Samples 171

Paired Dichotomous Response 177

Adjusting Sample Size to Compensate for Nonadherence 178

Sample Size Calculations for Continuous Response Variables 179

Two Independent Samples 180

Paired Data 181

Sample Size for Repeated Measures 183

Sample Size Calculations for “Time to Failure” 184

Sample Size for Testing “Equivalency” or Noninferiority of Interventions 188

Sample Size for Cluster Randomization 189

Multiple Response Variables 192

Estimating Sample Size Parameters 193

References 195

9 Baseline Assessment 201

Fundamental Point 201

Uses of Baseline Data 201

Description of Trial Participants 201

Baseline Comparability 202

Controlling for Imbalances in the Analysis 204

Subgrouping 205

What Constitutes a True Baseline Measurement?​ 207

Screening for Participants 207

Regression Toward the Mean 208

Interim Events 210

Uncertainty About Qualifying Diagnosis 210

Contamination of the Intervention 211

Changes of Baseline Measurement 212

References 213

10 Recruitment of Study Participants 215

Fundamental Point 215

Considerations Before Participant Enrollment 216

Selection of Study Sample 216

Common Recruitment Problems 217

Planning 219

Recruitment Sources 221

Conduct 223

Monitoring 225

Approaches to Lagging Recruitment 229

References 230

11 Data Collection and Quality Control 233

Fundamental Point 234

Problems in Data Collection 235

Major Types 235

Minimizing Poor Quality Data 238

Design of Protocol and Manual 238

Development of Forms and Data Entry Tools 239

Training and Certification 239

Pretesting 241

Techniques to Reduce Variability Including Central Adjudication of Events 241

Data Entry 243

Electronic Source Data 243

Quality Monitoring 244

Monitoring of Data 246

Monitoring of Procedures 246

Monitoring of Drug Handling 247

Audits 248

References 250

12 Assessment and Reporting of Harm 255

Fundamental Point 256

Assessment of Harm 256

Strengths 256

Limitations 257

Identification of Harm in Clinical Trials 259

Classification of Adverse Events 260

Ascertainment 262

Prespecified Adverse Events 263

Characteristics of Adverse Events 264

Length of Follow-up 264

Analyzing Adverse Events 266

Standard Reporting 266

Prespecified Analysis 267

Post Hoc Analysis 267

Meta-analysis 268

Reporting of Harm 269

Scientific Journal Publication 271

Regulatory Considerations 272

Recommendations for Assessing and Reporting Harm 273

References 274

13 Assessment of Health Related Quality of Life 279

Fundamental Point 279

Types of HRQL Measures 280

Primary Measures 280

Additional Measures 281

Uses of HRQL Measures 282

Methodological Issues 284

Design Issues 284

Study Population 285

Type of Intervention 285

Frequency of Assessment (Acute Versus Chronic) 286

Protocol Considerations 287

Modifying and Mediating Factors 288

Selection of HRQL Instruments 288

Types of Measures 289

Scoring of HRQL Measures 290

Determining the Significance of HRQL Measures 290

Utility Measures/​Preference Scaling and Comparative Effectiveness Research 291

References 292

14 Participant Adherence 297

Fundamental Point 298

Definitions 298

Medication Adherence 299

Considerations Before Participant Enrollment 300

Design Factors 300

Participant Factors 302

Maintaining Good Participant Adherence 306

Adherence Monitoring 311

Dealing with Low Adherence 314

Special Populations 315

References 315

15 Survival Analysis 319

Fundamental Point 319

Estimation of the Survival Curve 320

Cutler-Ederer Estimate 324

Kaplan-Meier Estimate 324

Comparison of Two Survival Curves 328

Point-by-Point Comparison 329

Comparison of Median Survival Times 329

Total Curve Comparison 330

Generalizations 335

Covariate Adjusted Analysis 337

References 340

16 Monitoring Committee Structure and Function 343

Fundamental Point 346

Monitoring Committee 346

Repeated Testing for Significance 350

Decision for Early Termination 352

Decision to Extend a Trial 363

Accelerated Approval Paradigm 367

References 368

17 Statistical Methods Used in Interim Monitoring 373

Fundamental Point 373

Classical Sequential Methods 374

Group Sequential Methods 375

Flexible Group Sequential Procedures:​ Alpha Spending Functions 378

Applications of Group Sequential Boundaries 381

Asymmetric Boundaries 384

Curtailed Sampling and Conditional Power Procedures 386

Other Approaches 391

Trend Adaptive Designs and Sample Size Adjustments 392

References 395

18 Issues in Data Analysis 403

Fundamental Point 404

Which Participants Should Be Analyzed?​ 404

Ineligibility 406

Nonadherence 410

Missing or Poor Quality Data 414

Competing Events 421

Composite Outcomes 422

Covariate Adjustment 424

Surrogates as a Covariate 425

Baseline Variables as Covariates 428

Subgroup Analyses 431

Not Counting Some Events 438

Comparison of Multiple Variables 439

Use of Cutpoints 440

Noninferiority Trial Analysis 442

Analysis Following Trend Adaptive Designs 444

Meta-analysis of Multiple Studies 445

Rationale and Issues 445

Statistical Methods 449

Analysis for Harmful Effects 451

References 453

19 Closeout 463

Fundamental Point 463

Termination Procedures 463

Planning 463

Scheduling of Closeout Visits 464

Final Response Ascertainment 465

Transfer of Post-trial Care 468

Data and Other Study Material 469

Cleanup and Verification 469

Storage 470

Dissemination of Results 471

Post Study Follow-up 473

References 475

20 Reporting and Interpreting of Results 479

Fundamental Point 480

Guidelines for Reporting 480

Authorship 482

Duplicate Publication 483

Disclosure of Conflict of Interest 483

Presentation of Data 483

Interpretation 484

Publication Bias 485

Did the Trial Work as Planned?​ 487

Baseline Comparability 487

Blinding 487

Adherence and Concomitant Treatment 488

What Are the Limitations?​ 488

What Kinds of Analyses?​ 489

How Do the Findings Compare with Those from Other Studies?​ 491

What Are the Clinical Implications of the Findings?​ 492

Data Sharing 493

References 494

21 Multicenter Trials 501

Fundamental Point 502

Reasons for Multicenter Trials 502

Conduct of Multicenter Trials 504

Globalization of Trials 511

Large, Simple Trials 514

References 515

22 Regulatory Issues 519

Fundamental Point 520

Background 521

Overview 521

History 521

Regulatory Requirements 523

Trial Phases 523

Pretrial Requirements 525

Conduct 526

Interventions:​ Drugs 528

Interventions:​ Devices 529

Interventions:​ Biologics 532

Post-trial Requirements 533

Documents for FDA submission 534

Advisory Committee Meeting 535

Post-approval Issues and Postmarketing Investigations 535

Key Links 537

International Conference on Harmonisation 537

U.​S.​ Food and Drug Administration 537

European Medicines Agency 538

Health Canada 538

Pharmaceuticals and Medical Devices Agency, Japan 538

Bioethics Resources 538

References 538

Erratum E1

Index543

About the Authors

Lawrence M. Friedman

received his M.D. from the University of Pittsburgh. After training in internal medicine, he went to the National Heart, Lung, and Blood Institute of the National Institutes of Health. During his many years there, Dr. Friedman was involved in numerous clinical trials and epidemiology studies, having major roles in their design, management, and monitoring. While at the NIH and subsequently, he served as a consultant on clinical trials to various NIH institutes and to other governmental and nongovernmental organizations. Dr. Friedman has been a member of many data monitoring and other safety committees.

Curt D. Furberg

is Professor Emeritus of the Division of Public Health Sciences of the Wake Forest University School of Medicine. He received his M.D. and Ph.D. at the University of Umea, Sweden, and is a former chief, Clinical Trials Branch and Associate Director, Clinical Applications and Prevention Program, National Heart, Lung, and Blood Institute. Dr. Furberg established the Department of Public Health Sciences and served as its chair from 1986 to 1999. He has played major scientific and administrative roles in numerous multicenter clinical trials and has served in a consultative or advisory capacity on others. Dr. Furberg’s research activities include the areas of clinical trials methodology and cardiovascular epidemiology. He is actively involved in the debate about the need for better documentation of meaningful clinical efficacy and long-term safety for drugs intended for chronic use.

David L. DeMetsPh.D.

is currently the Max Halperin Professor of Biostatistics and former Chair of the Department of Biostatistics and Medical Informatics at the University of Wisconsin—Madison He has co-authored numerous papers on statistical methods and four texts on clinical trials, two specifically on data monitoring. He has served on many NIH and industry-sponsored data monitoring committees for clinical trials in diverse disciplines. He served on the Board of Directors of the American Statistical Association, as well as having been President of the Society for Clinical Trials and President of the Eastern North American Region (ENAR) of the Biometric Society. In addition he was Elected Fellow of the International Statistics Institute, the American Statistical Association, the Association for the Advancement of Science, the Society for Clinical Trials and the American Medical Informatics Association. In 2013, he was elected as a member of the Institute of Medicine.

David M. Reboussin

is a Professor in the Department of Biostatistical Science at the Wake Forest University School of Medicine, where he has worked since 1992. He has a master’s degree in Statistics from the University of Chicago and received his doctorate in Statistics from the University of Wisconsin at Madison. He is currently Principal Investigator for the Systolic Blood Pressure Intervention Trial Coordinating Center and has been a co-investigator in the coordinating centers for several NIH and industry funded clinical trials including Action to Control Cardiovascular Risk in Diabetes (ACCORD), Action for Health in Diabetes (Look AHEAD), the Combined Oral and Nutritional Treatment of Late-Onset Diabetes Trial (CONTROL DM) and the Estrogen Replacement and Atherosclerosis (ERA) Trial. Dr. Reboussin has served on the data and safety monitoring boards for many National Institutes of Health trials within areas including cardiology, diabetes, nephrology, pulmonology, liver disease, psychiatry, pediatrics, weight loss and smoking cessation. His work in statistical methodology has included techniques and software for sequential monitoring of clinical trials.

Christopher B. Granger

is Professor of Medicine at Duke University, where he is an active clinical cardiologist and a clinical trialist at the Duke Clinical Research Institute. He received his M.D. at University of Connecticut and his residency training at the University of Colorado. He has had Steering Committee, academic leadership, and operational responsibilities for many clinical trials in cardiology. He has been on numerous data monitoring committees. He serves on the National Heart, Lung, and Blood Institute Board of External Experts. He works with the Clinical Trials Transformation Initiative, a partnership between the U.S. Food and Drug Administration and Duke aiming to increase the quality and efficiency of clinical trials. He is a founding member of the Sensible Guidelines for the Conduct of Clinical Trials group, a collaboration between McMaster, Oxford, and Duke Universities.

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_1

1. Introduction to Clinical Trials

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

The evolution of the modern clinical trial dates back at least to the eighteenth century [1, 2]. Lind, in his classical study on board the Salisbury, evaluated six treatments for scurvy in 12 patients. One of the two who was given oranges and lemons recovered quickly and was fit for duty after 6 days. The second was the best recovered of the others and was assigned the role of nurse to the remaining ten patients. Several other comparative studies were also conducted in the eighteenth and nineteenth centuries. The comparison groups comprised literature controls, other historical controls, and concurrent controls [2].

The concept of randomization was introduced by Fisher and applied in agricultural research in 1926 [3]. Probably the first clinical trial that used a form of random assignment of participants to study groups was reported in 1931 by Amberson et al. [4]. After careful matching of 24 patients with pulmonary tuberculosis into comparable groups of 12 each, a flip of a coin determined which group received sanocrysin, a gold compound commonly used at that time. The British Medical Research Council trial of streptomycin in patients with tuberculosis, reported in 1948, used random numbers in the allocation of individual participants to experimental and control groups [5, 6].

The principle of blinding was also introduced in the trial by Amberson et al. [4]. The participants were not aware of whether they received intravenous injections of sanocrysin or distilled water. In a trial of cold vaccines in 1938, Diehl and coworkers [7] referred to the saline solution given to the subjects in the control group as a placebo.

One of the early trials from the National Cancer Institute of the National Institutes of Health in 1960 randomly assigned patients with leukemia to either 6-azauracil or placebo. No treatment benefit was observed in this double-blind trial [8].

In the past several decades, the randomized clinical trial has emerged as the preferred method in the evaluation of medical interventions. Techniques of implementation and special methods of analysis have been developed during this period. Many of the principles have their origins in work by Hill [9–12]. For a brief history of key developments in clinical trials, see Chalmers [13].

The original authors of this book have spent their careers at the U.S. National Institutes of Health, in particular, the National Heart, Lung, and Blood Institute, and/or academia. The two new authors have been academically based throughout their careers. Therefore, many of the examples reflect these experiences. We also cite papers which review the history of clinical trials development at the NIH [14–18].

The purpose of this chapter is to define clinical trials, review the need for them, discuss timing and phasing of clinical trials, and present an outline of a study protocol.

Fundamental Point

A properly planned and executed clinical trial is the best experimental technique for assessing the effectiveness of an intervention. It also contributes to the identification of possible harms.

What Is a Clinical Trial?

We define a clinical trial as a prospective study comparing the effects and value of intervention (s) against a control in human beings. Note that a clinical trial is prospective, rather than retrospective. Study participants must be followed forward in time. They need not all be followed from an identical calendar date. In fact, this will occur only rarely. Each participant however, must be followed from a well-defined point in time, which becomes time zero or baseline for that person in the study. This contrasts with a case-control study, a type of retrospective observational study in which participants are selected on the basis of presence or absence of an event or condition of interest. By definition, such a study is not a clinical trial. People can also be identified from medical records or other data sources and subsequent records can be assessed for evidence of new events. With the increasing availability of electronic health records, this kind of research has become more feasible and may involve many tens of thousands of individuals. It is theoretically possible that the participants can be identified at the specific time they begin treatment with one or another intervention selected by the clinician, and then followed by means of subsequent health records. This type of study is not considered to be a clinical trial because it is unlikely that it is truly prospective. That is, many of the participants would have been identified after initiation of treatment and not directly observed from the moment of initiation. Thus, at least some of the follow-up data are retrospective. It also suffers from the major limitation that treatment is not chosen with an element of randomness. Thus associations between treatment and outcome are nearly always influenced by confounding factors, some of which are measured (and thus can be accounted for with adjustment) and others unmeasured (that cannot be). Of course, electronic records and registries can work effectively in collaboration with randomization into clinical trials. As exemplified by the Thrombus Aspiration in ST-Elevation Myocardial Infarction in Scandinavia (TASTE) trial [19], electronic registries greatly simplified the process of identifying and obtaining initial information on those people eligible for the trial. As noted by Lauer and D’Agostino [20], however, translating this approach into other settings will not be easy.

A clinical trial must employ one or more intervention techniques. These may be single or combinations of diagnostic, preventive, or therapeutic drugs, biologics, devices, regimens, procedures, or educational approaches. Intervention techniques should be applied to participants in a standard fashion in an effort to change some outcome. Follow-up of people over a period of time without active intervention may measure the natural history of a disease process, but it does not constitute a clinical trial. Without active intervention the study is observational because no experiment is being performed.

Early phase studies may be controlled or uncontrolled. Although common terminology refers to phase I and phase II trials, because they are sometimes uncontrolled, we will refer to them as clinical studies. A trial, using our definition, contains a control group against which the intervention group is compared. At baseline, the control group must be sufficiently similar in relevant respects to the intervention group in order that differences in outcome may reasonably be attributed to the action of the intervention. Methods for obtaining an appropriate control group are discussed in Chaps. 5 and 6. Most often a new intervention is compared with, or used along with, best current standard therapy. Only if no such standard exists or, for several reasons discussed in Chap. 2, is not available, is it appropriate for the participants in the intervention group to be compared to participants who are on no active treatment. “No active treatment” means that the participant may receive either a placebo or no treatment at all. Obviously, participants in all groups may be on a variety of additional therapies and regimens, so-called concomitant treatments, which may be either self-administered or prescribed by others (e.g., other physicians).

For purposes of this book, only studies in human beings will be considered as clinical trials. Certainly, animals (or plants) may be studied using similar techniques. However, this book focuses on trials in people, and each clinical trial must therefore incorporate participant safety considerations into its basic design. Equally important is the need for, and responsibility of, the investigator to inform fully potential participants about the trial, including information about potential benefits, harms, and treatment alternatives [21–24]. See Chap. 2 for further discussion of ethical issues.

Unlike animal studies, in clinical trials the investigator cannot dictate what an individual should do. He can only strongly encourage participants to avoid certain medications or procedures which might interfere with the trial. Since it may be impossible to have “pure” intervention and control groups, an investigator may not be able to compare interventions, but only intervention strategies. Strategies refer to attempts at getting all participants to adhere, to the best of their ability, to their originally assigned intervention. When planning a trial, the investigator should recognize the difficulties inherent in studies with human subjects and attempt to estimate the magnitude of participants’ failure to adhere strictly to the protocol. The implications of less than perfect adherence are considered in Chap. 8.

As discussed in Chaps. 6 and 7, the ideal clinical trial is one that is randomized and double-blind. Deviation from this standard has potential drawbacks which will be discussed in the relevant chapters. In some clinical trials compromise is unavoidable, but often deficiencies can be prevented or minimized by employing fundamental features of design, conduct, and analysis.

A number of people distinguish between demonstrating “efficacy” of an intervention and “effectiveness” of an intervention. They also refer to “explanatory” trials, as opposed to “pragmatic” or “practical” trials. Efficacy or explanatory trials refer to what the intervention accomplishes in an ideal setting. The term is sometimes used to justify not using an “intention-to-treat” analysis. As discussed in Chaps. 8 and 18, that is insufficient justification. Effectiveness or pragmatic trials refer to what the intervention accomplishes in actual practice, taking into account inclusion of participants who may incompletely adhere to the protocol or who for other reasons may not respond to an intervention. Both sorts of trials may address relevant questions and both sorts need to be properly performed. Therefore, we do not consider this distinction between trials as important as the proper design, conduct, and analysis of all trials in order to answer important clinical or public health questions, regardless of the setting in which they are done.

The SPIRIT 2013 Statement (Standard Protocol Items: Recommendations for Interventional Trials) [25], as well as the various International Conference on Harmonisation (ICH) documents [26] devote considerable attention to the quality of trials, and the features that make for high quality. Poorly designed, conducted, analyzed, and reported trials foster confusion and even erroneous interpretation of results. People have argued over what key elements deserve the most attention versus those that expend resources better used elsewhere. However, unless certain characteristics such as unbiased assignment to treatment of sufficient numbers of adequately characterized participants, objective and reasonably complete assessment of the primary and secondary outcomes, and proper analysis are performed, the trial may not yield interpretable results. Much of the rest of this book expands on these issues.

Clinical Trial Phases

In this book we focus on the design and analysis of randomized trials comparing the effectiveness and adverse effects of two or more treatments. Several steps or phases of clinical research, however, must occur before this comparison can be implemented. Classically, trials of pharmaceutical agents have been divided into phases I through IV. Studies with other kinds of interventions, particularly those involving behavior or lifestyle change or surgical approaches, will often not fit neatly into those phases. In addition, even trials of drugs may not fit into a single phase. For example, some may blend from phase I to phase II or from phase II to phase III. Therefore, it may be easier to think of early phase studies and late phase studies. Nevertheless, because they are in common use, and because early phase studies, even if uncontrolled, may provide information essential for the conduct of late phase trials, the phases are defined below.

A good summary of phases of clinical trials and the kinds of questions addressed at each phase was prepared by the International Conference on Harmonisation [26]. Figure 1.1, taken from that document, illustrates that research goals can overlap with more than one study phase.

Fig. 1.1

Correlation between development phases and types of study [26]

Thus, although pharmacology studies in humans that examine drug tolerance, metabolism, and interactions, and describe pharmacokinetics and pharmacodynamics, are generally done as phase I, some pharmacology studies may be done in other trial phases. Therapeutic exploratory studies, which look at the effects of various doses and typically use biomarkers as the outcome, are generally thought of as phase II. However, sometimes, they may be incorporated into other phases. The usual phase III trial consists of therapeutic confirmatory studies, which demonstrate clinical usefulness and examine the safety profile. But such studies may also be done in phase II or phase IV trials. Therapeutic use studies, which examine the drug in broad or special populations and seek to identify uncommon adverse effects, are almost always phase IV (or post-approval) trials.

Phase I Studies

Although useful pre-clinical information may be obtained from in vitro studies or animal models, early data must also be obtained in humans. People who participate in phase I studies generally are healthy volunteers, but may be patients who have already tried and failed to improve on the existing standard therapies. Phase I studies attempt to estimate tolerability and characterize pharmacokinetics and pharmacodynamics. They focus on questions such as bioavailability and body compartment distribution of the drug and metabolites. They also provide preliminary assessment of drug activity [26]. These studies may also assess feasibility and safety of pharmaceutical or biologic delivery systems. For example, in gene transfer studies, the action of the vector is an important feature. Implantable devices that release an active agent require evaluation along with the agent to assess whether the device is safe and delivers the agent in appropriate doses.

Buoen et al. reviewed 105 phase I dose-escalation studies in several medical disciplines that used healthy volunteers [27]. Despite the development of new designs, primarily in the field of cancer research, most of the studies in the survey employed simple dose-escalation approaches.

Often, one of the first steps in evaluating drugs is to estimate how large a dose can be given before unacceptable toxicity is experienced by patients [28–33]. This is usually referred to as the maximally tolerated dose. Much of the early literature has discussed how to extrapolate animal model data to the starting dose in humans [34] or how to step up the dose levels to achieve the maximally tolerated dose.

In estimating the maximally tolerated dose, the investigator usually starts with a very low dose and escalates the dose until a prespecified level of toxicity is obtained. Typically, a small number of participants, usually three, are entered sequentially at a particular dose. If no specified level of toxicity is observed, the next predefined higher dose level is used. If unacceptable toxicity is observed in any of the three participants, additional participants, usually three, are treated at the same dose. If no further toxicity is seen, the dose is escalated to the next higher dose. If additional unacceptable toxicity is observed, then the dose escalation is terminated and that dose, or perhaps the previous dose, is declared to be the maximally tolerated dose. This particular design assumes that the maximally tolerated dose occurs when approximately one-third of the participants experience unacceptable toxicity. Variations of this design exist, but most are similar.

Some [32, 35–37] have proposed more sophisticated designs in cancer research that specify a sampling scheme for dose escalation and a statistical model for the estimate of the maximally tolerated dose and its standard error. The sampling scheme must be conservative in dose escalation so as not to overshoot the maximally tolerated dose by very much, but at the same time be efficient in the number of participants studied. Many of the proposed schemes utilize a step-up/step-down approach; the simplest being an extension of the previously mentioned design to allow step-downs instead of termination after unacceptable toxicity, with the possibly of subsequent step-ups. Further increase or decrease in the dose level depends on whether or not toxicity is observed at a given dose. Dose escalation stops when the process seems to have converged around a particular dose level. Once the data are generated, a dose response model is fit to the data and estimates of the maximally tolerated dose can be obtained as a function of the specified probability of a toxic response [32].

Bayesian approaches have also been developed [38, 39]. These involve methods employing continual reassessment [35, 40] and escalation with overdose control [41]. Bayesian methods involve the specification of the investigators’ prior opinions about the agent’s dose-toxicity profile, which is then used to select starting doses, and escalation rules. The most common Bayesian phase I design is called the continual reassessment method, [35] in which the starting dose is set to the prior estimate of the maximally tolerated dose. After the first cohort of participants (typically of size 1, 2, or 3, though other numbers are possible), the estimate is updated and the next participant(s) assigned to that estimate. The process is repeated until a prespecified number of participants have been assigned. The dose at which a hypothetical additional participant would be assigned constitutes the final estimate of the maximally tolerated dose. Bayesian methods that constrain the number of total toxicities have also been developed (escalation with overdose control) as have designs that allow for two or more treatments [42] and methods that allow for incomplete follow-up of long-term toxicities (time-to-event continual reassessment method) [43]. Many variations have been proposed. An advantage of Bayesian phase I designs is that they are very flexible, allowing risk factors and other sources of information to be incorporated into escalation decisions. A disadvantage is their complexity, leading to unintuitive dose assignment rules.

A detailed description of the design and conduct of dose escalating trials for treatments of cancer is found in Chaps. 1–5 of a book edited by Crowley and Ankerst [44]. A book edited by Ting contains a more general discussion of dose-selection approaches [45].

Phase II Studies

Once a dose or range of doses is determined, the next goal is to evaluate whether the drug has any biological activity or effect. The comparison may consist of a concurrent control group, historical controls, or pre-treatment status versus post-treatment status. Because of uncertainty with regard to dose-response, phase II studies may also employ several doses, with perhaps four or five intervention arms. They will look, for example, at the relationship between blood level and activity. Genetic testing is common, particularly when there is evidence of variation in rate of drug metabolism. Participants in phase II studies are usually carefully selected, with narrow inclusion criteria [26].

Although sometimes phase II studies are used for regulatory agency approval of a product, generally phase II studies are performed to make a decision as to whether to further develop a new drug or device. As such, the purpose is to refine an estimate of the probability of success in phase III. Success depends on a variety of factors, including estimated beneficial and adverse effects, feasibility, and event rates of the target population. Because phase II trials by definition do not have adequate power to define the effect on major clinical outcomes, the estimate of treatment effect and harm may depend on multiple inputs, including effects on biomarkers, on more common but less definitive clinical outcomes (like unstable angina rather than myocardial infarction) and on more minor safety signals (like minor bleeding or modest elevation in liver function tests).

The phase II design depends on the quality and adequacy of the phase I study. The results of the phase II study will, in turn, be used to design the phase III trial. The statistical literature for phase II studies, which had been rather limited [46–52] has expanded [53, 54] and, as with phase I studies, includes Bayesian methods [55, 56].

One of the traditional phase II designs in cancer is based on the work of Gehan [46], which is a version of a two stage design. In the first stage, the investigator attempts to rule out drugs which have no or little biologic activity. For example, he may specify that a drug must have some minimal level of activity, say, in 20% of patients. If the estimated activity level is less than 20%, he chooses not to consider this drug further, at least not at that maximally tolerated dose. If the estimated activity level exceeds 20%, he will add more participants to get a better estimate of the response rate. A typical study for ruling out a 20% or lower response rate enters 14 participants. If no response is observed in the first 14 participants, the drug is considered not likely to have a 20% or higher activity level. The number of patients added depends on the degree of precision desired, but ranges from 10 to 20. Thus, a typical cancer phase II study might include fewer than 30 people to estimate the response rate. As is discussed in Chap. 8, the precision of the estimated response rate is important in the design of the controlled trial. In general, phase II studies are smaller than they ought to be.

Some [32, 47, 57] have proposed designs which have more stages or a sequential aspect. Others [50, 58] have considered hybrids of phase II and phase III designs in order to enhance efficiency. While these designs have desirable statistical properties, the most vulnerable aspect of phase II, as well as phase I studies, is the type of person enrolled. Usually, phase II studies have more exclusion criteria than phase III comparative trials. Furthermore, the outcome in the phase II study (e.g., tumor response) may be different than that used in the definitive comparative trial (e.g., survival). Refinements may include time to failure [54] and unequal numbers of participants in the various stages of the phase II study [59]. Bayesian designs for phase II studies require prior estimates, as was the case for phase I studies, but differ in that they are priors of efficacy measures for the dose or doses to be investigated rather than of toxicity rates. Priors are useful for incorporating historical data into the design and analysis of phase II trials. Methods are available for continuous [60], bivariate [60], and survival outcomes [61]. These methods can account not only for random variations in participant responses within institutions but also for systematic differences in outcomes between institutions in multicenter trials or when several control groups are combined. They also acknowledge the fact that historical efficacy measures of the control are estimated with error. This induces larger sample sizes than in trials which assume efficacy of the control to be known, but with correspondingly greater resistance to false positive and false negative errors. Bayesian methods can also be used in a decision-theoretic fashion to minimize a prespecified combination of these errors for a given sample size [62, 63].

Although not generally considered phase II studies, some pilot (or feasibility or vanguard) studies may serve similar functions. Particularly for studies of non-pharmacologic interventions, these pilot studies can uncover possible problems in implementing and assessing an intervention. Here, we distinguish pilot studies conducted for this purpose from those done to see if a design for a later phase trial is feasible. For example, can participant screening and enrollment and maintenance of adherence be successfully implemented?

Phase III/IV Trials

The phase III and phase IV trials are the clinical trials defined earlier in the chapter. They are generally designed to assess the effectiveness of new interventions or existing interventions with new indications and thereby, their value in clinical practice. They also examine adverse effects, but, as described below and in Chap. 12, assessment of harm in clinical trials has limitations. The focus of most of this book is on these late phase trials. However, many design assumptions depend on information obtained from phase I and phase II studies, or some combination of early phase studies.

Phase III trials of chronic conditions or diseases often have a short follow-up period for evaluation, relative to the period of time the intervention might be used in practice. In addition, they focus on efficacy or effectiveness, but knowledge of safety is also necessary to evaluate fully the proper role of an intervention in clinical practice. A procedure or device may fail after a few years and have adverse sequelae for the patient. In 2014, the FDA warned that morcellation to treat uterine fibroids by laparoscopic means, a procedure that had been used for years, could lead to spreading of unsuspected uterine sarcoma [64]. Thus, long-term surveillance of an intervention believed to be effective in phase III trials is often necessary. Such long-term studies or studies conducted after regulatory agency approval of the drug or device are referred to as phase IV trials. Drugs may be approved on the basis of intermediate or surrogate outcomes or biomarkers, such as blood pressure or cholesterol lowering. They may also be approved after relatively short term studies (weeks or months), even though in practice, in the case of chronic conditions, they may be taken for years or even decades. Even late phase clinical trials are limited in size to several hundred or thousand (at most, a few tens of thousands) of participants. Yet the approved drugs or devices will possibly be used by millions of people. This combination of incomplete information about clinical outcomes, relatively short duration, and limited size means that sometimes the balance between benefit and harm becomes clear only when larger phase IV studies are done, or when there is greater clinical experience. One example is some of the cyclooxygenase 2 (COX 2) inhibitors, which had been approved for arthritis pain, but only disclosed cardiovascular problems after larger trials were done. These larger trials were examining the effects of the COX 2 inhibitors on prevention of colon cancer in those with polyps [65, 66]. Similarly, only after they had been on the market were thiazolidinediones, a class of drugs used for diabetes, found to be associated with an increase in heart failure [67].

Regulatory agency approval of drugs, devices, and biologics may differ because, at least in the United States, the regulations for these different kinds of interventions are based on different laws. For example, FDA approval of drugs depends greatly on at least one well-designed clinical trial plus supporting evidence (often, another clinical trial). Approval of devices relies less on clinical trial data and more on engineering characteristics of the device, including similarity with previously approved devices. (For further discussion of regulatory issues, see Chap. 22.) Devices, however, are often implanted, and unless explanted, may be present for the life of the participant. Therefore, there are urgent needs for truly long-term data on performance of devices in vivo. Assessment of devices also depends, more so than drugs, on the skill of the person performing the implantation. As a result, the results obtained in a clinical trial, which typically uses primarily well-trained investigators, may not provide an accurate balance of harm and benefit in general practice.

The same caution applies to clinical trials of procedures of other sorts, whether surgical or lifestyle intervention, where only highly skilled practitioners are investigators. But unlike devices, procedures may have little or no regulatory oversight, although those paying for care often consider the evidence.

Why Are Clinical Trials Needed?

Well-designed and sufficiently large randomized clinical trials are the best method to establish which interventions are effective and generally safe and thereby improve public health. Unfortunately, a minority of recommendations in clinical practice guidelines are based on evidence from randomized trials, the type of evidence needed to have confidence in the results [68]. Thus, although trials provide the essential foundation of evidence, they do not exist for many commonly used therapies and preventive measures. Improving the capacity, quality and relevance of clinical trials is a major public health priority.

Much has been written about the advent of individualized medicine, where an intervention (usually a drug or biologic) is used specifically in a person for whom it was designed or who has a specific genetic marker. We may someday reach the point where that is possible for many conditions and therapies. But we are not there yet. With rare exceptions, the best we can generally do is to decide to use or not use a treatment that has been evaluated in a clinical trial in a given population. Even when we better understand the genetic components of a condition, the interaction with the environment usually precludes full knowledge of a disease’s patterns and course. Therefore, almost always, a clinical trial is the most definitive method of determining whether an intervention has the postulated effect. Even when a drug is designed to be used in people with selected genetic markers, clinical trials are still commonly conducted. An example is trastuzumab, which is beneficial in women with HER2 receptors in breast cancer [69–71]. Even here, treatment is only partly successful and can have major adverse effects. Benefits of using pharmacogenetics in the decisions to achieve optimum dosing of warfarin have been claimed from some studies, but not in others [72–75]. Given the uncertain knowledge about disease course and the usual large variations in biological measures, it is often difficult to say on the basis of uncontrolled clinical observation whether a new treatment has made a difference to outcome, and if it has, what the magnitude is. A clinical trial offers the possibility of such judgment because there exists a control group which, ideally, is comparable to the intervention group in every way except for the intervention being studied.

The consequences of not conducting appropriate clinical trials at the proper time can be both serious and costly. An example was the uncertainty as to the efficacy and safety of digitalis in congestive heart failure. Only in the 1990s, after the drug had been used for over 200 years, was a large clinical trial evaluating the effect of digitalis on mortality mounted [76]. Intermittent positive pressure breathing became an established therapy for chronic obstructive pulmonary disease without good evidence of benefits. One trial suggested no major benefit from this very expensive procedure [77]. Similarly, high concentration of oxygen was used for therapy in premature infants until a clinical trial demonstrated that it could cause blindness [78].

A clinical trial can determine the incidence of adverse effects or complications of the intervention. Few interventions, if any, are entirely free of undesirable effects. However, drug toxicity might go unnoticed without the systematic follow-up measurements obtained in a clinical trial of sufficient size. The Cardiac Arrhythmia Suppression Trial documented that commonly used anti-arrhythmic drugs were harmful in patients who had a history of myocardial infarction, and raised questions about routine use of an entire class of anti-arrhythmic agents [79]. Corticosteroids had been commonly used to treat people with traumatic brain injury. Small clinical trials were inconclusive, and a meta-analysis of 16 trials showed no difference in mortality between corticosteroids and control [80]. Because of the uncertainty as to benefit, a large clinical trial was conducted. This trial, with far more participants than the others combined, demonstrated a significant 18% relative increase in mortality at 14 days [81] and a 15% increase at 6 months in the corticosteroid group [82]. As a result, an update of the meta-analysis recommended against the routine use of corticosteroids in people with head injury [83]. Niacin was widely believed to be a safe and effective treatment to improve lipid parameters and reduce coronary heart disease events for patients at risk [84, 85]. The Atherothrombosis Intervention in Metabolic Syndrome with Low HDL/High Triglycerides: Impact on Global Health Outcomes (AIM-HIGH) trial failed to show added benefit from long-acting niacin in 3,414 participants with cardiovascular disease receiving statin therapy [86]. A concern with that trial was that it might have been underpowered. The Heart Protection Study 2-Treatment of HDL to Reduce the Incidence of Vascular Events (HPS2-THRIVE) [87] was designed to provide definitive information regarding the clinical effects of a combination formulation of niacin and laropiprant, an agent to prevent flushing side effects, on top of simvastatin. That trial of 25,673 participants also showed no reduction in the primary outcome of vascular events, but increases in serious adverse gastrointestinal events, infection, and onset and poor control of diabetes.

In the final evaluation, an investigator must compare the benefit of an intervention with its other, often unwanted effects in order to decide whether, and under what circumstances, its use should be recommended. The financial implications of an intervention, particularly if there is limited benefit, must also be considered. Several studies have indicated that drug eluting stents have somewhat less restenosis than bare metal stents in percutaneous coronary intervention [88, 89]. The cost difference, however, can be considerable, especially since more than one stent is typically inserted. The Comparison of Age-Related Macular Degeneration Treatments Trials (CATT) showed that ranibizumab and bevacizumab were similarly effective at the 1-year point with respect to visual acuity in people with age-related macular degeneration [90]. Bevacizumab appeared to have various more serious adverse effects, but was one-fortieth the cost of ranibizumab. Whether the difference in the adverse events is real is uncertain, as another trial of the same agents in the same population did not show it [91]. In both examples, are the added benefits or possibly fewer adverse events, which may be defined and measured in different ways, of the more expensive interventions worth the extra cost? Such assessments are not statistical in nature. They must rely on the judgment of the investigator and the medical practitioner as well as on those who pay for medical care. Clinical trials rarely fully assess costs of the interventions and associated patient care, which change over time, and cannot replace clinical judgment; they can only provide data so that decisions are evidence-based.

People suffering from or being treated for life-threatening diseases for which there are no known effective therapies and those caring for them often argue that controlled clinical trials are not needed and that they have a right to experimental interventions. Because there may be little hope of cure or even improvement, patients and their physicians want to have access to new interventions, even if those interventions have not been shown to be safe and effective by means of the usual clinical trial. They want to be in studies of these interventions, with the expectation that they will receive the new treatment, rather than the control (if there is a control group). Those with the acquired immunodeficiency syndrome (AIDS) used to make the case forcefully that traditional clinical trials are not the sole legitimate way of determining whether interventions are useful [92–95]. This is undeniably true, and clinical trial researchers need to be willing to modify, when necessary, aspects of study design or management. Many have been vocal in their demands that once a drug or biologic has undergone some minimal investigation, it should be available to those with life-threatening conditions, should they desire it, even without late phase clinical trial evidence [96]. If the patient community is unwilling to participate in clinical trials conducted along traditional lines, or in ways that are scientifically “pure,” trials are not feasible and no information will be forthcoming. When the situation involves a rare, life-threatening genetic disorder in children, what level of evidence is needed for patients and their families, clinicians, and regulatory authorities to approve use of new agents? When should accelerated or “fast track” approval occur? Should there be interim approval based on less rigid criteria, with use restricted to specific cases and situations? When should post-approval trials be required? The U.S. FDA approved bedaquiline for drug-resistant tuberculosis on the basis of a randomized trial of 160 patients with time to culture conversion as the primary outcome, even though the study was too small to reliably detect clinical outcomes [97, 98]. This was done because of the urgent need for new drugs and with the requirement that a “confirmatory trial” would be conducted. Investigators need to involve the relevant communities or populations at risk, even though this could lead to some compromises in design and scientific purity. Investigators need to decide when such compromises so invalidate the results that the study is not worth conducting. It should be noted that the rapidity with which trial results are demanded, the extent of community involvement, and the consequent effect on study design, can change as knowledge of the disease increases, as at least partially effective therapy becomes available, and as understanding of the need for valid research designs, including clinical trials, develops. This happened to a great extent with AIDS trials.

Although investigators should design clinical trials using the fundamentals discussed in this book, they must consider the context in which the trial is being conducted. The nature of the disease or condition being studied and the population and setting in which it is being done will influence the outcomes that are assessed, the kind of control, the size, the duration, and many other factors.

Clinical trials are conducted because it is expected that they will influence practice and therefore improve health [99–104]. Traditionally, there has been considerable delay in adoption of evidence from trials, depending on the direction of the results, strength of the findings, methods of dissemination of results, and other evidence. There is indirect evidence, though, that the results of clinical trials can affect practice, which in turn may improve health outcomes. Ford et al. [105] estimated that about half of the reduction in death from coronary artery disease in the United States between 1980 and 2000 was due to better control of risk factors. The other half of the reduction was due to improved treatments, most of which were based on clinical trial results. A specific example of change in practice based on evidence from trials and improved survival comes from a national registry in Sweden during 1996–2007. Increase use of reperfusion therapy, revascularization, and medications such as aspirin, beta blockers, clopidogrel, and statins in treatment of ST segment elevation myocardial infarction was associated with a 50% decrease in mortality over this relatively short period [106]. In the United States, a registry that included 350 hospitals from 2001 to 2003 showed 11% lower in-hospital mortality for each 10% improvement in hospital-level adherence to guideline-based treatment, with most of those treatment recommendations based on clinical trial results [107].

There is no such thing as a perfect study. However, a well thought-out, well-designed, appropriately conducted and analyzed clinical trial is an effective tool. While even well designed clinical trials are not infallible, they generally provide a sounder rationale for intervention than is obtainable by other research methods. On the other hand, poorly designed, conducted, and reported trials can be misleading. Also, without supporting evidence, no single study ought to be definitive. When interpreting the results of a trial, consistency with data from laboratory, animal, epidemiological, and other clinical research must be considered.

Some have claimed that observational studies provide the “correct” answer more often than not and that therefore clinical trials are often superfluous [108, 109]. Others have pointed out that sometimes, results of observational studies and clinical trials are inconsistent. Observational studies, many of them large, suggested that use of antioxidants would reduce the risk of cancer and heart disease. These agents began to be widely used as a result. Later, large randomized controlled trials evaluating many of the antioxidants demonstrated no benefit or even harm [110]. Similarly, because of the results from observational studies, hormone therapy was advocated for post-menopausal women as a way to prevent or reduce heart disease. Results of large clinical trials [111–113] cast considerable doubt on the findings from the observational studies. Whether the differences are due to the inherent limitations of observational studies (see Chap. 5) or more specifically to the “healthy user bias” has been debated, but these and numerous other examples [114] support the belief that observational studies are unreliable in determining modest intervention effects.

We believe that pitting one kind of clinical research against another is inappropriate. Both observational epidemiology studies, including registries, and clinical trials have their strengths and weaknesses; both have their place [115]. Proper understanding of the strengths and weaknesses of clinical trials, and how the results of well-designed and conducted trials can be used in conjunction with other research methodologies, is by far the best way of improving public health and scientific understanding.

Problems in the Timing of a Trial

Once drugs and procedures of unproved clinical benefit have become part of general medical practice, performing an adequate clinical trial becomes difficult ethically and logistically. Some people advocate instituting clinical trials as early as possible in the evaluation of new therapies [116, 117]. The trials, however, must be feasible. Assessing feasibility takes into account several factors. Before conducting a trial, an investigator needs to have the necessary knowledge and tools. He must know something about the expected adverse effects of the intervention and what outcomes to assess and have the techniques to do so. Well run clinical trials of adequate magnitude are costly, and therefore almost always require sponsors willing to pay for them, and should be done only when preliminary evidence of the efficacy and harm of an intervention looks promising enough to warrant the effort and expense involved.

Another aspect of timing is consideration of the relative stability of the intervention. If active research will be likely to make the intended intervention outmoded in a short time, studying such an intervention may be inappropriate. This is particularly true in long-term clinical trials, or studies that take many months to develop. One of the criticisms of trials of surgical interventions has been that surgical methods are constantly being improved. Evaluating an operative technique of several years past, when a study was initiated, may not reflect the current status of surgery [118–120].

These issues were raised years ago in connection with the Veterans Administration study of coronary artery bypass surgery [121]. The trial showed that surgery was beneficial in subgroups of patients with left main coronary artery disease and three vessel disease, but not overall [121–123]. Critics of the trial argued that when the trial was started, the surgical techniques were still evolving. Therefore, surgical mortality in the study did not reflect what occurred in actual practice at the end of the long-term trial. In addition, there were wide differences in surgical mortality between the cooperating clinics [124] that may have been related to the experience of the surgeons. Defenders of the study maintained that the surgical mortality in the Veterans Administration hospitals was not very different from the national experience at the time [125]. In the Coronary Artery Surgery Study [126] surgical mortality was lower than in the Veterans Administration trial, suggesting better technique. The control group mortality, however, was also lower. Despite continuing evolving technology, including the development of drug-eluting stents, many trials of coronary stents have been successfully undertaken [127, 128]. The changes in stent design and use of medications to limit stent thrombosis have been incorporated into each new trial.

Review articles show that surgical trials have been successfully undertaken [129, 130] and, despite challenges, can and should be conducted [131, 132]. While the best approach might be to postpone a trial until a procedure has reached is the point where it is unlikely to change greatly, at least in the near term, such a postponement will probably mean waiting until the procedure has been widely accepted as efficacious for some indication, thus making it difficult, if not impossible to conduct the trial. However, as noted by Chalmers and Sacks [133], allowing for improvements in operative techniques in a clinical trial is possible. As in all aspects of conducting a clinical trial, judgment must be used in determining the proper time to evaluate an intervention.

Study Protocol

Every well-designed clinical trial requires a protocol. The study protocol can be viewed as a written agreement between the investigator, the participant, and the scientific community. The contents provide the background, specify the objectives, and describe the design and organization of the trial. Every detail explaining how the trial is carried out does not need to be included, provided that a comprehensive manual of procedures contains such information. The protocol serves as a document to assist communication among those working in the trial. It should also be made available to others upon request. Many protocols are now being published in on-line journals.

The protocol should be developed before the beginning of participant enrollment and should remain essentially unchanged except perhaps for minor updates. Careful thought and justification should go into any changes. Major revisions which alter the direction of the trial should be rare. If they occur, the rationale behind such changes and the process by which they are made need to be clearly described. An example is the Cardiac Arrhythmia Suppression Trial, which, on the basis of important study findings, changed intervention, participant eligibility criteria, and sample size [134].

Numerous registries of clinical trials now exist. The WHO International Clinical Trials Registry Platform (ICTRP) [135] lists those registries, including ClinicalTrials.gov [136], one of the original registries that are acceptable to the International Committee of Medical Journal Editors. Registration of all late phase trials and many early phase studies is now advocated, and indeed required by many journals and sponsors. Journals will not publish results of trials or study design papers unless the study has been registered at one of the many sites. The U.S. National Institutes of Health requires that trials that it funds be registered [137], as does the Food and Drug Administration for trials it oversees [138]. The registry sites have, at a minimum, information about the study population, intervention and control, response variables, and other key elements of the study design. Reasons for registering trials include reducing the likelihood that trial results are not published or otherwise made known, providing a way to compare the study design as initially described with what was published, and allowing other researchers to determine what else is happening in their area of interest. From the ClinicalTrials.gov registry, we know that the majority (62%) of registered trials enroll 100 or fewer participants, the majority of trials (66%) are single center, and there is substantial variability in use of randomization, blinding, and use of monitoring committees [139]. We applaud the practice of registration, and encourage all investigators to go further by including links to their protocols at the registry sites. See Chap. 22 for a further discussion of trial registration.

A guidance for developing a clinical trials protocol has been published by the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT 2013 Statement) [25]. Topic headings of a typical protocol which also serve as an outline of the subsequent chapters in this book are given below:

A.

Background of the study

B.

Objectives

1.

Primary question and response variable

2.

Secondary questions and response variables

3.

Subgroup hypotheses

4.

Adverse effects

C.

Design of the study

1.

Study population

(a)

Inclusion criteria

(b)

Exclusion criteria

2.

Sample size assumptions and estimates

3.

Enrollment of participants

(a)

Informed consent

(b)

Assessment of eligibility

(c)

Baseline examination

(d)

Intervention allocation (e.g., randomization method)

4.

Intervention(s)

(a)

Description and schedule

(b)

Measures of compliance

5.

Follow-up visit description and schedule

6.

Ascertainment of response variables

(a)

Training

(b)

Data collection

(c)

Quality control

7.

Assessment of Adverse Events

(a)

Type and frequency

(b)

Instruments

(c)

Reporting

8.

Data analysis

(a)

Interim monitoring, including data monitoring committee role

(b)

Final analysis

9.

Termination policy

D.

Organization

1.

Participating investigators

(a)

Statistical unit or data coordinating center

(b)

Laboratories and other special units

(c)

Clinical center(s)

2.

Study administration

(a)

Steering committees and subcommittees

(b)

Monitoring committee

(c)

Funding organization

Appendices

Definitions of eligibility criteria

Definitions of response variables

Informed Consent Form

References

1.

Bull JP. The historical development of clinical therapeutic trials. J Chronic Dis 1959;10:218–248.

2.

Lilienfeld AM. Ceteris paribus: the evolution of the clinical trial. Bull Hist Med 1982;56:1–18.

3.

Box JF. R. A. Fisher and the design of experiments, 1922–1926. Am Stat 1980;34:1–7.MathSciNetMATH

4.

Amberson JB, Jr, McMahon BT, Pinner M. A clinical trial of sanocrysin in pulmonary tuberculosis. Am Rev Tuberc 1931;24:401–435.

5.

Medical Research Council. Streptomycin treatment of pulmonary tuberculosis. Br Med J 1948;2:769–782.

6.

Hart PD. Letter to the Editor:Randomised controlled clinical trials. Br Med J 1991;302:1271–1272.

7.

Diehl HS, Baker AB, Cowan DW Cold vaccines; an evaluation based on a controlled study. JAMA 1938;111:1168–1173.

8.

Freireich EJ, Frei E, III, Holland JF, et al. Evaluation of a new chemotherapeutic agent in patients with “advanced refractory” acute leukemia: studies of 6–azauracil. Blood 1960;16:1268–1278.

9.

Hill AB. The clinical trial. Br Med Bull 1951;7:278–282.

10.

Hill AB. The clinical trial. N Engl J Med 1952;247:113–119.

11.

Hill AB. Statistical Methods of Clinical and Preventive Medicine. 1962; Oxford University Press, New York.

12.

Doll R. Clinical trials: Retrospect and prospect. Stat Med 1982;1:337–344.

13.

Chalmers I. Comparing like with like: some historical milestones in the evolution of methods to create unbiased comparison groups in therapeutic experiments. Int J Epidemiol 2001;30:1156–1164.

14.

Gehan EA, Schneiderman MA. Historical and methodological developments in clinical trials at the National Cancer Institute. Stat Med 1990;9:871–880.

15.

Halperin M, DeMets DL, Ware JH. Early methodological developments for clinical trials at the National Heart, Lung, and Blood Institute. Stat Med 1990;9:881–892.

16.

Greenhouse SW. Some historical and methodological developments in early clinical trials at the National Institutes of Health. Stat Med 1990;9:893–901.

17.

Byar DP. Discussion of papers on "historical and methodological developments in clinical trials at the National Institutes of Health." Stat Med 1990; 9:903–906.

18.

Organization, review, and administration of cooperative studies (Greenberg Report): A report from the Heart Special Project Committee to the National Advisory Heart Council, May 1967. Control Clin Trials 1988; 9:137–148.

19.

Frӧbert O, Lagerqvist B, Olivecrona GK, et al. Thrombus aspiration during ST-segment elevation myocardial infarction. N Engl J Med 2013;369:1587–1597.

20.

Lauer MS, D’Agostino RB. The randomized registry trial—the next disruptive technology in clinical research? N Engl J Med 2013;369:1579–1581.

21.

OPRR Reports. Code of Federal Regulations: (45 CFR 46) Protection of Human Subjects. National Institutes of Health, Department of Health and Human Services. Revised January 15, 2009. http://​www.​hhs.​gov/​ohrp/​humansubjects/​guidance/​45cfr46.​html

22.

National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. The Belmont Report: ethical principles and guidelines for the protection of human subjects of research. Federal Register 1979;44:23192-23197. http://​archive.​hhs.​gov/​ohrp/​humansubjects/​guidance/​belmont.​htm

23.

Nuremburg Code. http://​www.​hhs.​gov/​ohrp/​archive/​nurcode.​html

24.

World Medical Association Declaration of Helsinki. http://​www.​wma.​net/​en/​30publications/​10policies/​b3/​index.​html

25.

Chan A-W, Tetzlaff JM, Altman DG, et al. SPIRIT 2013 Statement: defining standard protocol items for clinical trials. Ann Intern Med 2013;158:200–207.

26.

International Harmonised Tripartite Guideline: General Considerations for Clinical Trials: E8. July 17, 1997. http://​www.​ich.​org/​fileadmin/​Public_​Web_​Site/​ICH_​Products/​Guidelines/​Efficacy/​E8/​Step4/​E8_​Guideline.​pdf.

27.

Buoen C, Bjerrum OJ, Thomsen MS. How first-time-in-human studies are being performed: a survey of phase 1 dose-escalation trials in healthy volunteers published between 1995 and 2004. J Clin Pharmacol 2005;45:1123–1136.

28.

Carbone PP, Krant MJ, Miller SP, et al. The feasibility of using randomization schemes early in the clinical trials of new chemotherapeutic agents:hydroxyurea (NSC-32065). Clin Pharmacol Ther 1965;6:17–24.

29.

Anbar D. Stochastic approximation methods and their use in bioassay and Phase I clinical trials. Comm Stat Series A. 1984;13:2451–2467.

30.

Williams DA. Interval estimation of the median lethal dose. Biometrics 1986;42:641-645; correction in: Biometrics 1987;43:1035.

31.

Storer B, DeMets D. Current phase I/II designs: are they adequate? J Clin Res Drug Devel 1987;1:121–130.

32.

Storer B. Design and analysis of phase I clinical trials. Biometrics 1989;45:925–937.MathSciNetMATH

33.

Gordon NH, Willson JK. Using toxicity grades in the design and analysis of cancer phase I clinical trials. Stat Med 1992;11:2063–2075.

34.

Schneiderman MA. Mouse to man: statistical problems in bringing a drug to clinical trial. Proceedings of the 5th Berkeley Symposium of Math and Statistical Problems, University of California 1967;4:855–866.

35.

O'Quigley J, Pepe M, Fisher L. Continual reassessment method: a practical design for Phase I clinical trials in cancer. Biometrics 1990;46:33–48.MathSciNetMATH

36.

O'Quigley J, Chevret S. Methods for dose finding studies in cancer clinical trials: a review and results of a Monte Carlo Study. Stat Med 1991;10:1647–1664.

37.

Wang O, Faries DE. A two-stage dose selection strategy in phase 1 trials with wide dose ranges. J Biopharm Stat 2000;10:319–333.

38.

Babb J, Rogatko A. Bayesian methods for cancer phase I clinical trials. In: N. Geller (Ed.), Advances in Clinical Trial Biostatistics. New York: Marcel Dekker, 2004, pages 1–39.

39.

Biswas S, Liu DD, Lee JJ, Berry DA. Bayesian clinical trials at the University of Texas M. D. Anderson Cancer Center. Clin Trials 2009;6:205–216.

40.

Garrett-Mayer E. The continual reassessment method for dose-finding studies: a tutorial. Clin Trials 2006;3:57–71.

41.

Babb J, Rogatko A, Zacks S. Cancer phase I clinical trials: efficient dose escalation with overdose control. Stat Med 1998;17:1103–1120.MATH

42.

Thall PF, Millikan RE, Mueller P, Lee S-J. Dose-finding with two agents in phase I oncology trials. Biometrics 2003;59:487–496.MathSciNetMATH

43.

Cheung Y K, Chappell R. Sequential designs for phase I clinical trials with late-onset toxicities. Biometrics 2000;56:1177–1182.MathSciNetMATH

44.

Crowley J, Hoering A (eds.) Handbook of Statistics in Clinical Oncology (third edition). Boca Raton, FL: Chapman and Hall/CRC, 2012.

45.

Ting N (ed.) Dose Finding in Drug Development. New York: Springer, 2006.MATH

46.

Gehan EA. The determination of the number of patients required in a follow-up trial of a new chemotherapeutic agent. J Chron Dis 1961;13:346–353.

47.

Fleming TR. One-sample multiple testing procedures for phase II clinical trials. Biometrics 1982;38:143–151.MATH

48.

Herson J. Predictive probability early termination plans for phase II clinical trials. Biometrics 1979;35:775–783.MATH

49.

Geller NL. Design of Phase I and II clinical trials in cancer: a statistician's view. Cancer Invest 1984;2:483–491.

50.

Whitehead J. Sample sizes for Phase II and Phase III clinical trials: an integrated approach. Stat Med 1986;5:459–464.

51.

Chang MN, Therneau TM, Wieand HS, Cha SS. Designs for group sequential phase II clinical trials. Biometrics 1987;43:865–874.MATH

52.

Simon R, Wittes RE, Ellenberg SS. Randomized phase II clinical trials. Cancer Treat Rep 1985;69:1375–1381.

53.

Jung S, Carey M, Kim K. Graphical search for two-stage designs for phase II clinical trials. Control Clin Trials 2001;22:367–372.

54.

Case LD, Morgan TM. Duration of accrual and follow-up for two-stage clinical trials. Lifetime Data Anal 2001;7:21–37.MathSciNetMATH

55.

Thall P, Simon R. Recent developments in the design of phase II clinical trials. In: Recent Advances in Clinical Trial Design and Analysis. P. Thall, (Ed.). New York: Springer Science+Business Media 1995, pages 49–72.

56.

Grieve AP, Krams M. ASTIN: a Bayesian adaptive dose-response trial in acute stroke. ClinTrials 2005;2:340–351.

57.

Lee YJ, Staquet M, Simon R, et al. Two-stage plans for patient accrual in phase II cancer clinical trials. Cancer Treat Rep 1979;63:1721–1726.

58.

Schaid DJ, Ingle JN, Wieand S, Ahmann DL. A design for phase II testing of anticancer agents within a phase III clinical trial. Control Clin Trials 1988;9:107–118.

59.

Simon R. Optimal two-stage designs for phase II clinical trials. Control Clin Trials 1989;10:1–10.

60.

Thall PF, Simon R. Incorporating historical control data in planning phase II clinical trials. Stat Med 1990;9:215–228.

61.

Schmidli H, Bretz F, Racine-Poon A. Bayesian predictive power for interim adaptation in seamless phase II/III trials where the endpoint is survival up to some specified timepoint Stat Med 2007;26:4925–4938.MathSciNet

62.

Sylvester RJ, Staquet MJ. Design of phase II clinical trials in cancer using decision theory. Cancer Treat Rep 1980;64:519–524.

63.

Berry D. Decision analysis and Bayesian methods in clinical trials. In: Recent Advances in Clinical Trial Design and Analysis. P Thall (Ed.). New York: Springer Science+Business Media, 1995, pages 125-154.

64.

Laparoscopic Uterine Power Morcellation in Hysterectomy and Myomectomy: FDA Safety Communication. http://​www.​fda.​gov/​MedicalDevices/​Safety/​AlertsandNotices​/​ucm393576.​htm.

65.

Solomon SD, McMurray JJV, Pfeffer MA, et al. Cardiovascular risk associated with celecoxib in a clinical trial for colorectal adenoma prevention. N Engl J Med 2005;352:1071–1080.

66.

Psaty BM, Furberg CD. COX-2 inhibitors—lessons in drug safety. N Engl J Med 2005;352:1133–1135.

67.

Bolen S, Feldman L, Vassy J, et al. Systematic review: comparative effectiveness and safety of oral medications for type 2 diabetes mellitus. Ann Intern Med 2007;147:386–399.

68.

Tricoci PL, Allen JM, Kramer JM, Califf RM, Smith SC Jr. Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA 2009;301:831-841; erratum in JAMA 2009;301:1544.69.

69.

Romond EH, Perez EA, Bryant J, et al. Trastuzumab plus adjuvant chemotherapy for operable HER2-positive breast cancer. N Engl J Med 2005;353:1673–1684.

70.

Piccart-Gebhart MJ, Proctor M, Leyland-Jones B, et al. Trastuzumab after adjuvant chemotherapy in HER2-positive breast cancer. N Engl J Med 2005;353:1659–1672.

71.

Smith I, Proctor M, Gelber RD, et al. 2-year follow-up of trastuzumab after adjuvant chemotherapy in HER2-positive breast cancer: a randomised controlled trial. Lancet 2007;369:29–36.

72.

Kimmel SE, French B, Kasner, et al. A pharmacogenetic versus a clinical algorithm for warfarin dosing. N Engl J Med 2013;369:2283–2293.

73.

Verhoef TI, Ragia G, de Boer A, et al. A randomized trial of genotype-dosing of acenocoumarol and phenprocoumon. N Engl J Med 2013;369:2304–2312.

74.

Pirmohamed M, Burnside G, Eriksson N, et al. A randomized trial of genotype-guided dosing of warfarin. N Engl J Med 2013;369:2294–2303.

75.

Zineh I, Pacanowski M, Woodcock J. Pharmacogenetics and coumarin dosing—recalibrating expectations. N Engl J Med 2013;369:2273–2275.

76.

The Digitalis Investigation Group. The effect of digoxin on mortality and morbidity in patients with heart failure. N Engl J Med 1997;336:525–533.

77.

The Intermittent Positive Pressure Breathing Trial Group. Intermittent positive pressure breathing therapy of chronic obstructive pulmonary disease-a clinical trial. Ann Intern Med 1983;99:612–620.

78.

Silverman WA. The lesson of retrolental fibroplasia. Sci Am 1977;236:100–107.

79.

Echt DS, Liebson PR, Mitchell LB, et al. Mortality and morbidity in patients receiving encainide, flecainide, or placebo. The Cardiac Arrhythmia Suppression Trial. N Engl J Med 1991;324:781–788.

80.

Alderson P, Roberts I. Corticosteroids for acute traumatic brain injury. Cochrane Database SystRev 2000;(2):CD000196.

81.

Roberts I, Yates D, Sandercock P, et al. Effect of intravenous corticosteroids on death within 14 days in 10008 adults with clinically significant head injury (MRC CRASH trial): randomised placebo-controlled trial. Lancet 2004;364:1321–1328.

82.

Edwards P, Arango M, Balica L, et al. Final results of MRC CRASH, a randomised placebo-controlled trial of intravenous corticosteroid in adults with head injury—outcomes at 6 months. Lancet 2005;365:1957–1959.

83.

Alderson P, Roberts I. Corticosteroids for acute traumatic brain injury. Cochrane Summaries 2009; (http://​summaries.​cochrane.​org/​CD000196/​INJ_​corticosteroids-to-treat-brain-injury.

84.

Stone NJ, Robinson J, Lichtenstein AH, et al.ACC/AHA guideline on the treatment of blood cholesterol to reduce atherosclerotic cardiovascular risk in adults: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol 2014;63:2889-2934; erratum in J Am Coll Cardiol 2014;63:3024–3025.

85.

Canner PL, Furberg CD, McGovern ME. Benefits of niacin in patients with versus without the metabolic syndrome and healed myocardial infarction (from the Coronary Drug Project). Am J Cardiol 2006;97:477–479.

86.

The AIM-HIGH Investigators. Niacin in patients with low HDL cholesterol levels receiving intensive statin therapy. N Engl J Med 2011;365:2255–2267.

87.

The HPS2-THRIVE Collaborative Group. Effects of extended-release niacin with laropiprant in high-risk patients. N Engl J Med 2014; 371:203–212.

88.

Stone GW, Lansky AJ, Pocock SJ, et al. Paclitaxel-eluting stents versus bare-metal stents in acute myocardial infarction. N Engl J Med 2009;360:1946–1959.

89.

James SK, Stenestrand U, Lindbäck J, et al. Long-term safety and efficacy of drug-eluting versus bare-metal stents in Sweden. N Engl J Med 2009;360:1933–1945.

90.

The CATT Research Group. Ranibizumab and bevacizumab for neovascular age-related macular degeneration. N Engl J Med 2011;364:1897–1908,

91.

IVAN Study Investigators. Chakravarthy U, Harding SP, Rogers CA, et al. Ranibizumab versus bevacizumab to treat neovascular age-related macular degeneration: one-year findings from the IVAN randomized trial. Ophthalmology 2012;119:1399-1411; erratum in Ophthalmology 2012;119:1508 and Ophthalmology 2013;120:1719.

92.

Byar DP, Schoenfeld DA, Green SB, et al. Design considerations for AIDS trials. N Engl J Med 1990;323:1343–1348.

93.

Levine C, Dubler NN, Levine RJ. Building a new consensus: ethical principles and policies for clinical research on HIV/AIDS. IRB 1991;13:1–17.

94.

Spiers HR. Community consultation and AIDS clinical trials, part I. IRB 1991;13:7–10.

95.

Emanuel EJ, Grady C. Four paradigms of clinical research and research oversight. In: The Oxford Textbook of Clinical Research Ethics. EJ Emamuel, C Grady, RA Crouch, RK Lie, FG Miller, D Wendler (Eds.). Oxford: Oxford University Press, 2008, pages 222–230.

96.

Abigail Alliance For Better Access to Developmental Drugs. http://​abigail-alliance.​org/​.

97.

Diacon AH, Pym A, Grobusch MP, et al. Multidrug-resistant tuberculosis and culture conversion with bedaquiline. N Engl J Med 2014;371:723–732.

98.

Cox E, Laessig K. FDA approval of bedaquiline—the benefit-risk balance for drug-resistant tuberculosis. N Engl J Med 2014;371:689–691.

99.

Furberg CD. The impact of clinical trials on clinical practice. Arzneim-Forsch./Drug Res 1989;39:986–988.

100.

Lamas GA, Pfeffer MA, Hamm P, et al. Do the results of randomized clinical trials of cardiovascular drugs influence medical practice? N Engl J Med 1992;327:241–247.

101.

Friedman L, Wenger NK, Knatterud GL. Impact of the Coronary Drug Project findings on clinical practice. Control Clin Trials 1983;4:513–522.

102.

Boissel JP. Impact of randomized clinical trials on medical practices. Control Clin Trials 1989;10:120S–134S.

103.

Schron E, Rosenberg Y, Parker A, Stylianou M. Awareness of clinical trials results and influence on prescription behavior: A survey of US Physicians. Control Clin Trials 1994;15:108S.

104.

Ayanian JZ, Haustman PJ, Guadagnoli E, et al. Knowledge and practices of generalist and specialist physicians regarding drug therapy for acute myocardial infarction. N Engl J Med 1994;331:1136–1142.

105.

Ford ES, Ajani UA, Croft JB, et al. Explaining the decrease in U.S. deaths from coronary disease, 1980-2000. N Engl J Med 2007;356:2388–2398.

106.

Jernberg T, Hohanson P, Held C, et al. Association between adoption of evidence-based treatment and survival for patients with ST-elevation myocardial infarction. JAMA 2011;305:1677–1684.

107.

Peterson ED, Roe MT, Mulgund J, et al. Association between hospital process performance and outcomes among patients with acute coronary syndromes. JAMA 2006;295:1912–1920.

108.

Benson K, Hartz AJ. A comparison of observational studies and randomized, controlled trials. N Engl J Med 2000;342:1878–1886.

109.

Concato J, Shah N, Horwitz RI. Randomized, controlled trials, observational studies, and the hierarchy of research designs. N Engl J Med 2000;342:1887–1892.

110.

Bjelakovic G, Nikolova D, Gluud LL, Simonetti RG, Gluud C. Mortality in randomized trials of antioxidant supplements for primary and secondary prevention: systematic review and meta-analysis. JAMA 2007;297:842–857.

111.

Hulley S, Grady D, Bush T, et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. JAMA 1998;280:605–613.

112.

Writing Group for the Women’s Health Initiative Investigators. Risks and benefits of estrogen plus progestin in healthy postmenopausal women. JAMA 2002;288:321–333.

113.

The Women’s Health Initiative Steering Committee. Effects of conjugated equine estrogen in postmenopausal women with hysterectomy. JAMA 2004;291:1701–1712.

114.

Granger CB, McMurray JJV. Using measures of disease progression to determine therapeutic effect: a sirens’ song. J Am Coll Cardiol 2006;48:434–437.

115.

Furberg BD, Furberg CD. Evaluating Clinical Research: All that Glitters is not Gold. (Second ed.) New York: Springer, 2007.

116.

Chalmers TC. Randomization of the first patient. Med Clin North Am 1975;59:1035–1038.

117.

Spodick DH. Randomize the first patient: scientific, ethical, and behavioral bases. Am J Cardiol 1983;51:916–917.

118.

Bonchek LI. Sounding Board: Are randomized trials appropriate for evaluating new operations? N Engl J Med 1979;301:44–45.

119.

Van der Linden W. Pitfalls in randomized surgical trials. Surgery 1980;87:258–262.

120.

Rudicel S, Esdail J. The randomized clinical trial in orthopaedics: obligation or option? J Bone Joint Surg 1985;67:1284–1293.

121.

Murphy ML, Hultgren HN, Detre K, et al. Treatment of chronic stable angina - a preliminary report of survival data of the randomized Veterans Administration cooperative study. N Engl J Med 1977;297:621–627.

122.

Takaro T, Hultgren HN, Lipton MJ, Detre KM. The VA cooperative randomized study of surgery for coronary arterial occlusive disease. 11. Subgroup with significant left main lesions. Circulation 1976;54:111–107.

123.

Detre K, Peduzzi P, Murphy M, et al. Effect of bypass surgery on survival in patients in low- and high-risk subgroups delineated by the use of simple clinical variables. Circulation 1981;63:1329–1338.

124.

Proudfit WL. Criticisms of the VA randomized study of coronary bypass surgery. Clin Res 1978;26:236–240.

125.

Chalmers TC, Smith H Jr, Ambroz A, et al. In defense of the VA randomized control trial of coronary artery surgery. Clin Res 1978;26:230–235.

126.

CASS Principal Investigators and their Associates. Myocardial infarction and mortality in the Coronary Artery Surgery Study (CASS) randomized trial. N Engl J Med 1984;310:750–758.

127.

Cutlip DE, Balm DS, Kalon KL, et al. Stent thrombosis in the modern era: a pooled analysis of multicenter coronary stent clinical trials. Circulation 2001;103:1967–1971.

128.

Babapulle MN, Joseph L, Bélisle P, Brophy JM, Eisenberg MJ. A hierarchical Bayesian meta-analysis of randomised clinical trials of drug-eluting stents. Lancet 2004;364:14–20.

129.

Strachan CJL,Oates GD. Surgical trials. F.N. Johnson and S. Johnson (Eds). In: Clinical Trials. Oxford: Blackwell Scientific: 1977.

130.

Bunker JP, Hinkley D, McDermott WV. Surgical innovation and its evaluation. Science 1978;200:937–941.

131.

Weil RJ. The future of surgical research. PLoS Med 2004;1:e13. doi:10.​1371/​journal.​pmed.​0010013.

132.

Cook JA. The challenges faced in the design, conduct and analysis of surgical randomised controlled trials. Trials 2009. 10:9doi:10.​1186/​1745-6215-10-9.

133.

Chalmers TC, Sacks H. Letter to the editor: Randomized clinical trials in surgery. N Engl J Med 1979;301:1182.

134.

Greene HL, Roden DM, Katz RJ, et al. The Cardiac Arrhythmia Suppression Trial: first CAST…then CAST-II. J Am Coll Cardiol 1992;19:894–898.

135.

World Health Organization International Clinical Trials Registry Platform Search Portal. http://​apps.​who.​int/​trialsearch/​

136.

ClinicalTrials.gov. http://​clinicaltrials.​gov/​

137.

Clinical Trials Registration in ClinicalTrials.gov (Public Law 110-85): Competing Applications and Non-Competing Progress Reports. http://​grants.​nih.​gov/​grants/​guide/​notice-files/​NOT-OD-08-023.​html.

138.

Federal Register: May 21, 2008 (Volume 73, Number 99). http://​edocket.​access.​gpo.​gov/​2008/​E8-11042.​htm

139.

Califf RM, Zarin DA, Kramer JM, et al. Characteristics of clinical trials registered in ClinicalTrials.gov, 2007-2010. JAMA. 2012;307:1838–1847. doi: 10.​1001/​jama.​2012.​3424.

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_2

2. Ethical Issues

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

People have debated the ethics of clinical trials for as long as trials have been conducted. The arguments have changed over the years and perhaps become more sophisticated, but many of them involve issues such as the physician’s obligations to the individual patient versus societal good; clinical equipoise; study design considerations such as randomization and the choice of control group, including use of placebo; informed consent; conduct of trials in underdeveloped areas and world regions; conflicts of interest; participant confidentiality and sharing of data and specimens; lack of publication; and publication bias.

A well-designed trial should answer important public health questions without impairing the welfare of participants. There may, at times, be conflicts between a physician’s perception of what is good for his or her patient and the design and conduct of the trial. In such instances, the needs of the participant must predominate.

Ethical issues apply in all stages of a clinical trial. In this chapter, we summarize some of the major factors involving ethics in design, conduct, and reporting of clinical trials. As will be noted, several of the issues are unsettled and have no easy solution. We expect, however, that investigators will at least consider these issues in the planning stages of trials, so that high ethical standards can be applied to all trials.

Emanuel et al. [1] listed seven criteria that they considered essential to the ethical conduct of clinical research. These criteria are value, scientific validity, fair selection of participants, favorable benefit/risk balance, independent review, informed consent, and respect for enrolled participants (Table 2.1). Independent review is generally conducted by ethics review committees specifically constituted for oversight of research with human subjects. In the United States, such committees are termed institutional review boards (IRBs). Other names used outside the United States are research ethics committees, ethics committees, or ethics review committees. Although the role of ethics review committees is discussed later in this chapter under Informed Consent, it must be emphasized that independent review by these committees and others, such as data monitoring boards, applies to several aspects of a trial.

Table 2.1

Requirements for an ethical clinical trial

Requirement

Explanation

Value

Evaluate an intervention that has the potential to be of social or scientific value

Scientific validity

Use methods that will produce reliable results

Fair selection of participants

Participant selection that avoids placing the vulnerable at undue risk and avoids preferential access of attractive interventions to the privileged

Favorable benefit/risk balance

Minimize risks and maximize potential benefits, with an estimate that benefits will likely outweigh risks

Independent review

Review of design by individuals not directly affiliated with the research (for example, ethics review committees)

Informed consent

Provide information about purpose of research, procedures, and potential risks and benefits to enable participants to make voluntary decisions in a way that respects participant autonomy

Respect for enrolled participants

Protect the rights and wellbeing of participants

Adapted from Emanuel et al. [1]

We encourage the reader to seek out any of the many books and journals devoted to ethical aspects of clinical research. Those go into the issues, including ones we do not address, in considerable depth. A particularly relevant book is The Oxford Textbook of Clinical Research Ethics, many chapters of which relate directly to clinical trials [2]. The reader is also referred to several key documents:

1.

Nuremberg Code. This was the first major international statement on the ethics of medical research, published in 1947 in response to unethical human experimentation on concentration camp prisoners in the Second World War [3]. This code outlined ethical standards for medical research with an emphasis on the requirement for voluntary consent to participation.

2.

Declaration of Helsinki. Issued by the World Medical Association in 1964, and periodically amended, the Declaration of Helsinki is a comprehensive statement of the ethics of human subject research [4].

3.

Belmont Report. Created by a United States federal commission in 1979, this report outlines ethical principles for clinical research [5]. The report is structured around three basic principles: respect for persons, beneficence, and justice.

4.

International Ethical Guidelines for Biomedical Research Involving Human Subjects, prepared by the Council for International Organizations of Medical Sciences in collaboration with the World Health Organization, first in 1982 and amended several times, including in 2002 [6]. This document includes 21 guidelines that address ethical responsibilities in human subject research, many of which apply to clinical trials.

Fundamental Point

Investigators and sponsors of clinical trials have ethical obligations to trial participants and to science and medicine.

Planning and Design

Ethics Training

All clinical trial investigators should have training in research ethics. Understanding ethical principles, and the related regulatory requirements (see Chap. 22), is essential for responsible conduct of clinical trials. An important part of training in ethics is a review of the history of abuses in clinical research that prompted many of the guidelines and regulations that followed. These include an experiment in Tuskegee, Alabama, when treatment was withheld from around 400 African-American men with syphilis to study the course of the disease as well as the abhorrent experiments of concentration camp prisoners in the Second World War. There are a number of resources for research ethics training, including several National Institutes of Health (NIH) websites [7–9].

Does the Question Require a Clinical Trial?

An early decision relates to whether a clinical trial is even necessary. Not all questions need to be answered, and not all of those that should be answered require clinical trials. Sometimes, other kinds of clinical studies may be able to address the question at least as well as, or even better than, a clinical trial. Even if the answer may not be quite as good, the added benefits from the trial may not be worth the added risk.

Because clinical trials involve administering something (a drug, device, biologic, or procedure) to someone, or attempting to change someone’s behavior, there may be adverse as well as positive results. Although some of the potential adverse consequences may be known before the trial is started, and therefore prevented or minimized, others may arise unexpectedly during the trial or be more serious than anticipated. The question being addressed by the clinical trial, therefore, must be important enough to justify the possible adverse events. The question must have relevant clinical, public health, and/or other scientific value. A trivial question should not expose study participants to risk of harm, either physical or emotional. Harm can be either a direct result of the intervention or indirect, like from withholding something beneficial. The study investigator, sponsor or funder, and institutions where the study will be performed must all ensure that the question is sufficiently important and the trial is appropriately conducted to justify those risks.

Though the question may be important, the clinical trial may be infeasible or unethical. An obvious example is cigarette smoking—providing non-smokers with cigarettes to prove that smoking is harmful is clearly unethical. Observational studies have given us sufficient evidence to answer that question, since the relative risk is so great. The Cardiac Arrhythmia Suppression Trial (CAST) [10] was designed to determine whether suppression of ventricular arrhythmias with antiarrhythmic agents in people with heart disease would lead to a reduction in sudden cardiac death. After two of the three antiarrhythmic drugs were found to be harmful and the trial was stopped, some asked whether the study might be continued but reconfigured to demonstrate that quinidine, a long-used drug with some properties similar to the two discontinued agents, would also be harmful. The CAST investigators quickly decided that designing a trial specifically to prove harm, especially serious harm, would be unethical. Although the outcome of a trial is uncertain, the primary response variable should always be one where either benefit or noninferiority is potentially achievable.

Two kinds of trials raise ethical issues because of concerns about the balance between potential benefits to society (and perhaps to participants) and the risks of harm and discomfort to participants. In both, the likelihood of immediate benefit to the study participants exists but is remote. One involves “marketing” (also termed “seeding”) trials. Such clinical trials are conducted to show that a new drug or new version of an old drug is at least as good as (i.e., noninferior to) a drug already proven to be beneficial. Other than enhancing the financial status of the industry sponsor, there may be little benefit from the new drug. Yet trial participants are being put at risk from a drug with unknown adverse effects, some of which might be serious. If the new drug has some potential improvement over the existing one, the trial might be justified. Perhaps the new drug is easier to take (e.g., once a day rather than twice a day administration, or taking a pill rather than an injection), is better tolerated, or causes fewer adverse events. One could also argue that having more than one drug with similar benefits is good for the economy, fostering lower medical care costs. But in the end, those conducting such trials should show how the question is important and how there will be meaningful benefits for patients.

A second kind of trial, the ethics of which have been debated, is the early phase study. If these studies are performed in healthy volunteers, there is a nontrivial chance that they will be harmed, but have little opportunity to benefit, other than from whatever payment they receive as a result of their participation and from the possible contribution they provide to advancing treatment. Some people regularly enroll in such studies for the payment [11]. It has been argued that with proper attention to study design and safety monitoring, appropriate evaluation by ethics review committees, and true informed consent, these studies are ethical [12]. As always, risk must be kept to a minimum and the payment must not be so great as to encourage participants to do something that would place them at serious risk. The pros and cons of various payment models for research participants are discussed by Dickert and Grady [13]. As with other clinical research, early phase studies are only ethical if investigators and sponsors do whatever is necessary to minimize risk. Unfortunately, instances when investigators may not have taken proper care have occurred and received widespread attention [14–16].

Some early phase studies are conducted with participants who have a disease or condition. Patients with cancer that have not responded to other therapies may volunteer for such trials, hoping that the experimental intervention will prove beneficial. Given the small size of these studies and the unfortunate fact that most interventions early in their development do not prove beneficial, there may be only a small chance of benefit. But even if there is only a slight possibility of improvement, as long as there is adequate informed consent and the expectation of benefit to society from the knowledge to be gained, most would agree that these trials can be conducted in an ethical manner [17, 18]. However, the strategy of commonly subjecting participants to experimental therapies without the ability to compare safety and harm to a control group in an unbiased way raises its own ethical issues.

On the other hand, most treatments used in medicine, including those recommended in clinical practice guidelines [19], do not have the clinical trial evidence to be certain that the benefit outweighs the risk. This suggests that we have a responsibility, when possible, to promote high-quality clinical trials to provide the evidence to guide clinical decision-making. It is ironic that consent is essential for a patient to be in a clinical trial comparing two commonly used treatments, and yet assignment to those treatments in clinical practice is routine and accepted without consent and without gaining knowledge about whether there is benefit or harm. If one accepts that randomized trials are the most reliable way to define modest treatment effects, then increasing the number and efficiency of trials should be a priority for the broader health care system, a goal of the Patient-Centered Outcome Research Institute (PCORI) [20].

Controversies in the approach to informed consent in trials that compare treatments commonly used in practice were highlighted by the Surfactant, Positive Pressure, and Oxygenation Randomized Trial (SUPPORT) [21]. This trial randomly assigned premature babies to supplemental oxygen to keep the arterial oxygen saturation at the lower end versus the higher end of standard recommendations. The six-page, single-spaced consent form included standard elements of informed consent, including a statement that lower levels of oxygen might reduce retinopathy, a known complication of higher oxygen levels. The trial showed less retinopathy with lower oxygen target, but unexpectedly higher mortality, and the results have changed practice. Meanwhile, the Office for Human Research Protections (OHRP) of the U.S. Department of Health and Human Services investigated the consent process in the trial and determined that institutional review boards failed to have the consent state that mortality might be increased in one of the treatment strategies [22]. This decision has caused concern among academic institutions about the risk of conducting trials as well as undermining attempts to streamline the consent process in pragmatic trials that are comparing standard therapies [23]. In fact, it has been argued that the participant risks involved with random assignment to commonly used standard treatments are not different than standard practice and that this should be acknowledged in the regulations [24].

It appears that most people are willing to volunteer for clinical trials, but most people are not approached to participate in trials [25]. Some have suggested that there should be a greater sense of social responsibility to participate in clinical research since current treatments are available only due to previous patients participating, and future advances will likewise depend on this participation [26]. This places the burden on clinical researchers to be responsible in designing trials that will provide reliable guidance for future care. In fact, most trials are too small to provide reliable information and many results of trials are never published [27]. Even if our current complex approach to conducting trials were simplified, the costs are still a major barrier. Moreover, relatively little funding is allocated to answering the questions that would have the greatest impact on improving public health.

Randomization

In the typical “superiority trial” described in Chap. 5, randomization is usually done on top of standard or usual therapy, which all participants should receive. The special issues related to noninferiority trials are discussed in Chap. 5. Randomization can be a problem for physicians and other clinicians who feel pressure to be able to choose the treatment that has the greatest likelihood of benefit. The investigator, however, must acknowledge uncertainty when it exists. Therefore, an objection to random assignment should only apply if the investigator believes that there is reasonable certainty that a superior therapy exists. If that is the case, he or she should not participate in a trial that randomizes participants to a therapy other than the believed superior therapy. On the other hand, if he or she truly cannot say that one treatment is better than another, there should be no ethical problem with randomization. Such judgments regarding efficacy may vary among investigators, such that there is uncertainty for some but not others. Because it is unreasonable to expect that an individual investigator should have no preference, not only at the start of a trial but during its conduct, the concept of “clinical equipoise” among the expert clinical community has been proposed [28]. Some have maintained that until an intervention has been proven beneficial, randomization is the most ethical approach and one that will provide the correct answer soonest [29–32]. It may be that “equipoise” will change over the course of a trial, as was the case in the Second International Study of Infarct Survival (ISIS-2) trial testing streptokinase for myocardial infarction. During the period of recruitment, the data monitoring committee found that there was “proof beyond reasonable doubt” that streptokinase reduced mortality for patients 0–4 h after onset of pain, and this information was shared with investigators [33]. They were told that “patients can be randomized if the responsible physician remains, in the light of this and other evidence, uncertain as to whether streptokinase is indicated” [33]. However, is it ethically justifiable for a data monitoring committee to allow participants to be randomly assigned to an arm (in this case, placebo) for which there is “proof” of higher mortality? Many would argue that the committee should have recommended a change in the protocol with no further enrollment in this subset.

There are other situations in which consent is not possible in the traditional sense, including certain situations in which the patient is unable to provide consent (for example in the setting of cardiac arrest) and when the unit of randomization is not the patient (cluster randomized studies). An example of such a cluster randomized study is the Randomized Evaluation of Decolonization versus Universal Clearance to Eliminate MRSA (REDUCE MRSA) trial [34]. Forty-three hospitals were randomly assigned to 1 of 3 strategies of MRSA screening and patient isolation, targeted decolonization, or universal decolonization (of all patients without screening), to reduce rates of MRSA infection. Most hospitals used a central IRB. Since all regimens were standard of care and participation in the trial was anticipated to have a favorable benefit-risk balance, the requirement for patient consent was waived. Patients were given information sheets explaining the trial.

Control Group

Choice of the control group is a major design issue in clinical trials. If there is a known best therapy, one would generally expect the new intervention to be compared with that therapy, or added to it. But the optimal therapy may not be widely used for various reasons. These could include cost, unavailability of the therapy or lack of sufficient clinicians competent to administer it, lack of acceptance by the practicing clinical community, socioeconomic and cultural differences, or other factors. Depending on these circumstances, some trials may not use the best known therapy or standard of care as the control. They may rely on what the practicing communities typically do, or usual therapy [35]. Investigators and ethics review committees need to judge whether the usual therapy deprives participants of a proven better treatment that they would otherwise receive. If so, serious ethical concerns arise. A major area of disagreement has been the degree of responsibility of investigators to ensure that all participants receive the best proven therapy as a control or background care, even if usual care in the community in which the trial is being conducted is not up to that standard [36]. The appropriate control and background therapy depends, in part, on the purpose of the trial. (See also the section below, “Trials in Low- and Middle-Income Countries.”)

Considerable confusion has arisen when people talk about placebo-controlled trials, as they may refer to different kinds of designs. Often, a new intervention is added to usual care or standard care and compared against that care plus placebo. Sometimes, a new intervention is seen as a possible replacement for an existing therapy, yet for various reasons, it is not thought appropriate to compare the new intervention against the existing therapy. The commonly used therapy, for example, may not have been proven to be beneficial, or it may be poorly tolerated. Therefore, a placebo comparator is used instead of the existing therapy. Often, a blinded placebo control provides the most complete information about the risks and benefits of a new therapy as an inert placebo is the best approximation of a neutral control. The SYMPLICITY HTN-3 (Renal Denervation in Patients With Uncontrolled Hypertension-3) trial of renal denervation for control of severe refractory hypertension is a good example of the importance of a placebo (in this case a sham procedure) [37]. Earlier randomized trials of renal denervation compared with no renal denervation, on top of optimal medical therapy, showed a major (22–32 mmHg) reduction in systolic blood pressure with renal denervation that led to widespread enthusiasm and adoption of this treatment in Europe, where the device to perform the procedure was approved based on those results. However, the sham-controlled trial found similar 12–14 mmHg reductions in systolic blood pressure with renal denervation and with the sham procedure.

Even if a proven therapy exists, whether short-term discontinuation of that therapy for the purpose of conducting a placebo-controlled trial is harmful depends on the condition being studied. Exposing participants to serious harm by withholding beneficial treatment is unethical even in the short term. For conditions causing only mild to moderate discomfort, it may be acceptable. For example, investigators evaluating new analgesic agents might choose to use a placebo control, as long as any pain or discomfort is treated promptly. As always, there will be borderline cases that require discussion and review by ethics review committees [38].

Freedman et al. [39, 40] acknowledged that many factors are considered in deciding whether to use a placebo control. They argued that if an accepted treatment exists, much of the time a placebo control is unethical and, indeed, unnecessary. Rothman and Michels [41, 42] also maintained that in many cases a placebo, in lieu of the proven therapy, has been used inappropriately because a proven therapy existed. This debate occurred with the Enhanced Suppression of the Platelet IIb/IIIa Receptor with Integrilin (ESPRIT) trial [43–45]. The decision to use a placebo control, rather than another proven IIb/IIIa receptor inhibitor, was only allowed after it was shown that many cardiologists were not persuaded by the prior evidence, and even then only with declaration by the investigators that they were uncertain as to the benefits of IIb/IIIa inhibitors. We think that this is a valid argument as the participating investigators were informed about the current evidence and made the decision to conduct another placebo-controlled trial because they questioned the applicability of that evidence. History has supported their decision, since IIb/IIIa inhibitors are no longer strongly recommended in guidelines nor used as a standard in practice. Ethics review committees must have full knowledge and informed consent must contain the relevant information.

Before an investigator uses a placebo control, which will often be the best design, he or she should assess whether it will provide the basis for a better assessment of the active therapy and should determine that its use will not cause serious harm (due to withholding a proven effective alternative). Importantly, all participants must be told that there is a specified probability (e.g., 50%) of their receiving placebo. The World Medical Association Declaration of Helsinki (as amended in 2013) [4], the Council for International Organizations of Medical Sciences (CIOMS) [6], regulatory bodies [46], and others have guidelines for use of placebo. Miller summarizes the issues that should be considered by investigators [47]. If life-saving treatment is available, patients should not be assigned to placebo versus an active treatment. For example, once streptokinase was shown to save lives of patients with myocardial infarction, it would no longer be ethical to compare a new fibrinolytic agent, like alteplase, with placebo. Likewise, if assignment to placebo (versus available active therapy) would likely result in any significant pain or harm, then a placebo would be unethical. A placebo control is particularly important when studying conditions with a variable course and/or frequent spontaneous remissions, when existing therapies are inconsistently effective or have serious side effects, or when frequency of the condition is so low that an equivalence trial would be impractical [47].

Protection from Conflicts of Interest

A widely expressed concern in clinical research is the potential for conflicts of interest on the part of the investigators. In the context of ethical issues, conflicts of interest can lead to bias in design, conduct, data analysis, interpretation, and reporting of findings. Conflicts of interest are generally considered in the financial context, but intellectual or other conflicts also exist [48]. Ideally, no investigator should have any interests other than the well-being of the study participants and the generation of new knowledge that will improve clinical care and public health. That is unrealistic, however, given that investigators must receive research funding to conduct research, and this funding may come from government, industry, research foundations, private investors, or others who have considerable interest in the outcome of the study. Many investigators have also spent a career attempting to advance the science, and could be disappointed if or fail to accept that their theory is incorrect. Therefore, most clinical trials find it more realistic to manage conflicts of interest rather than to avoid them completely.

The practice of disclosing financial relationships to participants and others has been reviewed and recommendations have been proposed [49]. Among these recommendations, it was noted that because many participants may not fully appreciate the impact that financial relationships might have on research design, conduct, and analysis, in addition to disclosure, IRBs and others should “play a significant role in determining the acceptability of these relationships” [49]. We think that disclosure and IRB or other oversight may be sufficient for early phase studies. It may not be sufficient, however, for late phase trials—those that are designed to have major implications for clinical practice. Most late phase clinical trials are sponsored by industry, and although the investigators enrolling and following participants may not stand to gain financially from the results of the trial, the sponsors clearly do. Therefore, analysis should be conducted, or at least validated, by groups independent of the industry sponsor. Ideally, this should also occur in trials sponsored by others. Any investigators who have significant economic interests in the outcome either should not participate or should not have opportunities to affect and publish the trial results. This may mean that the lead investigator in multi-investigator studies or the investigator in single-investigator studies should have no major financial conflicts if the study is one likely to change practice and increase sales. Financial conflicts may also contribute to the problem of “negative” trials being less likely to be published or having their publication delayed (see Chap. 20). Trials with positive results are published more often (see Chap. 20). Other key investigators with major conflicts should also be barred from such trials. If the investigators have limited roles or only small financial investments, it may be acceptable for them to participate. We recognize that the situation is more complicated when those designing and overseeing, and perhaps co-authoring publications, are employees of the company sponsoring the trial. Nevertheless, complete openness and data analysis by an independent group remain important. The use of external independent oversight bodies and clear lines of authority may mitigate conflicts of interest. In the end, however, clinical trial results must be believed and accepted by the clinical communities. To the extent that conflicts of interest (real or perceived) lessen that acceptance, the study is impaired. Therefore, all appropriate ways of minimizing and managing conflicts should be used.

Informed Consent

Proper informed consent is essential to ethical trial conduct. Partly as a result of terrible things done in the name of clinical research, various bodies developed guidelines such as the Nuremberg Code [3], Declaration of Helsinki [4], Belmont Report [5], and International Ethical Guidelines for Biomedical Research Involving Human Subjects [6]. These guidelines lay out standards for informed consent that are commonly followed internationally. In parallel to the Belmont Report, the United States Congress passed laws that require adherence to informed consent regulations by those receiving government support—the so-called Common Rule, or Title 45 of the Code of Federal Regulations, part 46 (45 CFR 46) [50]—and those evaluating agents under the auspices of the U.S. Food and Drug Administration [51]. These regulations require that clinical research studies be reviewed by IRBs, and establish the membership and other procedures that IRBs must follow.

One of the primary roles of the IRB is to ensure that there is true, voluntary informed consent. The Common Rule and 21 CFR 50 [52] require consent forms to contain basic elements. Table 2.2 lists these as well as other elements that may be added as appropriate. Simply adhering to legal requirements does not ensure adequate informed consent [53–55]. Informed consent is a process that can take considerable time and effort; it is not simply a matter of getting a form signed. In many, perhaps most, clinical trial settings, true informed consent can be obtained. Potential participants have the capacity to understand what is being requested of them, they have adequate time to consider the implications of joining a trial, ask questions, and take information home to review and discuss with their families and personal physicians, and they are familiar with the concepts of research and voluntary consent. As discussed in the “Privacy and Confidentiality” section below, investigators may share data and biospecimens with other researchers, while following federal guidelines. If such sharing is planned or required by the sponsor, the informed consent material must make it clear that sharing will occur and that the data may be used for purposes other than those of the trial for which the person is volunteering.

Table 2.2

Informed consent checklist—basic and additional elements

Basic elements

 A statement that the study involves research

 An explanation of the purposes of the research

 The expected duration of the subject’s participation

 A description of the procedures to be followed

 Identification of any procedures that are experimental

 A description of any reasonably foreseeable risks or discomforts to the subject

 A description of any benefits to the subject or to others that may reasonably be expected from the research

 A disclosure of appropriate alternative procedures or courses of treatment, if any, that might be advantageous to the subject

 A statement describing the extent, if any, to which confidentiality of records identifying the subject will be maintained

 For research involving more than minimal risk, an explanation as to whether any compensation and any medical treatments are available if injury occurs and, if so, what they consist of, or where further information may be obtained

 An explanation of whom to contact for answers to pertinent questions about the research and research subjects’ rights, and whom to contact in the event of a research-related injury to the subject

 A statement that participation is voluntary, refusal to participate will involve no penalty or loss of benefits to which the subject is otherwise entitled, and the subject may discontinue participation at any time without penalty or loss of benefits to which the subject is otherwise entitled

Additional elements, as appropriate

 A statement that the particular treatment or procedure may involve risks to the subject (or to the embryo or fetus, if the subject is or may become pregnant), which are currently unforeseeable

 Anticipated circumstances under which the subject’s participation may be terminated by the investigator without regard to the subject’s consent

 Any additional costs to the subject that may result from participation in the research

 The consequences of a subject’s decision to withdraw from the research and procedures for orderly termination of participation by the subject

 A statement that significant new findings developed during the course of the research, which may relate to the subject’s willingness to continue participation, will be provided to the subject

 The approximate number of subjects involved in the study

From the Code of Federal Regulations [50]

Sometimes people may not understand that a clinical trial is a research endeavor. They may believe that they are receiving therapy for their condition. This may happen in early phase trials of new drugs that are being developed for serious, untreatable diseases, or in any clinical trial testing a promising intervention for a serious or chronic condition. Patients may view the trial as the last or best possibility for cure. Sometimes clinicians are also researchers and may seek to enroll their own patients into clinical trials. These situations can lead to what has been termed “therapeutic misconception” [56]. The distinction between research, an experiment in essence, and clinical care may blur. Extra effort must be made to provide patients with the information needed to judge the merits of volunteering for research, separate from their clinical care.

The situations where participant enrollment must be done immediately, in comatose patients, or in highly stressful circumstances and where the prospective participants are minors or not fully competent to understand the study are more complicated and may not have optimal solutions. In the United States, FDA [57] and Department of Health and Human Services [58] guidelines allow for research in emergency situations, when informed consent is not possible. Under these regulations, IRBs may approve the study without informed consent as long as a series of special conditions has been met, including that there has been community consultation and a safety committee is formed to monitor accumulating data. Similar research is also allowed in Canada [59] and under the European Medicines Agency (EMA) Guidelines for Good Clinical Practice [60]. A trial of fibrinolytic therapy versus placebo in the context of resuscitation for cardiac arrest was successfully conducted under the EMA guidelines [61] and a trial of therapeutic hypothermia for patients with out-of-hospital cardiac arrest was conducted under Department of Health and Human Services guidelines [62]. In these trials, local ethics committees agreed that the trial could be done without informed consent prior to enrollment. Instead, consent was later given by surviving participants or their family members or others.

Some have questioned research in emergency settings because of the lack of prior informed consent, and several such clinical trials have been quite controversial. An example is a trial of a product intended to be used as a blood substitute in trauma patients [63]. Because patients were unconscious at the time of administration of the blood substitute, consent could not be obtained. Therefore, community consultation was obtained before local IRBs approved the study. However, there were allegations that safety problems noted in earlier trials of the agent were not published or otherwise disclosed to those bodies. We do not take a position on the merits of this particular trial, and we support the concept of being able to conduct important research in settings where full informed consent before enrollment is not possible. The sponsors and investigators, though, must be completely open about all data relevant to the conduct of such studies and must follow all local regulations [64]. Failure to do so harms not only the unwitting participants, but the entire field of research in emergency settings.

For pragmatic, simple trials that are comparing treatments that are each standard of care, a streamlined approach to consent has been proposed [65]. Just as a “learning health care system” integrates clinical research with care, a simple consent process could be integrated into patient care with an explanation of the research, of the fact that either treatment is approved and standard, and that it is uncertain which is better.

Research on prisoners is restricted [66] due to a history of violation of ethical principals in the population and since informed consent free of the appearance of possible coercion is difficult to establish.

Also contentious is the practice of obtaining consent from participant surrogates when the study participant is unable to provide fully informed consent. This typically happens with research in minors, when parents or other guardians make the decision. Special review is required for pediatric research; requirements vary depending on the expected risks from the study [50]. Other situations, such as research in emotionally or mentally impaired individuals, also have generated discussion and guidelines regarding use of surrogate consent [67, 68]. Less clear is the use of surrogate consent for potential study participants who are temporarily unable to understand the nature of the study and give consent. This issue arose in research in people with acute respiratory distress syndrome [69]. Suggestions for accommodating research in such situations include risk assessment, determination of patient capacity, and reconsent [70]. As in all such situations, judgment on the part of investigators, sponsors, IRBs, and others will be required and second-guessing will inevitably occur.

The right to withdraw consent to continue in a trial, including withdrawing consent to continue to receive study interventions and undergo study procedures, is another important ethics principle. Less clear is to what extent participants have the right or option to refuse to have any type of follow-up, since determining major outcomes as well as serious adverse outcomes, including death, is essential in many trials to interpret the results and entails minimal risk to participants. If the initial consent declares that participants may withdraw from intervention and all study procedures but that vital status will be obtained at the end of the study regardless, this may be an appropriate compromise. This can protect the contributions of others who have placed themselves at some risk with the understanding that their participation may help future patients, while minimizing risk and discomfort to those who wish to withdraw.

Conduct

Trials in Low- and Middle-Income Countries

Many large multicenter clinical outcome trials are international, and they are becoming more so [71] (see Chap. 21). Most diseases are global. The ability to enroll and follow participants in more than one country assists in enrollment and may help in generating results that are generalizable to different populations and settings. However, trials that are conducted in low- and middle-income countries can raise ethical issues. Are they conducted in those regions because the disease of interest is prevalent there, and the results relevant to the region? Or are the countries or regions selected primarily for convenience, low cost, or fewer administrative and regulatory burdens? The control group may be receiving less than optimal care, and thus may have a higher event rate, permitting a smaller, shorter, and less expensive trial. If the trial is conducted for those reasons, it may be unethical. Some have said that the investigators are obligated to ensure that all participants receive optimal care without regard to usual practice in the country where the trial is being conducted. Others have maintained that it is sufficient if the participants receive care at least as good as what they would receive had they not been in the trial, or care that is better than standard care for their setting. This was the argument of the investigators in a tamoxifen trial of adjuvant oophorectomy and tamoxifen in the treatment of breast cancer in Vietnamese and Chinese women. State-of-the-art treatment by United States standards (including radiation) was not available and not likely to be available. What was being tested was whether a simple and affordable treatment like tamoxifen would be better than what was available [72].

Extrapolation of study results from less developed regions to highly developed countries with very different health care systems and standards of care, and vice versa, has also been questioned. While it is clear that risk and event rates tend to be higher in low-income countries [73], some studies have suggested that the treatment effects may indeed be different [74, 75].

After the trial ends, what is the obligation of the investigators to provide an intervention shown to be beneficial, both to the study participants and to the broader population in a low-income country? This and other similar issues have no easy answers. We believe, however, that trials should only be conducted in places and with participants likely to benefit from the results and with informed consent procedures that clearly describe what will be done at the end of the trial. The results from the trial must be able to be applied to clinical practice in the population from which the participants came [76].

Recruitment

Recruitment of trial participants is often one of the more challenging aspects of conducting a clinical trial (see Chap. 10). Unless an adequate number of participants are enrolled to generate the number of outcomes needed, the trial will not be able to answer the questions about benefit and harm. Therefore, there is great pressure to recruit an adequate number of participants and to do so as quickly as possible. The use of some financial incentives, such as “finder’s fees” (i.e. payment to physicians for referring participants to a clinical trial investigator), is inappropriate in that it might lead to undue pressure on a prospective participant [77]. This differs from the common and accepted practice of paying investigators a certain amount for the cost and effort of recruiting each enrolled participant. Even this practice becomes questionable if the amount of payment is so great as to induce the investigator to enroll inappropriate participants [13].

Study participants may (and at times should) be paid for their involvement in clinical trials. Typically, payment is meant to compensate them for the time, effort, and expense of attending clinic visits. Studies that enroll healthy volunteers (usually phase I studies) will often provide payment beyond reimbursement for expenses. The amount generally depends on the time required and the amount of pain and risks involved in any procedures. As with paying investigators, when the amount is such that people, whether they are healthy volunteers or patients, might make unwise or dangerous decisions, it becomes excessive. Participants should not be paid more for taking on more risk. Ethics review committees often have guidelines as to appropriate payment amounts for various kinds of studies and procedures and must ensure that the amount provided does not create an undue influence.

As discussed in Chap. 9, many potentially eligible trial participants may be on medication. This treatment may be for the condition that will be studied or some other reason. In order to assess the participant’s condition at baseline, the investigator may be tempted to withdraw medication, at least temporarily. For example, one might be interested in enrolling people at high risk of cardiovascular disease, and thus try to accrue those with hypertension. But an accurate baseline blood pressure might not be obtainable in those already on treatment. It might not even be clear that the participant already on antihypertensive drugs would have met the eligibility criteria if not on medication. Should one withdraw the drug or simply accept that those on treatment probably truly had hypertension, especially if while on treatment they still have high normal blood pressures? Usually, the latter is the better course of action.

Safety and Efficacy Monitoring

Occasionally, during a trial, important information relevant to informed consent derives either from other studies or from the trial being conducted. In such cases, the investigator is obligated to update the consent form and notify current participants in an appropriate manner. A trial of antioxidants in Finnish male smokers (the Alpha-Tocopherol Beta Carotene Cancer Prevention Study) indicated that beta carotene and vitamin E may have been harmful with respect to cancer or cardiovascular diseases, which was contrary to earlier observational studies [78]. Because of those findings, investigators of the ongoing Carotene and Retinol Efficacy Trial (CARET) informed the participants of the results and the possible risks [79]. CARET was subsequently stopped earlier than planned because of adverse events similar to those seen in the Finnish trial. The investigator of a third trial of antioxidants, the Age-Related Eye Disease Study (AREDS), then notified participants (with a focus on the smokers) of the findings from both the Finnish study and CARET [80, 81].

Five trials of warfarin in patients with atrial fibrillation were being conducted at approximately the same time [82] in the late 1980s. After the first three ended, showing clear benefit from warfarin in the reduction of strokes, the remaining two found it difficult ethically to continue. Interim results from the Heart and Estrogen/progestin Replacement Study (HERS) [83] and a Women’s Health Initiative (WHI) [84] evaluation of estrogen suggested that thromboembolic adverse events that had not been clearly presented in the informed consent were occurring. In both studies, the data and safety monitoring boards debated whether the studies should stop or continue with additional actions taken. The trials continued, but participants in those trials and medical communities were notified of these interim findings of embolic risk [85, 86]. Not only is such a practice an ethical stance, but a well-informed participant is usually a better trial participant. How much data should be provided to study participants and when, and the role of independent safety monitoring groups in this decision, are still areas of debate [87].

The issue of how to handle accumulating data from an ongoing trial is a difficult one, and is further discussed in Chap. 16. With advance understanding by both participants and investigators that they will not be told interim results unless they show conclusive benefit or harm, and that there is a responsible safety monitoring group, ethical concerns should be lessened if not totally alleviated.

Early Termination for Other Than Scientific or Safety Reasons

Clinical trials are only ethical if there are adequate resources to conduct them and see them to completion. Trials may (and should) be stopped early if there are safety concerns or if there are scientific reasons to do so (see Chap. 16). It is inappropriate, however, to stop a trial early because the sponsor changes its mind about research agendas or marketing priorities, or failed to adequately plan for sufficient resources. In such cases, participants who enrolled did so with the understanding that they would be helping to advance medical knowledge. In the process, they put themselves at possibly considerable risk based, in part, on that understanding. To fail to complete the study is a serious breach of ethics. An example of when this happened is the Controlled Onset Verapamil Investigation of Cardiovascular End Points (CONVINCE) trial [88]. Partway through follow-up, the sponsor ended the study for reasons other than scientific or safety concerns. As noted in an editorial by Psaty and Rennie [89], “the responsible conduct of medical research involves a social duty and a moral responsibility that transcends quarterly business plans….”

In another situation, an investigator with inadequate funds to complete his trial solicited money from participants in the trial so that he could continue purchasing the experimental drug [90]. Because the trial was being conducted in patients with a fatal condition, amyotrophic lateral sclerosis, the study participants viewed the trial as a last hope and were therefore under considerable pressure to donate. We view such actions as completely unethical. Plans for conducting the trial, including obtaining experimental agents, must be in place before the trial begins.

With all trials, investigators need to plan in advance how they will handle end-of-study issues such as whether participants will have continued access to the intervention and transition to appropriate medical care.

Privacy and Confidentiality

The issues of privacy and confidentiality have received considerable attention. The widespread uses of electronic media have made many people concerned about the privacy of their medical records, including research records. Electronic medical records have simplified the tasks of finding potentially eligible participants for trials, conducting international multicenter studies, following up on participants during and after the studies, and sharing data with other researchers. They have also led to laws restricting what kinds of medical records can be shared and with whom, in the absence of clear permission from the patients. In the United States, the Health Insurance Portability and Accountability Act (HIPAA) primarily addresses privacy issues in clinical practice [91]. However, there are clinical research provisions that affect how investigators identify, contact, and obtain informed consent from prospective participants, and how study data are maintained and provided to others [91] (see also Chap. 10). These laws, in turn, have generated articles pointing out the increased difficulty in conducting clinical research. Policies encouraging or mandating sharing of data and biospecimens from research studies [92–94] may conflict with the objectives of maintaining confidentiality. If data are shared with other researchers for unspecified purposes, might participants who volunteered for a trial object to their data being used for goals of which they might not approve? If the original informed consent does not allow for use of the biospecimens by others or for purposes different from the stated ones, either the biospecimens cannot be shared or new informed consents must be obtained. The increasing availability and use of genetic material adds to this conflict. Fear of employment or health insurance discrimination based on genetic information may make some people unwilling to participate in trials if complete confidentiality cannot be ensured. It is probably not possible to share data and specimens that are useful to the recipient investigator while also completely removing all participant identifiers. Some compromises are inevitable. At the current time, there are no clear solutions to these issues, but trial participants must have a right to make informed choices. Clinical trial investigators need to be aware of the concerns, and to the extent possible, plan to address them before the study starts.

Data Falsification

There has been concern about falsification of data and entry of ineligible, or even phantom, participants in clinical trials (see Chap. 10). A case of possible falsification that gained considerable attention was a trial of bone morphogenetic protein-2 in the management of fractures due to combat injuries [95]. An editorial in the journal that published the article, which had purported to show benefit from treatment, said that “much of the paper was essentially false” and announced the article’s withdrawal [96]. A trial of lumpectomy and radiation therapy for breast cancer was severely harmed because of falsified data on a small number of participants at one of many enrolling sites. The overall results were unchanged when the participants with the falsified data were not included [97, 98]. Nevertheless, the harm done to the study and to clinical trials in general was considerable. We condemn all data fabrication. It is important to emphasize that confidence in the integrity of the trial and its results is essential to every trial. If, through intentional or inadvertent actions, that confidence is impaired, not only have the participants and potentially others in the community been harmed, the trial loses its rationale and ability to influence science and medical practice. Chapter 11 reviews issues of ensuring data quality.

Reporting

Publication Bias, Suppression, and Delays

All investigators have the obligation to report trial results fully and in a timely fashion. As discussed in Chap. 20, it is well known that publication bias exists. Positive or exciting findings are more likely to be published than null results. In one survey of 74 trials of antidepressant agents, 38 were considered to have results favorable to the intervention. All but one of these were published. Of the 36 studies considered not to have favorable results, 22 were not published. Eleven others were published in ways that obscured the lack of favorable results [99]. Heres and colleagues examined trials of head-to-head comparisons of second-generation antipsychotic agents [100]. Ninety percent of the trials sponsored by industry were reported in favor of the sponsor’s drug. Interestingly, this occurred even with trials that compared the same drugs, but the outcome changed when the sponsor was a different company. Clearly bias and conflicts of interest can have important effects on publication and interpretation of results.

It is more probable that large, late phase trials will be published regardless of the results than will small, early stage trials. There are exceptions, however. As discussed in Chap. 5, the results of the second Prospective Randomized Amlodipine Survival Evaluation 2 (PRAISE-2) trial [101], although presented in 2000, were only published 13 years after the trial was completed [102]. The problem of delayed or absent publication is undoubtedly true of other trials with disappointing outcomes.

An important advance in ensuring publication is that many journals [103], sponsors such as the NIH [104], and the FDA [105] require that trials be registered at initiation in one of several accepted registration sites. Although it is not a complete solution to the problem of failure to make public the results of all trials, registration allows for easier tracking of trials that are initiated but perhaps never completed or never published. An analysis of trials registered on ClinicalTrials.gov [106] showed from a sample cohort that only 78 of 150 (52%) had associated publications within 2 years after results posting.

We take the position that the results of all clinical trials should be published in a timely way regardless of the findings. It is important that the totality of the information, pro and con, be available so that those designing other studies and clinicians can make informed decisions. If the study results are not published, it is also unfair to the participants who volunteered for a trial with the understanding that they would be helping medical research. So-called “gag clauses” in industry-sponsored trials [107] are both antithetical to academic freedom and contrary to ethical practice.

Conflicts of Interest and Publication

All researchers have biases of some sort. It is understandable that an investigator’s perspective will enter into a publication, even though best efforts are made to be objective in reporting and interpreting study results. For this reason, many journals, and most high-profile ones, require that authors disclose their potential conflicts of interest [108]. In addition, many multi-investigator studies have publication policies that exclude from authorship those with major conflicts of interest.

More extreme is “ghost authorship,” where the papers are written by employees of the sponsors, who are not listed as authors, and the academic-based investigators, who may have had little or no role in drafting the manuscript, are given authorship credit. We deplore this practice. We also deplore the practice of listing as authors anyone who did not truly contribute to the research. In response to concerns about “ghost authorship,” many journals now ask for the contribution of each listed author when the manuscript is submitted for publication (see Chap. 19 for further discussion of these issues).

References

1.

Emanuel EJ, Wendler D, Grady C. What makes clinical research ethical? JAMA 2000;283:2701-11.CrossRef

2.

Emanuel EJ, Grady C, Crouch RA, et al (eds.). The Oxford Textbook of Clinical Research Ethics. Oxford: Oxford University Press, 2008.

3.

U.S. Department of Health & Human Services website. The Nuremberg Code. Available at: http://​www.​hhs.​gov/​ohrp/​archive/​nurcode.​html. Accessed January 14, 2015.

4.

World Medical Association website. WMA Declaration of Helsinki – Ethical Principles for Medical Research Involving Human Subjects. Available at: http://​www.​wma.​net/​en/​30publications/​10policies/​b3/​index.​html. Accessed January 14, 2015.

5.

U.S. Department of Health & Human Services website. The Belmont Report: Ethical Principles and Guidelines for the Protection of Human Subjects of Research. The National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. April 18, 1979. Available at: http://​www.​hhs.​gov/​ohrp/​humansubjects/​guidance/​belmont.​html. Accessed January 14, 2015.

6.

Council for International Organizations of Medical Sciences, World Health Organization. International Ethical Guidelines for Biomedical Research Involving Human Subjects. Available at: http://​www.​cioms.​ch/​publications/​layout_​guide2002.​pdf. Accessed January 14, 2015.

7.

National Institutes of Health Clinical Center, U.S. Department of Health & Human Services. Ethics in Clinical Research. Available at: http://​clinicalcenter.​nih.​gov/​recruit/​ethics.​html. Accessed January 14, 2015.

8.

National Institutes of Health Clinical Center, U.S. Department of Health & Human Services. Clinical Research Training On-Line. Available at: http://​cc.​nih.​gov/​training/​training/​crt.​html. Accessed January 14, 2015.

9.

National Institutes of Health Clinical Center, U.S. Department of Health & Human Services. Ethical and Regulatory Aspects of Clinical Research. Available at: http://​www.​bioethics.​nih.​gov/​courses/​index.​shtml. Accessed January 14, 2015.

10.

Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. The Cardiac Arrhythmia Suppression Trial (CAST) Investigators. N Engl J Med 1989;321:406-412.

11.

Elliott C. Guinea-pigging: healthy human subjects for drug safety trials are in demand. But is it a living? New Yorker 2008;7:36-41.

12.

Jonsen AR, Miller FG. Research with healthy volunteers. In: Emanuel EJ, Grady C, Crouch RA, et al (eds.). The Oxford Textbook of Clinical Research Ethics. Oxford: Oxford University Press, 2008:481-487.

13.

Dickert N, Grady C. Incentives for research participants. In: Emanuel EJ, Grady C, Crouch RA, et al (eds.). The Oxford Textbook of Clinical Research Ethics. Oxford: Oxford University Press, 2008:386-396.

14.

Savulescu J, Spriggs M. The hexamethonium asthma study and the death of a normal volunteer in research. J Med Ethics 2002;28:3-4.CrossRef

15.

Suntharalingam G, Perry MR, Ward S, et al. Cytokine storm in a phase 1 trial of the anti-CD28 monoclonal antibody TGN1412. N Engl J Med 2006;355:1018-1028.CrossRef

16.

St Clair EW. The calm after the cytokine storm: lessons from the TGN1412 trial. J Clin Invest 2008;118:1344-1347 (correction J Clin Invest 2008;118:2365).

17.

Agrawal M, Emanuel EJ. Ethics of phase 1 oncology studies: reexamining the arguments and data. JAMA 2003;290:1075-1082.CrossRef

18.

Joffe S, Miller FG. Bench to bedside: mapping the moral terrain of clinical research. Hastings Cent Rep 2008;38:30-42.CrossRef

19.

Tricoci P, Allen JM, Kramer JM, et al. Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA 2009;301:831-841.CrossRef

20.

Patient-Centered Outcomes Research Institute website. Available at: http://​www.​pcori.​org/​. Accessed January 14, 2015.

21.

SUPPORT Study Group of the Eunice Kennedy Shriver NICHD Neonatal Research Network, Carlo WA, Finer NN, et al. Target ranges of oxygen saturation in extremely preterm infants. N Engl J Med 2010;362:1959-1969.

22.

Human Research Protections under Federalwide Assurance (FWA) 5960. http://​www.​hhs.​gov/​ohrp/​detrm_​letrs/​YR13/​mar13a.​pdf. Accessed January 14, 2015.

23.

Lantos JD. The weird divergence of ethics and regulation with regard to informed consent. Am J Bioeth 2013;13:31-33.CrossRef

24.

Lantos JD, Spertus JA. The concept of risk in comparative-effectiveness research. N Engl J Med 2014;371:2129-2130.CrossRef

25.

Research America: An Alliance for Discoveries in Health. Research Enterprise Survey, February 2010 (slide presentation). Available at: http://​www.​researchamerica.​org/​uploads/​ResearchEnterpri​sePoll.​pdf. Accessed January 14, 2015.

26.

Schaefer GO, Emanuel EJ, Wertheimer A. The obligation to participate in biomedical research. JAMA 2009;302:67-72.CrossRef

27.

Califf RM, Zarin DA, Kramer JM, et al. Characteristics of clinical trials registered in ClinicalTrials.gov, 2007-2010. JAMA 2012;307:1838-1847CrossRef

28.

Freedman B. Equipoise and the ethics of clinical research. N Engl J Med 1987;317:141-145.CrossRef

29.

Shaw LW, Chalmers TC. Ethics in cooperative clinical trials. Ann N Y Acad Sci 1970;169:487-495.CrossRef

30.

Byar DP, Simon RM, Friedewald WT, et al. Randomized clinical trials. Perspectives on some recent ideas. N Engl J Med 1976;295:74-80.CrossRef

31.

Spodick DH. The randomized controlled clinical trial. Scientific and ethical bases. Am J Med 1982;73:420-425.CrossRef

32.

Royall RM, Bartlett RH, Cornell RG, et al. Ethics and statistics in randomized clinical trials. Stat Sci 1991;6:52-88.MathSciNetCrossRef

33.

Intravenous streptokinase given within 0-4 hours of onset of myocardial infarction reduced mortality in ISIS-2. Lancet 1987;1:502.

34.

Huang SS, Septimus E, Kleinman K, et al. Targeted versus universal decolonization to prevent ICU infection. N Engl J Med 2013;368:2255-2265.CrossRef

35.

Dawson L, Zarin DA, Emanuel EJ, et al. Considering usual medical care in clinical trial design. PLoS Med 2009;6:e1000111.CrossRef

36.

Holm S, Harris J. The standard of care in multinational research. In: Emanuel EJ, Grady C, Crouch RA, et al (eds.). The Oxford Textbook of Clinical Research Ethics. Oxford: Oxford University Press, 2008:729-736.

37.

Bhatt DL, Kandzari DE, O’Neill WW, et al.; SYMPLICITY HTN-3 Investigators. A controlled trial of renal denervation for resistant hypertension. N Engl J Med 2014;370:1393-1401.CrossRef

38.

Temple RJ, Meyer R. Continued need for placebo in many cases, even when there is effective therapy. Arch Intern Med 2003;163:371-373.

39.

Freedman B, Weijer C, Glass KC. Placebo orthodoxy in clinical research. I: Empirical and methodological myths. J Law Med Ethics 1996;24:243-251.CrossRef

40.

Freedman B, Glass KC, Weijer C. Placebo orthodoxy in clinical research. II: Ethical, legal, and regulatory myths. J Law Med Ethics 1996;24:252-259.CrossRef

41.

Rothman KJ, Michels KB. The continuing unethical use of placebo controls. N Engl J Med 1994;331:394-398.CrossRef

42.

Michels KB, Rothman KJ. Update on unethical use of placebos in randomised trials. Bioethics 2003;17:188-204.CrossRef

43.

O’Shea JC, Hafley GE, Greenberg S, et al. Platelet glycoprotein IIb/IIIa integrin blockade with eptifibatide in coronary stent intervention: the ESPRIT trial: a randomized controlled trial. JAMA 2001;285:2468-2473.CrossRef

44.

Mann H, London AJ, Mann J. Equipoise in the enhanced suppression of the platelet IIb/IIIa receptor with integrilin trial (ESPRIT): a critical appraisal. Clin Trials 2005;2:233-241.CrossRef

45.

Tcheng J. Comment on Mann et al. Clin Trials 2005;2:242-243.

46.

International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use: ICH Harmonised Tripartite Guideline: Choice of Control Group and Related Issues in Clinical Trials E10 (July 2000). http://​www.​ich.​org/​fileadmin/​Public_​Web_​Site/​ICH_​Products/​Guidelines/​Efficacy/​E10/​Step4/​E10_​Guideline.​pdf

47.

Miller FG. The ethics of placebo-controlled trials. In: Emanuel EJ, Grady C, Crouch RA, et al (eds.). The Oxford Textbook of Clinical Research Ethics. Oxford: Oxford University Press, 2008:261-272.

48.

Levinsky NG. Nonfinancial conflicts of interest in research. N Engl J Med 2002;347:759-761.CrossRef

49.

Weinfurt KP, Hall MA, King NM, et al. Disclosure of financial relationships to participants in clinical research. N Engl J Med 2009;361:916-921.CrossRef

50.

Code of Federal Regulations, Title 45, Part 46. Available at: http://​www.​hhs.​gov/​ohrp/​humansubjects/​guidance/​45cfr46.​html. Accessed January 14, 2015.

51.

Code of Federal Regulations, Title 21. Available at: http://​www.​accessdata.​fda.​gov/​scripts/​cdrh/​cfdocs/​cfcfr/​CFRSearch.​cfm. Accessed January 14, 2015.

52.

Code of Federal Regulations, Title 21, Part 50. Available at: http://​www.​accessdata.​fda.​gov/​scripts/​cdrh/​cfdocs/​cfcfr/​CFRSearch.​cfm?​CFRPart=​50. Accessed January 14, 2015.

53.

Cassileth BR, Zupkis RV, Sutton-Smith K, et al. Informed consent -- why are its goals imperfectly realized? N Engl J Med 1980;302:896-900.

54.

Grunder TM. On the readability of surgical consent forms. N Engl J Med 1980;302:900-902.CrossRef

55.

Howard JM, DeMets D. How informed is informed consent? The BHAT experience. Control Clin Trials 1981;2:287-303.CrossRef

56.

Henderson GE, Churchill LR, Davis AM, et al. Clinical trials and medical care: defining the therapeutic misconception. PLoS Med 2007;4:e324.CrossRef

57.

U.S. Food and Drug Administration. Exception from informed consent requirements for emergency research. Available at: http://​www.​fda.​gov/​downloads/​RegulatoryInform​ation/​Guidances/​UCM249673.​pdf. Accessed January 14, 2015.

58.

Federal Register, volume 61, October 2, 1996. 45 CFR Part 46, pages 5131–3; Department of Health & Human Services, Waiver of Informed Consent Requirements in Certain Emergency Research. Available at: http://​www.​hhs.​gov/​ohrp/​policy/​hsdc97-01.​html. Accessed January 14, 2015.

59.

Tri-Council Policy Statement: Ethical conduct for research involving humans (amended 2005). http://​www.​ncehr-cnerh.​org/​english/​code_​2/​. Accessed January 14, 2015.

60.

European Medicines Agency ICH Topic E6 (R1) Guideline for Good Clinical Practice, January 1997 (corrected July 2002). Available at: http://​www.​ema.​europa.​eu/​docs/​en_​GB/​document_​library/​Scientific_​guideline/​2009/​09/​WC500002874.​pdf. Accessed January 14, 2015.

61.

Bottiger BW, Arntz HR, Chamberlain DA, et al. Thrombolysis during resuscitation for out-of-hospital cardiac arrest. N Engl J Med 2008;359:2651-2662.CrossRef

62.

Kim F, Nichol G, Maynard C, et al. Effect of prehospital induction of mild hypothermia on survival and neurological status among adults with cardiac arrest: a randomized clinical trial. JAMA 2014;311:45-52.CrossRef

63.

Burton TM. Despite heart attack deaths, PolyHeme still being tested on trauma patients. Wall Street Journal February 22, 2006.

64.

Kipnis K, King NM, Nelson RM. Trials and errors: barriers to oversight of research conducted under the emergency research consent waiver. IRB 2006;28:16-19.

65.

Kim SY, Miller FG. Informed consent for pragmatic trials--the integrated consent model. N Engl J Med 2014;370:769-772.

66.

U.S. Department of Health & Human Services. Frequently Asked Questions About Human Research. Available at: http://​answers.​hhs.​gov/​ohrp/​categories/​1568. Accessed January 14, 2015.

67.

Karlawish JH. Research involving cognitively impaired adults. N Engl J Med 2003;348:1389-1392.CrossRef

68.

Rosenstein DL, Miller FG. Research involving those at risk for impaired decision-making capacity. In: Emanuel EJ, Grady C, Crouch RA, et al (eds.). The Oxford Textbook of Clinical Research Ethics. Oxford: Oxford University Press, 2008:437-445.

69.

Steinbrook R. How best to ventilate? Trial design and patient safety in studies of the acute respiratory distress syndrome. N Engl J Med 2003;348:1393-1401.CrossRef

70.

Silverman HJ, Luce JM, Schwartz J. Protecting subjects with decisional impairment in research: the need for a multifaceted approach. Am J Respir Crit Care Med 2004;169:10-14.CrossRef

71.

Glickman SW, McHutchison JG, Peterson ED, et al. Ethical and scientific implications of the globalization of clinical research. N Engl J Med 2009;360:816-823.CrossRef

72.

Love RR, Duc NB, Allred DC, et al. Oophorectomy and tamoxifen adjuvant therapy in premenopausal Vietnamese and Chinese women with operable breast cancer. J Clin Oncol 2002;20:2559-2566.CrossRef

73.

Orlandini A, Díaz R, Wojdyla D, et al. Outcomes of patients in clinical trials with ST-segment elevation myocardial infarction among countries with different gross national incomes. Eur Heart J 2006;27:527-533.CrossRef

74.

Vickers A, Goyal N, Harland R, et al. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials 1998;19:159-166.CrossRef

75.

O’Shea JC, Califf RM. International differences in cardiovascular clinical trials. Am Heart J 2001;141:875-880.CrossRef

76.

London AL. Responsiveness to host community health needs. In: Emanuel EJ, Grady C, Crouch RA, et al (eds.). The Oxford Textbook of Clinical Research Ethics. Oxford: Oxford University Press, 2008:737-744.

77.

Lind SE. Finder’s fees for research subjects. N Engl J Med 1990;323:192-195.CrossRef

78.

The effect of vitamin E and beta carotene on the incidence of lung cancer and other cancers in male smokers. The Alpha-Tocopherol, Beta Carotene Cancer Prevention Study Group. N Engl J Med 1994;330:1029-1035.

79.

Miller AB, Buring J, Williams OD. Stopping the Carotene and Retinol Efficacy Trial: The Viewpoint of the Safety and Endpoint Monitoring Committee. In: DeMets DL, Furberg CD, Friedman LM (eds.). Data Monitoring in Clinical Trials: A Case Studies Approach. New York: Springer, 2006: 220-227.CrossRef

80.

Age-Related Eye Disease Study Research Group. The Age-Related Eye Disease Study (AREDS): design implications. AREDS report no. 1. Control Clin Trials 1999;20:573-600.

81.

Age-Related Eye Disease Study Research Group. A randomized, placebo-controlled, clinical trial of high-dose supplementation with vitamins C and E, beta carotene, and zinc for age-related macular degeneration and vision loss: AREDS report no. 8. Arch Ophthalmol 2001;119:1417-1436

82.

Tegeler CH, Furberg CD. Lessons from warfarin trials in atrial fibrillation: missing the window of opportunity. In: DeMets DL, Furberg CD, Friedman LM (eds.). Data Monitoring in Clinical Trials: A Case Studies Approach. New York: Springer, 2006:312-319.CrossRef

83.

Hulley S, Grady D, Bush T, et al. Randomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. Heart and Estrogen/progestin Replacement Study (HERS) Research Group. JAMA 1998;280:605-613.

84.

Rossouw JE, Anderson GL, Prentice RL, et al. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women’s Health Initiative randomized controlled trial. JAMA 2002;288:321-333.CrossRef

85.

Hulley SB, Grady D, Vittinghoff E, et al. Consideration of early stopping and other challenges in monitoring the Heart and Estrogen/Progestin Replacement Study. In: DeMets DL, Furberg CD, Friedman LM (eds.). Data Monitoring in Clinical Trials: A Case Studies Approach. New York: Springer, 2006:236-247.CrossRef

86.

Wittes J, Barrett-Connor E, Braunwald E, et al. Monitoring the randomized trials of the Women’s Health Initiative: the experience of the Data and Safety Monitoring Board. Clin Trials 2007;4:218-234.CrossRef

87.

Peppercorn J, Buss WG, Fost N, et al. The dilemma of data-safety monitoring: provision of significant new data to research participants. Lancet 2008;371:527-529.CrossRef

88.

Black HR, Elliott WJ, Grandits G, et al. Principal results of the Controlled Onset Verapamil Investigation of Cardiovascular End Points (CONVINCE) trial. JAMA 2003;289:2073-2082.CrossRef

89.

Psaty BM, Rennie D. Stopping medical research to save money: a broken pact with researchers and patients. JAMA 2003;289: 2128-2131.

90.

Marcus AD. Paying to keep your drug trial alive. Wall Street Journal. April 10, 2007.

91.

National Institutes of Health, U.S. Department of Health & Human Services. Health Insurance Portability and Accountability Act (HIPAA). Available at: http://​privacyruleandre​search.​nih.​gov/​. Accessed January 14, 2015.

92.

National Institutes of Health, U.S. Department of Health & Human Services. NIH Data Sharing Policy and Implementation Guidance (updated March 5, 2003). Available at: http://​grants.​nih.​gov/​grants/​policy/​data_​sharing/​data_​sharing_​guidance.​htm. Accessed January 14, 2015.

93.

Policy for Sharing of Data Obtained in NIH Supported or Conducted Genome-Wide Association Studies (GWAS). Available at: http://​grants.​nih.​gov/​grants/​guide/​notice-files/​NOT-OD-07-088.​html. Accessed January 14, 2015.

94.

Zarin DA, Tse T. Medicine. Moving toward transparency of clinical trials. Science 2008;319:1340-1342.CrossRef

95.

Wilson D, Meier B. Doctor falsified study on injured G.I.’s, Army says. The New York Times. May 12, 2009.

96.

Scott J. Withdrawal of a paper. J Bone Joint Surg Br 2009;91:285-286.CrossRef

97.

Fisher B, Anderson S, Redmond CK, et al. Reanalysis and results after 12 years of follow-up in a randomized clinical trial comparing total mastectomy with lumpectomy with or without irradiation in the treatment of breast cancer. N Engl J Med 1995;333:1456-1461.CrossRef

98.

Angell M, Kassirer JP. Setting the record straight in the breast cancer trials. N Engl J Med 1994;330:1448-1450.CrossRef

99.

Turner EH, Matthews AM, Linardatos E, et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008;358:252-260.

100.

Heres S, Davis J, Maino K, et al. Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics. Am J Psychiatry 2006;163:185-194.CrossRef

101.

Packer M, Carson P, Elkayam U, et al. Effect of amlodipine on the survival of patients with severe chronic heart failure due to a nonischemic cardiomyopathy: results of the PRAISE-2 study (prospective randomized amlodipine survival evaluation 2). JACC Heart Fail 2013;1:308-314.CrossRef

102.

Pfeffer MA, Skali H. PRAISE (prospective randomized amlodipine survival evaluation) and criticism. JACC Heart Fail 2013;1:315-317.CrossRef

103.

International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals: writing and editing for biomedical publication (updated October 2008). Available at: http://​www.​icmje.​org/​. Accessed January 14, 2015.

104.

Clinical Trials Registration in ClinicalTrials.gov (Public Law 110-85): Competing Applications and Non-Competing Progress Reports. Available at: http://​grants.​nih.​gov/​grants/​guide/​notice-files/​NOT-OD-08-023.​html. Accessed January 14, 2015.

105.

Federal Register, volume 73, Number 99, May 21, 2008. Available at: http://​edocket.​access.​gpo.​gov/​2008/​E8-11042.​htm. Accessed January 14, 2015.

106.

Zarin DA, Tse T, Williams RJ, et al. The ClinicalTrials.gov results database--update and key issues. N Engl J Med 2011;364:852-860.

107.

Steinbrook R. Gag clauses in clinical-trial agreements. N Engl J Med 2005;352:2160-2162.CrossRef

108.

Drazen JM, Van Der Weyden MB, Sahni P, et al. Uniform format for disclosure of competing interests in ICMJE journals. N Engl J Med 2009;361:1896-1897.CrossRef

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_3

3. What Is the Question?

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

The planning of a clinical trial depends on the question that the investigator is addressing. The general objective is usually obvious, but the specific question to be answered by the trial is often not stated well. Stating the question clearly and in advance encourages proper design. It also enhances the credibility of the findings. The reliability of clinical trial results derives in part from rigorous prospective definition of the hypothesis. This contrasts with observational studies where the analyses are often exploratory, may be part of an iterative process, and therefore more subject to chance [1]. One would like answers to a number of questions, but the study should be designed with only one major question in mind. This chapter discusses the selection of this primary question and appropriate ways of answering it. In addition, types of secondary and subsidiary questions are reviewed.

The first generation of clinical trials typically compared new interventions to placebo or no treatment on top of best current medical care. They addressed the straight-forward question of whether the new treatment was beneficial, neutral, or harmful compared to placebo or nothing. Since that time, the best medical care has improved dramatically, probably largely due to the contribution of randomized clinical trials (see Chap. 1).

Because of this success in developing beneficial therapies and preventive measures, new design challenges emerged. Prospective trial participants are likely to be on proven therapies. A new intervention is then either added to the existing one or compared against it. If a comparison between active treatments is performed in a clinical practice setting, the studies are often referred to as comparative effectiveness research. (Not all comparative effectiveness research involves clinical trials, but this book will be limited to a discussion of trials.) Due to the lower event rate in patients receiving best known care, whether in add-on trials or comparison trials, the margins for improvement with newer interventions became smaller. This statistical power issue has been addressed in three ways: first, sample sizes have been increased (see Chap. 8); second there has been an increased reliance on composite outcomes; and third, there has been an increased use of surrogate outcomes.

Another consequence of better treatment was the emergence of trials designed to answer a different type of question. In the past, as noted above, the typical question was: Is the new intervention better, or superior to, no treatment or standard treatment? Now, we frequently ask: Do alternative treatments that may be equal to, or at least no worse than, existing treatments with regard to the primary outcome convey other important advantages in terms of safety, adherence, patient convenience, and/or cost? These trials are often referred to as noninferiority trials and are discussed later in this chapter and in more detail in Chaps. 5, 8, and 18.

Fundamental Point

Each clinical trial must have a primary question. The primary question, as well as any secondary or subsidiary questions, should be carefully selected, clearly defined, and stated in advance.

Selection of the Questions

Primary Question

The primary question should be the one the investigators and sponsors are most interested in answering and that is capable of being adequately answered. It is the question upon which the sample size of the study is based, and which must be emphasized in the reporting of the trial results. The primary question may be framed in the form of testing a hypothesis because most of the time an intervention is postulated to have a particular outcome which, on the average, will be different from (or, in the case of noninferiority trials, not worse than) the outcome in a control group [2]. The outcome may be a clinical event such as improving survival, ameliorating an illness or disease complications, reducing symptoms, or improving quality of life; modifying an intermediate or surrogate characteristic such as blood pressure; or changing a biomarker such as a laboratory value.

Sometimes, trials are designed with more than one primary question. This may be appropriate, depending on the trial design. For example, factorial design trials are specifically conducted to answer more than one question. If done in the context of the usual parallel design trial, statistical adjustments might need to be made to account for the additional question(s) and the sample size made adequate. See Chap. 8 for further discussion of the issue of adjustments in parallel design trials.

Secondary Questions Regarding Benefit

There may also be a variety of subsidiary, or secondary questions that are usually related to the primary question. The study may be designed to help address these, or else data collected for the purpose of answering the primary question may also elucidate the secondary questions. They can be of two types. In the first, the response variable is different than that in the primary question. For example, the primary question might ask whether mortality from any cause is altered by the intervention. Secondary questions might relate to incidence of cause-specific death (such as cancer mortality), incidence of non-fatal renal failure, or incidence of stroke. Many investigators also assess patient-reported outcomes such as health-related quality of life (see Chap. 13).

The second type of secondary question relates to subgroup hypotheses. For example, in a study of cancer therapy, the investigator may want to look specifically at people by gender, age, stage of disease at entry into the trial or by presence or absence of a particular biomarker or genetic marker. Such subsets of people in the intervention group can be compared with similar people in the control group. Subgroup hypotheses should be 1) specified before data collection begins, 2) based on reasonable expectations, and 3) limited in number. In any event, the number of participants in most subgroups is usually too small to prove or disprove a subgroup hypothesis. One should not expect significant differences in subgroup unless the trial was specifically designed to detect them. Failure to find significant differences should not be interpreted to mean that they do not exist. Investigators should exercise caution in accepting subgroup results, especially when the overall trial results are not significant. A survey of clinical trialists indicated that inappropriate subgroup analyses were considered one of the two major sources of distortion of trial findings [3]. Generally, the most useful reasons for considering subgroups are to examine consistency of results across pre-defined subgroups and to create hypotheses that can be tested in future trials and meta-analyses.

There has been recognition that certain subgroups of people have not been adequately represented in clinical research, including clinical trials [4]. In the United States, this has led to requirements that women and minority populations be included in appropriate numbers in trials supported by federal government agencies [5]. The debate is whether the numbers of participants of each sex and racial/ethnic group must be adequate to answer the key questions that the trial addresses, or whether there must merely be adequate diversity of people. Many trials are international in scope. Whether one should examine outcome data by country or region has been debated [6]. Are observed differences in intervention effect by geographic region true or due to the play of chance [7, 8]? One might expect that culture, medical care system, genetic makeup, and other factors could affect the magnitude, or even existence of benefit from a new intervention. But, as has been noted [9, 10], the design and size of the trial should be driven by reasonable expectations that the intervention will or will not operate materially differently among the various subsets of participants. If such variability is expected, it is appropriate to design the trial to detect those differences. If not, adequate diversity with the opportunity to examine subgroup responses at the end of the trial (and conduct additional research if necessary) is more appropriate.

Secondary questions raise several trial methodological issues; for example, if enough statistical tests are done, a few will be significant by chance alone when there is no true intervention effect. An example was provided by the Second International Study of Infarct Survival (ISIS-2), a factorial design trial of aspirin and streptokinase in patients with acute myocardial infarction [11]. To illustrate the hazards of subgroup analyses, the investigators showed that participants born under the Gemini or Libra astrological birth signs had a somewhat greater incidence of vascular and total mortality on aspirin than on no aspirin, whereas for all other signs, and overall, there was an impressive and highly significant benefit from aspirin. Therefore, when a number of tests are carried out, results should be interpreted cautiously as they may well be due to chance. Shedding light or raising new hypotheses, and perhaps conducting meta-analyses, are more proper outcomes of these analyses than are conclusive answers. See Chap. 18 for further discussion of subgroup and meta-analyses.

Both primary and secondary questions should be important and relevant scientifically, medically, or for public health purposes. Participant safety and well-being must always be considered in evaluating importance. Potential benefit and risk of harm should be looked at by the investigator, as well as by local ethical review committees, and often, independent data monitoring committees.

Questions Regarding Harm

Important questions that can be answered by clinical trials concern adverse effects of or reactions to therapy (Chap. 12). Here, unlike the primary or secondary questions, it is not always possible to specify in advance the questions to be answered. What adverse effects might occur, and their severity, may be unpredictable. Furthermore, rigorous, convincing demonstration of serious toxicity is usually not achieved, because it is generally thought unethical to continue a study to the point at which a drug has been conclusively shown to be more harmful than beneficial [12–14]. Investigators traditionally monitor a variety of laboratory and clinical measurements, look for possible adverse events, and compare these in the intervention and control groups. Some of the most serious adverse effects, however, are rare and do not occur commonly enough to be detected reliably in clinical trials. Statistical significance and the previously mentioned problem of multiple response variables become secondary to clinical judgment and participant safety. While this will lead to the conclusion that some purely chance findings are labeled as adverse effects, responsibility to the participants requires a conservative attitude toward safety monitoring, particularly if an alternative therapy is available. Trials have been stopped early for less than statistically convincing evidence of adverse effects [15–17]. In such cases, only other trials of the identical or related interventions noting the same adverse effect (as were the situations for these examples of antiarrhythmic therapy in people with heart disease, beta carotene in people at high risk of lung cancer, and an angiotensin-converting enzyme inhibitor in acute myocardial infarction) or convincing nonclinical studies will provide irrefutable evidence that the adverse finding is true. In the last case cited, other studies contradicted the finding.

Ancillary Questions

Often a clinical trial can be used to answer questions which do not bear directly on the intervention being tested, but which are nevertheless of interest. The structure of the trial and the ready access to participants may make it the ideal vehicle for such investigations. Large trials, in particular, create databases that offer opportunities to better understand the disease or condition, treatment, predictors of outcomes, and new hypotheses that can be tested. The Group Utilization of Streptokinase and Tissue Plasminogen Activator for Occluded Coronary Arteries (GUSTO-1) trial [18] provides an example of use of a dataset that yielded over 100 subsequent publications, including one identifying predictors of mortality [19]. The Assessment of Pexelizumab in Acute Myocardial Infarction (APEX AMI) trial [20] found no benefit from the complement inhibitor, pexelizumab, but so far, over 50 manuscripts regarding primary angioplasty in acute ST-elevation myocardial infarction have been published.

Clinical trials can also be used to examine issues such as how the intervention works. A small group of participants might undergo mechanistic studies (as long as they are not unduly burdensome or invasive). In the Studies of Left Ventricular Dysfunction (SOLVD) [21], the investigators evaluated whether an angiotensin converting enzyme inhibitor would reduce mortality in symptomatic and asymptomatic subjects with impaired cardiac function. In selected participants, special studies were done with the objective of getting a better understanding of the disease process and of the mechanisms of action of the intervention. These substudies did not require the large sample size of the main studies (over 6,000 participants). Thus, most participants in the main trials had a relatively simple and short evaluation and did not undergo the expensive and time-consuming procedures or interviews demanded by the substudies. This combination of a rather limited assessment in many participants, designed to address an easily monitored response variable, and detailed measurements in subsets, can be extremely effective. An angiographic substudy in the GUSTO trial helped explain how accelerated alteplase treatment resulted in more effective coronary perfusion [22]. The improved survival appeared to be fully explained by this impact on reperfusion [23]. In the Harmonizing Outcomes with Revascularization and Stents in Acute Myocardial Infarction (HORIZONS-AMI) trial [24], lower rates of bleeding with bivalirudin compared with unfractionated heparin plus a glycoprotein IIb/IIIa inhibitor appeared to explain only part of the lower subsequent mortality in the bivalirudin group [25].

Exploratory genetic studies are commonly conducted to examine possible mechanisms of action of the intervention. Genetic variants of the cytochrome P450 CYP2C19 metabolic pathway of clopidogrel were related to the level of the active metabolite and reduction in platelet aggregation for participants treated with clopidogrel in the database from the Trial to Assess Improvement in Therapeutic Outcomes by Optimizing Platelet Inhibition with Prasugrel-Thrombolysis in Myocardial Infarction (TRITON-TIMI) [26].

Kinds of Trials

Trials with Extensive Data Collection vs. Large, Simple

Traditionally, most trials of new interventions have collected extensive information about participants, have detailed inclusion and exclusion criteria, involve considerable quality assurance measures, and assess many, carefully measured outcomes. These sorts of trials, although they address major questions and are well-conducted, are quite expensive and often very time-consuming. Therefore, given the needed resources, trial sponsors can afford to address only some of the many important questions can be answered, often in limited kinds of participants and clinical settings.

As discussed by Tricoci et al. [27] with respect to clinical practice guidelines in cardiology, but undoubtedly similar in other medical fields, many of these guidelines are based on inadequate data. One of the rationales for large, simple clinical trials is that they can provide data relevant to clinical practice, since they are typically conducted in practice settings [28]. The general idea is that for common conditions, and important outcomes, such as total mortality, even modest benefits of intervention, particularly interventions that are easily implemented in a large population, are important. Because an intervention is likely to have similar effects (or at least effects that trend in the same direction) in most participants, extensive characterization of people at entry may be unnecessary. The study must have unbiased allocation of participants to intervention or control and unbiased and reasonably complete ascertainment of outcomes. Sufficiently large numbers of participants are more important in providing the statistical power necessary to answer the question(s) than careful attention to quality and completeness of data. This model depends upon a relatively easily administered intervention, brief forms, and an easily ascertained outcome, such as a fatal or unambiguous nonfatal event. Neither the trials that collect extensive information nor the simple ones are better. Rather, both types are essential. The proper design depends on the condition being studied, the nature of the question, and the kind of intervention.

Superiority vs. Noninferiority Trials

As mentioned in the introduction to this chapter, traditionally, most trials were designed to establish whether a new intervention on top of usual or standard care was superior to that care alone (or that care plus placebo). If there were no effective treatments, the new intervention was compared to just placebo. As discussed in Chap. 8, these trials were generally two-sided. That is, the trial was designed to see whether the new intervention was better or worse than the control.

With the development of effective therapies, many trials have been designed to demonstrate that a new intervention is not worse than the intervention previously shown to be beneficial, i.e., an active control, by some prespecified amount. As noted earlier, the motivation for such a question is that the new intervention might not be better than standard treatment on the primary or important secondary outcomes, but may be less toxic, more convenient, less invasive and/or have some other attractive feature, including lower cost. The challenge is to define what is meant by “not worse than.” This has been referred to as the “margin of indifference,” or δ, meaning that if the new intervention is not less effective than this margin, its use might be of value given the other features. In the analysis of this design, the 95% upper confidence limit would need to be less than this margin in order to claim noninferiority. Defining δ is challenging and will be discussed in Chap. 5.

The question in a noninferiority trial is different than in a superiority trial and affects both the design and conduct of the trial. For example, in the superiority trial, poor adherence will lead to a decreased ability, or power, to detect a meaningful difference. For a noninferiority trial, poor adherence will diminish real and important differences and bias the results towards a noninferiority claim. Thus, great care must be taken in defining the question, the sensitivity of the outcome measures to the intervention being evaluated, and the adherence to the intervention during the conduct of the trial.

Comparative Effectiveness Trials

As mentioned, major efforts are being devoted to conducting comparative effectiveness research. Although comparative effectiveness studies can be of various sorts, encompassing several kinds of clinical research, we will limit our discussion to clinical trials. Much medical care has not been rigorously evaluated, meaning that trials comparing ongoing preventive and treatment approaches are needed. And of course, when new interventions are developed, they must be compared against existing therapy. Additionally, the increasing cost burden of medical care means that even if several treatments are equally effective, we need to consider factors such as cost, tolerability, and ease of administration. Therefore, comparative effectiveness trials are commonly of the noninferiority sort.

Much of the literature on comparative effectiveness research advocates conducting the studies in usual practice settings (often called pragmatic trials) [29, 30] (see Chap. 4). Because these trials are conducted in clinical practice settings, they must be relatively simple, demanding little in the way of effort to screen and assess outcomes. The goal is to compare two interventions, both of which are considered standard care.

Intervention

When the question is conceived, investigators, at the very least have in mind a class or type of intervention. More commonly, they know the precise drug, procedure, or lifestyle modification they wish to study. In reaching such a decision, they need to consider several aspects.

First, the potential benefit of the intervention must be maximized, while possible harm is kept to a minimum. Thus, dose of drug or intensity of rehabilitation and frequency and route of administration are key factors that need to be determined. Can the intervention or intervention approach be standardized, and remain reasonably stable over the duration of the trial? Investigators must also decide whether to use a single drug, biologic, or device, fixed or adjustable doses of drugs, sequential drugs, or drug or device combinations. Devices in particular undergo frequent modifications and updates. Investigators need to be satisfied that any new version that appears during the course of the trial functions sufficiently similarly in important ways to the older versions so that combining data from the versions would be appropriate. Of course, an investigator can use only the version available at the onset of the trial (if it is still obtainable), but the trial will then be criticized for employing the outdated version. For example, coronary stents have evolved and the newer ones have lower risk of stent thrombosis [31]. This development may have altered their relative effectiveness vs. bypass surgery, therefore trials that continued to use the older versions of the stents have little credibility.

Sometimes, it is not only the active intervention, but other factors that apply. In gene transfer studies, the nature of the vector, as well as the actual gene, may materially affect the outcome, particularly when it comes to adverse effects. If the intervention is a procedure, other considerations must be considered. Surgical and other procedures or techniques are frequently modified and some practitioners are more skilled than others. Investigators need to think about learning curves, and at what point someone has sufficient skill to perform the intervention.

Not only the nature of the intervention, but what constitutes the control group regimen must also be considered for ethical reasons, as discussed in Chap. 2, and study design reasons, as discussed in Chap. 5.

Second, the availability of the drug or device for testing needs to be determined. If it is not yet licensed, special approval from the regulatory agency and cooperation or support by the manufacturer are required (see Chap. 22).

Third, investigators must take into account design aspects, such as time of initiation and duration of the intervention, need for special tests or laboratory facilities, and the logistics of blinding in the case of drug studies. Certain kinds of interventions, such as surgical procedures, device implantation, vaccines, and gene transfer may have long-term or even life-long effects. Therefore, investigators might need to incorporate plans for long-term assessment. There had been reports that drug-eluting stents, used in percutaneous coronary intervention, perhaps had a greater likelihood of restenosis than bare-metal stents [32, 33]. Follow-up studies seemed to assuage these concerns [34]. Nevertheless, investigators must consider incorporating plans for long-term assessment. Problems with metal-on-metal hip replacements were only uncovered years after many had been implanted [35, 36]. The rubbing of the metal ball against the metal cup causes metal particles to wear away, possibly leading to both local and systemic adverse effects.

Response Variables

Kinds of Response Variables

Response variables are outcomes measured during the course of the trial, and they define and answer the questions. A response variable may be total mortality, death from a specific cause, incidence of a disease, a complication or specific adverse effect, symptomatic relief, quality of life, a clinical finding, a laboratory measurement, or the cost and ease of administering the intervention. If the primary question concerns total mortality, the occurrence of deaths in the trial clearly answers the question. If the primary question involves severity of arthritis, on the other hand, extent of mobility or a measure of freedom from pain may be reasonably good indicators. In other circumstances, a specific response variable may only partially reflect the overall question. As seen from the above examples, the response variable may show a change from one discrete state (living) to another (dead), from one discrete state to any of several other states (changing from one stage of disease to another) or from one level of a continuous variable to another. If the question can be appropriately defined using a continuous variable, the required sample size may be reduced (Chap. 8). However, the investigator needs to be careful that this variable and any observed differences are clinically meaningful and relevant and that the use of a continuous variable is not simply a device to reduce sample size.

In general, a single response variable should be identified to answer the primary question. If more than one are used, the probability of getting a nominally significant result by chance alone is increased (Chap. 18). In addition, if the different response variables give inconsistent results, interpretation becomes difficult. The investigator would then need to consider which outcome is most important, and explain why the others gave conflicting results. Unless she has made the determination of relative importance prior to data collection, her explanations are likely to be unconvincing.

Although the practice is not advocated, there may be circumstances when more than one “primary” response variable needs to be looked at. This may be the case when an investigator truly cannot decide which of several response variables relates most closely to the primary question. Ideally, the trial would be postponed until this decision can be made. However, overriding concerns, such as increasing use of the intervention in general medical practice, may compel her to conduct the study earlier. In these circumstances, rather than arbitrarily selecting one response variable which may, in retrospect, turn out to be suboptimal or even inappropriate, investigators prefer to list several “primary” outcomes. An old example is the Urokinase Pulmonary Embolism Trial [37], where lung scan, arteriogram and hemodynamic measures were given as the “primary” response variables in assessing the effectiveness of the agents urokinase and streptokinase. Chapter 8 discusses the calculation of sample size when a study with several primary response variables is designed.

Commonly, investigators prepare an extensive list of secondary outcomes, allowing them to claim that they “prespecified” these outcomes when one or more turn out to reach nominally significant differences. Although prespecification provides some protection against accusations that the findings were data-derived, a long list does not protect against likely play of chance. Far better is a short list of outcomes that are truly thought to be potentially affected by the intervention. Combining events to make up a response variable might be useful if any one event occurs too infrequently for the investigator reasonably to expect a significant difference without using a large number of participants. In answering a question where the response variable involves a combination of events, only one event per participant should be counted. That is, the analysis is by participant, not by event.

One kind of combination response variable involves two kinds of events. This has been termed a composite outcome. It must be emphasized, however, that the composite outcome should be capable of meaningful interpretation such as where all components are related through a common underlying condition or respond to the same presumed mechanism of action of the agent. In a study of heart disease, combined events might be death from coronary heart disease plus nonfatal myocardial infarction. This is clinically meaningful since death from coronary heart disease and nonfatal myocardial infarction might together represent a measure of serious coronary heart disease. Unfortunately, as identified in a survey of 40 trials using composite outcomes by Cordoba et al. [38], there was considerable lack of clarity as to how components were combined and results reported. Difficulties in interpretation can arise if the results of each of the components in such a response variable are inconsistent [39]. In the Physicians’ Health Study report of aspirin to prevent cardiovascular disease, there was no difference in mortality, a large reduction in myocardial infarction, and an increase in stroke, primarily hemorrhagic [40]. In this case, cardiovascular mortality was the primary response variable, rather than a combination. If it had been a combination, the interpretation of the results would have been even more difficult than it was [41]. Even more troublesome is the situation where one of the components in the combination response variable is far less serious than the others. For example, if occurrence of angina pectoris or a revascularization procedure is added, as is commonly done, interpretation can be problematic. Not only are these less serious than cardiovascular death or myocardial infarction, they often occur more frequently. Thus, if overall differences between groups are seen, are these results driven primarily by the less serious components? What if the results for the more serious components (e.g., death) trend in the opposite directions? This is not just theoretical. For example, the largest difference between intervention and control in the Myocardial Ischemia Reduction with Aggressive Cholesterol Lowering (MIRACL) trial was seen in the least serious of the four components; the one that occurred most often in the control group [42]. A survey of published trials in cardiovascular disease that used composite response variables showed that half had major differences in both importance and effect sizes of the individual components [43]. Those components considered to be most important had, on average, smaller benefits than the more minor ones. See Chap. 18 for a discussion of analytic and interpretation issues if the components of the composite outcome go in different directions or have other considerable differences in the effect size.

When this kind of combination response variable is used, the rules for interpreting the results and for possibly making regulatory claims about individual components should be established in advance. A survey of the cardiovascular literature found that the use of composite outcomes (often with three or four components) is common, and the components vary in importance [44]. One possible approach is to require that the most serious individual components show the same trend as the overall result. Some have suggested giving each component weights, depending on the seriousness [45, 46]. However, this may lead to trial results framed as unfamiliar scores that are difficult to interpret by clinicians. Although it has sample size implications, it is probably preferable to include in the combined primary response variable only those components that are truly serious, and to assess the other components as secondary outcomes. If an important part of a composite outcome goes in the wrong direction, as occurred with death in the Sodium-Hydrogen Exchange Inhibition to Prevent Coronary Events in Acute Cardiac Conditions (EXPEDITION) trial [47], even benefit in the composite outcome (death or myocardial infarction), is insufficient to conclude that the intervention (in this case, sodium-hydrogen exchange inhibition by means of cariporide during coronary artery bypass graft surgery) should be used. Adding to the concern was an adverse trend for cerebrovascular events.

Another kind of combination response variable involves multiple events of the same sort. Rather than simply asking whether an event has occurred, the investigator can look at the frequency with which it occurs. This may be a more meaningful way of looking at the question than seeking a yes-no outcome. For example, frequency of recurrent transient ischemic attacks or epileptic seizures within a specific follow-up period might comprise the primary response variable of interest. Simply adding up the number of recurrent episodes and dividing by the number of participants in each group in order to arrive at an average would be improper. Multiple events in an individual are not independent, and averaging gives undue weight to those with more than one episode. One approach is to compare the number of participants with none, one, two, or more episodes; that is, the distribution of the number of episodes, by individual.

Sometimes, study participants enter a trial with a condition that is exhibited frequently. For example, they may have had several episodes of transient atrial fibrillation in the previous weeks or may drink alcohol to excess several days a month. Trial eligibility criteria may even require a minimum number of such episodes. A trial of a new treatment for alcohol abuse may require participants to have at least six alcoholic drinks a day for at least 7 days over the previous month. The investigator needs to decide what constitutes a beneficial response. Is it complete cessation of drinking? Reducing the number of drinks to some fixed level (e.g., no more than two on any given day)? Reducing alcohol intake by some percent, and if so, what percent? Does this fixed level or percent differ depending on the intake at the start of the trial? Decisions must be made based on knowledge of the disease or condition, the kind of intervention and the expectations of how the intervention will work. The clinical importance of improvement versus complete “cure” must also be considered.

Specifying the Question

Regardless of whether an investigator is measuring a primary or secondary response variable, certain rules apply. First, she should define and record the questions in advance, being as specific as possible. She should not simply ask, “Is A better than B?” Rather, she should ask, “In population W is drug A at daily dose X more efficacious in improving Z by Q amount over a period of time T than drug B at daily dose Y?” Implicit here is the magnitude of the difference that the investigator is interested in detecting. Stating the questions and response variables in advance is essential for planning of study design and calculation of sample size. As shown in Chap. 8, sample size calculation requires specification of the response variables as well as estimates of the intervention effect. In addition, the investigator is forced to consider what she means by a successful intervention. For example, does the intervention need to reduce mortality by 10 or 25% before a recommendation for its general use is made? Since such recommendations also depend on the frequency and severity of adverse effects, a successful result cannot be completely defined beforehand. However, if a 10% reduction in mortality is clinically important, that should be stated, since it has sample size implications. Specifying response variables and anticipated benefit in advance also eliminates the possibility of the legitimate criticism that can be made if the investigator looked at the data until she found a statistically significant result and then decided that that response variable was what she really had in mind all the time

Second, the primary response variable must be capable of being assessed in all participants. Selecting one response variable to answer the primary question in some participants, and another response variable to answer the same primary question in other participants is not a legitimate practice. It implies that each response variable answers the question of interest with the same precision and accuracy; i.e., that each measures exactly the same thing. Such agreement is unlikely. Similarly, response variables should be measured in the same way for all participants. Measuring a given variable by different instruments or techniques implies that the instruments or techniques yield precisely the same information. This rarely, if ever, occurs. If response variables can be measured only one way in some participants and another way in other participants, two separate studies are actually being performed, each of which is likely to be too small.

Third, unless there is a combination primary response variable in which the participant remains at risk of having additional events, participation generally ends when the primary response variable occurs. “Generally” is used here because, unless death is the primary response variable, the investigator may well be interested in certain events, including adverse events, subsequent to the occurrence of the primary response variable. These events will not change the analysis of the primary response variable but may affect the interpretation of results. For example, deaths taking place after a nonfatal primary response variable has already occurred, but before the official end of the trial as a whole, may be of interest. On the other hand, if a secondary response variable occurs, the participant should remain in the study (unless, of course, it is a fatal secondary response variable). He must continue to be followed because he is still at risk of developing the primary response variable. A study of heart disease may have, as its primary question, death from coronary heart disease and, as a secondary question, incidence of nonfatal myocardial infarction. If a participant suffers a nonfatal myocardial infarction, this counts toward the secondary response variable. However, he ought to remain in the study for analytic purposes and be at risk of developing the primary response variable and of having other adverse events. This is true whether or not he is continued on the intervention regimen. If he does not remain in the study for purposes of analysis of the primary response variable, bias may result. (See Chap. 18 for further discussion of participant withdrawal.)

Fourth, response variables should be capable of unbiased assessment. Truly double-blind studies have a distinct advantage over other studies in this regard. If a trial is not double-blind (Chap. 7), then, whenever possible, response variable assessment should be done by people who are not involved in participant follow-up and who are blinded to the identity of the study group of each participant. Independent reviewers are often helpful. Of course, the use of blinded or independent reviewers does not entirely solve the problem of bias. Unblinded investigators sometimes fill out forms and the participants may be influenced by the investigators. This may be the case during a treadmill exercise performance test, where the impact of the person administering the test on the results may be considerable. Some studies arrange to have the intervention administered by one investigator and response variables evaluated by another. Unless the participant is blinded to his group assignment (or otherwise unable to communicate), this procedure is also vulnerable to bias. One solution to this dilemma is to use only “hard,” or objective, response variables (which are unambiguous and not open to interpretation, such as total mortality or some imaging or laboratory measures read by someone blinded to the intervention assignment). This assumes complete and honest ascertainment of outcome. Double-blind studies have the advantage of allowing the use of softer response variables, since the risk of assessment bias is minimized.

Fifth, it is important to have response variables that can be ascertained as completely as possible. A hazard of long-term studies is that participants may fail to return for follow-up appointments. If the response variable is one that depends on an interview or an examination, and participants fail to return for follow-up appointments information will be lost. Not only will it be lost, but it may be differentially lost in the intervention and control groups. Death or hospitalizations are useful response variables because the investigator can usually ascertain vital status or occurrence of a hospital admission, even if the participant is no longer active in a study. However, only in a minority of clinical trials are they appropriate.

Sometimes, participants withdraw their consent to be in the trial after the trial has begun. In such cases, the investigator should ascertain whether the participant is simply refusing to return for follow-up visits but is willing to have his data used, including data that might be obtained from public records; is willing to have only data collected up to the time of withdrawal used in analyses; or is asking that all of his data be deleted from the study records.

All clinical trials are compromises between the ideal and the practical. This is true in the selection of primary response variables. The most objective or those most easily measured may occur too infrequently, may fail to define adequately the primary question, or may be too costly. To select a response variable which can be reasonably and reliably assessed and yet which can provide an answer to the primary question requires judgment. If such a response variable cannot be found, the wisdom of conducting the trial should be re-evaluated.

Biomarkers and Surrogate Response Variables

A common criticism of clinical trials is that they are expensive and of long duration. This is particularly true for trials which use the occurrence of clinical events as the primary response variable. It has been suggested that response variables which are continuous in nature might substitute for the binary clinical outcomes. Thus, instead of monitoring cardiovascular mortality or myocardial infarction an investigator could examine progress of atherosclerosis by means of angiography or ultrasound imaging, or change in cardiac arrhythmia by means of ambulatory electrocardiograms or programmed electrical stimulation. In the cancer field, change in tumor size might replace mortality. In AIDS trials, change in CD-4 lymphocyte level has been used as a response to treatment instead of incidence of AIDS in HIV positive patients or mortality. Improved bone mineral density has been used as a surrogate for reduction in fractures.

A rationale for use of these “surrogate response variables” is that since the variables are continuous, the sample size can be smaller and the study less expensive than otherwise. Also, changes in the variables are likely to occur before the clinical event, shortening the time required for the trial. Wittes et al. [48] discuss examples of savings in sample size by the use of surrogate response variables.

It has been argued that in the case of truly life-threatening diseases (e.g., AIDS in its early days, certain cancers, serious heart failure), clinical trials should not be necessary to license a drug or other intervention. Given the severity of the condition, lesser standards of proof should be required. If clinical trials are done, surrogate response variables ought to be acceptable, as speed in determining possible benefit is crucial. Potential errors in declaring an intervention useful may therefore not be as important as early discovery of a truly effective treatment.

Even in such instances, however, one should not uncritically use surrogate endpoints [49, 50]. It was known for years that the presence of ventricular arrhythmias correlated with increased likelihood of sudden death and total mortality in people with heart disease [51], as it was presumably one mechanism for the increased mortality. Therefore, it was common practice to administer antiarrhythmic drugs with the aim of reducing the incidence of sudden cardiac death [52, 53]. The Cardiac Arrhythmia Suppression Trial demonstrated, however, that three drugs that effectively treated ventricular arrhythmias were not only ineffective in reducing sudden cardiac death, but actually caused increased mortality [54, 15].

A second example concerns the use of inotropic agents in people with heart failure. These drugs had been shown to improve exercise tolerance and other symptomatic manifestations of heart failure [55]. It was expected that mortality would also be reduced. Unfortunately, clinical trials subsequently showed that mortality was increased [56, 57].

Another example from the cardiovascular field is the Investigation of Lipid Level Management to Understand its Impact in Atherosclerotic Events (ILLUMINATE). In this trial, the combination of torcetrapib and atorvastatin was compared with atorvastatin alone in people with cardiovascular disease or diabetes. Despite the expected impressive and highly statistically significant increase in HDL-cholesterol and decrease in LDL-cholesterol in the combination group, there was an increase in all-cause mortality and major cardiovascular events [58]. Thus, even though it is well-known that lowering LDL-cholesterol (and possibly increasing HDL-cholesterol) can lead to a reduction in coronary heart disease events, some interventions might have unforeseen adverse consequences. Recent studies looking at the raising of HDL-cholesterol have also been disappointing, despite the theoretical grounds to expect benefit [59]. The Atherothrombosis Intervention in Metabolic Syndrome with Low HDL/High Triglycerides and Impact on Global Health Outcomes (AIM-HIGH) trial [60] and the Second Hearth Protection Study (HPS-2 THRIVE) [61] did not reduce cardiovascular outcomes in the context of lowering LDL-cholesterol.

It was noted that the level of CD-4 lymphocytes in the blood is associated with severity of AIDS. Therefore, despite some concerns [62] a number of clinical trials used change in CD-4 lymphocyte concentration as an indicator of disease status. If the level rose, the drug was considered to be beneficial. Lin et al., however, argued that CD-4 lymphocyte count accounts for only part of the relationship between treatment with zidovudine and outcome [63]. Choi et al. came to similar conclusions [64]. In a trial comparing zidovudine with zalcitabine, zalcitabine was found to lead to a slower decline in CD-4 lymphocytes than did zidovudine, but had no effect on the death rate from AIDS [65]. Also troubling were the results of a large trial which, although showing an early rise in CD-4 lymphocytes, did not demonstrate any long-term benefit from zidovudine [66]. Whether zidovudine or another treatment was, or was not, truly beneficial is not the issue here. The main point is that the effect of a drug on a surrogate endpoint (CD-4 lymphocytes) is not always a good indicator of clinical outcome. This is summarized by Fleming, who noted that the CD-4 lymphocyte count showed positive results in seven out of eight trials where clinical outcomes were also positive. However, the CD-4 count was also positive in six out of eight trials in which the clinical outcomes were negative [50].

Similar seemingly contradictory results have been seen with cancer clinical trials. In trials of 5-fluorouracil plus leucovorin compared with 5-fluorouracil alone, the combination led to significantly better tumor response, but no difference in survival [67]. Fleming cites other cancer examples as well [50]. Sodium fluoride, because of its stimulation of bone formation, was widely used in the treatment of osteoporosis. Despite this, it was found in a trial in women with postmenopausal osteoporosis to increase bone fragility [68].

These examples do not mean that surrogate response variables should never be used in clinical trials. Nevertheless, they do point out that they should only be used after considering the advantages and disadvantages, recognizing that erroneous conclusions about interventions might occasionally be reached.

Prentice has summarized two key criteria that must be met if a surrogate response variable is to be useful [69]. First, the surrogate must correlate with the true clinical outcome, which most proposed surrogates would likely do. Second, for a surrogate to be valid, it must capture the full effect of the intervention. For example, a drug might lower blood pressure or serum LDL-cholesterol, but as in the ILLUMINATE trial example, have some other deleterious effect that would negate any benefit or even prove harmful.

Another factor is whether the surrogate variable can be assessed accurately and reliably. Is there so much measurement error that, in fact, the sample size requirement increases or the results are questioned? Additionally, will the evaluation be so unacceptable to the participant that the study will become infeasible? If it requires invasive techniques, participants may refuse to join the trial, or worse, discontinue participation before the end. Measurement can require expensive equipment and highly trained staff, which may, in the end, make the trial more costly than if clinical events are monitored. The small sample size of surrogate response variable trials may mean that important data on safety are not obtained [70]. Finally, will the conclusions of the trial be accepted by the scientific and medical communities? If there is insufficient acceptance that the surrogate variable reflects clinical outcome, in spite of the investigator’s conviction, there is little point in using such variables.

Many drugs have been approved by regulatory agencies on the basis of surrogate response variables, including those that reduce blood pressure and blood sugar. In the latter case, though, the Food and Drug Administration now requires new diabetes drugs to show that cardiovascular events are not increased [71]. We think that, except in rare instances, whenever interventions are approved by regulatory bodies on the basis of surrogate response variables, further clinical studies with clinical outcomes should be conducted afterward. As discussed by Avorn [72], however, this has not always been the case. He cites examples not only of major adverse effects uncovered after drugs were approved on the basis of surrogate outcomes, but lack of proven clinical benefit. In all decisions regarding approval, the issues of biologic plausibility, risk, benefits, and history of success must be considered.

When are surrogate response variables useful? The situation of extremely serious conditions has been mentioned. Particularly, when serious conditions are also rare, it may be difficult or even impossible to obtain enough participants to use a clinical outcome. We may be forced to rely on surrogate outcomes. Other than those situations, surrogate response variables are useful in early phase development studies, as an aid in deciding on proper dosage and whether the anticipated biologic effects are being achieved. They can help in deciding whether, and how best, to conduct the late phase trials which almost always should employ clinical response variables.

Changing the Question

Occasionally, investigators want to change the primary response variable partway through a trial. Reasons for this might be several, but usually it is because achieving adequate power for the original primary response variable is no longer considered feasible. The event rate might be less than expected, and even extension of the trial might not be sufficient by itself or might be too expensive. The Look AHEAD (Action for Health in Diabetes) trial was designed to see if weight loss in obese or overweight people with type 2 diabetes would result in a reduction in cardiovascular disease. The investigators were confronted with a much lower than expected rate of the primary outcome during the course of the trial, and after 2 years, the data monitoring board recommended expanding the primary outcome. It was changed from a composite of cardiovascular death, myocardial infarction, and stroke to one including hospitalization for angina. In addition, the duration of follow-up was lengthened [73]. As discussed in Chap. 10, recruitment of participants might be too slow to reach the necessary sample size. The Prevention of Events with Angiotensin Converting Enzyme Inhibition (PEACE) trial was seeking 14,100 patients with coronary artery disease, but after a year, fewer than 1,600 had been enrolled. Therefore, the original primary outcome of death due to cardiovascular causes or nonfatal myocardial infarction was changed to include coronary revascularization, reducing the sample size to 8,100 [74]. The Carvedilol Post-Infarct Survival Control in Left Ventricular Dysfunction (CAPRICORN) trial [75] had both poor participant recruitment and lower than expected event rate. To the original primary outcome of all-cause mortality was added a second primary outcome of all-cause mortality or hospitalization for cardiovascular reasons. In order to keep the overall type 1 error at 0.05, the α was divided between the two primary outcomes. Unfortunately, at the end of the trial, there was little difference between groups in the new primary outcome, but a reduction in the original outcome. Had it not been changed, requiring a more extreme result, it would have reached statistical significance [76].

In these examples, the rationale for the change was clearly stated. On occasion, however, the reported primary response variable was changed without clear rationale (or even disclosed in the publication) and after the data had been examined [77, 78]. A survey by Chan et al. [79] found that over 60% of trials conducted in Denmark in 1994–1995 had primary outcome changes between the original protocol and the publication.

Changing the primary outcome during the trial cannot be undertaken lightly and is generally discouraged. It should only be done if other approaches to completing the trial and achieving adequate power are not feasible or affordable. Importantly, it must be done without knowledge of outcome trends. One possible way is for the protocol to specify that if recruitment is below a certain level or overall event rate is under a certain percent, the primary outcome will be changed. Anyone aware of the outcome trends by study group should not be involved in the decision. This includes the data monitoring committee. Sometimes, an independent committee that is kept ignorant of outcome trends is convened to make recommendations regarding the proposed change.

General Comments

Although this text attempts to provide straightforward concepts concerning the selection of study response variables, things are rarely as simple as one would like them to be. Investigators often encounter problems related to design, data monitoring and ethical issues and interpretation of study results.

In long-term studies of participants at high-risk, when total mortality is not the primary response variable, many may nevertheless die. They are, therefore, removed from the population at risk of developing the response variable of interest. Even in relatively short studies, if the participants are seriously ill, death may occur. In designing studies, therefore, if the primary response variable is a continuous measurement, a nonfatal event, or cause-specific mortality, the investigator needs to consider the impact of total mortality for two reasons. First, it will reduce the effective sample size. One might allow for this reduction by estimating the overall mortality and increasing sample size accordingly.

Second, if mortality is related to the intervention, either favorably or unfavorably, excluding from study analysis those who die may bias results for the primary response variable.

One solution, whenever the risk of mortality is high, is to choose total mortality as the primary response variable. Alternatively, the investigator can combine total mortality with a pertinent nonfatal event as a combined primary response variable. Neither of these solutions may be appropriate and, in that case, the investigator should monitor total mortality as well as the primary response variable. Evaluation of the primary response variable will then need to consider those who died during the study, or else the censoring may bias the comparison.

Investigators need to monitor total mortality-as well as any other adverse occurrence-during a study, regardless of whether or not it is the primary response variable (see Chap. 16). The ethics of continuing a study which, despite a favorable trend for the primary response variable, shows equivocal or even negative results for secondary response variables, or the presence of major adverse effects, are questionable. Deciding what to do is difficult if an intervention is giving promising results with regard to death from a specific cause (which may be the primary response variable), yet total mortality is unchanged or increased. An independent data monitoring committee has proved extremely valuable in such circumstances (Chap. 16).

Finally, conclusions from data are not always clear-cut. Issues such as alterations in quality of life or annoying long-term adverse effects may cloud results that are clear with regard to primary response variables such as increased survival. In such circumstances, the investigator must offer her best assessment of the results but should report sufficient detail about the study to permit others to reach their own conclusions (Chap. 21).

References

1.

Moyé LA. Random research. Circulation 2001;103:3150–3153.CrossRef

2.

Cutler SJ, Greenhouse SW, Cornfield J, Schneiderman MA. The role of hypothesis testing in clinical trials: biometrics seminar. J Chronic Dis 1966;19:857–882.CrossRef

3.

Al-Marzouki S, Roberts I, Marshall T, Evans S. The effect of scientific misconduct on the results of clinical trials: a Delphi survey. Contemp Clin Trials 2005;26:331–337.CrossRef

4.

Angell M. Caring for women's health - what is the problem? (editorial). N Engl J Med 1993;329:271–272.CrossRef

5.

NIH Revitalization Act, Subtitle B, Part 1, Sec. 131–133, June 10, 1993.

6.

Simes RJ, O’Connell RL, Aylward PE, et al. Unexplained international differences in clinical outcomes after acute myocardial infarction and fibrinolytic therapy: lessons from the Hirulog and Early Reperfusion or Occlusion (HERO)-2 trial. Am Heart J 2010;159:988–997.CrossRef

7.

Cannon CP, Harrington RA, James S, et al. Comparison of ticagrelor with clopidogrel in patients with a planned invasive strategy for acute coronary syndromes (PLATO): a randomised double-blind study. Lancet 2010;375:283–293.CrossRef

8.

Mahaffey KW, Wojdyla DM, Carroll K, et al. Ticagrelor compared with clopidogrel by geographic region in the Platelet Inhibition and Patient Outcomes (PLATO) trial. Circulation 2011;124:544–554.CrossRef

9.

Freedman LS, Simon R, Foulkes MA, et al. Inclusion of women and minorities in clinical trials and the NIH Revitalization Act of 1993 - the perspective of NIH clinical trialists. Control Clin Trials 1995;16:277–285.CrossRef

10.

Piantadosi S, Wittes J. Letter to the editor. Control Clin Trials 1993;14:562–567.CrossRef

11.

ISIS-2 (Second International Study of Infarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17,187 cases of suspected acute myocardial infarction: ISIS-2. Lancet 1988; ii:349–360.

12.

Shimkin MB. The problem of experimentation on human beings. I. The research worker's point of view. Science 1953;117:205–207.

13.

Chalmers TC. Invited Remarks: National Conference on Clinical Trials Methodology. Clin Pharmacol Ther 1979;25:649–650.CrossRef

14.

Stamler J. Invited Remarks: National Conference on Clinical Trials Methodology. Clin Pharmacol Ther 1979;25:651–654.CrossRef

15.

The Cardiac Arrhythmia Suppression Trial II Investigators. Effect of the antiarrhythmic agent moricizine on survival after myocardial infarction. N Engl J Med 1992;327:227–233.CrossRef

16.

Miller AB, Buring J, Williams OD. Stopping the carotene and retinol efficacy trial: the viewpoint of the safety and endpoint monitoring committee. In DeMets DL, Furberg CD, Friedman LM.(eds). Data Monitoring in Clinical Trials: A Case Studies Approach. New York: Springer 2006, pp.220-227.

17.

Swedberg K, Held P, Kjekshus J, et al. Effects of the early administration of enalapril on mortality in patients with acute myocardial infarction – results of the Cooperative New Scandinavian Enalapril Survival Study II (Consensus II). N Engl J Med 1992;327:678–684.CrossRef

18.

The GUSTO Investigators. An international randomized trial comparing four thrombolytic strategies for acute myocardial infarction. N Engl J Med 1993;329:673–682.CrossRef

19.

Lee KL, Woodlief LH, Topol EJ, et al. Predictors of 30-day mortality in the era of reperfusion for acute myocardial infarction. Results from an international trial of 41,021 patients. GUSTO-1 Investigators. Circulation 1995;91:1659–1668.CrossRef

20.

APEX AMI Investigators, Armstrong PW, Granger CB, Adams PX, et al. Pexelizumab for acute ST-elevation myocardial infarction in patients undergoing primary percutaneous coronary intervention: a randomized controlled trial. JAMA 2007;297:43–51.

21.

The SOLVD Investigators. Effect of enalapril on survival in patients with reduced left ventricular ejection fractions and congestive heart failure. N Engl J Med 1991;325: 293–302CrossRef

22.

Ross AM, Coyne KS, Moreyra E, et al. Extended mortality benefit of early postinfarction reperfusion. GUSTO-1 Angiographic Investigators. Global Utilization of Streptokinase and Tissue Plasminogen Activator for Occluded Coronary Arteries Trial. Circulation 1998;97:1549–1556.

23.

Simes RJ, Topol EJ, Holmes DR Jr., et al. Link between the angiographic substudy and mortality outcomes in a large randomized trial of myocardial reperfusion. Importance of early and complete infarct artery reperfusion. GUSTO-1 Investigators. Circulation 1995;91:1923–1928.CrossRef

24.

Stone GW, Witzenbichler B, Guagliumi G, et al. Bivalirudin during primary PCI in acute myocardial infarction. N Engl J Med 2008;358:2218–2230.CrossRef

25.

Stone GW, Clayton T, Deliargyris EN, Prats J, Mehran R, Pocock SJ. Reduction in cardiac mortality with bivalirudin in patients with and without major bleeding: the HORIZONS-AMI trial (Harmonizing Outcomes with Revascularization and Stents in Acute Myocardial Infarction). J Am Coll Cardiol 2014;63:15–20.CrossRef

26.

Mega JL, Close SL, Wiviott SD, et al. Cytochrome P-450 polymorphisms and response to clopidogrel. N Engl J Med 2009;360:354–362.CrossRef

27.

Tricoci P, Allen JM, Kramer JM, Califf RM, Smith SC Jr. Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA 2009;301:831–841.CrossRef

28.

Yusuf S, Collins R, Peto R. Why do we need some large, simple randomized trials? Stat Med 1984;3:409–422.CrossRef

29.

Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 2003;290:1624–1632.CrossRef

30.

Eapen ZN, Lauer MS, Temple RJ. The imperative of overcoming barriers to the conduct of large, simple trials. JAMA 2014;311:1397–1398.CrossRef

31.

Palmerini T, Biondi-Zoccai G, Della Riva D, et al. Clinical outcomes with drug-eluting and bare-metal stents in patients with ST-segment elevation myocardial infarction: evidence from a comprehensive network meta-analysis. J Am Coll Cardiol 2013;62:496–504.CrossRef

32.

McFadden E, Stabile E, Regar E, et al. Late thrombosis in drug-eluting coronary stents after discontinuation of antiplatelet therapy. Lancet 2004;364:1519–1521.CrossRef

33.

Ong AT, McFadden EP, Regar E, et al. Late angiographic stent thrombosis (LAST) events with drug eluting stents. J Am Coll Cardiol 2005;45:2088–2092.CrossRef

34.

Mauri L, Hsieh W-H, Massaro JM, et al. Stent thrombosis in randomized clinical trials of drug-eluting stents. N Engl J Med 2007;356:1020–1029.CrossRef

35.

Metal-on-Metal Hip Implants: FDA Safety Communication. http://​www.​fda.​gov/​medicaldevices/​safety/​alertsandnotices​/​ucm335775.​htm

36.

Heneghan C, Langton D, Thompson M. Ongoing problems with metal-on-metal hip implants.BMJ 2012;344:e1349. http://​www.​bmj.​com/​content/​344/​bmj.​e1349

37.

Urokinase Pulmonary Embolism Trial Study Group. Urokinase Pulmonary Embolism Trial: phase I results. JAMA 1970;214:2163–2172.CrossRef

38.

Cordoba G, Schwartz L, Woloshin S, Bae H, Gøtzsche PC. Definition, reporting, and interpretation of composite outcomes in clinical trials: systematic review. BMJ 2010;341:c3920.CrossRef

39.

Tomlinson G, Detsky AS. Composite end points in randomized trials: there is no free lunch. JAMA 2010;303:267–268.CrossRef

40.

Steering Committee of the Physicians' Health Study Research Group. Final report on the aspirin component of the ongoing Physicians' Health Study. N Engl J Med 1989;321:129–135.CrossRef

41.

Cairns J, Cohen L, Colton T, et al. Data Monitoring Board of the Physicians' Health Study. Issues in the early termination of the aspirin component of the Physicians’ Health Study. Ann Epidemiol 1991;1:395–405.CrossRef

42.

Schwartz GG, Olsson AG, Ezekowitz MD, et al for the Myocardial Ischemia Reduction with Aggressive Cholesterol Lowering (MIRACL) Study Investigators. Effects of atorvastatin on early ischemic events in acute coronary syndromes: the MIRACL study: a randomized controlled trial. JAMA 2001;285:1711–1718.

43.

Ferreira-Gonzalez I, Busse JW, Heels-Ansdell D, et al. Problems with use of composite end points in cardiovascular trials: systematic review of randomised controlled trials. Br Med J 2007; 334:756–757.CrossRef

44.

Lim E, Brown A, Helmy A, et al. Composite outcomes in cardiovascular research: a survey of randomized trials. Ann Intern Med 2008;149:612–617.CrossRef

45.

Neaton JD, Wentworth DN, Rhame F, et al. Methods of studying intervention: considerations in choice of a clinical endpoint for AIDS clinical trials. Stat Med 1994;13:2107–2125.CrossRef

46.

Hallstrom AP, Litivin PE, Weaver WD. A method of assigning scores to the components of a composite outcome: an example from the MITI trial. Control Clinl Trials 1992;13:148–155.CrossRef

47.

Mentzer RM Jr, Bartels C, Bolli R, et al. Sodium-hydrogen exchange inhibition by cariporide to reduce the risk of ischemic cardiac events in patients undergoing coronary artery bypass grafting: results of the EXPEDITION study. Ann Thorac Surg 2008;85:1261–1270.CrossRef

48.

Wittes J, Lakatos E, Probstfield J. Surrogate endpoints in clinical trials: cardiovascular diseases. Stat Med 1989; 8:415–425.CrossRef

49.

Fleming T, DeMets DL. Surrogate endpoints in clinical trials: are we being misled? Ann Intern Med 1996;125:605–613.CrossRef

50.

Fleming TR. Surrogate markers in AIDS and cancer trials. Stat Med 1994;13:1423–1435.CrossRef

51.

Bigger JT Jr, Fleiss JL, Kleiger R, et al, and the Multicenter Post-Infarction Research Group. The relationships among ventricular arrhythmias, left ventricular dysfunction, and mortality in the 2 years after myocardial infarction. Circulation 1984;69:250–258.

52.

Vlay SC. How the university cardiologist treats ventricular premature beats: a nationwide survey of 65 university medical centers. Am Heart J 1985;110:904–912CrossRef

53.

Morganroth J, Bigger JT Jr, Anderson JL. Treatment of ventricular arrhythmias by United States cardiologists: a survey before the Cardiac Arrhythmia Suppression Trial (CAST) results were available. Am J Cardiol 1990;65:40–48.CrossRef

54.

The Cardiac Arrhythmia Suppression Trial (CAST) Investigators. Preliminary Report: Effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N Engl J Med 1989;321:406–412.CrossRef

55.

Packer M. Vasodilator and inotropic drugs for the treatment of chronic heart failure: distinguishing hype from hope. J Am Coll Cardiol. 1988;12:1299–1317.CrossRef

56.

Packer M, Carver JR, Rodehoffer RT, et al. Effect of oral milrinone on mortality in severe chronic heart failure. N Engl J Med 1991;325:1468–1475.CrossRef

57.

The Xamoterol in Severe Heart Failure Study Group. Xamoterol in severe heart failure. Lancet 1990;336:1–6; correction Lancet 1990;336:698.

58.

Barter PJ, Caulfield M, Eriksson M, et al for the ILLUMINATE Investigators. Effects of torcetrapib in patients at high risk for coronary events. N Engl J Med 2007;357:2109–2122.CrossRef

59.

Boden WE, Sidhu MS, Toth PP. The therapeutic role of niacin in dyslipidemia management. J Cardiovasc Pharmacol Ther 2014;19:141–158.CrossRef

60.

AIM-HIGH Investigators, Boden WE, Probstfield JL, Anderson T, et al. Niacin in patients with low HDL cholesterol levels receiving intensive statin therapy. N Engl J Med 2011;365:2255–2267.

61.

HPS2-THRIVE Collaborative Group. Landray MJ, Haynes R, Hopewell JC, et al. Effects of extended-release niacin with laropiprant in high-risk patients. N Engl J Med 2014;371:203–212.

62.

Cohen J. Searching for markers on the AIDS trail. Science 1992;258:388–390.CrossRef

63.

Lin DY, Fischl MA, Schoenfeld DA. Evaluating the role of CD-4 lymphocyte counts as surrogate endpoints in human immunodeficiency virus clinical trials. Stat Med 1993;12:835–842.CrossRef

64.

Choi S, Lagakos SW, Schooley RT, Volberding PA. CD4+ lymphocytes are an incomplete surrogate marker for clinical progression in persons with asymptomatic HIV infection taking zidovidine. Ann Intern Med 1993;118:674–680.CrossRef

65.

Fischl MA., Olson RM, Follansbee SE, et al. Zalcitabine compared with zidovudine in patients with advanced HIV-1 infection who received previous zidovudine therapy. Ann Intern Med 1993;118:762–769.CrossRef

66.

Aboulker JP, Swart AM. Preliminary analysis of the Concorde trial. Lancet 1993;341:889–890.CrossRef

67.

Advanced Colorectal Cancer Meta-Analysis Project. Modulation of fluoruracil by leucovorin in patients with advanced colorectal cancer. Evidence in terms of response rate. J Clin Oncol 1992;10:896–903.

68.

Riggs BL, Hodgson SF, O'Fallon WM, et al. Effect of fluoride treatment on the fracture rate in postmenopausal women with osteoporosis. N Engl J Med 1990;322:802–809.CrossRef

69.

Prentice RL. Surrogate endpoints in clinical trials: definitions and operational criteria. Stat Med 1989;8:431–440.CrossRef

70.

Ray WA, Griffin MR, Avorn J. Sounding Board: Evaluating drugs after their approval for clinical use. N Engl J Med 1993;329:2029–2032.CrossRef

71.

FDA Guidance for Industry. Diabetes Mellitus—Evaluating Cardiovascular Risk in New Antidiabetic Therapies to Treat Type 2 Diabetes. http://​www.​fda.​gov/​downloads/​drugs/​guidancecomplian​ceregulatoryinfo​rmation/​guidances/​ucm071627.​pdf

72.

Avorn J. Viewpoint: Approval of a tuberculosis drug based on a paradoxical surrogate measure. JAMA 2013;309:1349–1350.CrossRef

73.

Brancati FL, Evans M, Furberg CD, et al. Midcourse correction to a clinical trial when the event rate is underestimated: the Look AHEAD (Action for Health in Diabetes) Study. Clin Trials 2012;9:113–124.CrossRef

74.

The PEACE Trial Investigators. Angiotensin-converting-enzyme inhibition in stable coronary artery disease. N Engl J Med 2004;351:2058–2068.CrossRef

75.

Dargie HJ. Effect of carvedilol on outcome after myocardial infarction in patients with left-ventricular dysfunction: the CAPRICORN randomised trial. Lancet 2001;357:1385–1390.CrossRef

76.

Julian D. The data monitoring experience in the Carvedilol Post-Infarct Survival Control in Left Ventricular Dysfunction Study: Hazards of changing primary outcomes. In DeMets DL, Furberg CD, Friedman LM (eds.). Data Monitoring in Clinical Trials. New York: Springer, 2006, pp. 337–345.CrossRef

77.

The Anturane Reinfarction Trial Research Group. Sulfinpyrazone in the prevention of sudden death after myocardial infarction. N Engl J Med 1980;302:250–256.CrossRef

78.

Anturane Reinfarction Trial Policy Committee. The Anturane Reinfarction Trial: reevaluation of outcome. N Engl J Med 1982;306:1005–1008.CrossRef

79.

Chan A-W, Hróbjartsson A, Haahr MT, et al. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004;291:2457–2465.CrossRef

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_4

4. Study Population

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

Defining the study population in the protocol is an integral part of posing the primary question. Additionally, in claiming an intervention is or is not effective it is essential to describe the type of participants on which the intervention was tested. Thus, the description requires two elements: specification of criteria for eligibility and description of who was actually enrolled. This chapter focuses on how to define the study population. In addition, it considers two questions. First, what impact does selection of eligibility criteria have on participant recruitment, or, more generally, study feasibility? Second, to what extent will the results of the trial be generalizable to a broader population? This issue is also discussed in Chap. 10.

In reporting the study, the investigator needs to say what population was studied and how they were selected. The reasons for this are several. First, if an intervention is shown to be successful or unsuccessful, the medical and scientific communities must know to what population the findings apply [1].

Second, knowledge of the study population helps other investigators assess the study’s merit and appropriateness. Unfortunately, despite guidelines for reporting trial results [2], many publications contain inadequate characterization of the study participants [3]. Therefore, readers may be unable to assess fully the merit or applicability of the studies.

Third, in order for other investigators to be able to replicate the study, they need data descriptive of those enrolled. Before most research findings are widely accepted, they need to be confirmed by independent scientists. Although it is small trials that are more likely to be repeated, these are the ones, in general, that most need confirmation.

Fundamental Point

The study population should be defined in advance, stating unambiguous inclusion (eligibility) criteria. The impact that these criteria will have on study design, ability to generalize, and participant recruitment must be taken into account.

Definition of Study Population

The study population is the subset of the population with the condition or characteristics of interest defined by the eligibility criteria. The group of participants actually studied in the trial, which constitutes the trial participants, is selected from the study population. (See Fig. 4.1). There are two main types of exclusions. First, patients who have absolute or relative contraindications to the study intervention. Second, trial design issues that may interfere with the optimal conduct of the trial and factors that could interfere with participant adherence (see below).

Fig. 4.1

Relationship of study sample to study population and general population (those with and those without the condition under study)

The extent to which the obtained trial results can be generalized depends on its external validity [1]. External validity refers to the questions whether the trial findings are valid for participants other than those meeting the protocol definition of the study populations, but from a comparable clinical setting. Rothwell identified six issues that could potentially affect external validity—trial setting, selection of participants, characteristics of randomized participants, differences between the trial protocol and clinical practice, outcome measures and follow-up, and adverse effects of treatment. External validity is a measure of generalizability. The term internal validity refers to the question whether the trial results are valid for all participants meeting the eligibility criteria of the trial protocol, i.e., the definition of the study population.

Considerations in Defining the Study Population

Inclusion criteria and reasons for their selection should be stated in advance. Those criteria central to the study should be the most carefully defined. For example, a study of survivors of a myocardial infarction may exclude people with severe hypertension, requiring an explicit definition of myocardial infarction. However, with regard to hypertension, it may be sufficient to state that people with a systolic or diastolic blood pressure above a specified level will be excluded. Note that even here, the definition of severe hypertension, though arbitrary, is fairly specific. In a study of antihypertensive agents, however, the above definition of severe hypertension is inadequate. To include only people with diastolic blood pressure over 90 mmHg, the protocol should specify how often it is to be determined, over how many visits, when, with what instrument, by whom, and in what circumstances. It may also be important to know which, if any, antihypertensive agents participants were on before entering the trial. For any study of antihypertensive agents, the criterion of hypertension is central; a detailed definition of myocardial infarction, on the other hand, may be less important.

If age is a restriction, the investigator should ideally specify not only that a participant must be over age 41, for example, but when he must be over 41. If a subject is 40 at the time of a pre-baseline screening examination, but 41 at baseline, is he eligible? This should be clearly indicated. If valvular heart disease is an exclusion criterion for a trial of anticoagulation in atrial fibrillation, is this any significant valve abnormality, or is restricted to rheumatic heart disease? Does it apply to prior valve repair? Often there are no “correct” ways of defining inclusion and exclusion criteria and arbitrary decisions must be made. Regardless, they need to be as clear as possible, and, when appropriate, with complete specifications of the technique and laboratory methods.

In general, eligibility criteria relate to participant safety and anticipated effect of the intervention. It should be noted, however, that cultural or political issues, in addition to scientific, public health, or study design considerations, may affect selection of the study populations. Some have argued that too many clinical trials exclude, for example, women, the elderly, or minority groups, or that even if not excluded, insufficient attention is paid to enrolling them in adequate numbers [4–7]. Some patient groups may be underrepresented due to practical issues (the frail might not be able to attend frequent follow-up visits) and the need for informed consent might exclude individuals with cognitive dysfunction. Policies from the U.S. National Institutes of Health now require clinical trials to include certain groups in enough numbers to allow for “valid analysis” [8]. The effect of these kinds of policies on eligibility criteria, sample size, and analysis must be considered when designing a trial.

The following five categories outline the framework upon which to develop individual criteria:

Potential for Benefit

Participants who have the potential to benefit from the intervention are obviously candidates for enrollment into the study. The investigator selects participants on the basis of his scientific knowledge and the expectation that the intervention will work in a specific way on a certain kind of participants. For example, participants with a urinary infection are appropriate to enroll in a study of a new antibiotic agent known to be effective in vitro against the identified microorganism and thought to penetrate to the site of the infection in sufficient concentration. It should be evident from this example that selection of the participant depends on knowledge of the presumed mechanism of action of the intervention. Knowing at least something about the mechanism of action may enable the investigator to identify a well-defined group of participants likely to respond to the intervention. Thus, people with similar characteristics with respect to the relevant variable, that is, a homogeneous population, can be studied. In the above example, participants are homogeneous with regard to the type and strain of bacteria, and to site of infection. If age or renal or liver function is also critical, these too might be considered, creating an even more highly selected group.

Even if the mechanism of action of the intervention is known, however, it may not be feasible to identify a homogeneous population because the technology to do so may not be available. For instance, the causes of headache are numerous and, with few exceptions, not easily or objectively determined. If a potential therapy were developed for one kind of headache, it would be difficult to identify precisely the people who might benefit.

If the mechanism of action of the intervention is unclear, or if there is uncertainty at which stage of a disease a treatment might be most beneficial, a specific group of participants likely to respond cannot easily be selected. The Diabetic Retinopathy Study [9] evaluated the effects of photocoagulation on progression of retinopathy. In this trial, each person had one eye treated while the other eye served as the control. Participants were subgrouped on the basis of existence, location and severity of vessel proliferation. Before the trial was scheduled to end, it became apparent that treatment was dramatically effective in the four most severe of the ten subgroups. To have initially selected for study only those four subgroups who benefited was not possible given existing knowledge. This is an example, of which there are many, of the challenge in predicting differential intervention effects based on defined subgroups. For most interventions, there is uncertainty about the benefits and harms that makes enrolling a broader group of participants with the condition prudent.

Some interventions may have more than one potentially beneficial mechanism of action. For example, if exercise reduces mortality or morbidity, is it because of its effect on cardiac performance, its weight-reducing effect, its effect on the person’s sense of well-being, some combination of these effects, or some as yet unknown effect? The investigator could select study participants who have poor cardiac performance, or who are obese or who, in general, do not feel well. If he chose incorrectly, his study would not yield a positive result. If he chose participants with all three characteristics and then showed benefit from exercise, he would never know which of the three aspects was important.

One could, of course, choose a study population, the members of which differ in one or more identifiable aspects of the condition being evaluated; i.e., a heterogeneous group. These differences could include stage or severity of a disease, etiology, or demographic factors. In the above exercise example, studying a heterogeneous population may be preferable. By comparing outcome with presence or absence of initial obesity or sense of well-being, the investigator may discover the relevant characteristics and gain insight into the mechanism of action. Also, when the study group is too restricted, there is no opportunity to discover whether an intervention is effective in a subgroup not initially considered. The broadness of the Diabetic Retinopathy Study was responsible for showing, after longer follow-up, that the remaining six subgroups also benefited from therapy [10]. If knowledge had been more advanced, only the four subgroups with the most dramatic improvement might have been studied. Obviously, after publication of the results of these four subgroups, another trial might have been initiated. However, valuable time would have been wasted. Extrapolation of conclusions to milder retinopathy might even have made a second study difficult. Of course, the effect of the intervention on a heterogeneous group may be diluted and the ability to detect a benefit may be reduced. That is the price to be paid for incomplete knowledge about mechanism of action.

Large, simple trials are, by nature, more heterogeneous in their study populations, than other sorts of trials. There is a greater chance that the participants will more closely resemble the mix of patients in many clinical practices. It is assumed, in the design, that the intervention affects a diverse group, and that despite such diversity, the effect of the intervention is more similar among the various kinds of participants than not. In such trials, not only are the interventions relatively easy to implement, and the baseline and outcome variables limited, so too are the eligibility criteria. Definitions of eligibility criteria may not require repeated visits or special procedures. They may rely on previously measured variables that are part of a diagnostic evaluation, or on variables that are measured using any of several techniques, or on investigator judgment. For example, a detailed definition of myocardial infarction or hypertension may be replaced with, “Does the investigator believe a myocardial infarction has occurred?” or “Is hypertension present?” The advantage of this kind of criteria is their simplicity and greater generalizability. The disadvantage is the possible difficulty that a clinician reading the results of the trial will have in deciding if the results are applicable to specific patients under his care. It should be noted, however, that even with the large simple trial model, the criteria are selected and specified in advance.

Homogeneity and heterogeneity are matters of degree and knowledge. As scientific knowledge advances, ability to classify is improved. Today’s homogeneous group may be considered heterogeneous tomorrow. Patients with mutations in BRCA1 and BRCA2 genes discovered in the 1990s have different susceptibility and course of breast and ovarian cancer. Patients with breast cancer tissue with HER2 and/or estrogen receptors respond differently to chemotherapy treatments [11]. Thus, breast cancer is now defined and treated based on genomically defined subsets.

High Likelihood of Showing Benefit

In selecting participants to be studied, not only does the investigator require people in whom the intervention might work, but he also wants to choose people in whom there is a high likelihood of detecting the hypothesized effects of the intervention. Careful choice will enable investigators to detect results in a reasonable period of time, given a reasonable number of participants and a finite amount of funding.

For example, in a trial of an antianginal agent, an investigator would not wish to enroll a person who, in the past 2 years, has had only one brief angina pectoris episode (assuming such a person could be identified). The likelihood of finding an effect of the drug on this person is limited, since his likelihood of having many angina episodes during the expected duration of the trial is small. Persons with frequent episodes would be more appropriate. One option is to enrich the population with high risk patients, as was done in the ROCKET-AF trial of rivaroxaban versus warfarin for stroke prevention in atrial fibrillation [12]. Patients were required to have three risk factors for stroke that resulted in a population with higher risk and higher stroke rate than the general population with indication for oral anticoagulation. This allowed for a smaller sample size, since the calculation of sample size (Chap. 8) takes into account the expected incidence of the primary outcome. The results were consistent across the risk levels of patients enrolled, and the FDA provided approval for the drug across the spectrum of risk, including even lower risk patients who were not included in the trial. Although one might have somewhat less confidence that the treatment is safe and effective in lower risk patients, trials of related drugs have subsequently shown consistency across risk and thus it seems reasonable to extrapolate to the lower risk population.

Another approach is to begin with a higher risk population and if the results from a first trial are positive, the investigator can then enroll groups with lower risk levels. The initial Veterans Administration study of the treatment of hypertension [13] involved people with diastolic blood pressure from 115 through 129 mmHg. After therapy was shown to be beneficial in that group, a second trial was undertaken using people with diastolic blood pressures from 90 to 114 mmHg [14]. The latter study suggested that treatment should be instituted for people with diastolic blood pressure over 104 mmHg. Results were less clear for people with lower blood pressure. Subsequently, the Hypertension Detection and Follow-up Program [15] demonstrated benefit from treatment for people with diastolic blood pressure of 90 mmHg or above. The first trial of angiotensin converting enzyme inhibitors in heart failure, the Cooperative North Scandinavian Enalapril Survival Study (CONSENSUS) [16], enrolled 253 patients with advanced heart failure. There was a 40% relative risk reduction in mortality at 6 months with enalapril versus placebo. Subsequent larger trials defined the treatment effects in patients with less severe heart failure with lower event rates. Studies Of Left Ventricular Dysfunction (SOLVD) consisted of two individual trials. One involved symptomatic participants [17] and the other asymptomatic participants with reduced ejection fraction [18].

Medical conditions with low event rates represent a challenge. One example is the relapse-remitting disease, multiple sclerosis. Its attack or relapse rate is reported to average 0.54 episodes annually with a slightly higher rate in the first year [19]. Properly designed clinical trials in this population would have to be very large and/or have a long duration. Similarly, many people accept the hypothesis that LDL-cholesterol is a continuous variable in its impact on the risk of developing cardiovascular disease. Theoretically, an investigator could take almost any population with moderate or even relatively low LDL-cholesterol, attempt to lower it, and see if occurrence of cardiovascular disease is reduced. However, this would require studying an impossibly large number of people. From a sample size point of view it is, therefore, desirable to begin by studying people with greater levels of risk factors and a consequent high expected event rate.

Generally, if the primary response is continuous (e.g., blood pressure, blood sugar, body weight), change is easier to detect when the initial level is extreme. In a study to determine whether a new drug is antihypertensive, there might be a more pronounced drop of blood pressure in a participant with diastolic pressure of 100 mmHg than in one with diastolic pressure of 90 mmHg or less. There are exceptions to this rule, especially if a condition has multiple causes. The relative frequency of each cause might be different across the spectrum of values. For example, familial hypercholesterolemia is heavily represented among people with extremely high LDL-cholesterol. These lipid disorders may require alternative therapies or may even be resistant to usual methods of reducing LDL-cholesterol. In addition, use of participants with lower levels of a variable such as cholesterol might be less costly due to lower screening costs [20]. Therefore, while in general, use of higher risk participants is preferable, other considerations can modify this.

Sometimes, it may be feasible to enroll people with low levels of a risk factor if other characteristics elevate the absolute risk. For example, the Justification for the Use of Statins in Prevention: an Intervention Trial Evaluating Rosuvastatin (JUPITER) [21] used C-reactive protein to select those with LDL-cholesterol levels under 130 mg/dL (3.4 mmol/L) but who were likely to be at higher risk of developing coronary heart disease. The cholesterol-lowering agent rosuvastatin was shown to significantly lower the incidence of coronary heart disease.

The concept of enrichment has received considerable attention from the FDA (Guidance for Industry: Enrichment strategies for clinical trials to support approval of human drugs and biological products) [22]. Enrichment is used in order to enroll those participants with a high likelihood of demonstrating an effect from the intervention. Participants with characteristics, including genetic features, that put them at high risk, are entered into the trial. As discussed in Chap. 5, withdrawal studies are also a way of preferentially assessing participants who are more likely to show benefit from the intervention.

The increased FDA focus on fast-track approval has already had implications for the design of randomized clinical trials and their study populations [23]. Regulatory approval without proper phase 3 trials or only based on surrogate efficacy or pharmacodynamic markers limits sample sizes and places focus on highly selected populations. These trials provide limited information about the safety of the intervention. For specific information see Chap. 22 on Regulatory Issues.

Avoiding Adverse Effects

Most interventions are likely to have adverse effects. The investigator needs to weigh these against possible benefit when he evaluates the feasibility of doing the study. However, any person for whom the intervention is known to be harmful should not, except in unusual circumstances, be admitted to the trial. Pregnant women are often excluded from drug trials (unless, of course, the primary question concerns pregnancy) particularly if there is preclinical evidence of teratogenicity. Even without preliminary evidence the amount of additional data obtained may not justify the risk. Similarly, investigators would probably exclude from a study of almost any of the anti-inflammatory drugs people with a recent history of gastric bleeding. Gastric bleeding is a fairly straightforward and absolute contraindication for enrollment. Yet, an exclusion criterion such as “history of major gastric bleed,” leaves much to the judgment of the investigator. The word “major” implies that gastric hemorrhaging is not an absolute contraindication, but a relative one that depends upon clinical judgment. The phrase also recognizes the question of anticipated risk vs. benefit, because it does not clearly prohibit people with a mild bleeding episode in the distant past from being placed on an anti-inflammatory drug. It may very well be that such people take aspirin or similar agents—possibly for a good reason—and studying such people may prove more beneficial than hazardous.

Note that these exclusions apply only before enrollment into the trial. During a trial participants may develop symptoms or conditions which would have excluded them had any of these conditions been present prior to randomization. In these circumstances, the participant may be removed from the intervention regimen if it is contraindicated, but should be kept in the trial and complete follow-up should be obtained for purposes of analysis. As described in Chap. 18, being off the intervention does not mean that a participant is out of the trial.

Competing Risk

The issue of competing risk is generally of greater interest in long-term trials. Participants at high risk of developing conditions which preclude the ascertainment of the outcome of interest should be excluded from enrollment. The intervention may or may not be efficacious in such participants, but the necessity for excluding them from enrollment relates to design considerations. In many studies of people with heart disease, those who have cancer or severe kidney or liver disorders are excluded because these diseases might cause the participant to die or withdraw from the study before the primary response variable can be observed. However, even in short-term studies, the competing risk issue needs to be considered. For example, an investigator may be studying a new intervention for a specific congenital heart defect in infants. Such infants are also likely to have other life-threatening defects. The investigator would not want to enroll infants if one of these other conditions were likely to lead to the death of the infant before the effect of the intervention could be evaluated. This matter is similar to the one raised in Chap. 3, which presented the problem of the impact of high expected total mortality on a study in which the primary response variable is morbidity or cause-specific mortality. When there is competing risk, the ability to assess the true impact of the intervention is, at best, lessened. At worst, if the intervention somehow has either a beneficial or harmful effect on the coexisting condition, biased results for the primary question can be obtained.

Avoiding Poor Adherers

Investigators prefer, ordinarily, to enroll only participants who are likely to adhere to the study protocol. Participants are expected to take their assigned intervention (usually a drug) and return for scheduled follow-up appointments regardless of the intervention assignment. In unblinded studies, participants are asked to accept the random assignment, even after knowing its identity, and abide by the protocol. Moreover, participants should not receive the study intervention from sources outside the trial during the course of the study. Participants should also refrain from using other interventions that may compete with the study intervention. Nonadherence by participants reduces the opportunity to observe the true effect of intervention.

One approach of enrichment of patients who are more likely to adhere to study interventions is to use a run-in phase, either a passive run-in (in which all patients are assigned to placebo for a period of time), active run-in (in which all patients are assigned to the active treatment to assure that they tolerate and adhere to it), or a combination. The PARADIGM HF trial [24] used such an approach. In this trial, 10,521 patients were entered into run-in phase, of which 2,079 were discontinued prior to randomization during the two run-in phases that consisted of a 2 week treatment period with enalapril, followed by a 4 week treatment period with LCZ696 (valsartan-neprolysin inhibitor) in a dose escalation. This resulted in a population more likely to tolerate and adhere to the treatments, although at the potential cost of having to apply the findings to the large number of patients excluded due to early intolerance. For a further discussion of run-in, see Chap. 14.

An exception to this effort to exclude those less likely to take their medication or otherwise comply with the protocol is what some have termed “pragmatic” clinical trials [25, 26]. These trials are meant to mimic real-world practice, with inclusion of participants who reflect general practice and who may fail to adhere consistently to the intervention. To compensate for the lower expected difference between the intervention and control groups, these trials need to be quite big, and have other characteristics of large, simple trials.

Pharmacogenetics

The field of pharmacogenetics is growing rapidly and so is also its role in clinical trials. Pharmacogenetic markers have been used to identify subgroups of patients in whom an intervention is particularly beneficial or harmful. Many of these observations have been based on post-hoc analyses of markers identified in stored samples collected at baseline. There are also situations in which the markers were known and measured in advance and used to select the study population [27, 28] or for prespecified subgroups [29].

The regulatory agencies, in particular the FDA, are paying more attention to subgroups defined by pharmacogenetic markers in their review and labeling. These markers include specific alleles, deficient gene products, inherited familial conditions and patterns of drug metabolism, such as ultra-rapid, normal, intermediate and poor metabolizer phenotypes. Thus, a very large number of drug labels in the U.S. now contain information linked to these markers. The drug labeling according to the FDA may describe:

▪

Drug exposure and clinical response variability

▪

Risk for adverse effects

▪

Genotype-specific dosing

▪

Mechanisms of drug action

▪

Polymorphic drug target and disposition genes

An FDA website lists approximately 140 different drugs with labeling related to genetic markers [30]. The most prevalent therapeutic areas to date are oncology, psychiatry, infectious diseases and cardiology. The experience with pharmacogenomics and psychotropic medications has been thoroughly reviewed [31].

Many large trials today collect genetic materials at baseline and store them for possible future use. We recommend investigators and sponsors to consider collection of DNA samples from participants at baseline. In doing so it is important that the informed consent specifies that permission is given for comprehensive analyses and sharing of these data in large-scale databases [32, 33].

Generalization

Study samples of participants are usually non-randomly chosen from the study population, which in turn is defined by the eligibility criteria (Fig. 4.1). As long as selection of participants into a trial occurs, and as long as enrollment is voluntary, participants must not be regarded as truly representative of the study population. Therefore, investigators have the problem of generalizing from participants actually in the trial to the study population and then to the population with the condition in a comparable clinical setting (external validity). It is often forgotten that participants must agree to enroll in a study. What sort of person volunteers for a study? Why do some agree to participate while others do not? The requirement that study participants sign informed consent or return for periodic examinations is sufficient to make certain people unwilling to participate. Sometimes the reasons are not obvious. What is known, however, is that volunteers can be different from non-volunteers [34, 35]. They are usually in better health, and they are more likely to comply with the study protocol. However, the reverse could also be true. A person might be more motivated if she has disease symptoms. In the absence of knowing what motivates the particular study participants, appropriate compensatory adjustments cannot be made in the analysis. Because specifying how volunteers differ from others is difficult, an investigator cannot confidently identify those segments of the study population or the general population that these study participants supposedly represent. (See Chap. 10 for a discussion of factors that people cite for enrolling or not enrolling in trials.)

Defined medical conditions and quantifiable or discrete variables such as age, sex, or elevated blood sugar can be clearly stated and measured. For these characteristics, specifying in what way the study participants and study population are different from the population with the condition is relatively easy. Judgments about the appropriateness of generalizing study results can, therefore, be made. Other factors of the study participants are less easily characterized. Obviously, an investigator studies only those participants available. If he lives in Florida, he will not be studying people living in Maine. Even within a geographical area, many investigators are based at hospitals or universities. Furthermore, many hospitals are referral centers. Only certain types of participants come to the attention of investigators at these institutions. It may be impossible to decide whether these factors are relevant when generalizing to other geographical regions or patient care settings. Multicenter trials typically enhance the ability to generalize. The growth of international trials, however, raises the important issue of relevance of results from geographical areas with very different clinical care systems.

Many trials now involve participants from community or practice-based settings. Results from these “practical” or “pragmatic” trials may more readily be translated to the broader population. Even here, however, those who choose to become investigators likely differ from other practitioners in the kinds of patients they see.

Many trials of aspirin and other anti-platelet agents in those who have had a heart attack have shown that these agents reduce recurrent myocardial infarction and death in both men and women [36]. The Physicians’ Health Study, conducted in the 1980s, concluded that aspirin reduced myocardial infarction in men over age 50 without previously documented heart disease [37]. Although it was reasonable to expect that a similar reduction would occur in women, it was unproven. Importantly, aspirin was shown in the Physicians’ Health Study and elsewhere [38] to increase hemorrhagic stroke. Given the lower risk of heart disease in premenopausal women, whether the trade-off between adverse effects and benefit was favorable was far from certain. The U.S. Food and Drug Administration approved aspirin for primary prevention in men, but not women. The Women’s Health Study was conducted in the 1990s and early 2000s [39]. Using a lower dose of aspirin that was used in the Physicians’ Health Study, it found evidence of benefit on heart disease only in women at least 65 years old. Based on that, generalization of the Physicians’ Health Study results to primary prevention in all women would not have been prudent. A subsequent meta-analysis, however, suggested that the benefits of aspirin for primary prevention were similar in women and men. A trial published in 2014 found no overall benefit of low dose aspirin in a Japanese population of men and women [40]. We must always be open to consider new information in our interpretation of study results [41].

One approach to addressing the question of representativeness is to maintain a log or registry which lists prospective participants identified, but not enrolled, and the reasons for excluding them. This log can provide an estimate of the proportion of all potentially eligible people who meet study entrance requirements and can also indicate how many otherwise eligible people refused enrollment. In an effort to further assess the issue of representativeness, response variables in those excluded have also been monitored. In a study on timolol [42], people excluded because of contraindication to the study drug or competing risks had a mortality rate twice that of those who enrolled. The Coronary Artery Surgery Study included a randomized trial that compared coronary artery bypass surgery against medical therapy and a registry of people eligible for the trial but who declined to participate [43]. The enrolled and not enrolled groups were alike in most identifiable respects. Survival in the participants randomly assigned to medical care was the same as those receiving medical care but not in the trial. The findings for those undergoing surgery were similar. Therefore, in this particular case, the trial participants appeared to be representative of the study population.

With more attention being paid to privacy issues, however, it may not be possible to assess outcomes in those not agreeing to enter a trial. Some people may consent to allow follow-up, even if they do not enroll, but many will not. Thus, comparison of trial results with results in those refusing to enter a trial, in an effort to show that the trial can be generalized, may prove difficult.

A group of Finnish investigators conducted a retrospective chart review [44]. The typical eligibility criteria for clinical trials of patients with gastric ulcer were applied to 400 patients hospitalized with the diagnosis of gastric ulcer. Only 29% of the patients met the eligibility criteria and almost all deaths and serious complications such as gastric bleeding, perforation and stenosis during the first 5–7 years occurred among those patients who would have been ineligible. Clearly, the testing of H₂-blockers or other compounds for the prevention of long-term complications of gastric ulcer in low-risk patients should not be generalized to the entire ulcer population, as the external validity may be low.

Since the investigator can describe only to a limited extent the kinds of participants in whom an intervention was evaluated, a leap of faith is always required when applying any study findings to the population with the condition. In taking this jump, one must always strike a balance between making unjustifiably broad generalizations and being too conservative in one’s claims. Some extrapolations are reasonable and justifiable from a clinical point of view, especially in light of subsequent information.

Recruitment

The impact of eligibility criteria on recruitment of participants should be considered when deciding on these criteria. Using excessive restrictions in an effort to obtain a pure (or homogeneous) sample can lead to extreme difficulty in obtaining sufficient participants and may raise questions regarding generalization of the trial results. Age and sex are two criteria that have obvious bearing on the ease of enrolling subjects. The Coronary Primary Prevention Trial undertaken by the Lipid Research Clinics was a collaborative trial evaluating a lipid-lowering drug in men between the ages of 35 and 59 with severe hypercholesterolemia. One of the Lipid Research Clinics [45] noted that approximately 35,000 people were screened and only 257 participants enrolled. Exclusion criteria, all of which were perfectly reasonable and scientifically sound, coupled with the number of people who refused to enter the study, brought the overall trial yield down to less than 1%. As discussed in Chap. 10, this example of greater than expected numbers being screened, as well as unanticipated problems in reaching potential participants, is common to most clinical trials. We believe that exclusion criteria should include only those with clear rationale such that the negative impact on enrollment and generalizability will likely be outweighed by benefits of limiting the population.

One reason that large international trials are including a larger proportion of patients from low and middle income countries is to increase enrollment potential. This trend for globalization of trials raises a number of important issues as discussed in Chap. 22. For the results of trials to be applicable across countries and health care systems, inclusive enrollment is important. But ethical issues arise when therapies are developed in countries in which those treatments will not be used, often due to cost. And enrolled patients may be systematically different in certain countries. The TOPCAT trial enrolled patients from Russia with heart failure who, in retrospect based on B-type natriuretic peptide levels, may not have had the same degree of heart failure, and who appeared to have less treatment effect from spironolactone [46]. Careful consideration of the advantages and disadvantages of including different health care environments is needed.

If entrance criteria are properly determined in the beginning of a study, there should be no need to change them unless interim results suggest harm in a specific subgroup (see Chap. 16). The reasons for each criterion should be carefully examined during the planning phase of the study. As discussed earlier in this chapter, eligibility criteria are appropriate if they include participants with high likelihood of showing benefit and exclude those who might be harmed by the intervention, have competing risks, and conditions and are not likely to comply with the study protocol. If they do not fall into one of the above categories, they should be reassessed. Whenever an investigator considers changing criteria, he needs to look at the effect of changes on participant safety and study design. It may be that, in opening the gates to accommodate more participants, he increases the required sample size, because the participants admitted may have lower probability of developing the primary response variable. He can thus lose the benefits of added recruitment. In summary, capacity to recruit participants and to carry out the trial effectively could greatly depend on the eligibility criteria that are set. As a consequence, careful thought should go into establishing them.

References

1.

Rothwell PM. External validity of randomized controlled trials: “To whom do the results of this trial apply?” Lancet 2005;365:82–93.CrossRef

2.

CONSORT. http://​www.​consort-statement.​org

3.

Van Spall HGC, Toren A, Kiss A, Fowler RA. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA 2007;297:1233–1240.CrossRef

4.

Douglas PS. Gender, cardiology, and optimal medical care. Circulation 1986;74:917–919.CrossRef

5.

Bennett JC, for the Board on Health Sciences Policy of the Institute of Medicine. Inclusion of women in clinical trials – policies for population subgroups. N Engl J Med 1993;329:288–292.CrossRef

6.

Freedman LS, Simon R, Foulkes MA, et al. Inclusion of women and minorities in clinical trials and the NIH Revitalization Act of 1993 – the perspective of NIH clinical trialists. Control Clin Trials 1995;16:277–285.CrossRef

7.

Lee PY, Alexander KP, Hammill BG, et al. Representation of elderly persons and women in published randomized trials of acute coronary syndromes. JAMA 2001;286:708–713.

8.

NIH Policy and Guidelines on the Inclusion of Women and Minorities as Subjects in Clinical Research – Amended, October, 2001. http://​grants.​nih.​gov/​grants/​funding/​women_​min/​guidelines_​amended_​10_​2001.​htm

9.

Diabetic Retinopathy Study Research Group: Preliminary report on effects of photocoagulation therapy. Am J Ophthalmol 1976;81:383–396.CrossRef

10.

Diabetic Retinopathy Study Research Group. Photocoagulation treatment of proliferative diabetic retinopathy: the second report of diabetic retinopathy study findings. Ophthalmol 1978;85:82–106.CrossRef

11.

Wooster R, Neuhausen SL, Mangion J, et al. Localization of a breast cancer susceptibility gene, BRCA2, to chromosome 13q12-13. Science 1994;265:2088–2090.CrossRef

12.

Patel MR, Mahaffey KW, Garg J, et al. for ROCKET AF investigators. Rivaroxaban versus warfarin in nonvalvular atrial fibrillation. N Engl J Med 2011;365:883–891.CrossRef

13.

Veterans Administration Cooperative Study Group on Antihypertensive Agents. Effects of treatment on morbidity in hypertension: results in patients with diastolic blood pressures averaging 115 through 129 mm Hg. JAMA 1967;202:1028–1034.CrossRef

14.

Veterans Administration Cooperative Study Group on Antihypertensive Agents. Effects of treatment on morbidity in hypertension: II. Results in patients with diastolic blood pressure averaging 90 through 114 mm Hg. JAMA 1970;213:1143–1152.

15.

Hypertension Detection and Follow-up Program Cooperative Group. Five-year findings of the Hypertension Detection and Follow-up Program. 1. Reduction in mortality of persons with high blood pressure, including mild hypertension. JAMA 1979;242:2562–2571.

16.

The CONSENSUS Trial Study Group. Effects of enalapril on mortality in severe heart failure. N Engl J Med 1987;316:1429–1435.CrossRef

17.

The SOLVD Investigators. Effect of enalapril on survival in patients with reduced left ventricular ejection fractions and congestive heart failure. N Engl J Med 1991;325:293–302.CrossRef

18.

The SOLVD Investigators. Effect of enalapril on mortality and the development of heart failure in asymptomatic patients with reduced left ventricular ejection fractions. N Engl J Med 1992;327:685–691.CrossRef

19.

Vollmer T. The natural history of relapses in multiple sclerosis. J Neurol Sci 2007;256:S5-S13.CrossRef

20.

Sondik EJ, Brown BW, Jr., Silvers A. High risk subjects and the cost of large field trials. J Chronic Dis 1974; 27:177–187.CrossRef

21.

Ridker PM, Danielson E, Fonseca FAH, et al. Rosuvastatin to prevent vascular events in men and women with elevated C-reactive protein. N Engl J Med 2008;359:2195–2207.CrossRef

22.

http://​www.​fda.​gov/​downloads/​drugs/​guidancecomplian​ceregulatoryinfo​rmation/​guidances/​ucm332181.​pdf.

23.

Darrow JJ, Avorn J, Kesselheim AS. New FDA breakthrough-drug category—implications for patients. N Engl J Med 2014;370:1252–1258.CrossRef

24.

McMurray JJV, Packer M, Desai AS, et al. Angiotensin-neprilysin inhibition versus enalapril in heart failure. N Engl J Med 2014;371:993–1004.CrossRef

25.

Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA. 2003;290:1624–1632.CrossRef

26.

Thorpe KE, Swarenstein M, Oxman AD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol 2009;62:464–475.CrossRef

27.

Ridker PM and PREVENT Investigators. Long-term, low does warfarin among venous thrombosis patients with and without factor V Leiden mutation: rationale and design for the Prevention of Recurrent Venous Thromboembolism (PREVENT) trial. Vasc Med 1998;3:67–73.CrossRef

28.

Mooney MM, Welch J, Abrams JS. Clinical trial design and master protocols in NCI clinical treatment trials. [abstract]. Clin Cancer Res 2014;20(2Suppl):Abstract IA08.

29.

Hakonarson H, Thorvaldsson S, Helgadottir A, et al. Effects of a 5-lipoxygenase-activating protein inhibitor on biomarkers associated with risk of myocardial infarction: a randomized trial. JAMA 2005;293:2245–2256.CrossRef

30.

The U.S. Food and Drug Administration. Drugs. Table of pharmacogenomics biomarkers in labeling. www.​fda.​gov/​drugs/​scienceresearch/​researchareas/​pharmacogenetics​/​ucm083378.​htm.

31.

Mrazek DA. Psychiatric pharmacogenomics. New York: Oxford University Press, 2010.CrossRef

32.

Landrum MJ, Lee JM, Riley GR, et al. ClinVar: public archive of relationships among sequence variation and human phenotype. Nucleic Acids Res 2014;42 (Database issue):D980-5.CrossRef

33.

Mailman MD, Feolo M, Jin Y, et al. The NCBI dbGaP database of genotypes and phenotypes. Nat Genet 2007;39:1181–1186.CrossRef

34.

Wilhelmsen L, Ljungberg S, Wedel H, Werko L. A comparison between participants and non-participants in a primary preventive trial. J Chronic Dis. 1976;29:331–339.CrossRef

35.

Smith P, Arnesen H. Mortality in non-consenters in a post-myocardial infarction trial. J Intern Med 1990; 228:253–256.CrossRef

36.

Antithrombotic Trialists’ Collaboration. Collaborative meta-analysis of randomized clinical trials of antiplatelet therapy for prevention of death, myocardial infarction, and stroke in high risk patients. BMJ 2002;324:71–86; correction BMJ 2002;324:141.

37.

Steering Committee of the Physicians’ Health Study Research Group. Final report on the aspirin component of the ongoing Physicians’ Health Study. N Engl J Med 1989;321:129–135.CrossRef

38.

Peto R, Gray R, Collins R, et al. Randomized trial of prophylactic daily aspirin in British male doctors. Br Med J 1988;296:313–316.CrossRef

39.

Ridker PM, Cook NR, Lee I-M, et al. A randomized trial of low-dose aspirin in the primary prevention of cardiovascular disease in women. N Engl J Med 2005;352:1293–1304.CrossRef

40.

Ikeda Y, Shimada K, Teramoto T, et al. Low-dose aspirin for primary prevention of cardiovascular events in Japanese patients 60 years or older with atherosclerotic risk factors. A randomized clinical trial. JAMA. Published online November 17, 2014. doi:10.​1001/​jama.​2014.​15690.

41.

Berger JS, Roncaglioni MC, Avanzini F, et al. Aspirin for the primary prevention of cardiovascular events in women and men: a sex-specific meta-analysis of randomized controlled trials. JAMA 2006;295:306–313; correction JAMA 2006;295:2002.CrossRef

42.

Pedersen TR. The Norwegian Multicenter Study of timolol after myocardial infarction. Circulation 1983;67 (suppl 1):I-49-1-53.

43.

CASS Principal Investigators and Their Associates. Coronary Artery Surgery Study (CASS): a randomized trial of coronary artery bypass surgery. Comparability of entry characteristics and survival in randomized patients and nonrandomized patients meeting randomization criteria. J Am Coll Cardiol 1984;3:114–128.

44.

Kaariainen I, Sipponen P, Siurala M. What fraction of hospital ulcer patients is eligible for prospective drug trials? Scand J Gastroenterol 1991;186:73–76.CrossRef

45.

Benedict GW. LRC Coronary Prevention Trial: Baltimore. Clin Pharmacol Ther 1979;25:685–687.CrossRef

46.

Pitt B, Pfeffer MA, Assmann SF, et al. TOPCAT Investigators. Spironolactone for heart failure with preserved ejection fraction. N Engl J Med 2014;370:1383–1392.CrossRef

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_5

5. Basic Study Design

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

The foundations for the design of controlled experiments were established for agricultural application. They are described in several classical statistics textbooks [1–4]. From these sources evolved the basic design of controlled clinical trials.

Although the history of clinical experimentation contains several instances in which the need for control groups has been recognized [5, 6], this need was not widely accepted until the 1950s [7]. In the past, when a new intervention was first investigated, it was likely to be given to only a small number of people, and the outcome compared, if at all, to that in people with the same condition previously treated in a different manner. The comparison was informal and frequently based on memory alone. Sometimes, in one kind of what has been called a “quasi-experimental” study, people were evaluated initially and then reexamined after an intervention had been introduced. In such studies, the changes from the initial state were used as the measure of success or failure of the new intervention. What could not be known was whether the person would have responded in the same manner if there had been no intervention at all. However, then—and sometimes even today—this kind of observation has formed the basis for the use of new interventions.

Of course, some results are so highly dramatic that no comparison group is needed. Successful results of this magnitude, however, are rare. One example is the effectiveness of penicillin in pneumococcal pneumonia. Another example originated with Pasteur who in 1884 was able to demonstrate that a series of vaccine injections protected dogs from rabies [8]. He suggested that due to the long incubation time, prompt vaccination of a human being after infection might prevent the fatal disease. The first patient was a 9-year-old boy who had been bitten 3 days earlier by a rabid dog. The treatment was completely effective. Confirmation came from another boy who was treated within 6 days of having been bitten. During the next few years, hundreds of patients were given the anti-rabies vaccine. If given within certain time-limits, it was almost always effective.

Gocke reported on a similar, uncontrolled study of patients with acute fulminant viral hepatitis [9]. Nine consecutive cases had been observed, all of whom had a fatal outcome. The next diagnosed case, a young staff nurse in hepatic coma, was given immunotherapy in addition to standard treatment. The patient survived as did four others among eight given the antiserum. The author initially thought that this uncontrolled study was conclusive. However, in considering other explanations for the encouraging findings, he could not eliminate the possibility that a tendency to treat patients earlier in the course and more intensive care might be responsible for the observed outcome. Thus, he joined a double-blind, randomized trial comparing hyperimmune anti-Australia globulin to normal human serum globulin in patients with severe acute hepatitis. Nineteen of 28 patients (67.9%) randomized to control treatment died, compared to 16 of 25 patients (64%) randomized to treatment with exogenous antibody, a statistically nonsignificant difference [10].

A number of medical conditions are either of short duration or episodic in nature. Evaluation of therapy in these cases can be difficult in the absence of controlled studies. Snow and Kimmelman reviewed various uncontrolled studies of surgical procedures for Ménière’s disease [11]. They found that about 75% of patients improved, but noted that this is similar to the 70% remission rate occurring without treatment.

Given the wide spectrum of the natural history of almost any disease and the variability of an individual’s response to an intervention, most investigators recognize the need for a defined control or comparison group.

Fundamental Point

Sound scientific clinical investigation almost always demands that a control group be used against which the new intervention can be compared. Randomization is the preferred way of assigning participants to control and intervention groups.

Overview

Statistics and epidemiology textbooks and papers [12–31], cover various study designs in some detail. Green and Byar also present a “hierarchy of strength of evidence concerning efficacy of treatment” [32]. In their scheme, anecdotal case reports are weakest and confirmed randomized clinical trials are strongest, with various observational and retrospective designs in between. This chapter will discuss several major clinical trial designs.

Most trials use the so-called parallel design. That is, the intervention and control groups are followed simultaneously from the time of allocation to one or the other. Exceptions to the simultaneous follow-up are historical control studies. These compare a group of participants on a new intervention with a previous group of participants on standard or control therapy. A modification of the parallel design is the cross-over trial, which uses each participant at least twice, at least once as a member of the control group and at least once as a member of one or more intervention groups. Another modification is a withdrawal study, which starts with all participants on the active intervention and then, usually randomly, assigns a portion to be followed on the active intervention and the remainder to be followed off the intervention. Factorial design trials, as described later in this chapter, employ two or more independent assignments to intervention or control.

Regardless of whether the trial is a typical parallel design or some variant, one must select the kind of control group and the way participants are allocated to intervention or control. Controls may be on placebo, no treatment, usual or standard care, or a specified treatment. Randomized control and nonrandomized concurrent control studies both assign participants to either the intervention or the control group, but only the former makes the assignment by using a random procedure. Hybrid designs may use a combination of randomized and non-randomized controls. Large, simple trials or pragmatic trials generally have broader and simpler eligibility criteria than other kinds of trials, but as with other studies, can use any of the indicated controls. Allocation to intervention or control may also be done differently, even if randomized. Randomization may be by individual participant or by groups of participants (group or cluster assignment). Adaptive designs may adjust intervention or control assignment or sample size on the basis of participant characteristics or outcomes.

Finally, there are superiority trials and equivalence or noninferiority trials. A superiority trial, which for many years was the typical kind of trial, assesses whether the new intervention is different from (better or worse than) the control. An equivalence trial would assess if the new intervention is more or less equal to the control. A noninferiority trial evaluates whether the new intervention is no worse than the control by some margin, delta (δ). In both of these latter cases, the control group would be on a treatment that had previously been shown to be effective, i.e., have an active control.

Questions have been raised concerning the method of selection of the control group, but the major controversy in the past revolved around the use of historical versus randomized control [33–35]. With regard to drug evaluation, this controversy is less intense than in the past. It has been hotly contested, however, in the evaluation of new devices or procedures [36, 37]. While it is acknowledged that randomized controls provide the best evidence, devices that are relatively little used may be approved based on historical controls with post-marketing studies to further assess possible adverse effects. An example is a device used for closure of a cardiac chamber wall defect [38]. It should be noted that after marketing, rare, but serious adverse effects were reported [39]. No study design is perfect or can answer all questions. Each of the designs has advantages and disadvantages, but a randomized control design is the standard by which other studies should be judged. A discussion of sequential designs is postponed until Chap. 17 because the basic feature involves interim analyses.

For each of the designs it is assumed, for simplicity of discussion, that a single control group and a single intervention group are being considered. These designs can be extended to more than one intervention group and more than one control group.

Randomized Control Trials

Randomized control trials are comparative studies with an intervention group and a control group; the assignment of the participant to a group is determined by the formal procedure of randomization. Randomization, in the simplest case, is a process by which all participants are equally likely to be assigned to either the intervention group or the control group. The features of this technique are discussed in Chap. 6. There are three advantages of the randomized design over other methods for selecting controls [35].

First, randomization removes the potential of bias in the allocation of participants to the intervention group or to the control group. Such selection bias could easily occur, and cannot be necessarily prevented, in the non-randomized concurrent or historical control study because the investigator or the participant may influence the choice of intervention. This influence can be conscious or subconscious and can be due to numerous factors, including the prognosis of the participant. The direction of the allocation bias may go either way and can easily invalidate the comparison. This advantage of randomization assumes that the procedure is performed in a valid manner and that the assignment cannot be predicted (see Chap. 6).

Second, somewhat related to the first, is that randomization tends to produce comparable groups; that is, measured as well as unknown or unmeasured prognostic factors and other characteristics of the participants at the time of randomization will be, on the average, evenly balanced between the intervention and control groups. This does not mean that in any single experiment all such characteristics, sometimes called baseline variables or covariates, will be perfectly balanced between the two groups. However, it does mean that for independent covariates, whatever the detected or undetected differences that exist between the groups, the overall magnitude and direction of the differences will tend to be equally divided between the two groups. Of course, many covariates are strongly associated; thus, any imbalance in one would tend to produce imbalances in the others. As discussed in Chaps. 6 and 18, stratified randomization and stratified analysis are methods commonly used to guard against and adjust for imbalanced randomizations (i.e., “accidental” bias).

Third, the validity of statistical tests of significance is guaranteed. As has been stated [35], “although groups compared are never perfectly balanced for important covariates in any single experiment, the process of randomization makes it possible to ascribe a probability distribution to the difference in outcome between treatment groups receiving equally effective treatments and thus to assign significance levels to observed differences.” The validity of the statistical tests of significance is not dependent on the balance of the prognostic factors between the randomized groups. The chi-square test for two-by-two tables and Student’s t-test for comparing two means can be justified on the basis of randomization alone without making further assumptions concerning the distribution of baseline variables. If randomization is not used, further assumptions concerning the comparability of the groups and the appropriateness of the statistical models must be made before the comparisons will be valid. Establishing the validity of these assumptions may be difficult.

In 1977, randomized and nonrandomized trials of the use of anticoagulant therapy in patients with acute myocardial infarctions were reviewed by Chalmers et al. and the conclusions compared [40]. Of 32 studies, 18 used historical controls and involved a total of 900 patients, 8 used nonrandomized concurrent controls and involved over 3,000 patients, and 6 were randomized trials with a total of over 3,800 patients. The authors reported that 15 of the 18 historical control trials and 5 of the 8 nonrandomized concurrent control trials showed statistically significant results favoring the anticoagulation therapy. Only one of the six randomized control trials showed significant results in support of this therapy. Pooling the results of these six randomized trials yielded a statistically significant 20% reduction in total mortality, confirming the findings of the nonrandomized studies. Pooling the results of the nonrandomized control studies showed a reduction of about 50% in total mortality in the intervention groups, more than twice the decrease seen in the randomized trials. Peto [41] has assumed that this difference in reduction is due to bias. He suggests that since the presumed bias in the nonrandomized trials was of the same order of magnitude as the presumed true effect, the non-randomized trials could have yielded positive answers even if the therapy had been of no benefit. Of course, pooling results of several studies can be hazardous. As pointed out by Goldman and Feinstein [42], not all randomized trials of anticoagulants study the same kind of participants, use precisely the same intervention or measure the same response variables. And, of course, not all randomized trials are done equally well. The principles of pooled analysis, or meta-analysis, are covered in Chap. 18.

In the 1960s, Grace, Muench and Chalmers [43] reviewed studies involving portacaval shunt operations for patients with portal hypertension from cirrhosis. In their review, 34 of 47 non-randomized studies strongly supported the shunt procedure, while only one of the four randomized control trials indicated support for the operation. The authors concluded that the operation should not be endorsed.

Sacks and coworkers expanded the work by Chalmers et al. referenced above [40], to five other interventions [44]. They concluded that selection biases led historical control studies to favor inappropriately the new interventions. It was also noted that many randomized control trials were of inadequate size, and therefore may have failed to find benefits that truly existed [45]. Chalmers and his colleagues also examined 145 reports of studies of treatment after myocardial infarction [46]. Of the 57 studies that used a randomization process that had proper concealment of allocation to intervention or control, 14% had at least one significant (p < 0.05) maldistribution of baseline variables with 3.4% of all of the variables significantly different between treatment groups. Of these 57 studies, 9% found significant outcome differences between groups. Among the 43 reports where the control groups were selected by means of a nonrandom process, 58% had baseline variable differences and 34% of all of the variables were significantly different between groups. The outcomes between groups in the nonrandom studies were significantly different 58% of the time. For the 45 studies that used a randomized, but unblinded process to select the control groups, the results were in between; 28% had baseline imbalances, 7% of the baseline variables were significantly different, and 24% showed significant outcome differences.

The most frequent objections to the use of the randomized control clinical trial were stated by Ingelfinger [47], to be “emotional and ethical.” Many clinicians feel that they must not deprive a participant from receiving a new therapy or intervention which they, or someone else, believe to be beneficial, regardless of the validity of the evidence for that claim. The argument aimed at randomization is that in the typical trial it deprives about one-half the participants from receiving the new and presumed better intervention. There is a large literature on the ethical aspects of randomization. See Chap. 2 for a discussion of this issue.

Not all clinical studies can use randomized controls. Occasionally, the prevalence of the disease is so rare that a large enough population can not be readily obtained. In such an instance, only case-control studies might be possible. Such studies, which are not clinical trials according to the definition in this book, are discussed in standard epidemiology textbooks [15, 16, 22, 28].

Zelen proposed a modification of the standard randomized control study [48]. He argued that investigators are often reluctant to recruit prospective trial participants not knowing to which group the participant will be assigned. Expressing ignorance of optimal therapy compromises the traditional doctor-patient relationship. Zelen, therefore, suggested randomizing eligible participants before informing them about the trial. Only those assigned to active intervention would be asked if they wish to participate. The control participants would simply be followed and their outcomes monitored. Obviously, such a design could not be blinded. Another major criticism of this controversial design centers around the ethical concern of not informing participants that they are enrolled in a trial. The efficiency of the design has also been evaluated [49]. It depends on the proportion of participants consenting to comply with the assigned intervention. To compensate for this possible inefficiency, one needs to increase the sample size (Chap. 8). The Zelen approach has been tried with varying degrees of success [50, 51]. Despite having been proposed in 1979 it does not appear to have been widely used.

Nonrandomized Concurrent Control Studies

Controls in this type of study are participants treated without the new intervention at approximately the same time as the intervention group is treated. Participants are allocated to one of the two groups, but by definition this is not a random process. An example of a nonrandomized concurrent control study would be a comparison of survival results of patients treated at two institutions, one institution using a new surgical procedure and the other using more traditional medical care. Another example is when patients are offered either of two treatments and the patient selects the one that he or she thinks is preferable. Comparisons between the two groups is then made, adjusting for any observed baseline imbalances.

To some investigators, the nonrandomized concurrent control design has advantages over the randomized control design. Those who object to the idea of ceding to chance the responsibility for selecting a person’s treatment may favor this design. It is also difficult for some investigators to convince potential participants of the need for randomization. They find it easier to offer the intervention to some and the control to others, hoping to match on key characteristics.

The major weakness of the nonrandomized concurrent control study is the potential that the intervention group and control group are not strictly comparable. It is difficult to prove comparability because the investigator must assume that she has information on all the important prognostic factors. Selecting a control group by matching on more than a few factors is impractical and the comparability of a variety of other characteristics would still need to be evaluated. In small studies, an investigator is unlikely to find real differences which may exist between groups before the initiation of intervention since there is poor sensitivity statistically to detect such differences. Even for large studies that could detect most differences of real clinical importance, the uncertainty about the unknown or unmeasured factors is still of concern.

Is there, for example, some unknown and unmeasurable process that results in one type of participant’s being recruited more often into one group and not into the other? If all participants come from one institution, physicians may select participants into one group based on subtle and intangible factors. In addition, there exists the possibility for subconscious bias in the allocation of participants to either the intervention or control group. One group might come from a different socioeconomic class than the other group. All of these uncertainties will decrease the credibility of the concurrent but nonrandomized control study. For any particular question, the advantages of reduced cost, relative simplicity and investigator and participant acceptance must be carefully weighed against the potential biases before a decision is made to use a non-randomized concurrent control study. We believe this will occur very rarely.

Historical Controls and Databases

In historical control studies, a new intervention is used in a series of participants and the results are compared to the outcome in a previous series of comparable participants. Historical controls are thus, by this definition, nonrandomized and nonconcurrent.

Strengths of Historical Control Studies

The argument for using a historical control design is that all new participants can receive the new intervention. As presented by Gehan and Freireich [33] many clinicians believe that no participant should be deprived of the possibility of receiving a new therapy or intervention. Some clinicians require less supportive evidence than others to accept a new intervention as being beneficial. If an investigator is already of the opinion that the new intervention is beneficial, then she would most likely consider any restriction on its use unethical. Therefore, she would favor a historical control study. In addition, participants may be more willing to enroll in a study if they can be assured of receiving a particular therapy or intervention. Finally, since all new participants will be on the new intervention, the time required to complete recruitment of participants for the trial will be cut approximately in half. This allows investigators to obtain results faster or do more studies with given resources. Alternatively, the sample size for the intervention group can be larger, with increased power.

Gehan emphasized the ethical advantages of historical control studies and pointed out that they have contributed to medical knowledge [52]. Lasagna argued that medical practitioners traditionally relied on historical controls when making therapeutic judgments. He maintained that, while sometimes faulty, these judgments are often correct and useful [53].

Typically, historical control data can be obtained from two sources. First, control group data may be available in the literature. These data are often undesirable because it is difficult, and perhaps impossible, to establish whether the control and intervention groups are comparable in key characteristics at the onset. Even if such characteristics were measured in the same way, the information may not be published and for all practical purposes it will be lost. Second, data may not have been published but may be available on computer files or in medical charts. Such data on control participants, for example, might be found in a large center which has several ongoing clinical investigations. When one study is finished, the participants in that study may be used as a control group for some future study. Centers which do successive studies, as in cancer research, will usually have a system for storing and retrieving the data from past studies for use at some future time. The advent of electronic medical records may also facilitate access to historical data from multiple sources, although it does not solve the problem of nonstandard and variable assessment or missing information.

Limitations of Historical Control Studies

Despite the time and cost benefits, as well as the ethical considerations, historical control studies have potential limitations which should be kept in mind. They are particularly vulnerable to bias. Moertel [54] cited a number of examples of treatments for cancer which have been claimed, on the basis of historical control studies, to be beneficial. Many treatments in the past were declared breakthroughs on the basis of control data as old as 30 years. Pocock [55] identified 19 instances of the same intervention having been used in two consecutive trials employing similar participants at the same institution. Theoretically, the mortality in the two groups using the same treatment should be similar. Pocock noted that the difference in mortality rates between such groups ranged from negative 46% to plus 24%. Four of the 19 comparisons of the same intervention showed differences significant at the 5% level.

An improvement in outcome for a given disease may be attributed to a new intervention when, in fact, the improvement may stem from a change in the patient population or patient management. Shifts in patient population can be subtle and perhaps undetectable. In a Veterans Administration Urological Research Group study of prostate cancer [56], 2,313 people were randomized to placebo or estrogen treatment groups over a 7-year period. For those enrolled during the last 2–3 years, no differences were found between the placebo and estrogen groups. However, those assigned to placebo entering in the first 2–3 years had a shorter survival time than those assigned to estrogen entering in the last 2–3 years of the study. The reason for the early apparent difference is probably that the people randomized earlier were older than the later group and thus were at higher risk of death during the period of observation [35]. The results would have been misleading had this been a historical control study and had a concurrent randomized comparison group not been available.

A more recent example involves two trials evaluating the potential benefit of amlodipine, a calcium channel blocker, in patients with heart failure. The first trial, the Prospective Randomized Amlodipine Survival Evaluation trials, referred to as PRAISE-I [57], randomized participants to amlodipine or placebo, stratifying by ischemic or nonischemic etiology of the heart failure. The primary outcome, death plus hospitalization for cardiovascular reasons, was not significantly different between groups (p = 0.31), but the reduction in mortality almost reached significance (p = 0.07). An interaction with etiology was noted, with all of the benefit from amlodipine in both the primary outcome and mortality seen in those with nonischemic etiology. A second trial, PRAISE-2 [58], was conducted in only those with nonischemic causes of heart failure. The impressive subgroup findings noted in PRAISE-1 were not replicated. Of relevance here is that the event rates in the placebo group in PRAISE-2 were significantly lower than in the nonischemic placebo participants from the first trial (see Fig. 5.1).

Fig. 5.1

PRAISE 1 and 2 placebo arms

Even though the same investigators conducted both trials using the same protocol, the kinds of people who were enrolled into the second trial were markedly different from the first trial. Covariate analyses were unable to account for the difference in outcome.

On a broader scale, for both known and unknown reasons, in many countries trends in prevalence of various diseases occur [59]. Therefore, any clinical trial in those conditions, involving long-term therapy using historical controls would need to separate the treatment effect from the time trends, an almost impossible task. Examples are seen in Figs. 5.2 and 5.3.

Fig. 5.2

Death rates for selected causes of death for all ages, by sex: United States, 1998–2008

Fig. 5.3

Changes in incidence of hepatitis, by type, in the U.S. [61]

Figure 5.2 illustrates the changes over time, in rates of the leading causes of death in the United States [60]. A few of the causes exhibit quite large changes. Figure 5.3 shows incidence of hepatitis in the U.S. [61]. The big changes make interpretation of historical control trials difficult.

The method by which participants are selected for a particular study can have a large impact on their comparability with earlier participant groups or general population statistics. In the Coronary Drug Project [62], a trial of survivors of myocardial infarction initiated in the 1960s, an annual total mortality rate of 6% was anticipated in the control group based on rates from a fairly unselected group of myocardial infarction patients. In fact, a control group mortality rate of about 4% was observed, and no significant differences in mortality were seen between the intervention groups and the control group. Using the historical control approach, a 33% reduction in mortality might have been claimed for the treatments. One explanation for the discrepancy between anticipated and observed mortality is that entry criteria excluded those most seriously ill.

Shifts in diagnostic criteria for a given disease due to improved technology can cause major changes in the recorded frequency of the disease and in the perceived prognosis of those with the disease. The use of elevated serum troponin, sometimes to the exclusion of the need for other features of an acute myocardial infarction such as symptoms or electrocardiographic changes, has clearly led to the ability to diagnose more infarctions. Changes in the kinds of troponin measured and in how it is used to define myocardial infarction can also affect reported incidence. Conversely, the ability to abort an evolving infarction by means of percutaneous coronary intervention or thrombolytic therapy, can reduce the number of clearly diagnosed infarctions.

In 1993, the Centers for Disease Control and Prevention (CDC) in the U.S. implemented a revised classification system for HIV infection and an expanded surveillance case definition of AIDS. This affected the number of cases reported [63, 64]. See Fig. 5.4.

Fig. 5.4

AIDS cases, by quarter year of report—United States, 1984–1993 [64]

International coding systems and names of diseases change periodically and, unless one is aware of the modifications, prevalence of certain conditions can appear to change abruptly. For example, when the Eighth Revision of the International Classification of Diseases came out in 1968, almost 15% more deaths were assigned to ischemic heart disease than had been assigned in the Seventh Revision [65]. When the Ninth Revision appeared in 1979, there was a correction downward of a similar magnitude [66]. The transition to the Tenth Revision will also lead to changes in assignment of causes of deaths [67]. A common concern about historical control designs is the accuracy and completeness with which control group data are collected. With the possible exception of special centers which have many ongoing studies, data are generally collected in a nonuniform manner by numerous people with diverse interests in the information. Lack of uniform collection methods can easily lead to incomplete and erroneous records. Data on some important prognostic factors may not have been collected at all. Because of the limitations of data collected historically from medical charts, records from a center which conducts several studies and has a computerized data management system may provide the most reliable historical control data.

Role of Historical Controls

Despite the limitations of the historical control study, it does have a place in scientific investigation. As a rapid, relatively inexpensive method of obtaining initial impressions regarding a new therapy, such studies can be important. This is particularly so if investigators understand the potential biases and are willing to miss effective new therapies if bias works in the wrong direction. Bailar et al. [68] identified several features which can strengthen the conclusions to be drawn from historical control studies. These include an a priori identification of a reasonable hypothesis and planning for analysis.

In some special cases where the diagnosis of a disease is clearly established and the prognosis is well known or the disease highly fatal, a historical control study may be the only reasonable design. The results of penicillin in treatment of pneumococcal pneumonia were so dramatic in contrast to previous experience that no further evidence was really required. Similarly, the benefits of treatment of malignant hypertension became readily apparent from comparisons with previous, untreated populations [69–71].

The use of prospective registries to characterize patients and evaluate effects of therapy has been advocated [72–74]. Supporters say that a systematic approach to data collection and follow-up can provide information about the local patient population, and can aid in clinical decision making. They argue that clinical trial populations may not be representative of the patients actually seen by a physician. Moon et al. described the use of databases derived from clinical trials to evaluate therapy [75]. They stress that the high quality data obtained through these sources can reduce the limitations of the typical historical control study. Many hospitals and other large medical care systems have electronic health records. Other clinical care entities are more slowly converting to electronic systems. At least partly because of the existence of these systems and the relative ease of accessing huge computerized medical databases, the use of databases in outcomes research has burgeoned [76]. These kinds of analyses are much faster and cheaper than conducting clinical trials. Databases can also be used to identify adverse events. Examples are comparisons of different antihypertensive agents and risk of stroke [77] and cyclooxygenase 2 (COX 2) inhibitors and risk of coronary heart disease [78]. In addition, databases likely represent a much broader population than the typical clinical trial, and can therefore complement clinical trial findings. This information can be useful as long as it is kept in mind that users and non-users of a medication are different and therefore have different characteristics.

Others [32, 79–81] have emphasized limitations of registry studies such as potential bias in treatment assignment, multiple comparisons, lack of standardization in collecting and reporting data, and missing data. Another weakness of prospective database registries is that they rely heavily on the validity of the model employed to analyze the data [82].

Lauer and D’Agostino note the high cost of clinical trials and argue that large databases may be able to substitute for trials that otherwise would not be conducted [83]. They also point out that existing registries and electronic health records can assist in conducting clinical trials. One such trial was the Thrombus Aspiration in ST-Elevation Myocardial Infarction in Scandinavia (TASTE), conducted in Scandinavia, which has extensive electronic health records [84].

There is no doubt that analyses of large databases can provide important information about disease occurrence and outcomes, as well as suggestions that certain therapies are preferable. As noted above, they can help to show that the results of clinical trials conducted in selected populations appear to apply in broader groups. Given their inherent chances for bias, however, they are no substitute for a randomized clinical trial in evaluating whether one intervention is truly better than another.

Cross-Over Designs

The cross-over design is a special case of a randomized control trial and has some appeal to medical researchers. The cross-over design allows each participant to serve as his own control. In the simplest case, namely the two period cross-over design, each participant will receive either intervention or control (A or B) in the first period and the alternative in the succeeding period. The order in which A and B are given to each participant is randomized. Thus, approximately half of the participants receive the intervention in the sequence AB and the other half in the sequence BA. This is so that any trend from first period to second period can be eliminated in the estimate of group differences in response. Cross-over designs need not be simple; they need not have only two groups. and there may be more than two periods [85, 86]. Depending on the duration of expected action of the intervention (for example, drug half-life), a wash-out period may be used between the periods.

The advantages and disadvantages of the two-period cross-over design have been described [19, 21, 86–89]. The appeal of the cross-over design to investigators is that it allows assessment of how each participant does on both A and B. Since each participant is used twice, variability is reduced because the measured effect of the intervention is the difference in an individual participant’s response to intervention and control. This reduction in variability enables investigators to use smaller sample sizes to detect a specific difference in response. James et al. described 59 cross-over studies of analgesic agents. They concluded that if the studies had been designed using parallel or noncross-over designs, 2.4 times as many participants would have been needed [90]. Carriere showed that a three-period cross-over design is even more efficient than a two-period cross-over design [85].

In order to use the cross-over design, however, a fairly strict assumption must be made; the effects of the intervention during the first period must not carry over into the second period. This assumption should be independent of which intervention was assigned during the first period and of the participant response. In many clinical trials, such an assumption is clearly inappropriate, even if a wash-out is incorporated. If, for example, the intervention during the first period cures the disease, then the participant obviously cannot return to the initial state. In other clinical trials, the cross-over design appears more reasonable. If a drug’s effect is to lower blood pressure or heart rate, then a drug-versus-placebo cross-over design might be considered if the drug has no carryover effect once the participant is taken off medication. Obviously, a fatal event and many disease complications cannot serve as the primary response variable in a cross-over trial.

Mills et al. [91] reviewed 116 reports of cross-over trials, which consisted of 127 individual trials. Reporting of key design and conduct characteristics was highly variable, making it difficult to discern whether optimal designs were followed.

As indicated in the International Conference on Harmonisation document E9, Statistical Principles for Clinical Trials [92], cross-over trials should be limited to those situations with few losses of study participants. A typical and acceptable cross-over trial, for example, might compare two formulations of the same drug in order to assess bioequivalence in healthy participants. Similarly, different doses may be used to assess pharmacologic properties. In studies involving participants who are ill or otherwise have conditions likely to change, however, cross-over trials have the limitations noted above.

Although the statistical method for checking the assumption of no period-treatment interaction was described by Grizzle [93], the test is not as powerful as one would like. What decreases the power of the test is that the mean response of the AB group is compared to the mean response of the BA group. However, participant variability is introduced in this comparison, which inflates the error term in the statistical test. Thus, the ability to test the assumption of no period-intervention interaction is not sensitive enough to detect important violations of the assumption unless many participants are used. The basic appeal of the cross-over design is to avoid between-participant variation in estimating the intervention effect, thereby requiring a smaller sample size. Yet the ability to justify the use of the design still depends on a test for carryover that includes between-participant variability. This weakens the main rationale for the cross-over design. Because of this insensitivity, the cross-over design is not as attractive as it at first appears. Fleiss et al. noted that even adjusting for baseline variables may not be adequate if inadequate time has been allowed for the participant to return to baseline status at the start of the second period [94]. Brown [19, 21] and Hills and Armitage [95] discourage the use of the cross-over design in general. Only if there is substantial evidence that the therapy has no carryover effects, and the scientific community is convinced by that evidence, should a cross-over design be considered.

Withdrawal Studies

A number of studies have been conducted in which the participants on a particular treatment for a chronic disease are taken off therapy or have the dosage reduced. The objective is to assess response to discontinuation or dose reduction. This design may be validly used to evaluate the duration of benefit of an intervention already known to be useful. For example, subsequent to the Hypertension Detection and Follow-up Program [96], which demonstrated the benefits of treating mild and moderate hypertension, several investigators withdrew a sample of participants with controlled blood pressure from antihypertensive therapy [97]. Participants were randomly assigned to continue medication, stop medication yet initiate nutritional changes, or stop medication without nutritional changes. After 4 years, only 5% of those taken off medication without nutritional changes remained normotensive and did not need the re-instatement of medication. This compared with 39% who were taken off medication yet instituted weight loss and reductions in salt intake. Patients with severe chronic obstructive pulmonary disease (COPD) were prescribed a combination of tiotropium, salmeterol, and an inhaled glucocorticoid, fluticasone propionate for 6 weeks [98]. Because of the adverse effects of long term use of glucocorticoids, the investigators withdrew the fluticasone propionate over the subsequent 12 weeks. Despite a decrease in lung function, COPD exacerbations remained unchanged.

Withdrawal studies have also been used to assess the efficacy of an intervention that had not conclusively been shown to be beneficial in the long term. An early example is the Sixty Plus Reinfarction Study [99]. Participants doing well on oral anticoagulant therapy since their myocardial infarction, an average of 6 years earlier, were randomly assigned to continue on anticoagulants or assigned to placebo. Those who stayed on the intervention had lower mortality (not statistically significant) and a clear reduction in nonfatal reinfarction. A meta-analysis of prednisone and cyclosporine withdrawal trials (including some trials comparing withdrawal of the two drugs) in renal transplant patients has been conducted with graft failure or rejection as the response variables [100]. This meta-analysis found that withdrawal of prednisone was associated with increased risks of acute rejection and graft failure. Cyclosporine withdrawal led to an increase in acute rejection, but not graft failure. The Fracture Intervention Trial Long-term Extension (FLEX) assessed the benefits of continuing treatment with alendronate after 5 years of therapy [101]. The group that was randomized to discontinue alendronate had a modest increase in vertebral fractures but no increase in nonvertebral fractures.

One serious limitation of this type of study is that a highly selected sample is evaluated. Only those participants who physicians thought were benefiting from the intervention were likely to have been on it for several months or years. Anyone who had major adverse effects from the drug would have been taken off and, therefore, not been eligible for the withdrawal study. Thus, this design can overestimate benefit and underestimate toxicity. Another drawback is that both participants and disease states change over time.

If withdrawal studies are conducted, the same standards should be adhered to that are used with other designs. Randomization, blinding where feasible, unbiased assessment, and proper data analysis are as important here as in other settings.

Factorial Design

In the simple case, the factorial design attempts to evaluate two interventions compared to control in a single experiment [2–4, 102]. See Table 5.1.

Table 5.1

Two-by-two factorial design

Intervention X

Control

Marginals

Intervention YControlMarginals

aca + c

bdb + d

a + bc + d

Cell

Intervention

a

X + Y

b

Y + control

c

X + control

d

control + control

Effect of intervention X: a + c versus b + d

Effect of intervention Y: a + b versus c + d

Given the cost and effort in recruiting participants and conducting clinical trials, getting two (or more) experiments done at once is appealing. Examples of factorial designs are the Canadian transient ischemic attack study where aspirin and sulfinpyrazone were compared singly and together with placebo [103], the Third International Study of Infarct Survival (ISIS-3) that compared streptokinase, tissue plasminogen activator, and antistreplase plus aspirin plus heparin vs. aspirin alone [104], the Physicians’ Health Study of aspirin and beta carotene [105], and the Women’s Health Initiative (WHI) trial of hormone replacement, diet, and vitamin D plus calcium [106]. A review of analysis and reporting of factorial design trials [107] contains a list of 29 trials involving myocardial infarction and 15 other trials. Some factorial design studies are more complex than the 2 by 2 design, employing a third, or even a fourth level. It is also possible to leave some of the cells empty, that is, use an incomplete factorial design [108]. This was done in the Action to Control Cardiovascular Risk in Diabetes (ACCORD), which looked at intensive vs. less intensive glucose control plus either intensive blood pressure or lipid control [109]. This kind of design would be implemented if it is inappropriate, infeasible, or unethical to address every possible treatment combination. It is also possible to use a factorial design in a cross-over study [110].

The appeal of the factorial design might suggest that there really is a “free lunch.” However, every design has strengths and weaknesses. A concern with the factorial design is the possibility of the existence of interaction between the interventions and its impact on the sample size. Interaction means that the effect of intervention X differs depending upon the presence or absence of intervention Y, or vice versa. It is more likely to occur when the two drugs are expected to have related mechanisms of action.

If one could safely assume there were no interactions, with a modest increase in sample size, two experiments can be conducted in one; one which is considerably smaller than the sum of two independent trials under the same design specifications. However, if one cannot reasonably rule out interaction, one should statistically test for its presence. As is true for the cross-over design, the power for testing for interaction is less than the power for testing for the main effects of interventions (cells a + c vs. b + d or cells a + b vs. c + d). Thus, to obtain satisfactory power to detect interaction, the total sample size must be increased. The extent of the increase depends on the degree of interaction, which may not be known until the end of the trial. The larger the interaction, the smaller the increase in sample size needed to detect it. If an interaction is detected, or perhaps only suggested, the comparison of intervention X would have to be done individually for intervention Y and its control (cell a vs. b and cell c vs. d). The power for these comparisons is obviously less than for the a + c vs. b + d comparison.

As noted, in studies where the various interventions either act on the same response variable or possibly through the same or similar mechanism of action, as with the presumed effect on platelets of both drugs in the Canadian transient ischemic attack study [103], interaction can be more of a concern. Furthermore, there may be a limited amount of reduction in the response variable that can be reasonably expected, restricting the joint effect of the interventions.

In trials such as the Physicians’ Health Study [105], the two interventions, aspirin and beta carotene, were expected to act on two separate outcomes, cardiovascular disease and cancer. Thus, interaction was much less likely. But beta carotene is an antioxidant, and therefore might have affected both cancer and heart disease. It turned out to have no effect on either. Similarly, in the Women’s Health Initiative [106], dietary and hormonal interventions may affect more than one disease process. There, diet had little effect on cancer and heart disease, but hormonal therapy had effects on heart disease, stroke, and cancer, among other conditions [111, 112].

In circumstances where there are two separate outcomes, e.g., heart disease and cancer, but one of the interventions may have an effect on both, data monitoring may become complicated. If, during the course of monitoring response variables it is determined that an intervention has a significant or important effect on one of the outcomes in a factorial design study, it may be difficult ethically, or even impossible, to continue the trial to assess fully the effect on the other outcome. Chapter 17 reviews data monitoring in more detail.

The factorial design has some distinct advantages. If the interaction of two interventions is important to determine, or if there is little chance of interaction, then such a design with appropriate sample size can be very informative and efficient. However, the added complexity, impact on recruitment and adherence, and potential adverse effects of “polypharmacy” must be considered. Brittain and Wittes [113] discuss a number of settings in which factorial designs might be useful or not, and raise several cautions. In addition to the issue of interaction, they note that less than full adherence to the intervention can exacerbate problems in a factorial design trial.

Group Allocation Designs

In group or cluster allocation designs, a group of individuals, a clinic or a community are randomized to a particular intervention or control [114–118]. The rationale is that the intervention is most appropriately or more feasibly administered to an entire group (for example, if the intervention consists of a broad media campaign). This design may also be better if there is concern about contamination. That is, when what one individual does might readily influence what other participants do. In the Child and Adolescent Trial for Cardiovascular Health, schools were randomized to different interventions [119]. Investigators randomized villages in a trial of vitamin A versus placebo on morbidity and mortality in children in India [120]. The Rapid Early Action for Coronary Treatment (REACT) trial involved ten matched pairs of cities. Within each pair, one city was randomly allocated to community education efforts aimed at reducing the time between symptoms of myocardial infarction and arrival at hospital [121]. Despite 18 months of community education, delay time was not different from that in the control cities. Communities have been compared in other trials [122, 123]. These designs have been used in cancer trials where a clinic or physician may have difficulty approaching people about the idea of randomization. The use of such designs in infectious disease control in areas with high prevalence of conditions such as tuberculosis and AIDS has become more common [124]. It should be noted that this example is both a group allocation design and a factorial design. Variations of group allocation, including cross-over and modification of cross-over, such as stepped wedge designs, where groups cross-over sequentially, rather than all at once, have been implemented [125, 126]. In the group allocation design, the basic sampling units and the units of analysis are groups, not individual participants. This means that the effective sample is substantially less than the total number of participants. Chapters 8 and 18 contain further discussions of the sample size determination and analysis of this design.

Hybrid Designs

Pocock [127] has argued that if a substantial amount of data is available from historical controls, then a hybrid, or combination design could be considered. Rather than a 50/50 allocation of participants, a smaller proportion could be randomized to control, permitting most to be assigned to the new intervention. A number of criteria must be met in order to combine the historical and randomized controls. These include the same entry criteria and evaluation factors, and participant recruitment by the same clinic or investigator. The data from the historical control participants must also be fairly recent. This approach, if feasible, requires fewer participants to be entered into a trial. Machin, however, cautions that if biases introduced from the non-randomized participants (historical controls) are substantial, more participants might have to be randomized to compensate than would be the case in a corresponding fully randomized trial [128].

Large, Simple and Pragmatic Clinical Trials

Advocates of large, simple trials maintain that for common medical conditions, it is important to uncover even modest benefits of intervention, particularly short-term interventions that are easily implemented in a large population. They also argue that an intervention is unlikely to have very different effects in different sorts of participants (i.e., subgroups). Therefore, careful characterization of people at entry and of interim response variables, both of which add to the already considerable cost of trials, are unnecessary. The important criteria for a valid study are unbiased (i.e., randomized) allocation of participants to intervention or control and unbiased assessment of outcomes. Sufficiently large numbers of participants are more important than modest improvements in quality of data. The simplification of the study design and management allows for sufficiently large trials at reasonable cost. Examples of successfully completed large, simple trials are ISIS-3 [104], Gruppo Italiano per lo Studio della Streptochinasi nell’Infarto Miocardico (GISSI) [129], Global Utilization of Streptokinase and Tissue Plasminogen Activator for Occluded Coronary Arteries (GUSTO) [130], a study of digitalis [131], the MICHELANGELO Organization to Assess Strategies in Acute Ischemic Syndromes (OASIS)-5 [132], and the Thrombus Aspiration in ST-Elevation Myocardial Infarction in Scandinavia (TASTE) trial [84]. It should be noted that with the exception of the digitalis trial, these studies were relatively short-term. The questions addressed by these trials may be not only of the sort, “What treatment works better?” but “What is the best way of providing the treatment?” Can something shown to work in an academic setting be translated to a typical community medical care setting? Several have advocated conducting pragmatic or practical clinical trials. These kinds of trials, as noted in Chap. 3, are conducted in clinical practices, often far from academic centers. They address questions perceived as relevant to those practices [133–136]. Because of the broad involvement of many practitioners, the results of the trial may be more widely applied than the results of a trial done in just major medical settings. Thus, they may address a common criticism that the kinds of participants normally seen in academic centers, and therefore enrolled in many academic-based trials, are not the sort seen in typical clinical practices.

As indicated, these models depend upon a relatively easily administered intervention and an easily ascertained outcome. If the intervention is complex, requiring either special expertise or effort, particularly where adherence to protocol must be maintained over a long time, these kinds of studies are less likely to be successful. Similarly, if the response variable is a measure of morbidity that requires careful measurement by highly trained investigators, large simple or pragmatic trials are not feasible.

In recent years, the concept of comparative effectiveness research has become popular. Although trials comparing one agent against another have been conducted for many years, certain features of comparative effectiveness research should be mentioned. First, much of the research consists of other than clinical trials comparisons of interventions (e.g., use of databases as discussed in the sections above on nonrandomized control studies). In the clinical trial arena, much of the comparative effectiveness literature emphasizes studies done in collaboration with clinical practices (i.e., large, simple trials). They compare two or more interventions that are commonly used and involve outcome measures, including cost, that are of particular relevance to practitioners or to the participants [137].

It has also been pointed out that baseline characteristics may be useful for subgroup analysis. The issue of subgroup analysis is discussed more fully in Chap. 18. Although in general, it is likely that the effect of an intervention is qualitatively the same across subgroups, exceptions may exist. In addition, important quantitative differences may occur. When there is reasonable expectation of such differences, appropriate baseline variables need to be measured. Variables such as age, gender, past history of a particular condition, or type of medication currently being taken can be assessed in a simple trial. On the other hand, if an invasive laboratory test or a measurement that requires special training is necessary at baseline, such characterization may make a simple or pragmatic trial infeasible.

The investigator also needs to consider that the results of the trial must be persuasive to others. If other researchers or clinicians seriously question the validity of the trial because of inadequate information about participants or inadequate documentation of quality control, then the study has not achieved its purpose.

There is no doubt that many clinical trials are too expensive and too cumbersome, especially multicenter ones. The advent of the large, simple trial or the pragmatic trial is an important step in enabling many meaningful medical questions to be addressed in an efficient manner. In other instances, however, the use of large numbers of participants may not compensate for reduced data collection and quality control. As always, the primary question being asked dictates the optimal design of the trial.

With increased understanding of genetic influences, the concept that interventions are likely to work similarly in all or at least most participants may no longer hold. There are differential effects of interventions in human epidermal growth factor receptor (HER-2) breast cancer, for example [138]. The concept of “personalized medicine” argues against the concept of large, simple trials and some have designed clinical trials to take advantage of biomarkers [139]. For most common conditions, however, we do not yet have the understanding required to implement personalized medicine, and large, simple trials will remain important for some time.

Studies of Equivalency and Noninferiority

Many clinical trials are designed to demonstrate that a new intervention is better than or superior to the control. However, not all trials have this goal. New interventions may have little or no superiority to existing therapies, but, as long as they are not materially worse, may be of interest because they are less toxic, less invasive, less costly, require fewer doses, improve quality of life, or have some other value to patients. In this setting, the goal of the trial would be to demonstrate that the new intervention is not worse, in terms of the primary response variable, than the standard by some predefined margin.

In studies of equivalency, the objective is to test whether a new intervention is equivalent to an established one. Noninferiority trials test whether the new intervention is no worse than, or at least as good as, some established intervention. Sample size issues for these kinds of trials are discussed in Chap. 8. It should also be noted that although the following discussion assumes one new intervention and one established intervention (the control), there is no reason why more complicated designs involving multiple new interventions, for example, could not be implemented. This occurred in the Comparison of Age-Related Macular Degeneration Treatments Trials (CATT), where four groups (one standard therapy—monthly administration of intravitreal injections of ranibizumab—and three unproven therapies—as needed injections of ranibizumab and monthly and as needed injections of bevicizumab) were compared using a noniferiority design [140].

In equivalency and noninferiority trials, several design aspects need to be considered [141–148]. The control or standard treatment must have been shown conclusively to be effective; that is, truly better than placebo or no therapy. The circumstances under which the active control was found to be useful (i.e., similarity of populations, concomitant therapy, and dosage) ought to be reasonably close to those of the planned trial. These requirements also mean that the trials that demonstrated efficacy of the standard should be recent and properly designed, conducted, analyzed, and reported.

Table 5.2 shows the key assumptions for these trials. First, the active control that is selected must be one that is an established standard for the indication being studied and not a therapy that is inferior to other known ones. It must be used with the dose and formulation proven effective. Second, the studies that demonstrated benefit of the control against either placebo or no treatment must be sufficiently recent such that no important medical advances or other changes have occurred, and in populations similar to those planned for the new trial. Third, the evidence that demonstrated the benefits of the control must be available so that a control group event rate can be estimated. Fourth, the response variable used in the new trial must be sensitive to the postulated effects of the control and intervention. The proposed trial must be able to demonstrate “assay sensitivity,” or the ability to show a difference if one truly exists. As emphasized in Chap. 8, the investigator must specify what she means by equivalence.

Table 5.2

Noninferiority design assumptions

–  Proper control arm

–  Constancy over time and among participants

–  Availability of data from prior studies of the control

–  Assay sensitivity to demonstrate a true difference

It cannot be shown statistically that two therapies are identical, as an infinite sample size would be required. Therefore, if the intervention falls sufficiently close to the standard, as defined by reasonable boundaries, the intervention is claimed to be “the same” as the control (in an equivalence trial) or no worse than the control (in a noninferiority trial). Selecting the margin of indifference or noninferiority, δ, is a challenge. Ideally, the relative risk of the new intervention compared to the control should be as close to 1 as possible. For practical reasons, the relative risk is often set in the range of 1.2–1.4. This means that in the worst case, the new intervention may be 20–40% inferior to standard treatment and yet be considered equivalent or noninferior. Some have even suggested that any new intervention could be approved by regulatory agencies as being noninferior to a standard control intervention if it retains as least 50% of the control versus placebo effect. Further, there are options as to what 50% (or 40% or 20%) means. For example, one could choose either the point estimate from the control versus placebo comparison, or the lower confidence interval estimate of that comparison. Also, the choice of the metric or scale must be selected, such as a relative risk, or hazard ratio or perhaps an absolute difference. Of course, if an absolute difference that might seem reasonable with a high control group event rate is chosen, it might not seem so reasonable if the control group event rate turns out to be much lower than expected. This happened with a trial comparing warfarin against a new anticoagulant agent, where the observed control group event rate was less than that originally expected. Thus, with a predetermined absolute difference for noninferiority, the relative margin of noninferiority was larger than had been anticipated when the trial was designed [149].

It should be emphasized that new interventions are often hailed as successes if they are shown to be 20 or 25% better than placebo or a standard therapy. To turn around and claim that anything within a margin of 40 or 50% is equivalent to, or noninferior to a standard therapy would seem illogical. But the impact on sample size of seeking to demonstrate that a new intervention is at most 20% worse than a standard therapy, rather than 40%, is considerable. As is discussed in Chap. 8, it would not be just a twofold increase in sample size, but a fourfold increase if the other parameters remained the same. Therefore, all design considerations and implications must be carefully considered.

Perhaps even more than in superiority trials, the quality, the size and power of the new trial, and how well the trial is conducted, including how well participants adhere to the assigned therapy, are crucial. A small sample size or poor adherence with the protocol, leading to low statistical power, and therefore lack of significant difference, does not imply equivalence.

To illustrate the concepts around noninferiority designs, consider the series of trials represented in Fig. 5.5, which depicts estimates with 95% confidence intervals for the intervention effect.

Fig. 5.5

Possible results of noninferiority trials

The heavy vertical line (labeled Delta) indicates the amount of worse effect of the intervention compared to the control that was chosen as tolerable. The thin vertical line indicates zero difference (a relative risk of 1). Trial A shows a new intervention that is superior to control (i.e. the upper confidence interval excludes zero difference). Trial B has an estimate of the intervention effect that is favorable but the upper limit of the confidence interval does not exclude zero. It is less than the margin of indifference, however, and thus meets the criterion of being noninferior. Trial C is also noninferior, but the point estimate of the effect is slightly in favor of the control. Trial D does not conclusively show superiority or noninferiority, probably because it is too small or there were other factors that led to low power. Trial E indicates inferiority for the new intervention.

As discussed above, the investigator must consider several issues when designing an equivalence or noninferiority trial. First, the constancy assumption that the control versus placebo effect has not changed over time is often not correct. This can be seen, for example, in two trials of the same design conducted back to back with essentially the same protocol and investigators, the PRAISE-1 and PRAISE-2 trials [57, 58] discussed in the section on Historical Controls and Databases. In PRAISE-1, the trial was stratified according to etiology, ischemic and non-ischemic heart failure. Most of the favorable effect of the drug on mortality was seen in the nonischemic stratum, contrary to expectation. To validate that subgroup result, PRAISE-2 was conducted in non-ischemic heart failure patients using the same design. In this second trial, no benefit of amlodipine was observed. The comparison of the placebo arms from PRAISE-1 and PRAISE-2 (Fig. 5.1), indicates that the two populations of nonischemic heart failure patients were at substantially different risk, despite being enrolled close in time, with the same entry criteria and same investigators. No covariate analysis could explain this difference in risk. Thus, the enrolled population itself is not constant, challenging the constancy assumption.

In addition, as background therapy changes, the effect of the control or placebo may also change. With more therapeutic options, the effect of one drug or intervention alone may no longer be as large as it was when placebo was the total background. Practice and referral patterns change.

Even if the data from prior trials of the selected control are available, the estimates of active control vs. placebo may not be completely accurate. As with all trials, effect of treatment depends at least partly on the sample of participants who were identified and volunteered for the study. The observed effect is not likely to reflect the effect exactly in some other population. It is also possible that the quality of the trials used to obtain the effect of the control may not have been very good. And of course, the play of chance may have affected the observed benefit.

Many of the assumptions about the active control group event rates that go into the design of a noninferiority or equivalence trial are unlikely to be valid. At the end of the trial, investigators obtain seemingly more precise estimates of the margin and imputed “efficacy,” when in fact they are based on a model that has considerable uncertainty and great care must be used in interpreting the results.

If I is the new intervention, C is the control or standard treatment, and P is placebo or not treatment, for the usual superiority trial, the goal is to show that the new intervention is better than placebo or no treatment, or that new intervention plus control is better than control alone.

For noninferiority trials, the margin of indifference, δ, is specified, where I-C < δ. Efficacy imputation requires an estimate of the relative risk (RR) of the new intervention to control, RR(I/C) and of the control to placebo or no treatment, RR(C/P). Therefore, the estimated relative risk of the new intervention compared with placebo is

Rather than focus on the above assumption-filled model, an alternative approach might be considered. The first goal is to select the best control. This might be the one that, based on prior trials, was most effective. It might also be the one that the academic community considers as the standard of care, the one recommended in treatment guidelines, or the treatment that is most commonly used in practice. The selection will depend on the nature of the question being posed in the new trial. There might also be several possible best controls, all considered to be similar, as, for example, one of several beta blockers or statins. The choice might be influenced by regulatory agencies. The margin of noninferiority should use the data from the prior trials of the active control to get some estimate for initiating discussion but should not use it as a precise value. Once that estimate has been obtained, investigators, with input from others, including, as appropriate, those from regulatory agencies, should use their experience and clinical judgment to make a final determination as to what margin of noninferiority would support using a new intervention. These decisions depend on factors such as the severity of the condition being studied, the known risks of the standard or control intervention, the trade-offs that might be achieved with the new intervention, whether it is 50% or 20%, or some other relative risk, or an absolute difference, and the practicality of obtaining the estimated sample size. Having set the margin, effort must be on conducting the best trial, with as high participant adherence and complete follow-up as feasible. When the noninferiority trial has been completed, the attention should be given to the interpretation of trial results, keeping in mind the entirety of the research using the new intervention and the active control and the relevance of the findings to the specific clinical practice setting (see Chaps. 18 and 20).

Adaptive Designs

There is a great deal of interest in designs which are termed adaptive, but there are different designs that are adaptive and have different meanings of the term. Clinical trials have used forms of adaptive designs for many years. As discussed in Chap. 1, early phase studies have designs that allow for modifications as the data accrue. Many late phase trials are adaptive in the sense that the protocol allows for modification of the intervention in order to achieve a certain goal, typically using an interim variable. For example, trials of antihypertensive agents, with the primary response variable of stroke or heart disease, will allow, and even encourage, changes in dose of the agent, or addition or substitution of agent in order to reach a specified blood pressure reduction or level. A trial in people with depression changed antidepression drugs based on interim success or lack of success as judged by depression questionnaires [150]. Some have proposed re-randomizing either all participants or those failing to respond adequately to the first drug to other agents [151, 152].

Some trials, by design, will adjust the sample size to retain a desired power if the overall event rate is lower than expected, the variability is higher than planned, or adherence is worse than expected. In such cases, the sample size can be recalculated using the updated information (see Chap. 8). An event-driven adaptive design continues until the number of events thought necessary to reach statistical significance, given the hypothesized intervention effect, accumulates. In trials where time to event is the outcome of interest, the length of follow-up or the number of study participants, or both, may be increased in order to obtain the predetermined number of outcome events, In other adaptive designs, the randomization ratio may be modified to keep the overall balance between intervention and control arms level on some risk score (see Chap. 6).

Various designs are called response adaptive. Traditionally, if the effect of the intervention was less than expected, or other factors led to a less than desirable conditional power, the study either continued to the end without providing a clear answer or was stopped early for futility (see Chap. 17). Some studies, particularly where the outcome occurred relatively quickly, allowed for modification of the randomization ratio between intervention and control arm, depending on the response of the most recent participant or responses of all accumulated participants.

Because of concerns about inefficiencies in study design, several trend adaptive approaches have been developed. At the beginning of the trial, the investigator may have inadequate information about the rate at which the outcome variable will occur and be unable to make a realistic estimate of the effect of the intervention. Rather than continue to conduct an inappropriately powered trial or terminate early an otherwise well designed study, the investigator may wish to modify the sample size. After a trial is underway and better estimates become available, these trend adaptive approaches adjust sample size based on the observed trend in the primary outcome, in order to maintain the desired power. Trend adaptive designs require some adjustment of the analysis to assess properly the significance of the test statistic. A criticism of these designs had been that they can introduce bias during the implementation of the adjustment. Some newer approaches, however, now allow for modifying sample size based on observed trends [153, 154]. They may also, however, provide sufficient information to allow people not privy to the accumulating data to make reasonable guesses as to the trend. See Chap. 18 for a further discussion of these methods.

Group sequential designs, in common use for many years, are also considered to be response adaptive in that they facilitate early termination of the trial when there is convincing evidence of benefit or harm. Response adaptive and trend adaptive designs will be considered further in Chaps. 17 and 18.

References

1.

Fisher RA. Statistical Methods for Research Workers. Edinburgh: Oliver and Boyd, 1925.MATH

2.

Fisher RA. The Design of Experiments. Edinburgh: Oliver and Boyd, 1935.

3.

Cochran WG, Cox GM Experimental Designs (2nd edition). New York: John Wiley and Sons, 1957.

4.

Cox DR. Planning of Experiments. New York: John Wiley and Sons, 1958.MATH

5.

Bull JP. The historical development of clinical therapeutic trials. J Chronic Dis 1959;10:218–248.

6.

Eliot MM. The control of rickets: preliminary discussion of the demonstration in New Haven. JAMA 1925;85:656–663.

7.

Hill AB. Observation and experiment. N Engl J Med 1953;248:995–1001.

8.

Macfarlane G. Howard Florey: The Making of a Great Scientist. Oxford: Oxford University Press, 1979, pp11–12.

9.

Gocke DJ. Fulminant hepatitis treated with serum containing antibody to Australia antigen. N Engl J Med 1971;284:919.

10.

Acute Hepatic Failure Study Group. Failure of specific immunotherapy in fulminant type B hepatitis. Ann Intern Med 1977;86:272–277.

11.

Snow JB Jr, Kimmelman CP. Assessment of surgical procedures for Ménière’s disease. Laryngoscope 1979;89:737–747.

12.

Armitage P, Berry G, Matthews JNS. Statistical Methods in Medical Research (4th edition). Malden, MA: Blackwell Publishing, 2002.

13.

Brown BW, Hollander M. Statistics: A Biomedical Introduction. New York: John Wiley and Sons, 1977.MATH

14.

Feinstein AR. Clinical Biostatistics. St Louis: The C.V. Mosby Company, 1977.

15.

MacMahon B, Trichopoulos D. Epidemiology: Principles and Methods (2nd edition). Lippincott Williams & Wilkins, 1996.

16.

Lilienfeld DE, Stolley PD. Foundations of Epidemiology (3rd edition). New York: Oxford University Press, 1994.

17.

Srivastava JN (ed.). A Survey of Statistical Design and Linear Models. Amsterdam: North-Hollard, 1975.

18.

Peto R, Pike MC, Armitage P, et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. 1. Introduction and design. Br J Cancer 1976;34:585–612.

19.

Brown BW Jr. Statistical controversies in the design of clinical trials—some personal views. Control Clin Trials 1980;1:13–27.

20.

Pocock SJ. Allocation of patients to treatment in clinical trials. Biometrics 1979;35:183–197.

21.

Brown BW Jr. The crossover experiment for clinical trials. Biometrics 1980;36:69–79.MATH

22.

Hennekens CH, Buring JC. Epidemiology in Medicine. SL Mayrent (ed.). Boston: Little, Brown, 1987.

23.

Byar DP. Some statistical considerations for design of cancer prevention trials. Prev Med 1989;18:688–699.

24.

Geller NL (ed.). Advances in Clinical Trial Biostatistics. New York: Marcel Dekker, 2003.MATH

25.

Piantadosi S. Clinical Trials: A Methodologic Perspective (2nd edition). New York: John Wiley and Sons, 2005.

26.

Machin D, Day S, Green S. Textbook of Clinical Trials (2nd edition). West Sussex: John Wiley and Sons, 2006.

27.

Green S, Benedetti J, Crowley J. Clinical Trials in Oncology (3rd edition). Boca Raton: CRC Press, 2012.

28.

Hulley SB, Cummings SR, Browner WS, et al. Designing Clinical Research (4th edition). New York: Wolters Kluwer/Lippincott Williams & Wilkins, 2013.

29.

Meinert CL. Clinical Trials: Design, Conduct, and Analysis (2nd edition). New York: Oxford University Press, 2012.

30.

Cook TD, DeMets DL (eds). Introduction to Statistical Methods for Clinical Trials. Boca Raton: Chapman & Hall/CRC, Taylor & Francis Group, LLC, 2008.

31.

Chow S-C, Shao J. Statistics in Drug Research: Methodologies and Recent Developments. New York: Marcel Dekker, 2002.

32.

Green SB, Byar DP. Using observational data from registries to compare treatments: the fallacy of omnimetrics. Stat Med 1984;3:361–373.

33.

Gehan EA, Freireich EJ. Non-randomized controls in cancer clinical trials. N Engl J Med 1974;290:198–203.

34.

Weinstein MC. Allocation of subjects in medical experiments. N Engl J Med 1974;291:1278–1285.

35.

Byar DP, Simon RM, Friedewald WT, et al. Randomized clinical trials: perspectives on some recent ideas. N Engl J Med 1976;295:74–80.

36.

Sapirstein W, Alpert S, Callahan TJ. The role of clinical trials in the Food and Drug Administration approval process for cardiovascular devices. Circulation 1994;89:1900–1902.

37.

Hlatky MA. Perspective: Evidence-based use of cardiac procedures and devices. N Engl J Med 2004;350:2126–2128.

38.

AMPLATZER® Septal Occluder. http://​www.​fda.​gov/​MedicalDevices/​ProductsandMedic​alProcedures/​DeviceApprovalsa​ndClearances/​Recently-ApprovedDevices/​ucm083978.​htm

39.

St. Jude Amplatzer Atrial Septal Occluder (ASO): Safety communication—reports of tissue erosion. http://​www.​fda.​gov/​safety/​medwatch/​safetyinformatio​n/​safetyalertsforh​umanmedicalprodu​cts/​ucm371202.​htm

40.

Chalmers TC, Matta RJ, Smith H, Kunzier AM. Evidence favoring the use of anticoagulants in the hospital phase of acute myocardial infarction. N Engl J Med 1977;297:1091–1096.

41.

Peto R. Clinical trial methodology. Biomedicine (Special issue) 1978;28:24–36.

42.

Goldman L, Feinstein AR. Anticoagulants and myocardial infarction: the problems of pooling, drowning, and floating. Ann Intern Med 1979;90:92–94.

43.

Grace ND, Muench H, Chalmers TC. The present status of shunts for portal hypertension in cirrhosis. Gastroenterology 1966;50:684–691.

44.

Sacks H, Chalmers TC, Smith H Jr. Randomized versus historical controls for clinical trials. Am J Med 1982;72:233–240.

45.

Sacks HS, Chalmers TC, Smith H Jr. Sensitivity and specificity of clinical trials: randomized v historical controls. Arch Intern Med 1983;143:753–755.

46.

Chalmers TC, Celano P, Sacks HS, Smith H Jr. Bias in treatment assignment in controlled clinical trials. N Engl J Med 1983;309:1358–1361.

47.

Ingelfinger FJ. The randomized clinical trial (editorial). N Engl J Med 1972;287:100–101.

48.

Zelen M. A new design for randomized clinical trials. N Engl J Med 1979;300:1242–1245.

49.

Anbar D. The relative efficiency of Zelen’s prerandomization design for clinical trials. Biometrics 1983;39:711–718.

50.

Ellenberg SS. Randomization designs in comparative clinical trials. N Engl J Med 1984;310:1404–1408.

51.

Zelen M. Randomized consent designs for clinical trials: an update. Stat Med 1990;9:645–656.

52.

Gehan EA. The evaluation of therapies: historical control studies. Stat Med 1984;3:315–324.

53.

Lasagna L. Historical controls: the practitioner’s clinical trials. N Engl J Med 1982;307:1339–1340.

54.

Moertel CG. Improving the efficiency of clinical trials: a medical perspective. Stat Med 1984;3:455–465.

55.

Pocock SJ. Letter to the editor. Br Med J 1977;1:1661.

56.

Veterans Administration Cooperative Urological Research Group. Treatment and survival of patients with cancer of the prostate. Surg Gynecol Obstet 1967;124:1011–1017.

57.

Packer M, O’Connor CM, Ghali JK, et al. for the Prospective Randomized Amlodipine Survival Evaluation Study Group. Effect of amlodipine on morbidity and mortality in severe chronic heart failure. N Engl J Med 1996;335:1107–1114.

58.

Packer M, Carson P, Elkayam U, et al. Effect of amlodipine on the survival of patients with severe chronic heart failure due to a nonischemic cardiomyopathy. JACC:Heart Failure 2013;1:308–314.

59.

Havlik RJ, Feinleib M (eds.). Proceedings of the Conference on the Decline in Coronary Heart Disease Mortality. Washington, D.C.: NIH Publication No. 79-1610, 1979.

60.

Health, United States, 2011, With Special Feature on Socioeconomic Status and Health. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics. http://​www.​cdc.​gov/​nchs/​data/​hus/​hus11.​pdf, page 32, figure 3.

61.

Health, United States, 2008, With Special Feature on the Health of Young Adults. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics. http://​www.​cdc.​gov/​nchs/​data/​hus/​hus08.​pdf, page 37, figure 9.

62.

Coronary Drug Project Research Group. Clofibrate and niacin in coronary heart disease. JAMA 1975;231:360–381.

63.

Castro KG, Ward JW, Slutsker L, et al. 1993 revised classification system for HIV infection and expanded surveillance case definitions for AIDS among adolescents and adults. MMWR Recomm Rep, December 18, 1992.

64.

Current trends update: trends in AIDS diagnosis and reporting under the expanded surveillance definition for adolescents and adults—United States, 1993. MMWR Weekly 1994;43:826–831.

65.

Rosenberg HM, Klebba AJ. Trends in cardiovascular mortality with a focus on ischemic heart disease: United States, 1950-1976. In Havlik R, Feinleib M (eds). Proceedings of the Conference on the Decline in Coronary Heart Disease Mortality. Washington, D.C.: NIH Publication No. 79-1610, 1979.

66.

Morbidity and Mortality Chartbook on Cardiovascular, Lung, and Blood Diseases. National Heart, Lung, and Blood Institute, U.S. Department of Health and Human Services, Public Health Service. May 1994.

67.

Centers for Disease Control and Prevention. International Classification of Diseases, (ICD-10-CN/PCS) Transition. http://​www.​cdc.​gov/​nchs/​icd/​icd10cm_​pcs_​impact.​htm

68.

Bailar JC III, Louis TA, Lavori PW, Polansky M. Studies without internal controls. N Engl J Med 1984;311:156–162.

69.

Dustan HP, Schneckloth RE, Corcoran AC, Page IH. The effectiveness of long-term treatment of malignant hypertension. Circulation 1958;18:644–651.

70.

Bjork S, Sannerstedt R, Angervall G, Hood B. Treatment and prognosis in malignant hypertension: clinical follow-up study of 93 patients on modern medical treatment. Acta Med Scand 1960;166:175–187.

71.

Bjork S, Sannerstedt R, Falkheden T, Hood B. The effect of active drug treatment in severe hypertensive disease: an analysis of survival rates in 381 cases on combined treatment with various hypotensive agents. Acta Med Scand 1961;169:673–689.

72.

Starmer CF, Lee KL, Harrell FE, Rosati RA. On the complexity of investigating chronic illness. Biometrics 1980;36:333–335.

73.

Hlatky MA, Lee KL, Harrell FE Jr, et al. Tying clinical research to patient care by use of an observational database. Stat Med 1984;3:375–387.

74.

Hlatky MA, Califf RM, Harrell FE Jr, et al. Clinical judgment and therapeutic decision making. J Am Coll Cardiol 1990;15:1–14.

75.

Moon TE, Jones SE, Bonadonna G, et al. Using a database of protocol studies to evaluate therapy: a breast cancer example. Stat Med 1984;3:333–339.

76.

Anderson C. Measuring what works in health care. Science 1994;263:1080–1082.

77.

Klungel OH, Heckbert SR, Longstreth WT, et al. Antihypertensive drug therapies and the risk of ischemic stroke. Arch Intern Med 2001;161:37–43.

78.

Graham DJ, Campen D, Hui R, et al. Risk of acute myocardial infarction and sudden cardiac death in patients treated with cyclo-oxygenase 2 selective and non-selective non-steroidal anti-inflammatory drugs: nested case-control study. Lancet 2005;365:475–481.

79.

Byar, DP. Why databases should not replace randomized clinical trials. Biometrics 1980;36:337–342.

80.

Dambrosia JM, Ellenberg JH. Statistical considerations for a medical database. Biometrics 1980;36:323–332.

81.

Sheldon TA. Please bypass the PORT. Br Med J 1994;309:142–143.

82.

Mantel N. Cautions on the use of medical databases. Stat Med 1983;2:355–362.

83.

Lauer MS, D’Agostino RB, Sr. The randomized registry trial—the next disruptive technology in clinical research? N Engl J Med 2013;369:1579–1581.

84.

Frӧbert O, Lagerqvist B, Olivecrona GK, et al. Thrombus aspiration during ST-elevation myocardial infarction. N Engl J Med 2013:369:1587–1597; correction. N Engl J Med 2014;371:786.

85.

Carriere KC. Crossover designs for clinical trials. Stat Med 1994;13:1063–1069.

86.

Koch GG, Amara IA, Brown BW Jr, et al. A two-period crossover design for the comparison of two active treatments and placebo. Stat Med 1989;8:487–504.

87.

Fleiss JL. A critique of recent research on the two treatment crossover design. Control Clin Trials 1989;10:237–243.

88.

Woods JR, Williams JG, Tavel M. The two-period crossover design in medical research. Ann Intern Med 1989;110:560–566.

89.

Louis TA, Lavori PW, Bailar JC III, Polansky M. Crossover and self-controlled designs in clinical research. N Engl J Med 1984;310:24–31.

90.

James KE, Forrest WH, Jr, Rose RL. Crossover and noncrossover designs in four-point parallel line analgesic assays. Clin Pharmacol Ther 1985;37:242–252.

91.

Mills EJ, Chan A-W, Wu P, et al. Design, analysis, and presentation of crossover trials. Trials 2009;10:27 doi:10.​1186/​1745-6215-10-27.

92.

International Conference on Harmonisation: E9 Statistical principles for clinical trials. http://​www.​fda.​gov/​downloads/​RegulatoryInform​ation/​Guidances/​UCM129505.​pdf.

93.

Grizzle JE. The two period change-over design and its use in clinical trials. Biometrics 1965;21:467–480.

94.

Fleiss JL, Wallenstein S, Rosenfeld R. Adjusting for baseline measurements in the two-period crossover study: a cautionary note. Control Clin Trials 1985;6:192–197.

95.

Hills M, Armitage P. The two-period cross-over clinical trial. Br J Clin Pharmacol 1979;8:7–20.

96.

Hypertension Detection and Follow-up Program Cooperative Group. Five-year findings of the Hypertension Detection and Follow-Up Program. 1. Reduction in mortality of persons with high blood pressure, including mild hypertension. JAMA 1979;242:2562–2571.

97.

Stamler R, Stamler J, Grimm R, et al. Nutritional therapy for high blood pressure-Final report of a four-year randomized controlled trial—The Hypertension Control Program. JAMA 1987;257:1484–1491.

98.

Magnussen H, Disse B, Rodriguez-Roisin R, et al. Withdrawal of inhaled glucocorticoids and exacerbations of COPD. N Engl J Med 2014;371:1285–1294.

99.

Report of the Sixty Plus Reinfarction Study Research Group. A double-blind trial to assess long-term oral anticoagulant therapy in elderly patients after myocardial infarction. Lancet 1980;316:989–994.

100.

Kasiske BL, Chakkera HA, Louis TA, Ma JZ. A meta-analysis of immunosuppression withdrawal trials in renal transplantation. J Am Soc Nephrol 2000;11:1910–1917.

101.

Black DM, Schwartz AV, Ensrud KE, et al, for the FLEX Research Group. Effects of continuing or stopping alendronate after 5 years of treatment: The Fracture Intervention Trial Long-term Extension (FLEX): a randomized trial. JAMA 2006;296:2927–2938.

102.

Montgomery AA, Peters TJ, Little P. Design, analysis and presentation of factorial randomized controlled trials. BMC Med Res Methodol 2003;3:26doi:10.​1186/​1471-2288-3-26.

103.

The Canadian Cooperative Study Group. A randomized trial of aspirin and sulfinpyrazone in threatened stroke. N Engl J Med 1978;299:53–59.

104.

ISIS-3 (Third International Study of Infarct Survival) Collaborative Group. ISIS-3: a randomized study of streptokinase vs plasminogen activator vs anistrephase and of aspirin plus heparin vs aspirin alone among 41,299 cases of suspected acute myocardial infarction. Lancet 1992;339:753–770.

105.

Stampfer MJ, Buring JE, Willett W, et al. The 2 x 2 factorial design: its application to a randomized trial of aspirin and carotene in U.S. physicians. Stat Med 1985;4:111–116.

106.

Design of the Women’s Health Initiative clinical trial and observational study. The Women’s Health Initiative Study Group. Control Clin Trials 1998;19:61–109.

107.

McAlister FA, Straus SE, Sackett DL, Altman DG. Analysis and reporting of factorial trials: a systematic review. JAMA 2003;289:2545–2553.

108.

Byar DP, Herzberg AM, Tan W-Y. Incomplete factorial designs for randomized clinical trials. Stat Med 1993;12:1629–1641.

109.

Action to Control Cardiovascular Risk in Diabetes Study Group. Effects of intensive glucose lowering in Type 2 diabetes. N Engl J Med 2008;358:2545–2559.

110.

Fletcher DJ, Lewis SM, Matthews JNS. Factorial designs for crossover clinical trials. Stat Med 1990;9:1121–1129.

111.

Writing Group for the Women’s Health Initiative Investigators. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women’s Health Initiative randomized controlled trial. JAMA 2002;288:321–333.

112.

The Women’s Health Initiative Steering Committee. Effects of conjugated equine estrogen in postmenopausal women with hysterectomy. JAMA 2004;291:1701–1712.

113.

Brittain E, Wittes J. Factorial designs in clinical trials: the effects of non-compliance and subadditivity. Stat Med 1989;8:161–171.

114.

Hayes RJ, Moulton LH. Cluster Randomized Trials: A Practical Approach. Chapman & Hall/CRC, Taylor & Francis Group, 2009.

115.

Donner A, Birkett N, Buck C. Randomization by cluster: sample size requirements and analysis. Am J Epidemiol 1981;114:906–914.

116.

Armitage P. The role of randomization in clinical trials. Stat Med 1982;1:345–352.

117.

Simon R. Composite randomization designs for clinical trials. Biometrics 1981;37:723–731.MATH

118.

Cornfield J. Randomization by group: a formal analysis. Am J Epidemiol 1978;108:100–102.

119.

Zucker DM, Lakatos E, Webber LS, et al. Statistical design of the Child and Adolescent Trial for Cardiovascular Health (CATCH): implications of cluster randomization. Control Clin Trials 1995;16:96–118.

120.

Vijayaraghavan K, Radhaiah G, Prakasam BS, et al. Effect of massive dose vitamin A on morbidity and mortality in Indian children. Lancet 1990;336:1342–1345.

121.

Luepker RV, Raczynski JM, Osganian S, et al. Effect of a community intervention on patient delay and emergency medical service use in acute coronary heart disease: the Rapid Early Action for Coronary Treatment (REACT) trial. JAMA 2000;284:60–67.

122.

Farquhar JW, Fortmann SP, Flora JA, et al. Effects of community-wide education on cardiovascular disease risk factors. The Stanford Five-City Project. JAMA 1990;264:359–365.

123.

Gail MH, Byar DP, Pechacek TF, Corle DK, for COMMIT Study Group. Aspects of statistical design for the Community Intervention Trial for Smoking Cessation. Control Clin Trials 1992;13:6–21.

124.

Sismanidis C, Moulton LH, Ayles H, et al. Restricted randomization of ZAMSTAR: a 2x2 factorial cluster randomized trial. Clin Trials 2008;5:316–327.

125.

Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials 2007;28:182–191.

126.

Woertman W, de Hoop E, Moerbeek M, et al. Stepped wedge designs could reduce the required sample size in cluster randomized trials. J Clin Epidemiol 2013;66:752–758.

127.

Pocock SJ. The combination of randomized and historical controls in clinical trials. J Chronic Dis 1976;29:175–188.

128.

Machin D. On the possibility of incorporating patients from nonrandomising centres into a randomised clinical trial. J Chronic Dis 1979;32:347–353.

129.

Gruppo Italiano per lo Studio della Streptochinasi nell’ Infarto Miocardico (GISSI). Effectiveness of intravenous thrombolytic treatment in acute myocardial infarction. Lancet 1986;i:397–402.

130.

The GUSTO Investigators. An international randomized trial comparing four thrombolytic strategies for acute myocardial infarction. N Engl J Med 1993;329:673–682; correction N Engl J Med 1994;331:277.

131.

The Digitalis Investigation Group. Rationale, design, implementation, and baseline characteristics of patients in the DIG Trial: a large, simple, long-term trial to evaluate the effect of digitalis on mortality in heart failure. Control Clin Trials 1996;17:77–97.

132.

MICHELANGELO OASIS 5 Steering Committee. Design and rationale of the MICHELANGELO Organization to Assess Strategies in Acute Ischemic Syndromes (OASIS)-5 trial program evaluating fondaparinux, a synthetic factor Xa inhibitor, in patients with non-ST-segment elevation acute coronary syndromes. Am Heart J 2005;150:1107–1114.

133.

Tunis SR, Stryer DB, Clancy CM. Practical clinical trials: increasing the value of clinical research for decision making in clinical and health policy. JAMA 2003;290:1624–1632.

134.

March JS, Silva SG, Compton S, et al. The case for practical clinical trials in psychiatry. Am J Psychiatry 2005;162:836–846.

135.

Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol 2009;62:464–475.

136.

Johnson KE, Tachibana C, Coronado GD, et al. Research Methods & Reporting: A guide to research partnerships for pragmatic trials. BMJ 2014;349:g6826 doi:10.​1136/​bmj.​g6826.

137.

Mailankody S, Prasad V. Perspective: Comparative effectiveness questions in oncology. N Engl J Med 2014; 370:1478–1481.

138.

Ross JS, Slodkowska EA, Symmans WF, et al. The HER-2 receptor and breast cancer: ten years of targeted anti-HER-2 therapy and personalized medicine. Oncologist 2009;14:320–368.

139.

Lai TL, Lavori PW, Shih MC, Sikic BI. Clinical trial designs for testing biomarker-based personalized therapies. Clin Trials 2012;9:141–154.

140.

The CATT Research Group. Ranibizumab and bevacizumab for neovascular age-related macular degeneration. N Engl J Med 2011;364:1897–1908.

141.

Blackwelder WC. “Proving the null hypothesis” in clinical trials. Control Clin Trials 1982;3:345–353.

142.

Hung JHM, Wang SJ, Tsong Y, et al. Some fundamental issues with non-inferiority testing in active controlled trials. Stat Med 2003;30:213–225.

143.

Fleming TR. Current issues in non-inferiority trials. Stat Med 2008;27:317–332.MathSciNet

144.

D’Agostino RB Sr, Massaro JM, Sullivan LM. Non-inferiority trials: design concepts and issues—the encounters of academic consultants in statistics. Stat Med 2003;22:169–186.

145.

Kaul S, Diamond GA. Making sense of noninferiority: a clinical and statistical perspective on its application to cardiovascular clinical trials. Prog Cardiovasc Dis 2007;49:284–299.

146.

Mulla SM, Scott IA, Jackevicius CA, You JJ, Guyatt GH. How to use a noninferiority trial. JAMA 2012;308:2605–2611.

147.

Schumi J, Wittes JT. Through the looking glass: understanding non-inferiority. Trials 2011;12:106 doi:10.​1186/​1745-6215-12-106.

148.

DeMets DL, Friedman L. Some thoughts on challenges for noninferiority study designs. Therapeutic Innovation & Regulatory Science 2012;46:420–427.

149.

SPORTIF Executive Steering Committee for the SPORTIF V Investigators. Ximelagatran vs warfarin for stroke prevention in patients with nonvalvular atrial fibrillation: a randomized trial. JAMA 2005;293:690–698.

150.

Trivedi MH, Rush AJ, Wisniewski SR, et al, for the STAR*D Study Team. Evaluation of outcomes with citalopram for depression using measurement-based care in STAR*D: implications for clinical practice. Am J Psychiatry 2006;163:28–40.

151.

Murphy SA, Oslin DW, Rush AJ, Zhu J, for MCATS. Methodological challenges in constructing effective treatment sequences for chronic psychiatric disorders (Perspective). Neuropsychopharmacology 2007;32:257–262.

152.

Lavori PW, Dawson D. Improving the efficiency of estimation in randomized trials of adaptive treatment strategies. Clin Trials 2007;4:297–308.

153.

Levin GP, Emerson SC, Emerson SS. Adaptive clinical trial designs with prespecified rules for modifying the sample size: understanding efficient types of adaptation. Stat Med 2013;32:1259–1275.MathSciNet

154.

Mehta C. Adaptive clinical trial designs with pre-specified rules for modifying the sample size: a different perspective. Stat Med 2013;32:1276–1279.MathSciNet

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_6

6. The Randomization Process

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

The randomized controlled clinical trial is the standard by which all trials are judged. In the simplest case, randomization is a process by which each participant has the same chance of being assigned to either intervention or control. An example would be the toss of a coin, in which heads indicates intervention group and tails indicates control group. Even in the more complex randomization strategies, the element of chance underlies the allocation process. Of course, neither trial participant nor investigator should know what the assignment will be before the participant’s decision to enter the study. Otherwise, the benefits of randomization can be lost. The role that randomization plays in clinical trials has been discussed in Chap. 5 as well as by numerous authors [1–12]. While not all accept that randomization is essential [10, 11], most agree it is the best method for achieving comparability between study groups, and the most appropriate basis for statistical inference [1, 3].

Fundamental Point

Randomization tends to produce study groups comparable with respect to known as well as unknown risk factors, removes investigator bias in the allocation of participants, and guarantees that statistical tests will have valid false positive error rates.

Several methods for randomly allocating participants are used [6, 9, 12–14]. This chapter will present the most common of these methods and consider the advantages and disadvantages of each. Unless stated otherwise, it can be assumed that the randomization strategy will allocate participants into two groups, an intervention group and a control group. However, many of the methods described here can easily be generalized for use with more than two groups.

Two forms of experimental bias are of concern. The first, selection bias, occurs if the allocation process is predictable [5, 15–18]. In this case, the decision to enter a participant into a trial may be influenced by the anticipated treatment assignment. If any bias exists as to what treatment particular types of participants should receive, then a selection bias might occur. All of the randomization procedures described avoid selection bias by not being predictable. A second bias, accidental bias, can arise if the randomization procedure does not achieve balance on risk factors or prognostic covariates. Some of the allocation procedures described are more vulnerable to accidental bias, especially for small studies. For large studies, however, the chance of accidental bias is negligible [5].

Whatever randomization process is used, the report of the trial should contain a brief, but clear description of that method. In the 1980s, Altman and Doré [15] reported a survey of four medical journals where 30% of published randomized trials gave no evidence that randomization had in fact been used. As many as 10% of these “randomized” trials in fact used non-random allocation procedures. Sixty percent did not report the type of randomization that was used. In one review in the 1990s, only 20–30% of trials provided fair or adequate descriptions, depending on the size of the trial or whether the trial was single center or multicenter [18]. More recently, a review of 253 trials published in five major medical journals after the release of the Consolidated Standards for Reporting Trials (CONSORT) [19] recommendations found little improvement in reports of how randomization was accomplished [20]. Descriptions need not be lengthy to inform the reader, publications should clearly indicate the type of randomization method and how the randomization was implemented.

Fixed Allocation Randomization

Fixed allocation procedures assign the interventions to participants with a prespecified probability, usually equal, and that allocation probability is not altered as the study progresses. A number of methods exist by which fixed allocation is achieved [6, 9, 12, 14, 21–25], and we will review three of these—simple, blocked, and stratified.

Our view is that allocation to intervention and control groups should be equal unless there are compelling reasons to do otherwise. Peto [7] among others, has suggested an unequal allocation ratio, such as 2:1, of intervention to control. The rationale for such an allocation is that the study may slightly lose sensitivity but may gain more information about participant responses to the new intervention, such as toxicity and side effects. In some instances, less information may be needed about the control group and, therefore, fewer control participants are required. If the intervention turns out to be beneficial, more study participants would benefit than under an equal allocation scheme. However, new interventions may also turn out to be harmful, in which case more participants would receive them under the unequal allocation strategy. Although the loss of sensitivity or power may be less than 5% for allocation ratios approximately between 1/2 and 2/3 [8, 21], equal allocation is the most powerful design and therefore generally recommended. We also believe that equal allocation is more consistent with the view of indifference or equipoise toward which of the two groups a participant is assigned (see Chap. 2). Unequal allocation may indicate to the participants and to their personal physicians that one intervention is preferred over the other. In a few circumstances, the cost of one treatment may be extreme so that an unequal allocation of 2:1 or 3:1 may help to contain costs while not causing a serious loss of power. Thus, there are tradeoffs that must be considered. In general, equal allocation will be presumed throughout the following discussion unless otherwise indicated.

Simple Randomization

The most elementary form of randomization, referred to as simple or complete randomization, is best illustrated by a few examples [9, 12]. One simple method is to toss an unbiased coin each time a participant is eligible to be randomized. For example, if the coin turns up heads, the participant is assigned to group A; if tails, to group B. Using this procedure, approximately one half of the participants will be in group A and one half in group B. In practice, for small studies, instead of tossing a coin to generate a randomization schedule, a random digit table on which the equally likely digits 0 to 9 are arranged by rows and columns is usually used to accomplish simple randomization. By randomly selecting a certain row (column) and observing the sequence of digits in that row (column) A could be assigned, for example, to those participants for whom the next digit was even and B to those for whom the next digit was odd. This process produces a sequence of assignments which is random in order, and each participant has an equal chance of being assigned to A or B.

For large studies, a more convenient method for producing a randomization schedule is to use a random number producing algorithm, available on most computer systems. A simple randomization procedure might assign participants to group A with probability p and participants to group B with probability 1 − p. One computerized process for simple randomization is to use a uniform random number algorithm to produce random numbers in the interval from 0.0 to 1.0. Using a uniform random number generator, a random number can be produced for each participant. If the random number is between 0 and p, the participant would be assigned to group A; otherwise to group B. For equal allocation, the probability cut point, p, is one-half (i.e., p = 0.50). If equal allocation between A and B is not desired (p ≠ 1/2), then p can be set to the desired proportion in the algorithm and the study will have, on the average, a proportion p of the participants in group A.

This procedure can be adapted easily to more than two groups. Suppose, for example, the trial has three groups, A, B and C, and participants are to be randomized such that a participant has a 1/4 chance of being in group A, a 1/4 chance of being in group B, and a 1/2 chance of being in group C. By dividing the interval 0 to 1 into three pieces of length 1/4, 1/4, and 1/2, random numbers generated will have probabilities of 1/4, 1/4 and 1/2, respectively, of falling into each subinterval. Specifically, the intervals would be <0.25, 0.25–0.50, and ≥0.50. Then any participant whose random number is less than 0.25 is assigned A, any participant whose random number falls between 0.25 and 0.50 is assigned B and the others, C. For equal allocation, the interval would be divided into thirds and assignments made accordingly.

The advantage of this simple randomization procedure is that it is easy to implement. The major disadvantage is that, although in the long run the number of participants in each group will be in the proportion anticipated, at any point in the randomization, including the end, there could be a substantial imbalance [23]. This is true particularly if the sample size is small. For example, if 20 participants are randomized with equal probability to two treatment groups, the chance of a 12:8 split (i.e., 60% A, 40% B) or worse is approximately 50%. For 100 participants, the chance of the same ratio (60:40 split) or worse is only 5%. While such imbalances do not cause the statistical tests to be invalid, they do reduce ability to detect true differences between the two groups. In addition, such imbalances appear awkward and may lead to some loss of credibility for the trial, especially for the person not oriented to statistics. For this reason primarily, simple randomization is not often used, even for large studies. In addition, interim analysis of accumulating data might be difficult to interpret with major imbalances in number of participants per arm, especially for smaller trials.

Some investigators incorrectly believe that an alternating assignment of participants to the intervention and the control groups (e.g., ABABAB …) is a form of randomization. However, no random component exists in this type of allocation except perhaps for the first participant. A major criticism of this method is that, in a single-blind or unblinded study, the investigators know the next assignment, which could lead to a bias in the selection of participants. Even in a double-blind study, if the blind is broken on one participant as sometimes happens, the entire sequence of assignments is known. Therefore, this type of allocation method should be avoided.

Blocked Randomization

Blocked randomization, sometimes called permuted block randomization, was described by Hill [4] in 1951. It avoids serious imbalance in the number of participants assigned to each group, an imbalance which could occur in the simple randomization procedure. More importantly, blocked randomization guarantees that at no time during randomization will the imbalance be large and that at certain points the number of participants in each group will be equal [9, 12, 26]. This protects against temporal trends during enrollment, which is often a concern for larger trials with long enrollment phases.

If participants are randomly assigned with equal probability to groups A or B, then for each block of even size (for example, 4, 6 or 8) one half of the participants will be assigned to A and the other half to B. The order in which the interventions are assigned in each block is randomized, and this process is repeated for consecutive blocks of participants until all participants are randomized. For example, the investigators may want to ensure that after every fourth randomized participant, the number of participants in each intervention group is equal. Then a block of size 4 would be used and the process would randomize the order in which two A’s and two B’s are assigned for every consecutive group of four participants entering the trial. One may write down all the ways of arranging the groups and then randomize the order in which these combinations are selected. In the case of block size 4, there are six possible combinations of group assignments: AABB, ABAB, BAAB, BABA, BBAA, and ABBA. One of these arrangements is selected at random and the four participants are assigned accordingly. This process is repeated as many times as needed.

Another method of blocked randomization may also be used. In this method for randomizing the order of assignments within a block of size b, a random number between 0 and 1 for each of the b assignments (half of which are A and the other half B) is obtained. The example below illustrates the procedure for a block of size four (2As and 2Bs). Four random numbers are drawn between 0 and 1 in the order shown.

Assignment

Random number

Rank

A

0.069

1

A

0.734

3

B

0.867

4

B

0.312

2

The assignments then are ranked according to the size of the random numbers. This leads to the assignment order of ABAB. This process is repeated for another set of four participants until all have been randomized.

The advantage of blocking is that balance between the number of participants in each group is guaranteed during the course of randomization. The number in each group will never differ by more than b/2 when b is the length of the block. This can be important for at least two reasons. First, if the type of participant recruited for the study changes during the entry period, blocking will produce more comparable groups. For example, an investigator may use different sources of potential participants sequentially. Participants from these sources may vary in severity of illness or other crucial respects. One source, with the more seriously ill participants, may be used early during enrollment and another source, with healthier participants, late in enrollment [3]. If the randomization were not blocked, more of the seriously ill participants might be randomized to one group. Because the later participants are not as sick, this early imbalance would not be corrected. A second advantage of blocking is that if the trial should be terminated before enrollment is completed, balance will exist in terms of number of participants randomized to each group.

A potential, but solvable problem with basic blocked randomization is that if the blocking factor b is known by the study staff and the study is not double-blind, the assignment for the last person entered in each block is known before entry of that person. For example, if the blocking factor is 4 and the first three assignments are ABB, then the next assignment must be A. This could, of course, permit a bias in the selection of every fourth participant to be entered. Clearly, there is no reason to make the blocking factor known. However, in a study that is not double-blind, with a little ingenuity the staff can soon discover the blocking factor. For this reason, repeated blocks of size 2 should not be used. On a few occasions, perhaps as an intellectual challenge, investigators or their clinic staff have attempted to break the randomization scheme [27]. This curiosity is natural but nevertheless can lead to selection bias. To avoid this problem in the trial that is not double-blind, the blocking factor can be varied as the recruitment continues. In fact, after each block has been completed, the size of the next block could be determined in a random fashion from a few possibilities such as 2, 4, 6, and 8. The probabilities of selecting a block size can be set at whatever values one wishes with the constraint that their sum equals 1.0. For example, the probabilities of selecting block sizes 2, 4, 6, and 8 can be 1/6, 1/6, 1/3, and 1/3 respectively. Randomly selecting the block size makes it very difficult to determine where blocks start and stop and thus determine the next assignment.

A disadvantage of blocked randomization is that, from a strictly theoretical point of view, analysis of the data is more complicated than if simple randomization were used. Unless the data analysis performed at the end of the study reflects the randomization process actually performed [26, 28–30] it may be incorrect since standard analytical methods assume a simple randomization. In their analysis of the data most investigators ignore the fact that the randomization was blocked. Matts and Lachin [26] studied this problem and concluded that the measurement of variability used in the statistical analysis is not exactly correct if the blocking is ignored. Usually the analysis ignoring blocks is conservative, though it can be anticonservative especially when the blocks are small (e.g. a block size of two). That is, the analysis ignoring blocks will have probably slightly less power than the correct analysis, and understate the “true” significance level. Since blocking guarantees balance between the two groups and, therefore, increases the power of a study, blocked randomization with the appropriate analysis is more powerful than not blocking at all or blocking and then ignoring it in the analysis [26]. Also, the correct treatment of blocking would be difficult to extend to more complex analyses. Being able to use a single, straightforward analytic approach that handles covariates, subgroups, and other secondary analyses simplifies interpretation of the trial as a whole. Performing the most correct analysis is even more problematic for adaptive designs, as discussed in the next section.

Stratified Randomization

One of the objectives in allocating participants is to achieve between group comparability of certain characteristics known as prognostic or risk factors [12, 31–44]. These are baseline factors which correlate with subsequent participant response or outcome. Investigators may become concerned when prognostic factors are not evenly distributed between intervention and control groups. As indicated previously, randomization tends to produce groups which are, on the average, similar in their entry characteristics, known or unknown, or unmeasured. This is a concept likely to be true for large studies or for many small studies when averaged. For any single study, especially a small study, there is no guarantee that all baseline characteristics will be similar in the two groups. In the multicenter Aspirin Myocardial Infarction Study [45] which had 4,524 participants, the top 20 cardiovascular prognostic factors for total mortality identified in the Coronary Drug Project [43] were compared in the intervention and control groups and no major differences were found (Furberg CD, unpublished data). However, individual clinics, with an average of 150 participants, showed considerable imbalance for many variables between the groups. Imbalances in prognostic factors can be dealt with either after the fact by using stratification in the analysis (Chap. 18) or can be prevented by using stratification in the randomization. Stratified randomization is a method which helps achieve comparability between the study groups for those factors considered.

Stratified randomization requires that the prognostic factors be measured either before or at the time of randomization. If a single factor is used, it is divided into two or more subgroups or strata (e.g., age 30–34 years, 35–39 years, 40–44 years). If several factors are used, a stratum is formed by selecting one subgroup from each of them. The total number of strata is the product of the number of subgroups in each factor. The stratified randomization process involves measuring the level of the selected factors for a participant, determining to which stratum she belongs and performing the randomization within that stratum.

Within each stratum, the randomization process itself could be simple randomization, but in practice most clinical trials use some blocked randomization strategy. Under a simple randomization process, imbalances in the number in each group within the stratum could easily happen and thus defeat the purpose of the stratification. Blocked randomization is, as described previously, a special kind of stratification. However, this text will restrict use of the term blocked randomization to stratifying over time, and use stratified randomization to refer to stratifying on factors other than time. Some confusion may arise here because early texts on design used the term blocking as this book uses the term stratifying. However, the definition herein is consistent with current usage in clinical trials.

As an example of stratified randomization with a block size of 4, suppose an investigator wants to stratify on age, sex and smoking history. One possible classification of the factors would be three 10-year age levels and three smoking levels.

Age (years)

Sex

Smoking history

1. 40–49

Male

Current smoker

2. 50–59

Female

Ex-smoker

3. 60–69

Never smoked

Thus, the design has 3 × 2 × 3 = 18 strata. The randomization for this example appears in Table 6.1.

Table 6.1

Stratified randomization with block size of 4

Strata

Age

Sex

Smoking

Group assignment

1

40–49

M

Current

ABBA BABA…

2

40–49

M

Ex

BABA BBAA…

3

40–49

M

Never

etc.

4

40–49

F

Current

5

40–49

F

Ex

6

40–49

F

Never

7

50–59

M

Current

8

50–59

M

Ex

9

50–59

M

Never

10

50–59

F

Current

11

50–59

F

Ex

12

50–59

F

Never

(etc.)

Participants who were between 40 and 49 years old, male and current smokers, that is, in stratum 1, would be assigned to groups A or B in the sequences ABBA BABA .... Similarly, random sequences would appear in the other strata.

Small studies are the ones most likely to require stratified randomization, because in large studies, the magnitude of the numbers increases the chance of comparability of the groups. In the example shown above, with three levels of the first factor (age), two levels of the second factor (sex), and three levels of the third factor (smoking history), 18 strata have been created. As factors are added and the levels within factors are refined, the number of strata increase rapidly. If the example with 18 strata had 100 participants to be randomized, then only five to six participants would be expected per stratum if the study population were evenly distributed among the levels. Since the population is most likely not evenly distributed over the strata, some strata would actually get fewer than five to six participants. If the number of strata were increased, the number of participants in each stratum would be even fewer. Pocock and Simon [41] showed that increased stratification in small studies can be self-defeating because of the sparseness of data within each stratum. Thus, only important variables should be chosen and the number of strata kept to a minimum.

In addition to making the two study groups appear comparable with regard to specified factors, the power of the study can be increased by taking the stratification into account in the analysis. Stratified randomization, in a sense, breaks the trial down into smaller trials. Participants in each of the “smaller trials” belong to the same stratum. This reduces variability in group comparisons if the stratification is used in the analysis. Reduction in variability allows a study of a given size to detect smaller group differences in response variables or to detect a specified difference with fewer participants [22, 26].

Sometimes the variables initially thought to be most prognostic and, therefore used in the stratified randomization, turn out to be unimportant. Other factors may be identified later which, for the particular study, are of more importance. If randomization is done without stratification, then analysis can take into account those factors of interest and will not be complicated by factors thought to be important at the time of randomization. It has been argued that there usually does not exist a need to stratify at randomization because stratification at the time of analysis will achieve nearly the same expected power [7]. This issue of stratifying pre- versus post-randomization has been widely discussed [35–38, 42]. It appears for a large study that stratification after randomization provides nearly equal efficiency to stratification before randomization [39, 40]. However, for studies of 100 participants or fewer, stratifying the randomization using two or three prognostic factors may achieve greater power, although the increase may not be large.

Stratified randomization is not the complete solution to all potential problems of baseline imbalance. Another strategy for small studies with many prognostic factors is considered below in the section on adaptive randomization.

In multicenter trials, centers vary with respect to the type of participants randomized as well as the quality and type of care given to participants during follow-up. Thus, the center may be an important factor related to participant outcome, and the randomization process should be stratified accordingly [33]. Each center then represents, in a sense, a replication of the trial, though the number of participants within a center is not adequate to answer the primary question. Nevertheless, results at individual centers can be compared to see if trends are consistent with overall results. Another reason for stratification by center is that if a center should have to leave the study, the balance in prognostic factors in other centers would not be affected.

One further point might need consideration. If in the stratified randomization, a specific proportion or quota is intended for each stratum, the recruitment of eligible participants might not occur at the same rate. That is, one stratum might meet the target before the others. If a target proportion is intended, then plans need to be in place to close down recruitment for that stratum, allowing the others to be completed.

Adaptive Randomization Procedures

The randomization procedures described in the sections on fixed allocation above are non-adaptive strategies. In contrast, adaptive procedures change the allocation probabilities as enrollment progresses. Two types of adaptive procedures will be considered here. First, we will discuss methods which adjust or adapt the allocation probabilities according to imbalances in numbers of participants or in baseline characteristics between the two groups. Second, we will briefly review adaptive procedures that adjust allocation probabilities according to the responses of participants to the assigned intervention.

Baseline Adaptive Randomization Procedures

Two common methods for adaptive allocation which are designed to make the number of participants in each study group equal or nearly equal are biased coin randomization and urn randomization. Both make adaptations based only on the number of participants in each group, though they can be modified to perform allocation within strata in the same way as blocked randomization, and operate by changing the allocation probability over time.

The Biased Coin Randomization procedure, originally discussed by Efron [46], attempts to balance the number of participants in each treatment group based on the previous assignments, but does not take participant responses into consideration. Several variations to this approach have been discussed [47–63]. The purpose of the algorithm is basically to randomize the allocation of participants to groups A and B with equal probability as long as the number of participants in each group is equal or nearly equal. If an imbalance occurs and the difference in the number of participants is greater than some prespecified value, the allocation probability (p) is adjusted so that it is higher for the group with fewer participants. The investigator can determine the value of the allocation probability. The larger the value of p, the faster the imbalance will be corrected, while the nearer p is to 0.5, the slower the correction. Efron suggests an allocation probability of p = 2/3 when a correction is indicated. Since much of the time p is greater than 1/2, the process has been named the “biased coin” method. As a simple example, suppose n A and n B represent the number of participants in groups A and B respectively. If n A is less than n B and the difference exceeds a predetermined value, D, then we allocate the next participant to group A with probability p = 2/3. If n A is greater than n B by an amount of D, we allocate to group B with probability p = 2/3. Otherwise, p is set at 0.50. This procedure can be modified to include consideration of the number of consecutive assignments to the same group and the length of such a run. Some procedures for which the allocation probability also depend on differences in baseline characteristics, as discussed below, are sometimes also called “biased coin” designs.

Another similar adaptive randomization method is referred to as the Urn Design, based on the work of Wei and colleagues [64–67]. This method also attempts to keep the number of participants randomized to each group reasonably balanced as the trial progresses. The name Urn Design refers to the conceptual process of randomization. Imagine an urn filled with m red balls and m black balls. If a red ball is drawn at random, assign the participant to group A, return the red ball, and add one (or more than one) black ball to the urn. If a black ball is drawn, assign the participant to group B, return that ball, and add one (or more than one) red ball to the urn. This process will tend to keep the number of participants in each group reasonably close because, like the biased coin procedure it adjusts the allocation probability to be higher for the smaller group. How much imbalance there might be over time depends on m and how many balls are added after each draw.

Since the biased coin and urn procedures are less restrictive than block randomization, they can be less susceptible to selection bias, but by the same token they do not control balance as closely. If there are temporal trends in the recruitment pool during enrollment, imbalances can create difficulties. This happened in the Stop Atherosclerosis in Native Diabetics Study (SANDS), a trial comparing intensive intervention for cholesterol and blood pressure with less intensive intervention in people with diabetes [68, 69]. Randomization was done using a stratified urn design, but partway through the trial there was in imbalance in the intervention groups at the same time new and more aggressive guidelines regarding lipid lowering treatment in people who had known coronary heart disease came out. The participants in SANDS who met those guidelines could no longer be treated with the less intensive regimen and no new participants with a history of prior cardiovascular events could be enrolled. Not only was there a possibility of imbalance between study groups, the sample size needed to be reconsidered because of the lower average risk level of the participants.

The most correct analysis of a randomized trial from a theoretical point of view is based on permutation distributions modeling the randomization process. For adaptive procedures this requires that the significance level for the test statistic be determined by considering all possible sequences of assignments which could have been made in repeated experiments using the same allocation rule, assuming no group differences. How well population models approximate the permutation distribution for adaptive designs in general is not well understood [6, 14, 70]. Efron [46] argues that it is probably not necessary to take the biased coin randomization into account in the analysis, especially for larger studies. Mehta and colleagues [71] compared analyses ignoring and incorporating biased coin and urn procedures and concluded that the permutation distribution should not be ignored. Smythe and Wei [30, 46] and Wei and Lachin [46, 66] indicate conditions under which test statistics from urn designs are asymptotically normal, and show that if this randomization method is used, but ignored in the analyses, the p-value will be slightly conservative, that is, slightly larger than if the strictly correct analysis were done. Thus the situation for analysis of biased coin and urn designs is similar to that for permuted block designs. Ignoring the randomization is conservative, though not likely to be excessively conservative. Unlike the permuted block design, however, strong temporal trends can create problems for adaptive randomization, and make the permutation-based analysis more important. Although the biased coin method does not appear to be as widely used, stratified urn procedures have been used successfully, as in the multicenter Diabetes Control and Complication Trial [72, 73].

Minimization

In the Enforcing Underage Drinking Laws (EUDL) randomized community trial, 68 communities in five states were selected to receive either an intervention or a control condition. Matched pairs were created using community characteristics including population size, median family income, percentage of the population currently in college, and percentages that were black, Hispanic and spoke Spanish. The specific set of pairings used was determined by sampling from all possible pairings and selecting the set of pairs with the smallest Mahalanobis distance measure. One community in each pair was then randomly assigned to receive the intervention [74]. In this situation, all the communities to be randomized and the key prognostic covariates are known in advance. The treatment and control groups are guaranteed to be well-balanced, and randomization provides a foundation for later statistical inference using standard population models. This type of a priori matching is a common feature of group-randomized trials [75].

Unfortunately, this is almost never possible in a clinical setting, where patients typically arrive sequentially and must be treated immediately. To accommodate the sequential nature of participant enrollment, some compromise between manipulation of allocation to achieve balance of prognostic covariates and a less restrictive treatment allocation must be made. Stratified block designs can balance a small number of selected prognostic covariates, and randomization will tend to balance unselected as well as unmeasured covariates, but such methods do not perform well when it is important to balance a large number of prognostic covariates in a small sample. For such settings, procedures which adapt allocation to achieve balance on prognostic covariates have been developed.

The biased coin and urn procedures achieve balance in the number of randomizations to each arm. Other stratification methods are adaptive in the sense that intervention assignment probabilities for a participant are a function of the distribution of baseline covariates for participants already randomized. This concept was suggested by Efron [46] as an extension of the biased coin method and also has been discussed in depth by Pocock and Simon [41], and others [47, 48, 51, 52, 59, 63, 76, 77]. In a simple example, if age is a prognostic factor and one study group has more older participants than the other, this allocation scheme is more likely to randomize the next several older participants to the group which currently has more younger participants. Various methods can be used as the measure of imbalance in prognostic factors. In general, adaptive stratification methods incorporate several prognostic factors in making an “overall assessment” of the group balance or lack of balance. Participants are then assigned to a group in a manner which will tend to correct an existing imbalance or cause the least imbalance in prognostic factors. Proschan and colleagues [70] distinguish between minimization procedures which are deterministic [59, 68], as ‘strict minimization’, reserving the term mimimization for the more general procedure described by Pocock and Simon [41] [see Appendix]. Generalization of this strategy exists for more than two study groups. Development of these methods was motivated in part by the previously described problems with non-adaptive stratified randomization for small studies. Adaptive methods do not have empty or near empty strata because randomization does not take place within a stratum although prognostic factors are used. Minimization gives unbiased estimates of treatment effect and slightly increased power relative to stratified randomization [68]. These methods are being used, especially in clinical trials of cancer where several prognostic factors need to be balanced, and the sample size is typically 100–200 participants.

The major advantage of this procedure is that it protects against a severe baseline imbalance for important prognostic factors. Overall marginal balance is maintained in the intervention groups with respect to a large number of prognostic factors. One disadvantage is that minimization is operationally more difficult to carry out, especially if a large number of factors are considered. Although White and Freedman [63] initially developed a simplified version of the minimization method by using a set of specially arranged index cards, today any small programmable computer can easily carry out the calculations. Unlike blocked, biased coin and urn procedures, however, the calculations for minimization cannot be done in advance. In addition, the population recruited needs to be stable over time, just as for other adaptive methods. For example, if treatment guidelines change during a long recruitment period, necessitating a change in the inclusion or exclusion criteria, the adaptive procedure may not be able to correct imbalances that developed beforehand, as with the SANDS example cited above.

For minimization, assuming that the order of participant enrollment is random and applying the allocation algorithm to all permutations or the order can provide a null distribution for the test statistic [14, 70]. Considerable programming and computing resources are required to do this, and biostatisticians prefer to use conventional tests and critical values to determine significance levels. Unfortunately, for minimization there are no general theoretical results on how well the standard analysis approximates the permutation analysis [6, 14, 70], though there are some simulation-based results for specific cases [78].

General advice for stratified block randomization and minimization is to include the baseline variables used to determine the allocation as covariates in the analysis [51, 79]. This seems to produce reliable results in most actual trials using stratified block randomization, and in most trials using minimization, though trials using minimization designs rarely examine the permutation distribution. Proschan et al. [70] however, report an example of an actual trial using minimization for which conventional analysis greatly overstated the significance of the intervention effect relative when compared to the permutation distribution. The use of unequal allocation contributed to the discrepancy in this case, but the Proschan et al. recommend that the permutation test be used to control type 1 error whenever allocation is done using minimization. Several regulatory guidelines make the similar recommendations [80–83].

Despite the appeal of improved balance on more prognostic covariates, most biostatisticians approach minimization and other dynamic allocation plans with caution. As conditions vary considerably from trial to trial, it is expected that the best choice for method of allocation also varies, with the primary goal of avoiding a method which is poorly suited for the given situation.

Response Adaptive Randomization

Response adaptive randomization uses information on participant response to intervention during the course of the trial to determine the allocation of the next participant. Examples of response adaptive randomization models are the Play the Winner [84] and the Two-Armed Bandit [85] models. These models assume that the investigator is randomizing participants to one of two interventions and that the primary response variable can be determined quickly relative to the total length of the study. Bailar [86] and Simon [87] reviewed the uses of these allocation methods. Additional modifications or methods were developed [88–94].

The Play the Winner procedure may assign the first participant by the toss of a coin. The next participant is assigned to the same group as the first participant if the response to the intervention was a success; otherwise, the participant is assigned to the other group. That is, the process calls for staying with the winner until a failure occurs and then switching. The following example illustrates a possible randomization scheme where S indicates intervention success and F indicates failure:

Assignment

Participant

1

2

3

4

5

6

7

8

…

Group A

S

F

S

F

Group B

S

S

F

S

Another response adaptive randomization procedure is the Two Armed Bandit method which continually updates the probability of success as soon as the outcome for each participant is known. That information is used to adjust the probabilities of being assigned to either group in such a way that a higher proportion of future participants would receive the currently “better” or more successful intervention.

Both of these response adaptive randomization methods have the intended purpose of maximizing the number of participants on the “superior” intervention. They were developed in response to ethical concerns expressed by some clinical investigators about the randomization process. Although these methods do maximize the number of participants on the “superior” intervention, the possible imbalance will almost certainly result in some loss of power and require more participants to be enrolled into the study than would a fixed allocation with equal assignment probability [92]. A major limitation is that many clinical trials do not have an immediately occurring response variable. They also may have several response variables of interest with no single outcome easily identified as being the one upon which randomization should be based. Furthermore, these methods assume that the population from which the participants are drawn is stable over time. If the nature of the study population should change and this is not accounted for in the analysis, the reported significance levels could be biased, perhaps severely [93]. Here, as before, the data analysis should ideally take into account the randomization process employed. For response adaptive methods, that analysis will be more complicated than it would be with simple randomization. Because of these disadvantages, response adaptive procedures are not commonly used.

One application of response adaptive allocation can be found in a trial evaluating extra-corporeal membrane oxygenator (ECMO) in a neonatal population suffering from respiratory insufficiency [95–99]. This device oxygenates the blood to compensate for the inability or inefficiency of the lungs to achieve this task. In this trial, the first infant was allocated randomly to control therapy. The result was a failure. The next infant received ECMO which was successful. The next ten infants were also allocated to ECMO and all outcomes were successful. The trial was then stopped. However, the first infant was much sicker than the ECMO-treated infants. Controversy ensued and the benefits of ECMO remain unclear. This experience does not offer encouragement to use this adaptive randomization methodology.

Mechanics of Randomization

The manner in which the chosen randomization method is actually implemented is very important [100]. If this aspect of randomization does not receive careful attention, the entire randomization process can easily be compromised, thus voiding any of the advantages for using it. To accomplish a valid randomization, it is recommended that an independent central unit be responsible for developing the randomization process and making the assignments of participants to the appropriate group [27, 101]. For a single center trial, this central unit might be a biostatistician or clinician not involved with the care of the participants. In the case of a multicenter trial, the randomization process is usually handled by the data coordinating center. Ultimately, however, the integrity of the randomization process will rest with the investigator.

Chalmers and colleagues [102] reviewed the randomization process in 102 clinical trials, 57 where the randomization was unknown to the investigator and 45 where it was known. The authors reported that in 14% of the 57 studies, at least one baseline variable was not balanced between the two groups. For the studies with known randomization schedules, twice as many, or 26.7%, had at least one prognostic variable maldistributed. For 43 non-randomized studies, such imbalances occurred four times as often or in 58%. The authors emphasized that those recruiting and entering participants into a trial should not be aware of the next intervention assignment.

In many cases when a fixed proportion randomization process is used, the randomization schedules are made before the study begins [103–107]. The investigators may call a central location, and the person at that location looks up the assignment for the next participant [103]. Another possibility, used historically and still sometimes in trials involving acutely ill participants, is to have a scheme making available sequenced and sealed envelopes containing the assignments [106]. As a participant enters the trial, she receives the next envelope in the sequence, which gives her the assignment. Envelope systems, however, are more prone to errors and tampering than the former method [27, 101]. In one study, personnel in a clinic opened the envelopes and arranged the assignments to fit their own preferences, accommodating friends and relatives entering the trial. In another case, an envelope fell to the bottom of the box containing the envelopes, thus changing the sequence in which they were opened. Many studies prefer web-based or telephone systems to protect against this problem. In an alternative procedure that has been used in several double-blind drug studies, medication bottles are numbered with a small perforated tab [105]. The bottles are distributed to participant in sequence. The tab, which is coded to identify the contents, is torn off and sent to the central unit. This system is also subject to abuse unless an independent person is responsible for dispensing the bottles. Many clinical trials using a fixed proportion randomization schedule require that the investigator access a website or call the central location to verify that a participant is eligible to be in the trial before any assignment is made. This increases the likelihood that only eligible participants will be randomized.

For many trials, especially multicenter and multinational trials, logistics require a central randomization operations process. Web-based approaches to randomization and other aspects of trial management predominate now [108]. In some cases, the clinic may register a participant by dialing into a central computer and entering data via touchtone, with a voice response. These systems, referred to as Interactive Voice Response Systems or IVRS, or Interactive Web Response Systems, IWRS, are effective and can be used to not only assign intervention but can also capture basic eligibility data. Before intervention is assigned, baseline data can be checked to determine eligibility. This concept has been used in a pediatric cancer cooperative clinical trial network [109] and in major multicenter trials [110, 111].

Whatever system is chosen to communicate the intervention assignment to the investigator or the clinic, the intervention assignment should be given as closely as possible to the moment when both investigator and participant are ready to begin the intervention. If the randomization takes place when the participant is first identified and the participant withdraws or dies before the intervention actually begins, a number of participants will be randomized before being actively involved in the study. An example of this occurred in a non-blinded trial of alprenolol in survivors of an acute myocardial infarction [112]. In that trial, 393 participants with a suspected myocardial infarction were randomized into the trial at the time of their admission to the coronary care unit. The alprenolol or placebo was not initiated until 2 weeks later. Afterwards, 231 of the randomized participants were excluded because a myocardial infarction could not be documented, death had occurred before therapy was begun, or various contraindications to therapy were noted. Of the 162 participants who remained, 69 were in the alprenolol group and 93 were in the placebo group. This imbalance raised concerns over the comparability of the two groups and possible bias in reasons for participant exclusion. By delaying the randomization until initiation of therapy, the problem of these withdrawals could have been avoided.

Problems of implementation can also affect the integrity of the randomization procedure. Downs and colleagues [101] relate their experiences with problems caused by errors in programming, incomplete and missing data for stratification variables, and other problems. They also recommend testing of the proposed procedure before the trial begins, and monitoring of the allocation after it begins.

Recommendations

For large studies involving more than several hundred participants, the randomization should be blocked. If a large multicenter trial is being conducted, randomization should be stratified by center. Randomization stratified on the basis of other factors in large studies is usually not necessary, because randomization tends to make the study groups quite comparable for all risk factors. The participants can still, of course, be stratified once the data have been collected and the study can be analyzed accordingly.

For small studies, the randomization should also be blocked, and stratified by center if more than one center is involved. Since the sample size is small, a few strata for important risk factors may be defined to assure that balance will be achieved for at least those factors. For a larger number of prognostic factors, the adaptive stratification techniques should be considered and the appropriate analyses performed. As in large studies, stratified analysis can be performed even if stratified randomization was not done. For many situations, this will be satisfactory.

Appendix: Adaptive Randomization Algorithm

Adaptive randomization can be used for more than two intervention groups, but for the sake of simplicity only two will be used here. In order to describe this procedure in more detail, a minimum amount of notation needs to be defined. First, let

x ik  = the number of participants already assigned intervention k

(k = 1, 2) who have the same level of prognostic factor i

(i = 1, 2, … , f) as the new participant.

and define

The x t ik represents the change in balance of allocation if the new participant is assigned intervention t. Finally, let

B(t) = function of the x t ik ’s, which measures the “lack of balance” over all prognostic factors if the next participant is assigned intervention t.

Many possible definitions of B(t) can be identified. As an illustrative example, let

where w i  = the relative importance of factor i to the other factors and the range is the absolute difference between the largest and smallest values of x t i1 and x t i2 .

The value of B(t) is determined for each intervention (t = 1 and t = 2). The intervention with the smaller B(t) is preferred, because allocation of the participant to that intervention will cause the least imbalance. The participant is assigned, with probability p > 1/2, to the intervention with the smaller score, B(1) or B(2). The participant is assigned, with probability (1 − p), to the intervention with the larger score. These probabilities introduce the random component into the allocation scheme. Note that if p = 1 and, therefore, 1 − p = 0, the allocation procedure is deterministic (no chance or random aspect) and has been referred to by the term “minimization” [51, 59].

As a simple example of the adaptive stratification method, suppose there are two groups and two prognostic factors to control. The first factor has two levels and the second factor has three levels. Assume that 50 participants have already been randomized and the following table summarizes the results (Table 6.A1).

Table 6.A1

Fifty randomized participants by group and level of factor (x ik ’s)a

Factor

1

2

Total

Level

1

2

1

2

3

Group

1

16

10

13

9

4

26

2

14

10

12

6

6

24

30

20

25

15

10

50

aAfter Pocock and Simon [41]

In addition, the function B(t) as defined above will be used with the range of the x ik ’s as the measure of imbalance, where w 1  = 3 and w 2  = 2; that is, the first factor is 1.5 times as important as the second as a prognostic factor. Finally, suppose p = 2/3 and 1 − p = 1/3.

If the next participant to be randomized has the first level of the first factor and the third level of the second factor, then this corresponds to the first and fifth columns in the table. The task is to determine B(1) and B(2) for this participant as shown below.

1.

Determine B(1)

(a)

Factor 1, Level 1

K

x 1k

x 1 1k

Range (x 1 11 , x 1 12 )

Group

1

16

17

$$ \left|17\hbox{--} 14\right| $$ = 3

2

14

14

(b)

Factor 2, Level 3

K

x 2k

x 1 2k

Range (x 1 21 , x 1 22 )

Group

1

4

5

$$ \left|5\hbox{--} 6\right| $$ = 1

2

6

6

Using the formula given, B(1) is computed as 3 × 3 + 2 × 1 = 11.

2.

Determine B(2)

(a)

Factor 1, Level 1

K

x 1k

x 2 1k

Range (x 2 11 , x 2 12 )

Group

1

16

16

$$ \left|16\hbox{--} 15\right| $$ = 1

2

14

15

(b)

Factor 2, Level 3

K

x 2k

x 1 1k

Range (x 1 21 , x 1 22 )

Group

1

4

4

$$ \left|4\hbox{--}7\right| $$ = 3

2

6

7

Then B(2) is computed as 3 × 1 + 2 × 3 = 9.

3.

Now rank B(1) and B(2) from smaller to larger and assign with probability p the group with the smaller B(t).

t

B(t)

Probability of assigning t

2

B(2) = 9

p = 2/3

1

B(1) = 11

1 − p = 1/3

Thus, this participant is randomized to Group 2 with probability 2/3 and to Group 1 with probability 1/3. Note that if minimization were used (p = 1), the assignment would be Group 2.

References

1.

Armitage P. The role of randomization in clinical trials. Statist Med 1982;1:345–352.CrossRef

2.

Brown BW. Statistical Controversies in the Design of Clinical-Trials—Some Personal Views. Control Clin Trials 1980;1:13–27.CrossRef

3.

Byar DP, Simon RM, Friedewald WT, et al. Randomized Clinical Trials. N Engl J Med 1976;295:74–80.CrossRef

4.

Hill AB. The Clinical Trial. Brit M Bull 1951;7:278–282.

5.

Lachin JM. Statistical properties of randomization in clinical trials. Control Clin Trials 1988;9:289–311.CrossRef

6.

Lachin JM, Matts JP, Wei LJ. Randomization in clinical trials: Conclusions and recommendations. Control Clin Trials 1988;9:365–374.CrossRef

7.

Peto R. Clinical trial methodology. Biomedicine 1978;28:24–36.

8.

Peto R, Pike MC, Armitage P, et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. British Journal of Cancer 1976;34:585.CrossRef

9.

Pocock SJ. Allocation of Patients to Treatment in Clinical Trials. Biometrics 1979;35:183–197.CrossRef

10.

Royall RM. Ethics and statistics in randomized clinical trials. Statistical Science 1991;6:52–62.MathSciNetCrossRefMATH

11.

Weinstein MC. Allocation of Subjects in Medical Experiments. N Engl J Med 1974;291:1278–1285.CrossRef

12.

Zelen M. The randomization and stratification of patients to clinical trials. J Chronic Dis 1974;27:365–375.CrossRef

13.

Bather JA. On the allocation of treatments in sequential medical trials. International Statistical Review/Revue Internationale de Statistique 1985;1–13.

14.

Kalish LA, Begg CB. Treatment allocation methods in clinical trials: a review. Statist Med 1985;4:129–144.CrossRef

15.

Altman DG, Dore CJ. Randomisation and baseline comparisons in clinical trials. Lancet 1990;335:149–153.CrossRef

16.

Stigler SM. The use of random allocation for the control of selection bias. Biometrika 1969;56:553–560.MathSciNetCrossRefMATH

17.

Wei LJ. On the Random Allocation Design for the Control of Selection Bias in Sequential Experiments. Biometrika 1978;65:79–84.MathSciNetCrossRefMATH

18.

Williams DH, Davis CE. Reporting of assignment methods in clinical trials. Control Clin Trials 1994;15:294–298.CrossRef

19.

Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. The Lancet 2001;357:1191–1194.CrossRef

20.

Mills EJ, Wu P, Gagnier J, Devereaux PJ. The quality of randomized trial reporting in leading medical journals since the revised CONSORT statement. Contemp Clin Trials 2005;26:480–487.CrossRef

21.

Brittain E, Schlesselman JJ. Optimal Allocation for the Comparison of Proportions. Biometrics 1982;38:1003–1009.CrossRef

22.

Kalish LA, Harrington DP. Efficiency of Balanced Treatment Allocation for Survival Analysis. Biometrics 1988;44:815–821.CrossRefMATH

23.

Lachin JM. Properties of simple randomization in clinical trials. Control Clin Trials 1988;9:312-326.CrossRef

24.

Louis TA. Optimal allocation in sequential tests comparing the means of two Gaussian populations. Biometrika 1975;62:359–369.MathSciNetCrossRefMATH

25.

Louis TA. Sequential Allocation in Clinical Trials Comparing Two Exponential Survival Curves. Biometrics 1977;33:627–634.MathSciNetCrossRef

26.

Matts JP, Lachin JM. Properties of permuted-block randomization in clinical trials. Control Clin Trials 1988;9:327–344.CrossRef

27.

Schulz KF, Grimes DA. Allocation concealment in randomised trials: defending against deciphering. The Lancet 2002;359:614–618.CrossRef

28.

Kalish LA, Begg CB. The impact of treatment allocation procedures on nominal significance levels and bias. Control Clin Trials 1987;8:121–135.CrossRef

29.

Matts JP, McHugh RB. Analysis of accrual randomized clinical trials with balanced groups in strata. J Chronic Dis 1978;31:725–740.CrossRef

30.

Smythe RT, Wei LJ. Significance tests with Restricted Randomization Design. Biometrika 1983;70:496-500.MathSciNetCrossRefMATH

31.

Ducimetiere P. Stratification; in Boissel JP, Klimt CR (eds): Multi-Center Controlled Trials, Principles and Problems. Paris, Institut National de la Sante et de la Recherche Medicale, 1979.

32.

Feinstein AR, Landis JR. The role of prognostic stratification in preventing the bias permitted by random allocation of treatment. J Chronic Dis 1976;29:277–284.CrossRef

33.

Fleiss JL. Multicentre clinical trials: Bradford Hill’s contributions and some subsequent developments. Statist Med 1982;1:353–359.CrossRef

34.

Green SB, Byar DP. The effect of stratified randomization on size and power of statistical tests in clinical trials. J Chronic Dis 1978;31:445–454.CrossRef

35.

Grizzle JE. A note on stratifying versus complete random assignment in clinical trials. Control Clin Trials 1982;3:365–368.CrossRef

36.

Mantel N, McHugh R, Matts J. Pre-Stratification or Post-Stratification. Biometrics 1984;40:256–258.

37.

McHugh R, Matts J. Post-Stratification in the Randomized Clinical Trial. Biometrics 1983;39:217–225.MathSciNetCrossRefMATH

38.

Meier P. Stratification in the design of a clinical trial. Control Clin Trials 1981;1:355–361.CrossRef

39.

Palta M, Amini SB. Consideration of covariates and stratification in sample size determination for survival time studies. J Chronic Dis 1985;38:801–809.CrossRef

40.

Palta M, Amini SB. Magnitude and likelihood of loss resulting from non-stratified randomization. Statist Med 1982;1:267–275.CrossRef

41.

Pocock SJ, Simon R. Sequential Treatment Assignment with Balancing for Prognostic Factors in the Controlled Clinical Trial. Biometrics 1975;31:103–115.CrossRef

42.

Simon R. Restricted Randomization Designs in Clinical Trials. Biometrics 1979;35:503–512.MathSciNetCrossRef

43.

The Coronary Drug Project Research Group: Factors influencing long-term prognosis after recovery from myocardial infarctionΓÇöThree-year findings of the Coronary Drug Project. J Chronic Dis 1974;27:267–285.CrossRef

44.

Zelen M. Aspects of the planning and analysis of clinical trials in cancer. A Survey of Statistical Design and Linear Models Amsterdam: North-Holland 1975.

45.

Aspirin Myocardial Infarction Study Research Group: A randomized, controlled trial of aspirin in persons recovered from myocardial infarction. JAMA 1980;243:661–669.CrossRef

46.

Efron B. Forcing a Sequential Experiment to be Balanced. Biometrika 1971;58:403–417.MathSciNetCrossRefMATH

47.

Atkinson AC. Optimum biased coin designs for sequential clinical trials with prognostic factors. Biometrika 1982;69:61–67.MathSciNetCrossRefMATH

48.

Begg CB, Iglewicz B. A Treatment Allocation Procedure for Sequential Clinical Trials. Biometrics 1980;36:81–90.CrossRefMATH

49.

Begg CB. On Inferences from Wei’s Biased Coin Design for Clinical Trials. Biometrika 1990;77:467–478.MathSciNetCrossRefMATH

50.

Efron B. Randomizing and balancing a complicated sequential experiment. Biostatistics Casebook 1980;19–30.

51.

Forsythe AB, Stitt FW. Randomization or minimization in the treatment assignment of patient trials: validity and power of tests. Health Sciences Computing Facility, University of California, 1977.

52.

Freedman LS, White SJ. On the Use of Pocock and Simon’s Method for Balancing Treatment Numbers over Prognostic Factors in the Controlled Clinical Trial. Biometrics 1976;32:691–694.CrossRefMATH

53.

Halpern J, Brown BW. Sequential treatment allocation procedures in clinical trials-with particular attention to the analysis of results for the biased coin design. Statist Med 1986;5:211–229.CrossRef

54.

Hannigan JF, Brown BW. Adaptive Randomization Biased Coin-Design: Experience in a Cooperative Group Clinical Trial. 74. 1982. Department of Statistics, Stanford University.

55.

Klotz JH. Maximum Entropy Constrained Balance Randomization for Clinical Trials. Biometrics 1978;34:283–287.CrossRef

56.

Raghavarao D. Use of distance function in sequential treatment assignment for prognostic factors in the controlled clinical trial. Calcutta Statist Assoc Bull 1980;29:99–102.CrossRef

57.

Smith RL. Sequential Treatment Allocation Using Biased Coin Designs. J R Stat Soc Series B Stat Methodol 1984;46:519–543.MathSciNetMATH

58.

Soares JF, Jeff Wu CF. Some restricted randomization rules in sequential designs. Commun StatTheory Methods 1983;12:2017–2034.MathSciNetCrossRefMATH

59.

Taves DR. Minimization: a new method of assigning patients to treatment and control groups. Clinical pharmacology and therapeutics 1974;15:443.CrossRef

60.

Wei LJ. A class of treatment assignment rules for sequential experiments. Commun Stat Theory Methods 1978;7:285–295.MathSciNetCrossRefMATH

61.

Wei LJ. The Adaptive Biased Coin Design for Sequential Experiments. Ann Stat 978;6:92–100.

62.

Wei LJ. A class of designs for sequential clinical trials. J Am Stat Assoc 1977;72:382–386.CrossRef

63.

White SJ, Freedman LS. Allocation of patients to treatment groups in a controlled clinical study. Br J Cancer 1978;37:849.CrossRef

64.

Wei LJ. An application of an urn model to the design of sequential controlled clinical trials. J Am Stat Assoc 1978;73:559–563.MathSciNetCrossRefMATH

65.

Wei LJ, Smythe RT, Smith RL. K-treatment comparisons with restricted randomization rules in clinical trials. Ann Stat 1986;265–274.

66.

Wei LJ, Lachin JM. Properties of the urn randomization in clinical trials. Control Clin Trials 1988;9:345–364.CrossRef

67.

Wei LJ, Smythe RT, Lin DY, Park TS. Statistical inference with data-dependent treatment allocation rules. J Am Stat Assoc 1990;85:156–162.MathSciNetCrossRef

68.

Birkett NJ. Adaptive allocation in randomized controlled trials. Control Clin Trials 1985;6:146–155.CrossRef

69.

Russell M, Fleg JL, Galloway W, et al. Examination of lower targets for low-density lipoprotein cholesterol and blood pressure in diabetes—the Stop Atherosclerosis in Native Diabetics Study (SANDS). American Heart Journal 2006;152:867–875.CrossRef

70.

Proschan MA, Brittain E, Kammerman L. Minimize the Use of Minimization with Unequal Allocation. Biometrics 2011;67:1135–1141.MathSciNetCrossRefMATH

71.

Mehta CR, Patel NR, Wei LJ. Constructing Exact Significance Tests with Restricted Randomization Rules. Biometrika 1988;75:295–302.CrossRefMATH

72.

The DCCT Research Group: The Diabetes Control and Complications Trial (DCCT): Design and Methodologic Considerations for the Feasibility Phase. Diabetes 1986;35:530–545.

73.

The DCCT Research Group: Diabetes Control and Complications Trial (DCCT): Results of Feasibility Study. The DCCT Research Group. Diabetes Care 1987;10:1–19.

74.

Reboussin BA, Preisser JS, Song EY, Wolfson M. Sample size estimation for alternating logistic regressions analysis of multilevel randomized community trials of under-age drinking. R Stat Soc Ser A Stat Soc 2012;175:691–712.MathSciNetCrossRef

75.

Murray DM. Design and Analysis of Group-randomized Trials. Oxford University Press, 1998.

76.

Gail MH, Wieand S, Piantadosi S. Biased Estimates of Treatment Effect in Randomized Experiments with Nonlinear Regressions and Omitted Covariates. Biometrika 1984;71:431–444.MathSciNetCrossRefMATH

77.

Begg CB, Kalish LA. Treatment Allocation for Nonlinear Models in Clinical Trials: The Logistic Model. Biometrics 1984;40:409–420.CrossRef

78.

Aickin M. A Simulation Study of the Validity and Efficiency of Design-Adaptive Allocation to Two Groups in the Regression Situation. Int J Biostat 2009;5:Article 19.

79.

Kahan BC, Morris TP. Improper analysis of trials randomised using stratified blocks or minimisation. Statist Med 2012;31:328–340.MathSciNetCrossRef

80.

Committee for Proprietary Medicinal Products (CPMP: Committee for Proprietary Medicinal Products (CPMP): points to consider on adjustment for baseline covariates. Statist Med 2004;23:701.

81.

FDA: International Conference on Harmonization—Efficacy: Statistical principles for clinical trials. U S Food and Drug Administration.

82.

Food and Drug Administration: Guidance for industry: Adaptive design clinical trials for drugs and biologics. Washington DC, USA: Food and Drug Administration 2010.

83.

Green H, McEntegart DJ, Byrom B, et al. Minimization in crossover trials with non-prognostic strata: theory and practical application. J Clin Pharm Ther 2001;26:121–128.CrossRef

84.

Zelen M. Play the Winner Rule and the Controlled Clinical Trial. J Am Stat Assoc 1969;64:131–146.MathSciNetCrossRef

85.

Robbins H. Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society 1952;58:527–535.MathSciNetCrossRefMATH

86.

Bailar JC. Patient assignment algorithms: an overview; in Proceedings of the 9th International Biometric Conference, Boston, August 22-27, 1976: Invited Papers: Biometric Society, 1976, pp 189–206.

87.

Simon R. Adaptive Treatment Assignment Methods and Clinical Trials. Biometrics 1977;33:743–749.CrossRef

88.

Armitage P. The search for optimality in clinical trials. International Statistical Review/Revue Internationale de Statistique 1985;15–24.

89.

Bather JA. Randomized Allocation of Treatments in Sequential Experiments. J R Stat Soc Series B Stat Methodol 1981;43:265–292.MathSciNetMATH

90.

Berry DA. Modified Two-Armed Bandit Strategies for Certain Clinical Trials. J Am Stat Assoc 1978;73:339–345.CrossRefMATH

91.

Nordbrock E. An Improved Play-the-Winner Sampling Procedure for Selecting the Better of Two Binomial Populations. J Am Stat Assoc 1976;71:137–139.MathSciNetCrossRefMATH

92.

Simon R, Weiss GH, Hoel DG. Sequential Analysis of Binomial Clinical Trials. Biometrika 1975;62:195–200.MathSciNetCrossRefMATH

93.

Simon R, Hoel DG, Weiss GH. The use of covariate information in the sequential analysis of dichotomous response experiments. Commun Stat Theory Methods 1977;6:777–788.CrossRefMATH

94.

Wei LJ. Exact two-sample permutation tests based on the randomized play-the-winner rule. Biometrika 1988;75:603–606.MathSciNetCrossRefMATH

95.

Bartlett RH, Roloff DW, Cornell RG, et al. Extracorporeal Circulation in Neonatal Respiratory Failure: A Prospective Randomized Study. Pediatrics 1985;76:479–487.

96.

O’Rourke PP, Crone RK, Vacanti JP, et al. Extracorporeal Membrane Oxygenation and Conventional Medical Therapy in Neonates With Persistent Pulmonary Hypertension of the Newborn: A Prospective Randomized Study. Pediatrics 1989;84:957–963.

97.

Paneth N, Wallenstein S. Extracorporeal Membrane Oxygenation and the Play the Winner Rule. Pediatrics 1985;76:622–623.

98.

Ware JH, Epstein MF. Extracorporeal Circulation in Neonatal Respiratory Failure: A Prospective Randomized Study. Pediatrics 1985;76:849–851.

99.

Ware JH. Investigating Therapies of Potentially Great Benefit: ECMO. Stat Sci 1989;4:298–306.MathSciNetCrossRefMATH

100.

Pocock SJ, Lagakos SW. Practical experience of randomization in cancer trials: an international survey. Br J Cancer 1982;46:368–375.CrossRef

101.

Downs M, Tucker K, Christ-Schmidt H, Wittes J. Some practical problems in implementing randomization. Clin Trials 2010;7:235–245.CrossRef

102.

Chalmers TC, Celano P, Sacks HS, Smith H. Bias in Treatment Assignment in Controlled Clinical Trials. N Engl J Med 1983;309:1358–1361.CrossRef

103.

Beta-Blocker Heart Attack Trial Research Group. A randomized trial of propranolol in patients with acute myocardial infarction: I. mortality results. JAMA 1982;247:1707–1714.

104.

CASS Principle Investigators and Their Associates. Coronary artery surgery study (CASS): a randomized trial of coronary artery bypass surgery. Survival data. Circulation 1983;68:939–950.CrossRef

105.

Collaborative Group on Antenatal Steroid Therapy. Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome. Am J Obstet Gynecol 1981;141:276–287.

106.

Hypertension Detection and Follow-up Program Cooperative Group: Five-year findings of the hypertension detection and follow-up program: I. reduction in mortality of persons with high blood pressure, including mild hypertension. JAMA 1979;242:2562–2571.

107.

Multiple Risk Factor Intervention Trial Research Group. Multiple risk factor intervention trial: Risk factor changes and mortality results. JAMA 1982;248:1465–1477.CrossRef

108.

Reboussin D, Espeland MA. The science of web-based clinical trial management. Clin Trials 2005;2:1–2.CrossRef

109.

Krischer JP, Hurley C, Pillalamarri M, et al. An automated patient registration and treatment randomization system for multicenter clinical trials. Control Clin Trials 1991;12:367–377.CrossRef

110.

Kjekshus J, Apetrei E, Barrios V, et al. Rosuvastatin in Older Patients with Systolic Heart Failure. N Engl J Med 2007;357:2248–2261.CrossRef

111.

SPORTIF Executive Steering Committee for the SPORTIF: Ximelagatran vs warfarin for stroke prevention in patients with nonvalvular atrial fibrillation: A randomized trial. JAMA 2005;293:690–698.CrossRef

112.

Ahlmark G, Saetre H. Long-term treatment with ß-blockers after myocardial infarction. Eur J Clin Pharmacol 1976;10:77–83.CrossRef

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_7

7. Blinding

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

In any clinical trial bias in determining treatment effects is one of the main concerns. Bias may be defined as systematic error, or “difference between the true value and that actually obtained due to all causes other than sampling variability” [1]. It can be caused by conscious factors, subconscious factors, or both. Bias can occur at a number of places in a clinical trial, from the initial design through data analysis, interpretation and reporting. One general solution to the problem of bias is to keep the participants and the investigators blinded, or masked, to the identity of the assigned intervention. One can also blind several other aspects of a trial including the assessment, classification and evaluation of the response variables. A large sample size does not reduce bias.

The history of blind assessment in medicine goes back more than 200 years [2]. In its simplest form, the investigators used blindfolds or curtains so that the participants would not know the nature or timing of the intervention. Dummy interventions were also utilized from the inception. The first series of blind assessment was directed at mesmerism, an intervention based on a new “healing fluid” in nature called “animal magnetism.” A group of women was involved in the first blindfold experiment. Its conclusion was the “while the woman was permitted to see the operation, she placed her sensations precisely in the part towards which it was directed; that on the other hand, when she did not see the operation, she placed them at hazard, and in parts very distant from those which were the object of magnetism.” In another type of experiment, women were told that they were receiving mesmerism from an adjoining room through a paper curtain over a door. The knowledge of intervention produced sensations. When they received treatment but were not told they were mesmerized, nothing happened. Blinding eliminated the effects of mesmerism, and sham worked as well as “real” mesmerism.

The first clinical trial that in modern time applied the principle of blinding was published in 1931 by Amberson et al. [3]. This trial was probably also the first trial that employed a form of random assignment of participants to the study groups.

Fundamental Point

A clinical trial should, ideally, have a double-blind design in order to limit potential problems of bias during data collection and assessment. In studies where such a design is impossible, other measures to reduce potential bias are advocated.

Who Is Blinded?

The blinding terminology is not well understood. A survey of 91 internal medicine physicians in Canada from 2001 [4] showed that 75% knew the definition of single-blind. Approximately 40% understood the proper definition of double-blind. A more recent survey showed that the understanding of the blinding terminology has not improved [5]. Among 66 single-blind trials, the investigators were asked who was blinded. Twenty-six said the patients, 22 the outcome assessors and 16 the data analysts/investigators. Viergever and Ghersi [5] also reviewed to what extent information of blinding was part of registered records of clinical trials. They concluded that this information was often not provided or was of poor quality in trial publications. The authors concluded that the term double-blind was found to be common despite the lack of clarity on its exact meaning.

The meaning of the term double-blind has been addressed in recent publications. Responders to a survey of 200 blinded RCTs from the Cochrane Central Register of Controlled Trials provided their operational meanings of the term [6]. The authors were asked which of the following six categories of key trial persons had been blinded: (1) patients, (2) health care providers responsible for care, (3) data collectors, (4) assessors of outcome (including the data monitoring committee), (5) data analysts or (6) manuscript writers. Fifteen different answers were given for the term “double-blind”. The most common answers included patients (97%), health care providers (89%), data collectors (90%) and outcome assessors (69%).

The use of the terms single-blind and double-blind is particularly inconsistent in trials of non-pharmaceutical interventions [7].

Types of Blinding

Unblinded

In an unblinded or open trial, both the participant and all investigators know to which intervention the participant has been assigned. Some kinds of trials are primarily conducted in this manner and those include most surgical procedures, comparisons of devices and medical treatment, changes in lifestyle (e.g. eating habits, exercise, cigarette smoking) or learning techniques. Approaches to blinding elements of non-pharmacologic interventions are discussed below.

An unblinded study is appealing for two reasons. First, investigators are likely to be more comfortable making decisions, such as whether or not to continue a participant on the assigned study medication if they know its identity. Second, all other things being equal, it is often simpler to execute than other studies. The usual drug trial may be easier to design and carry out, and consequently less expensive, if blinding is not an issue. Also, it has been argued that it more accurately reflects clinical practice [8]. However, an unblinded trial need not be simple. For example, trials that simultaneously attempt to induce lifestyle changes and test drug interventions can be fairly complex. An example is the Women’s Health Initiative [9] which had three distinct interventions: hormone replacement therapy, calcium and vitamin D supplementation and an unblinded dietary intervention.

The main disadvantage of an unblinded trial is the possibility of bias. Participant reporting of symptoms and side effects and prescription of concomitant or compensatory treatment are all susceptible to bias. Other problems of biased data collection and assessment by the investigator are addressed in Chap. 11. Since participants when joining a trial have sincere hopes and expectations about beneficial effects, they may become dissatisfied and drop-out of the trial in disproportionately large numbers if not on the new or experimental intervention. The benefit of blinding in trials of a short intervention (such as treatment with a fibrinolytic agent for acute myocardial infarction) where differential drop-out is unlikely, and with an outcome (like all-cause mortality) that is not subject to ascertainment bias can be debated. However, even in these trials assessment of other adverse events will be protected from bias with blinding.

A trial of the possible benefits of ascorbic acid (vitamin C) in the common cold was designed as a double-blind study [10, 11]. However, it soon became apparent that many of the participants, most of whom were medical staff, discovered mainly by tasting whether they were on ascorbic acid or placebo. As more participants became aware of their medication’s identity, the dropout rate in the placebo group increased. Since evaluation of severity and duration of colds depended on the participants’ reporting of their symptoms, this unblinding was important. Among those participants who claimed not to know the identity of the treatment at the end of the trial, ascorbic acid showed no benefit over placebo. In contrast, among participants who knew or guessed what they were on, ascorbic acid did better than placebo. Therefore, preconceived notions about the benefit of a treatment, coupled with a subjective response variable, may have yielded biased reporting. The investigators’ willingness to share this experience provided us with a nice illustration of the importance of maintaining blinding.

In a trial of coronary artery bypass surgery versus medical treatment [12], the number of participants who smoked was equal in the two study groups at baseline. During the early part of follow-up, there were significantly fewer smokers in the surgical group than in the medical group. A possible explanation could have been that the unblinded surgeons gave more anti-smoking advice to those randomized to surgery. The effect of this group difference on the outcome of the trial is difficult, if not impossible, to assess.

Single-Blind

The established definition of a single-blind study is that only the participants are unaware of which intervention they are receiving. The advantages of this design are similar to those of an unblinded study—it is usually simpler to carry out than a double-blind design, and knowledge of the intervention may help the investigators exercise their best judgment when caring for the participants. Indeed, certain investigators are reluctant to participate in studies in which they do not know the study group assignment. They may recognize that bias is partially reduced by keeping the participant blinded but feel that the participant’s health and safety are best served if they themselves are not blinded.

The disadvantages of a single-blind design are similar to, though not so pronounced as, those of an unblinded design. The investigator avoids the problems of biased participant reporting, but she herself can affect the administration of non-study therapy, data collection, and data assessment. For example, a single-blind study reported benefits from zinc administration in a group of people with taste disorders [13]. Because of the possibility of bias in a study using a response variable as subjective and hard to measure as taste, the study was repeated, using a type of crossover, double-blind design [14]. This second study showed that zinc, when compared with placebo, did not relieve the taste disorders of the study group. The extent of the blinding of the participants did not change; therefore, presumably, knowledge of drug identity by the investigator was important. The results of treatment cross-over were equally revealing. In the single-blind study, participants who did not improve when given placebo as the first treatment, “improved” when placed on zinc. However, in all four double-blind, cross-over procedures (placebo to zinc, placebo to placebo, zinc to zinc, zinc to placebo), the participants who had previously shown no improvement on the first treatment did show benefit when given the second medication. Thus, the expectation that the participants who failed to respond to the first drug were now being given an active drug may have been sufficient to produce a positive response.

Another example comes from two noninferiority trials comparing ximelagatran, a novel oral direct thrombin inhibitor, to warfarin for the prevention of thromboembolic events in people with nonvalvular atrial fibrillation [15]. The first trial, SPORTIF III, was single-blind with blinded events assessment, while the second trial, SPORTIF V, was double-blind. The primary response variable was all strokes and systemic embolic events. The observed risk ratio in the single-blind SPORTIF III was 0.71 (95% CI, 0.48–1.07) while the result trended in the opposite direction in the double-blind SPORTIF V with a risk ratio of 1.38 (95% CI, 0.91–2.10). One cannot be sure how much bias may have played a role, but, in general, more confidence ought to be placed on trials with a double-blind design. A more recent example is the open label trials of renal artery denervation for resistant hypertension that reported large treatment benefits not seen in a subsequent sham-controlled blinded trial [16].

Both unblinded and single-blind trials are vulnerable to another source of potential bias introduced by the investigators. This relates to group differences in compensatory and concomitant treatment. Investigators may feel that the control group is not being given the same opportunity as the intervention group and, as a result, may prescribe additional treatment as “compensation.” This may be in the form of advice or therapy. For example, several studies have attempted blood pressure lowering as either the sole intervention, or as part of a broader effort. In general, the investigators would make an intensive effort to persuade participants in the intervention group to take their study medication. To persuade successfully the investigators themselves had to be convinced that blood pressure reduction was likely beneficial. When they were seeing participants who had been assigned to the control group, this conviction was difficult to suppress. Therefore, participants in the control group were likely to have been instructed about non-pharmacological ways by which to lower their blood pressure or other preventive treatments. The result of compensatory treatment is a diminution of the difference in blood pressure or even hypertension-related outcomes between the intervention group and the “untreated,” control group. This may also have been a factor in a heart failure trial of dronedarone, an antiarrhythmic drug, in which the intervention group had a higher mortality rate [17]. This finding in the dronedarone group would in part be due to a lower use of ACE-inhibitors, a drug class known to reduce mortality.

Working against this is the fact that investigators typically prefer to be associated with a study that gives positive findings. Favorable results published in a reputable journal are likely to lead to more invitations to present the findings at scientific meetings and grand rounds and can also support academic promotions. Investigators may, therefore, subconsciously favor the intervention group when they deal with participants, collect data, and assess and interpret results, although this may perhaps be less of an issue in multicenter trials.

Concomitant treatment means any non-study therapy administered to participants during a trial. If such treatment is likely to influence the response variable, this needs to be considered when determining sample size. Of more concern is the bias that can be introduced if concomitant treatment is applied unequally in the two groups. In order to bias the outcome of a trial, concomitant treatment must be effective, and it must be used in a high proportion of the participants. When this is the case, bias is a possibility and may occur in either direction, depending on whether the concomitant treatment is preferentially used in the control, or in the intervention group. It is usually impossible to determine the direction and magnitude of such bias in advance or its impact after it has occurred.

Double-Blind

In a double-blind study, neither the participants nor the investigators or more specifically the team of investigators responsible for following the participants, collecting data, and assessing outcomes should know the identity of the intervention assignment. Such designs are usually restricted to trials of drugs or biologics. It is theoretically possible to design a study comparing two surgical procedures or implantation of two devices in which the surgeon performing the operation knows the type of surgery or device, but neither the study investigator nor the participant knows. Similarly, one might be able to design a study comparing two diets in which the food looks identical. However, such trials are uncommon.

The main advantage of a truly double-blind study is that the risk of bias is reduced. Preconceived ideas of the investigators will be less important, because they will not know which intervention a particular participant is receiving. Any effect of their actions, therefore, would theoretically occur equally in the intervention and control groups. As discussed later, the possibility of bias may never be completely eliminated. However, a well designed and properly run double-blind study can minimize bias. As in the example of the trial of zinc and taste impairment, double-blind studies have at times led to results that differ from unblinded or single blind studies. Such cases illustrate the role of bias as a factor in clinical trials.

In a double-blind trial certain functions, which in open or single-blind studies could be accomplished by the investigators, might sometimes be taken over by others in order to maintain the blinding. These functions include participant care if it is important for patient care to know the intervention, collection of efficacy and safety data that might disclose the nature of the intervention, and assessment and monitoring of treatment outcomes. Typically, an outside body needs to monitor the data for toxicity and benefit, especially in long-term trials. Chapter 17 discusses data monitoring in greater detail. A person other than the investigator who sees the participants needs to be responsible for assigning the interventions to the participants. Treatments that require continuous dose adjustment, such as warfarin, are difficult to blind, but it can be accomplished. In one trial [18], an unblinded pharmacist or physician adjusted the warfarin doses according to an algorithm for maintaining the International Normalized Ratio (INR), a measure of anticoagulation, within a pre-specified range but also adjusted the placebo doses randomly. The authors concluded that “placebo warfarin dose adjustment schedules can protect blinding adequately” for participants and investigators and recommended their use for future trials of warfarin. A similar approach was employed in the Coumadin Aspirin Reinfarction Study [19]. An INR control center adjusted the doses in the three treatment arms to keep the INR values below the prespecified safety limits and to maintain the double-blind. In another trial [20] a point of care device was used that encrypted result that was a true INR for participants on warfarin and a sham INR for those not on warfarin. These INR values were used for dose adjustments. The system seemed to work well to maintain blinding.

The double-blind design is no protection against imbalances in use of concomitant medications. A placebo-controlled trial of a long-acting inhaled anticholinergic medication in participants with chronic obstructive pulmonary disease allowed the use of any other available drug treatment for this condition as well as a short-acting inhaled anticholinergic agent for acute exacerbations [21]. The extent of this co-intervention is likely to differ between the actively treated and the placebo groups, but the findings regarding concomitant drug use by study group were not presented. Moreover, it may have influenced symptomology as well as risks of disease events and made it very difficult to determine the true effects of the long-acting anticholinergic inhaler. Reporting the proportion of participants given a co-intervention at any time over the four years of the trial by treatment group would have helped the interpretation of results, even if the frequency and intensity of its use were not reported.

In many single- and double-blind drug trials the control group is placed on a matched placebo. Much debate has centered on the ethics of using a placebo. See Chap. 2 for a further discussion of this issue.

Triple-Blind

A triple-blind study is an extension of the double-blind design; the committee monitoring response variables is not told the identity of the groups. The committee is simply given data for groups A and B. A triple-blind study has the theoretical advantage of allowing the monitoring committee to evaluate the response variable results more objectively. This assumes that appraisal of efficacy and harm, as well as requests for special analyses, may be biased if group identity is known. However, in a trial where the monitoring committee has an ethical responsibility to ensure participant safety, such a design may be counterproductive. When hampered in the safety-monitoring role, the committee cannot carry out its responsibility to minimize harm to the participants, since monitoring is often guided by the constellation of trends and their directions. In addition, even if the committee could discharge its duties adequately while being kept blinded, many investigators would be uneasy participating in such a study. Though in most cases the monitoring committee looks only at group data and can rarely make informed judgments about individuals, the investigators still rely on the committee to safeguard their study participants. This may not be a completely rational approach because, by the time many monitoring committees receive data, often any emergency situation has long passed. Nevertheless, the discomfort many investigators feel about participating in double-blind studies would be magnified should the data monitoring committee also be kept blinded.

Finally, people tend not to accept beneficial outcomes unless a statistically significant difference has been achieved. Rarely, though, will investigators want to continue a study in order to achieve a clearly significant difference in an adverse direction; that is, until the intervention is statistically significantly worse or more harmful than the control. Therefore, many monitoring committees demand to know which study groups are on which intervention.

A triple-blind study can be conducted ethically if the monitoring committee asks itself at each meeting whether the direction of observed trends matters. If it does not matter, then the triple-blind can be maintained, at least for the time being. This implies that the monitoring committee can ask to be unblinded at any time it chooses. In the Randomized Aldactone Evaluation Study (RALES), the Data and Safety Monitoring Board was split and several members argued against being blinded [22]. However, triple-blind was employed initially. For most outcome variables, the treatment groups were labeled A and B. Since increased rates of gynecomastia and hyperkalemia, which might occur with aldactone, would unmask the A and B assignments, these adverse events were labeled X and Y. Using different labels for these adverse events prevented unblinding of the other outcome variables.

Triple blinding may at times be useful, but if trends in important clinical outcomes or an imbalance in adverse effects develop, it is no longer appropriate.

Protecting the Double-Blind Design

Double-blind studies are more difficult to carry out than other trials. One must ensure that the investigator team remains blinded and that any data which conceivably might endanger blinding be kept from them during the study. An effective data monitoring scheme must be set up, and emergency unblinding procedures must be established. These requirements pose their own problems and can increase the cost of a study. Page and Persch [23] discuss strategies for blinding health care providers and data collectors. The latter ought to be different from those providing medical care for the participants.

An old illustration is the Aspirin Myocardial Infarction Study [24], a double-blind trial of aspirin in people with coronary heart disease, in which the investigators wished to monitor the action of aspirin on platelets. A postulated beneficial effect of aspirin relates to its ability to reduce the aggregation of platelets. Therefore, measuring platelet aggregation provided both an estimate of whether the aspirin treated group was getting a sufficient dose and a basis for measurement of participant adherence. However, tests of platelet aggregation needed to be performed shortly after the blood sample was drawn. The usual method used to have a laboratory technician insert the specimen in an aggregometer, add a material such as epinephrine (which, in the absence of aspirin, causes platelets to aggregate) and analyze a curve which is printed on a paper strip. In order to maintain the blind, the study needed to find a way to keep the technician from seeing the curve. Therefore, a cassette tape-recorder was substituted for the usual paper strip recorder and the indicator needle was covered. These changes required a modification of the aggregometer. All of the 30 clinics required this equipment, so the adjustment was expensive. However, it helped ensure the maintenance of the blind.

A double-blind design is a particular problem in clinical trials of treatments other than the use of pharmaceuticals. Methods of blinding procedures in 123 reports of nonpharmacological trials were systematically reviewed by Boutron et al. [25]. Three categories were classified: surgical or technical procedures, participative interventions, and devices. Most of the reports used some form of sham procedure. For surgical interventions the sham procedure was a simulating intervention. The controls in participative interventions were either an attention-control intervention or a differently administered placebo. The device trials used sham prosthesis, identical apparatus, and simulation of a device. A small number of nonpharmacological trials blinded the participants to the study hypothesis. A critical approach employed in one-third of the trials was a blinded, centralized assessment of the primary outcome.

Protecting the double-blind can be a special problem in active-control trials, i.e. trials comparing active interventions. The adverse drug effect patterns for the drugs being compared can be distinctly different. When the selective serotonin receptor inhibitors were introduced they were compared to tricyclic antidepressants. The latter are anticholinergic and commonly cause dryness of mouth, blurred vision and tachycardia. The occurrence of these adverse effects unblinded treatment in a large number of participants in 20 comparative trials [26].

Naturally, participants want to be on the “better” intervention. In a drug trial, the “better” intervention usually is presumed to be the new one; in the case of a placebo-control trial it is presumed to be the active medication. Investigators may also be curious about a drug’s identity. For these reasons, consciously or subconsciously, both participants and investigators may try to unblind the medication. Unblinding can be done deliberately by going so far as to have the drug analyzed, or in a less purposeful manner by “accidentally” breaking open capsules, holding pills up to the light, carefully testing them, or by taking any of numerous other actions. In the first case, which may have occurred in the vitamin C study discussed earlier, little can be done to ensure blinding absolutely. Curious participants and investigators can discover many ways to unblind the trial, whatever precautions are taken. Probably, however, the less purposeful unblinding is more common.

We strongly recommend that the assessment of trial outcomes be as objective as possible. This means that the person at the clinic making these assessments be blinded. At times, this may be done at a central location.

Matching of Drugs

Drug studies, in particular, lend themselves to double-blind designs. One of the surest ways to unblind a drug study is to have dissimilar appearing medications. When the treatment identity of one participant becomes known to the investigator, the whole trial is unblinded. Thus, matching of drugs is essential.

Proper matching has received little attention in the literature. A notable exception is the vitamin C study [10, 11] in which of the double-blind was not maintained throughout the trial. One possible reason given by the investigators was that, in the rush to begin the study, the contents of the capsules were not carefully produced. The lactose placebo could easily be distinguished from ascorbic acid by taste, as the study participants quickly discovered. An early report showed similar concern [27]. The authors noted that, of 22 studies surveyed, only five had excellent matching between the drugs being tested. A number of features of matching must be considered. A review of 191 randomized placebo-controlled trials from leading general medicine and psychiatry trials showed that 81 (42%) trials reported on the matching of drug characteristics [28]. Only 19 (10%) commented on more than one of the matching features and appearance was, by far, the most commonly reported characteristic. Thus, most reports of drug studies do not indicate how closely tablets or capsules resembled one another, or how great a problem was caused by imperfect matching.

Cross-over studies, where each subject sees both medications, require the most care in matching. Visual discrepancies can occur in size, shape, color, and texture. Ensuring that these characteristics are identical may not be simple. In the case of tablets, dyes or coatings may adhere differently to the active ingredient than to the placebo, causing slight differences in color or sheen. Agents can also differ in odor. The taste and the local action on the tongue of the active medication are likely to be different than those of the placebo. For example, propranolol is a topical anesthetic which causes lingular numbness if held in the mouth. Farr and Gwaltney reported on problems in matching zinc lozenges against placebo [29]. Because zinc lozenges are difficult to blind, the authors questioned whether studies using zinc for common cold prevention were truly valid. They conducted trials illustrating that if a placebo is inadequately matched, the “unpleasant side effects of zinc” may reduce the perception of cold symptoms.

Drug preparations should be pretested if it is possible. One method is to have a panel of observers unconnected with the study compare samples of the medications. Perfect matches are almost impossible to obtain and some differences are to be expected. Preparing placebos for trial of herbal medicines can be a challenge. One way is the use of appropriately matched placebo capsules, an approach applied successfully [30]. However, beyond detecting differences, it is important to assess whether the observers can actually identify the agents. If not, slightly imperfect matches may be tolerated. The investigator must remember that, except in cross-over studies, the participant has only one drug and is therefore not able to make a comparison. On the other hand, participants may meet and talk in waiting rooms, or in some other way compare notes or pills. Of course, staff always have the opportunity to compare different preparations and undermine the integrity of a study.

Differences may become evident only after some time, due to degradation of the active ingredient. Freshly prepared aspirin is relatively odor free, but after a while, tell-tale acetic acid accumulates. Ginkgo biloba has a distinct odor and a bitter taste. In one trial of Ginkgo, the investigators used coated tablets to mask both odor and taste [31]. The tablets were placed in blister packs to reduce the risk of odor. Quinine was added to the placebo tablets to make them as bitter as the active drug. This approach prevented any known blind-breaking.

Use of substances to mask characteristic taste, color, or odor, as was done in the ginkgo biloba trial mentioned above, is often advocated. Adding vanilla to the outside of tablets may mask an odor; adding dyes will mask dissimilar colors. A substance such as quinine or quassin will impart a bitter taste to the preparations. Not only will these chemical substances mask differences in taste, but they will also effectively discourage participants from biting into a preparation more than once. However, the possibility that they may have toxic effects after long-term use or even cause allergic reactions in a small percent of the participants must be considered. It is usually prudent to avoid using extra substances unless absolutely essential to prevent unblinding of the study.

Less obviously, the weight or specific gravity of the tablets may differ. Matching the agents on all of these characteristics may be impossible. However, if a great deal of effort and money are being spent on the trial, a real attempt to ensure matching makes sense. The investigator also needs to make sure that the containers are identical. Bottles and vials need to be free of any marks other than codes which are indecipherable except with the key.

Sometimes, two or more active drugs are being compared. The ideal method of blinding is to have the active agents look alike, either by formulating them appropriately or possibly by enclosing them in identical capsules. The former may not be possible, and the latter may be expensive or require capsules too large to be practical. In addition, enclosing tablets in capsules may change the rate of absorption and the time to treatment response. In a comparative acute migraine trial, one manufacturer benefitted from encapsulating a competitor’s FDA-approved tablet in a gelatin capsule [32]. A better, simpler and more common option is to implement a “double-dummy.” Each active agent has a placebo identical to it. Each study participant would then take two medications. This is a good approach when the administration of the two drugs being compared is different, for example, when a once daily drug is being compared to a twice daily drug. A pharmaceutical sponsor may sometimes have problems finding a matching placebo for a competitor’s product.

Sometimes, if two or more active agents are being compared against placebo, it may not be feasible to make all drugs appear identical. As long as each active agent is not being compared against another, but only against placebo, one option is to create a placebo for each active drug or a so-called “double-dummy.” Another option is to limit the number of placebos. For example, assume the trial consists of active drugs A, B, and C and placebo groups. If each group is the same size, one third of placebo groups will take a placebo designed to look like active drug A, one third will take a placebo designed to look like drug B, and one third, like active drug C. This design was successfully implemented in at least one reported study [33].

Coding of Drugs

By drug coding is meant the labeling of individual drug bottles or vials so that the identity of the drug is not disclosed. Coding is usually done by means of assigning a random set of numbers to the active drug and a different set to the control. Each participant should have a unique drug code which remains with him for the duration of the trial. If only one code is used for each study group, unblinding a single participant would result in unblinding everybody. Furthermore, many drugs have specific side effects. One side effect in one participant may not be attributable to the drug, but a constellation of several side effects in several participants with the same drug code may easily unblind the whole study.

In large studies it is possible through use of computer programs to make up and stock drugs under hundreds or thousands of unique codes. Bar coding of the bottles with study medication is now common. This type of coding has no operational limits on the number of unique codes, it simplifies keeping an accurate and current inventory of all study medications and helps assure that each participant is dispensed his assigned study medication.

Official Unblinding

A procedure should be developed to break the blind quickly for any individual participant at any time should it be in his best interest. Such systems include having labels on file in the hospital pharmacy or other accessible locations, or having an “on call” 24 hour-a-day process so that the assignment can be decoded. In order to avoid needless breaking of the code, someone other than the investigator could hold a list that reveals the identity of each drug code. Alternatively, each study medication bottle label might have a sealed tear-off portion that would be filed in the pharmacy or with the participant’s records. In an emergency, the seal could be torn and the drug identity revealed. In one study, the sealed labels attached to the medication bottles were transparent when held up to strong light. Care should be taken to ensure that the sealed portion is of appropriate color and thickness to prevent reading through it.

Official breaking of the blind may be necessary. There are bound to be situations that require disclosures, especially in long-term studies. Perhaps the study drug requires tapering the dosage. Children may get hold of study pills and swallow them. In an emergency, knowledge that a participant is or is not on the active drug would indicate whether tapering is necessary. Usually, most emergencies can be handled by withdrawing the medication without breaking the blind. When the treating physician is different from the study investigator, a third party can obtain the blinded information from the pharmacy or central data repository and relate the information to the treating physician. In this way, the participant and the study investigator need not be unblinded. Knowledge of the identity of the study intervention seldom influences emergency care of the participant. This information is important for treating physicians to know since it can help reduce the frequency of unnecessary unblinding. When unblinding does occur, the investigator should review and report the circumstances which led to it in the results paper.

In summary, double-blind trials require careful planning and constant monitoring to ensure that the blind is maintained and that participant safety is not jeopardized.

Inadvertent Unblinding

The phrase “truly double-blind study” was used earlier. While many studies are designed as double- or single-blind, it is unclear how many, in fact, are truly and completely blind. Drugs have side effects, some of which are fairly characteristic. Known pharmaceutical effects of the study medication may lead to unblinding. Inhalation of short-acting beta-agonists causes tremor and tachycardia within minutes in most users. Even the salt of the active agent can cause side effects that lead to unblinding. For example, the blinded design was broken in a clinical trial comparing the commonly used ranitidine hydrochloride to a new formulation of ranitidine bismuth citrate. The bismuth-containing compound colored the tongue of its users black [34]. Rifampin, a treatment for tuberculosis, causes the urine to change color. Existence or absence of such side effects does not necessarily unblind drug assignment, since not all people on drugs do develop reactions and some people on placebo develop events which can be mistaken for drug side effects. In trials of warfarin vs. oral anticoagulants, healthcare providers often check the INR in the event of significant bleeding. Since it is elevated with warfarin, this is likely to lead to unblinding. It is well known that aspirin is associated with gastrointestinal problems. In the Women’s Health Study [35], 2.7% of the participants in the low-aspirin group developed peptic ulcer. On the other hand, 2.1% of the placebo participants had the same condition. This difference is highly significant (p < 0.001), but for a participant having an ulcer, in itself, would not unblind.

Occasionally, accidental unblinding occurs. In some studies, a special center labels and distributes drugs to the clinic where participants are seen. Obviously, each carton of drugs sent from the pharmaceutical company to this distribution center must contain a packing slip identifying the drug. The distribution center puts coded labels on each bottle and removes the packing slip before sending the drugs to the investigator. In one instance, one carton contained two packing slips by mistake. The distribution center, not realizing this, shipped the carton to the investigator with the second packing slip enclosed.

Laboratory errors have also occurred. These are particularly likely when, to prevent unblinding, only some laboratory results are given to the investigators. Occasionally investigators have received the complete set of laboratory results. This usually happens at the beginning of a study before “bugs” have been worked out, or when the laboratory hires new personnel who are unfamiliar with the procedures. If a commercial laboratory performs the study determinations, the tests should be done in a special area of the laboratory, with safeguards to prevent study results from getting intermingled with routine work. Routine laboratory panels obtained during regular clinical care of patients may include laboratory results that could lead to unblinding. In a large, long-term trial of a lipid-lowering drug, the investigators were discouraged from getting serum cholesterol determination on their coronary patients. It is difficult to know how many complied.

In addition, monitoring the use of study medication prescribed outside the study is essential. Any group differences might be evidence of a deficiency in the blind. Another way of estimating the success of a double-blind design is to monitor specific intermediate effects of the study medication. The use of platelet aggregation in the Aspirin Myocardial Infarction Study is an example. An unusually large number of participants with non-aggregating platelets in the placebo group would raise the suspicion that the blind had been broken.

Assessment and Reporting of Blinding

The importance of blinding in avoiding bias is well established in clinical trials. However, the assessment and reporting of blinding do not always receive proper attention. Readers of trial reports are often given incomplete information about the type of blinding during the trial. This is a potential concern since randomized trials with inadequate blinding, on average, show larger treatment effects than properly blinded trials [36].

In their systematic review of 819 articles of blinded randomized trials assessing pharmacologic treatment, Boutron et al. [25] considered three blinding methods—(1) the initial blinding of participants and investigators, (2) the maintenance of this blinding and, (3) the blinding of those assessing trial outcomes. Overall, only 472 of the blinded reports (58%) described the method of blinding, while 13% gave some information and 29% none at all. The methods to establish blinding were presented in 41% of the reports. These included different types of matching, the use of a “double-dummy” procedure, sham interventions and masking of the specific taste of the active treatments. The methods for blinded assessment were described in 14% of the reports. They are especially useful in trials when blinding cannot be established. The main method was a centralized assessment of the primary outcome by blinded classification committees.

In a survey of 191 placebo-controlled double-blind trials published in 1998–2001, the authors evaluated how often the success of blinding was reported [28]. Only 15 (8%) reported evidence of success, and of these 15 trials, blinding was imperfect in nine. A similar survey of 1,599 blinded randomized trials from 2001 reported that only 2% of the trials reported tests for the success of blinding [37]. Interestingly, many investigators had conducted, but not published such tests. A report on the quality of reporting of randomized clinical trials in medical oncology between 2005 and 2009 according to the 2001 CONSORT statement showed that numerous items remained unreported. Only 41% of the 347 trials clearly reported whether and how blinding was applied [38]. A similar review of 442 trials in the psychiatric literature showed that the reporting of how blinding was accomplished and evaluated decreased following the publication of the CONSORT statement [39].

Although the success of blinding may be important, it is not easy to access and few trial publications provide this information. If done, there are different views as to whether and when to assess blinding—early after randomization, throughout the trial or at the end [40]. Early assessment in a double-blind trial would be a measure of the initial success of blinding. Repeated questioning may trigger the curiosity of the study participants. Assessment at the end “confounds failures in blinding with successes in pre-trial hunches about efficacy” [41]. If study participants do well, there is a tendency for them to predict that they received active treatment; if they have suffered events or perceived no improvement, their prediction is more likely to be placebo. Similarly, investigators’ hunches about efficacy can also be influenced by their preconceived expectations as illustrated by Sackett [42]. He concluded that “We neither can nor need to test for blindness during and after trial, …”. The CONSORT 2010 statement eliminated the 2001 recommendation to assess how the success of blinding was assessed [43].

The CONSORT 2010 [43] and the SPIRIT 2013 [44] guidelines have a checklist of items recommended to be included in trial protocols Both have a similar item for blinded trials asking who was blinded after assignment to interventions (e.g. participants, care providers, outcome assessors, data analysts) and how. The former has a second item asking for “if relevant, description of the similarity of interventions”. The latter has a second item asking for a description of “If blinded, circumstances under which unblinding is permissible and procedure for revealing a participant’s allocated intervention during the trial”.

Debriefing of Participants

Typically, participants randomized to blinded trials are never informed which treatment they received [39]. Various reasons are given by the investigators for not debriefing trial participants. There is strong and consistent evidence from recent large randomized clinical trials that trial participants would welcome being told which treatment they received. Whether investigators have ethical or other obligations to provide feedback is debated. No guidelines exist regarding debriefing of treatment received. However, it has been emphasized that clinically useful findings ought to be shared.

Debriefing to placebo allocation appears to raise more issues. Three theoretical concerns have been brought up: first, participants who benefited from placebo treatment might relapse on debriefing; second, the debriefing may engender mistrust and harm future doctor-patient relationships; and, third, the debriefing may have negative consequences for participants. However, the support for these concerns has been mixed [45].

A survey of participants in 14 randomized clinical trials in Parkinson’s disease reported that 54% remembered being surprised or shocked [46]. Twenty-eight percent felt “disappointed”. However, the respondents were overall positive and, most importantly, were willing to consider participating in future trials.

We favor that the investigators debrief the trial participants in person at the trial completion about the trial findings and their treatment group assignments. This ought to be part of transfer of post-trial care (see Chap. 20 on Closeout).

References

1.

Mausner JS, Bahn AK. Epidemiology: An Introductory Text. Philadelphia: W.B. Saunders, 1974.

2.

Kaptchuk TJ. Intentional Ignorance: A history of blind assessment and placebo controls in medicine. Bull Hist Med 1998;72:389–433.CrossRef

3.

Amberson JB, Jr, McMahon BT, Pinner M. A clinical trial of sanocrysin in pulmonary tuberculosis. Am Rev Tuberc 1931;24:401–435.

4.

Devereaux PJ, Manns BJ, Ghali WA, et al. Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials. JAMA 2001;285:2000–2003.CrossRef

5.

Viergever RF, Ghersi D. Information on blinding in registered records of clinical trials. Trials 2012;13:210.CrossRef

6.

Haahr MT, Hrobjartsson A. Who is blinded in randomized clinical trials? A study of 200 trials and a survey of authors. Clin Trials 2006;3:360–365.CrossRef

7.

Park J, White AR, Stevinson C, Ernst E. Who are we blinding? A systematic review of blinded clinical trials. Perfusion 2001;14:296–304.

8.

Hansson L, Hedner T, Dahlöf B. Prospective randomized open blinded end-point (PROBE) study. A novel design for intervention trials. Prospective Randomized Open Blinded End-Point. Blood Press 1992;1:113–119.CrossRef

9.

The Women’s Health Initiative Study Group. Design of the Women’s Health Initiative clinical trial and observational study. Control Clin Trials 1998;19:61–109.CrossRef

10.

Karlowski TR, Chalmers TC, Frenkel LD, et al. Ascorbic acid for the common cold. A prophylactic and therapeutic trial. JAMA 1975;231:1038–1042.CrossRef

11.

Lewis TL, Karlowski TR, Kapikian AZ, et al. A controlled clinical trial of ascorbic acid for the common cold. Ann NY Acad Sci 1975;258:505–512.CrossRef

12.

European Coronary Surgery Study Group. Coronary-artery bypass surgery in stable angina pectoris: survival at two years. Lancet 1979;i:889–893.

13.

Schechter PJ, Friedewald WT, Bronzert DA, et al. Idiopathic hypogeusia: a description of the syndrome and a single-blind study with zinc sulfate. Int Rev Neurobiol 1972;(suppl 1):125–140.

14.

Henkin RI, Schechter PJ, Friedewald WT, et al. A double blind study of the effects of zinc sulfate on taste and smell dysfunction. Am J Med Sci 1976;272:285–299.CrossRef

15.

Halperin JL, and The Executive Steering Committee on behalf of the SPORTIF III and V Study Investigators. Ximelagatran compared with warfarin for prevention of thromboembolism in patients with nonvalvular atrial fibrillation: Rationale, objectives, and design of a pair of clinical studies and baseline patient characteristics (SPORTIF III and V). Am Heart J 2003;146:431–438.CrossRef

16.

Bhatt DL, Kandzari DE, O’Neill WW, et al. for the SYMPLICITY HTN-3 Investigators. A controlled trial of renal denervation for resistant hypertension. N Engl J Med 2014;370:1393–1401.

17.

Køber L, Torp-Pedersen C, McMurray JJV, et al. for the Dronedarone Study Group. Increased mortality after dronedarone therapy for severe heart failure. N Engl J Med 2008;358:2678–2682.CrossRef

18.

Hertzberg V, Chimowitz M, Lynn M, et al. Use of dose modification schedules is effective for blinding trials of warfarin: evidence from the WASID study. Clin Trials 2008;5:25–30.

19.

Coumadin Aspirin Reinfarction Study (CARS) Investigators. Randomised double-blind trial of fixed low-dose warfarin with aspirin after myocardial infarction. Lancet 2008;350:389–396.

20.

SPORTIF Executive Steering Committee for the SPORTIF V Investigators. Ximelagatran vs warfarin for stroke prevention in patients with nonvalvular atrial fibrillation. A randomized trial. JAMA 2005;293:690–698.

21.

Tashkin DP, Celli B, Senn S, et al. for the UPLIFT Study Investigators. A 4-year trial of tiotropium in chronic obstructive pulmonary disease. N Engl J Med 2008;359:1543–1554.CrossRef

22.

Wittes J, Boissel J-P, Furberg CD, et al. Stopping the Randomized Aldactone Evaluation Study early for efficacy. In DeMets DL, Furberg CD, Friedman LM, (eds.). Data Managing in Clinical Trials. A Case Study Approach. New York: Springer, 2006, pp. 148–157.

23.

Page SJ, Persch AC. Recruitment, retention, and blinding in clinical trials. Am J Occup Ther 2013;67:154–161.CrossRef

24.

Aspirin Myocardial Infarction Study Research Group. A randomized, controlled trial of aspirin in persons recovered from myocardial infarction. JAMA 1980;243:661–669.CrossRef

25.

Boutron I, Guittet L, Estellat C, et al. Reporting methods of blinding in randomized trials assessing nonpharmacological treatments. PLoS Med 2007;4:370–378.CrossRef

26.

von Wolff A, Hölzel LP, Westphal A, et al. Selective serotonin reuptake inhibitors and tricyclic antidepressants in the acute treatment of chronic depression and dysthymia: a systematic review and meta-analysis. J Affect Disord 2013;144:7–15.CrossRef

27.

Hill LE, Nunn AJ, Fox W. Matching quality of agents employed in “double-blind” controlled clinical trials. Lancet 1976;i:352–356.

28.

Fergusson D, Glass KC, Waring D, Shapiro S. Turning a blind eye: the success of blinding reported in a random sample of randomised, placebo controlled trials. Br Med J 2004;328:432.CrossRef

29.

Farr BM, Gwaltney JM Jr. The problems of taste in placebo matching: an evaluation of zinc gluconate for the common cold. J Chronic Dis 1987;40:875–879.CrossRef

30.

Fai CK, De Qi G, Wei DA, Chung LP. Placebo preparation for the proper clinical trial of herbal medicine – requirements, verification and quality control. Recent Pat Inflamm Allergy Drug Discov 2011;5:169–174.CrossRef

31.

DeKosky ST, Williamson JD, Fitzpatrick AL, et al. for the Ginkgo Evaluation of Memory (GEM) Study Investigators. Ginkgo biloba for prevention of dementia. A randomized controlled trial. JAMA 2008;300:2253–2262.CrossRef

32.

Mathew NT, Schoenen J, Winner P, et al. Comparative efficacy of eletriptan 40 mg versus sumatriptan 100 mg. Headache 2003;43:214–222.CrossRef

33.

The CAPS Investigators. The Cardiac Arrhythmia Pilot Study. Am J Cardiol 1986;57:91–95.CrossRef

34.

Pacifico L, Osborn JF, Anania C, et al. Review article: bismuth-based therapy for helicobacter pylori eradication in children. Aliment Pharmacol Ther 2012;35:1010–1026.

35.

Ridker PM, Cook NP, Lee I-M, et al. A randomized trial of low-dose aspirin in the primary prevention of cardiovascular disease in women. N Engl J Med 2005;352:1293–1304.CrossRef

36.

Schulz KF, Chalmers I, Hayes R, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 1995;273:408–412.CrossRef

37.

Hróbjartsson A, Forfang E, Haahr MT, et al. Blinded trials taken to the test: an analysis of randomized clinical trials that report tests for the success of blinding. Int J Epidemiol 2007;36:654–663.CrossRef

38.

Peron J, Pond GR, Gan HK, et al. Quality of reporting of modern randomized controlled trials in medical oncology: a systematic review. J Natl Cancer Inst 2012;104:982–989.CrossRef

39.

Han C, Kwak K, Marks DM, et al. The impact of the CONSORT statement on reporting of randomized clinical trials in psychiatry. Comtemp Clin Trials 2009;30:116–122.CrossRef

40.

Boutron I, Estellat C, Ravaud P. A review of blinding in randomized controlled trials found results inconsistent and questionable. J Clin Epidemiol 2005;58:1220–1226.CrossRef

41.

Sackett DL. Turning a blind eye. Why we don’t test for blindness at the end of our trials. (Letter) Br Med J;328:1136.

42.

Sackett DL. Commentary: Measuring the success of blinding in RCTs: don’t, must, can’t or needn’t? Int J Epidemiol 2007;36:664–665.CrossRef

43.

Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. BMJ 2010;340:698–702.CrossRef

44.

SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ 2013;346:e7586.

45.

Bishop FL, Jacobson EE, Shaw J, Kaptchuk TJ. Participants’ experiences of being debriefed to placebo allocation in a clinical trial. Qual Health Res 2012;22:1138—1149.CrossRef

46.

Goetz CG, Janko K, Blasucci L, Jaglin JA. Impact of placebo assignment in clinical trials of Parkinson’s disease. Movement Disorders 2003;18:1146–1149.CrossRef

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_8

8. Sample Size

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

The size of the study should be considered early in the planning phase. In some instances, no formal sample size is ever calculated. Instead, the number of participants available to the investigators during some period of time determines the size of the study. Many clinical trials that do not carefully consider the sample size requirements turn out to lack the statistical power or ability to detect intervention effects of a magnitude that has clinical importance. In 1978, Freiman and colleagues [1] reviewed the power of 71 published randomized controlled clinical trials which failed to find significant differences between groups. “Sixty-seven of the trials had a greater than 10% risk of missing a true 25% therapeutic improvement, and with the same risk, 50 of the trials could have missed a 50% improvement.” The situation was not much improved in 1994, when a similar survey found only 16% of negative trials had 80% power for a 25% effect, and only 36% for a 50% effect [2]. In other instances, the sample size estimation may assume an unrealistically large intervention effect. Thus, the power for more realistic intervention effects will be low or less than desired. The danger in studies with low statistical power is that interventions that could be beneficial are discarded without adequate testing and may never be considered again. Certainly, many studies do contain appropriate sample size estimates, but in spite of many years of critical review many are still too small [3, 4].

This chapter presents an overview of sample size estimation with some details. Several general discussions of sample size can be found elsewhere [5–21]. For example, Lachin [11] and Donner [9] have each written a more technical discussion of this topic. For most of the chapter, the focus is on sample size where the study is randomizing individuals. In the some sections, the concept of sample size for randomizing clusters of individuals or organs within individuals is presented.

Fundamental Point

Clinical trials should have sufficient statistical power to detect differences between groups considered to be of clinical importance. Therefore, calculation of sample size with provision for adequate levels of significance and power is an essential part of planning.

Before a discussion of sample size and power calculations, it must be emphasized that, for several reasons, a sample size calculation provides only an estimate of the needed size of a trial [6]. First, parameters used in the calculation are estimates, and as such, have an element of uncertainty. Often these estimates are based on small studies. Second, the estimate of the relative effectiveness of the intervention over the control and other estimates may be based on a population different from that intended to be studied. Third, the effectiveness is often overestimated since published pilot studies may be highly selected and researchers are often too optimistic. Fourth, during the final planning stage of a trial, revisions of inclusion and exclusion criteria may influence the types of participants entering the trial and thus alter earlier assumptions used in the sample size calculation. Assessing the impact of such changes in criteria and the screening effect is usually quite difficult. Fifth, trial experience indicates that participants enrolled into control groups usually do better than the population from which the participants were drawn. The reasons are not entirely clear. One factor could be that participants with the highest risk of developing the outcome of interest are excluded in the screening process. In trials involving chronic diseases, because of the research protocol, participants might receive more care and attention than they would normally be given, or change their behavior because they are part of a study, thus improving their prognosis, a phenomenon sometimes called the Hawthorne or trial effect [22]. Also, secular trends toward improved care may result in risk estimates from past studies being higher than what will be found in current patient populations [23]. Participants assigned to the control group may, therefore, be better off than if they had not been in the trial at all. Finally, sample size calculations are based on mathematical models that may only approximate the true, but unknown, distribution of the response variables.

Due to the approximate nature of sample size calculations, the investigator should be as conservative as can be justified while still being realistic in estimating the parameters used in the calculation. If a sample size is drastically overestimated, the trial may be judged as unfeasible. If the sample size is underestimated, there is a good chance the trial will fall short of demonstrating any differences between study groups or be faced with the need to justify an increase in sample size or an extension of follow-up [24–26]. In general, as long as the calculated sample size is realistically obtainable, it is better to overestimate the size and possibly terminate the trial early (Chap. 16) than to modify the design of an ongoing trial, or worse, to arrive at incorrect conclusions.

Statistical Concepts

An understanding of the basic statistical concepts of hypothesis testing, significance level, and power is essential for a discussion of sample size estimation. A brief review of these concepts follows. Further discussion can be found in many basic medical statistics textbooks [27–37] as well as textbooks on sample size [17–21]. Those with no prior exposure to these basic statistical concepts might find these resources helpful.

Except where indicated, trials of one intervention group and one control group will be discussed. With some adjustments, sample size calculations can be made for studies with more than two groups [8]. For example, in the Coronary Drug Project (CDP), five active intervention arms were each compared against one control arm [38]. The Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack trial (ALLHAT) compared four active intervention arms: three newer drugs to an older one as first line therapy for hypertension [39]. Both trials used the method of Dunnett [40], where the number of participants in the control group is equal to the number assigned to each of the active intervention groups times the square root of the number of active groups. The optimal size of the control arm in the CDP was determined to be 2.24 times the size of each individual active intervention arm [38]. In fact, the CDP used a factor of 2.5 in order to minimize variance. Other approaches are to use the Bonferroni adjustment to the alpha level [41]; that is, divide the overall alpha level by the number of comparisons, and use that revised alpha level in the sample size comparison.

Before computing sample size, the primary response variable used to judge the effectiveness of intervention must be identified (see Chap. 3). This chapter will consider sample size estimation for three basic kinds of outcomes: (1) dichotomous response variables, such as success and failure (2), continuous response variables, such as blood pressure level or a change in blood pressure, and (3) time to failure (or occurrence of a clinical event).

For the dichotomous response variables, the event rates in the intervention group (p I) and the control group (p C) are compared. For continuous response variables, the true, but unknown, mean level in the intervention group (μI) is compared with the mean level in the control group (μC). For survival data, a hazard rate, λ, is often compared for the two study groups or at least is used for sample size estimation. Sample size estimates for response variables which do not exactly fall into any of the three categories can usually be approximated by one of them.

In terms of the primary response variable, p I will be compared with p C or μI will be compared with μC. This discussion will use only the event rates, p I , and p C , although the same concepts will hold if response levels μI and μC are substituted appropriately. Of course, the investigator does not know the true values of the event rates. The clinical trial will give him only estimates of the event rates, $$ \widehat{p_{I\ }}\ \mathrm{and}\ \widehat{p_C} $$. Typically, an investigator tests whether or not a true difference exists between the event rates of participants in the two groups. The traditional way of indicating this is in terms of a null hypothesis, denoted H ₀, which states that no difference between the true event rates exists (H ₀: p C − p I = 0). The goal is to test H ₀ and decide whether or not to reject it. That is, the null hypothesis is assumed to be true until proven otherwise.

Because only estimates of the true event rates are obtained, it is possible that, even if the null hypothesis is true (p C − p I = 0), the observed event rates might by chance be different. If the observed differences in event rates are large enough by chance alone, the investigator might reject the null hypothesis incorrectly. This false positive finding, or Type I error, should be made as few times as possible. The probability of this Type I error is called the significance level and is denoted by α. The probability of observing differences as large as, or larger than the difference actually observed given that H ₀ is true is called the “p-value,” denoted as p. The decision will be to reject H ₀ if p ≤ α. While the chosen level of α is somewhat arbitrary, the ones used and accepted traditionally are 0.01, 0.025 or, most commonly, 0.05. As will be shown later, as α is set smaller, the required sample size estimate increases.

If the null hypothesis is not in fact true, then another hypothesis, called the alternative hypothesis, denoted by H A, must be true. That is, the true difference between the event rates p C and p I is some value δ where δ ≠ 0. The observed difference $$ \widehat{p_{\mathrm{C}\ }}-\widehat{p_{\mathrm{I}}} $$ can be quite small by chance alone even if the alternative hypothesis is true. Therefore, the investigator could, on the basis of small observed differences, fail to reject H ₀ even when it is not true. This is called a Type II error, or a false negative result. The probability of a Type II error is denoted by β. The value of β depends on the specific value of δ, the true but unknown difference in event rates between the two groups, as well as on the sample size and α. The probability of correctly rejecting H ₀ is denoted 1 − β and is called the power of the study. Power quantifies the potential of the study to find true differences of various values δ. Since β is a function of α, the sample size and δ, 1 − β is also a function of these parameters. The plot of 1 − β versus δ for a given sample size is called the power curve and is depicted in Fig. 8.1. On the horizontal axis, values of δ are plotted from 0 to an upper value, δA (0.25 in this figure). On the vertical axis, the probability or power of detecting a true difference δ is shown for a given significance level and sample size. In constructing this specific power curve, a sample size of 100 in each group, a one-sided significance level of 0.05 and a control group event rate of 0.5 (50%) were assumed. Note that as δ increases, the power to detect δ also increases. For example, if δ = 0.10 the power is approximately 0.40. When δ = 0.20 the power increases to about 0.90. Typically, investigators like to have a power (1 − β) of at least 0.80, but often around 0.90 or 0.95 when planning a study; that is to have an 80%, 90% or 95% chance of finding a statistically significant difference between the event rates, given that a difference, δ, actually exists.

Fig. 8.1

A power curve for increasing differences (δ) between the control group rate of 0.5 and the intervention group rate with a one-sided significance level of 0.05 and a total sample size (2N) of 200

Since the significance level α should be small, say 0.05 or 0.01, and the power (1 − β) should be large, say 0.90 or 0.95, the only quantities which are left to vary are δ, the size of the difference being tested for, and the total sample size. In planning a clinical trial, the investigator hopes to detect a difference of specified magnitude δ or larger. One factor that enters into the selection of δ is the minimum difference between groups judged to be clinically important. In addition, previous research may provide estimates of δ. This is part of the question being tested as discussed in Chap. 3. The exact nature of the calculation of the sample size, given α, 1 − β and δ is considered here. It can be assumed that the randomization strategy will allocate an equal number (N) of participants to each group, since the variability in the responses for the two groups is approximately the same; equal allocation provides a slightly more powerful design than unequal allocation. For unequal allocation to yield an appreciable increase in power, the variability needs to be substantially different in the groups [42]. Since equal allocation is usually easier to implement, it is the more frequently used strategy and will be assumed here for simplicity.

Before a sample size can be calculated, classical statistical theory says that the investigator must decide whether he is interested in differences in one direction only (one-sided test)—say improvements in intervention over control—or in differences in either direction (two-sided test). This latter case would represent testing the hypothesis that the new intervention is either better or worse than the control. In general, two-sided tests should be used unless there is a very strong justification for expecting a difference in only one direction. An investigator should always keep in mind that any new intervention could be harmful as well as helpful. However, as discussed in Chap. 16, some investigators may not be willing to prove the intervention harmful and would terminate a study if the results are suggestive of harm. A classic example of this issue was provided by the Cardiac Arrhythmia Suppression Trial or CAST [43]. This trial was initially designed as a one-sided, 0.025 significance level hypothesis test that anti-arrhythmic drug therapy would reduce the incidence of sudden cardiac death. Since the drugs were already marketed, harmful effects were not expected. Despite the one-sided hypothesis in the design, the monitoring process used a two-sided, 0.05 significance level approach. In this respect, the level of evidence for benefit was the same for either the one-sided 0.025 or two-sided 0.05 significance level design. As it turned out, the trial was terminated early due to increased mortality in the intervention group (see Chaps. 16 and 17).

If a one-sided test of hypothesis is chosen, in most circumstances the significance level ought to be half what the investigator would use for a two-sided test. For example, if 0.05 is the two-sided significance level typically used, 0.025 would be used for the one-sided test. As done in the CAST trial, this requires the same degree of evidence or scientific documentation to declare a treatment effective, regardless of the one-sided vs. two-sided question. In this circumstance, a test for negative or harmful effects might also be done at the 0.025 level. This in effect, provides two one-sided 0.025 hypothesis tests for an overall 0.05 significance level.

As mentioned above, the total sample size 2N (N per arm) is a function of the significance level (α), the power (1 − β) and the size of the difference in response (δ) which is to be detected. Changing either α, 1 − β or δ will result in a change in 2N. As the magnitude of the difference δ decreases, the larger the sample size must be to guarantee a high probability of finding that difference. If the calculated sample size is larger than can be realistically obtained, then one or more of the parameters in the design may need to be reconsidered. Since the significance level is usually fixed at 0.05, 0.025, or 0.01, the investigator should generally reconsider the value selected for δ and increase it, or keep δ the same and settle for a less powerful study. If neither of these alternatives is satisfactory, serious consideration should be given to abandoning the trial.

Rothman [44] argued that journals should encourage using confidence intervals to report clinical trial results instead of significance levels. Several researchers [44–46] discuss sample size formulas from this approach. Confidence intervals are constructed by computing the observed difference in event rates and then adding and subtracting a constant times the standard error of the difference. This provides an interval surrounding the observed estimated difference obtained from the trial. The constant is determined so as to give the confidence interval the correct probability of including the true, but unknown difference. This constant is related directly to the critical value used to evaluate test statistics. Trials often use a two-sided α level test (e.g., α = 0.05) and a corresponding (1 − α) confidence interval (e.g., 95%). If the 1 − α confidence interval excludes zero or no difference, we would conclude that the intervention has an effect. If the interval contains zero difference, no intervention effect would be claimed. However, differences of importance could exist, but might not be detected or not be statistically significant because the sample size was too small. For testing the null hypothesis of no treatment effect, hypothesis testing and confidence intervals give the same conclusions. However, confidence intervals provide more information on the range of the likely difference that might exist. For sample size calculations, the desired confidence interval width must be specified. This may be determined, for example, by the smallest difference between two event rates that would be clinically meaningful and important. Under the null hypothesis of no treatment effect, half the desired interval width is equal to the difference specified in the alternative hypothesis. The sample size calculation methods presented here do not preclude the presentation of results as confidence intervals and, in fact, investigators ought to do so. However, unless there is an awareness of the relationship between the two approaches, as McHugh and Le [46] have pointed out, the confidence interval method might yield a power of only 50% to detect a specified difference. This can be seen later, when sample size calculations for comparing proportions are presented. Thus, some care needs to be taken in using this method.

So far, it has been assumed that the data will be analyzed only once at the end of the trial. However, as discussed in Chaps. 16 and 17, the response variable data may be reviewed periodically during the course of a study. Thus, the probability of finding significant differences by chance alone is increased [47]. This means that the significance level α may need to be adjusted to compensate for the increase in the probability of a Type I error. For purposes of this discussion, we assume that α carries the usual values of 0.05, 0.025 or 0.01. The sample size calculation should also employ the statistic which will be used in data analysis. Thus, there are many sample size formulations. Methods that have proven useful will be discussed in the rest of this chapter.

Dichotomous Response Variables

We shall consider two cases for response variables which are dichotomous, that is, yes or no, success or failure, presence or absence. The first case assumes two independent groups or samples [48–59]. The second case is for dichotomous responses within an individual, or paired responses [60–64].

Two Independent Samples

Suppose the primary response variable is the occurrence of an event over some fixed period of time. The sample size calculation should be based on the specific test statistic that will be employed to compare the outcomes. The null hypothesis H ₀ (p C − p I = 0) is compared to an alternative hypothesis H A (p C − p I ≠ 0). The estimates of p I and p C are $$ \widehat{p_{\mathrm{C}\ }}-\widehat{p_{\mathrm{I}}} $$ where $$ \widehat{p_{\mathrm{I}}}={r}_{\mathrm{I}}/{N}_{\mathrm{I}} $$ and $$ \widehat{p_{\mathrm{C}}}={r}_{\mathrm{C}}/{N}_{\mathrm{C}} $$ with r I and r C being the number of events in the intervention and control groups and N I and N C being the number of participants in each group. The usual test statistic for comparing such dichotomous or binomial responses is

where $$ \overline{p}=\left({r}_{\mathrm{I}}+{r}_{\mathrm{C}}\right)/\left({N}_{\mathrm{I}}+{N}_{\mathrm{C}}\right) $$. The square of the Z statistic is algebraically equivalent to the chi-square statistic, which is often employed as well. For large values of N I and N C, the statistic Z has approximately a normal distribution with mean 0 and variance 1. If the test statistic Z is larger in absolute value than a constant Z α, the investigator will reject H ₀ in the two-sided test.

The constant Z α is often referred to as the critical value. The probability of a standard normal random variable being larger in absolute value than Z α is α. For a one-sided hypothesis, the constant Z α is chosen such that the probability that Z is greater (or less) than Z α is α. For a given α, Z α is larger for a two-sided test than for a one-sided test (Table 8.1). Z α for a two-sided test with α = 0.10 has the same value as Z α for a one-sided test with α = 0.05. While a smaller sample size can be achieved with a one-sided test compared to a two-sided test at the same α level, we in general do not recommend this approach as discussed earlier.

Table 8.1

Z α for sample size formulas for various values of α

α

Z α

One-sided test

Two-sided test

0.10

1.282

1.645

0.05

1.645

1.960

0.025

1.960

2.240

0.01

2.326

2.576

The sample size required for the design to have a significance level α and a power of 1 − β to detect true differences of at least δ between the event rates p I and p C can be expressed by the formula [11]:

where 2N = total sample size (N participants/group) with $$ \overline{p}=\left({p}_{\mathrm{C}}+{p}_{\mathrm{I}}\right)/2 $$; Z α is the critical value which corresponds to the significance level α; and Z β is the value of the standard normal value not exceeded with probability β. Z β corresponds to the power 1 − β (e.g., if 1 − β = 0.90, Z β = 1.282). Values of Z α and Z β are given in Tables 8.1 and 8.2 for several values of α and 1 − β. More complete tables may be found in most introductory texts textbooks [27–29, 31, 33–37, 51], sample size texts [17–21, 65], or by using software packages and online resources [66–73]. Note that the definition of $$ \overline{p} $$ given earlier is equivalent to the definition of $$ \overline{p} $$ given here when N I = N C; that is, when the two study groups are of equal size. An alternative to the above formula is given by

Table 8.2

Z β for sample size formulas for various values of power (1 − β)

1 − β

Z β

0.50

0.00

0.60

0.25

0.70

0.53

0.80

0.84

0.85

1.036

0.90

1.282

0.95

1.645

0.975

1.960

0.99

2.326

These two formulas give approximately the same answer and either may be used for the typical clinical trial.

Example: Suppose the annual event rate in the control group is anticipated to be 20%. The investigator hopes that the intervention will reduce the rate to 15%. The study is planned so that each participant will be followed for 2 years. Therefore, if the assumptions are accurate, approximately 40% of the participants in the control group and 30% of the participants in the intervention group will develop an event. Thus, the investigator sets p C = 0.40, p I = 0.30, and, therefore, $$ \overline{p}=\left(0.4+0.3\right)/2=0.35 $$. The study is designed as two-sided with a 5% significance level and 90% power. From Tables 8.1 and 8.2, the two-sided 0.05 critical value is 1.96 for Z β and 1.282 for Z β. Substituting these values into the right-hand side of the first sample size formula yields 2N to be

Evaluating this expression, 2N equals 952.3. Using the second formula, 2N is 4(1.96 + 1.202)2 (0.35)(0.65)/(0.4 − 0.3)2 or 2N = 956. Therefore, after rounding up to the nearest ten, the calculated total sample size by either formula is 960, or 480 in each group.

Sample size estimates using the first formula are given in Table 8.3 for a variety of values of p I and p C, for two-sided tests, and for α = 0.01, 0.025 and 0.05 and 1 − β = 0.80 or 0.90. For the example just considered with α = 0.05 (two-sided), 1 − β = 0.90, p C = 0.4 and p I = 0.3, the total sample size using Table 8.3 is 960. This table shows that, as the difference in rates between groups increases, the sample size decreases.

Table 8.3

Sample size

Alpha/power

2α (Two-sided)

0.01

0.025

0.05

p C

p I

0.90

0.80

0.90

0.80

0.90

0.80

0.6

0.5

1470

1160

1230

940

1040

780

0.4

370

290

310

240

260

200

0.3

160

130

140

110

120

90

0.20

90

70

80

60

60

50

0.5

0.40

1470

1160

1230

940

1040

780

0.30

360

280

300

230

250

190

0.25

220

180

190

140

160

120

0.20

150

120

130

100

110

80

0.4

0.30

1360

1060

1130

870

960

720

0.25

580

460

490

370

410

310

0.20

310

250

260

200

220

170

0.3

0.20

1120

880

930

710

790

590

0.15

460

360

390

300

330

250

0.10

240

190

200

150

170

130

0.2

0.15

3440

2700

2870

2200

2430

1810

0.10

760

600

630

490

540

400

0.05

290

230

240

190

200

150

0.1

0.05

1650

1300

1380

1060

1170

870

The event rate in the intervention group can be written as p I = (1 − k) p C where k represents the proportion that the control group event rate is expected to be reduced by the intervention. Figure 8.2 shows the total sample size 2N versus k for several values of p C using a two-sided test with α = 0.05 and 1 − β = 0.90. In the example where p C = 0.4 and p I = 0.3, the intervention is expected to reduce the control rate by 25% or k = 0.25. In Fig. 8.2, locate k = 0.25 on the horizontal axis and move up vertically until the curve labeled p C = 0.4 is located. The point on this curve corresponds to a 2N of approximately 960. Notice that as the control group event rate p C decreases, the sample size required to detect the same proportional reduction increases. Trials with small event rates (e.g., p C = 0.1) require large sample sizes unless the interventions have a dramatic effect.

Fig. 8.2

Relationship between total sample size (2N) and reduction in event rate (k) for several control group event rates (p C), with a two-sided significance level of 0.05 and power of 0.90

In order to make use of the sample size formula or table, it is necessary to know something about p C and k. The estimate for p C is usually obtained from previous studies of similar people. In addition, the investigator must choose k based on preliminary evidence of the potential effectiveness of the intervention or be willing to specify some minimum difference or reduction that he wants to detect. Obtaining this information is difficult in many cases. Frequently, estimates may be based on a small amount of data. In such cases, several sample size calculations based on a range of estimates help to assess how sensitive the sample size is to the uncertain estimates of p C, k, or both. The investigator may want to be conservative and take the largest, or nearly largest, estimate of sample size to be sure his study has sufficient power. The power (1 − β) for various values of δ can be compared for a given sample size 2N, significance level α, and control rate p C. By examining a power curve such as in Fig. 8.1, it can be seen what power the trial has for detecting various differences in rates, δ. If the power is high, say 0.80 or larger, for the range of values δ that are of interest, the sample size is probably adequate. The power curve can be especially helpful if the number of available participants is relatively fixed and the investigator wants to assess the probability that the trial can detect any of a variety of reductions in event rates.

Investigators often overestimate the number of eligible participants who can be enrolled in a trial. The actual number enrolled may fall short of goal. To examine the effects of smaller sample sizes on the power of the trial, the investigator may find it useful to graph power as a function of various sample sizes. If the power falls far below 0.8 for a sample size that is very likely to be obtained, he can expand the recruitment effort, hope for a larger intervention effect than was originally assumed, accept the reduced power and its consequences or abandon the trial.

To determine the power, the second sample size equation in this section is solved for Z β:

where $$ \overline{p} $$ as before is (p C + p I)/2. The term Z β can be translated into a power of 1 − β by use of Table 8.2. For example, let p C = 0.4 and p I = 0.3. For a significance level of 0.05 in a two-sided test of hypothesis, Z α is 1.96. In a previous example, it was shown that a total sample of approximately 960 participants or 480 per group is necessary to achieve a power of 0.90. Substituting Z α = 1.96, N = 480, p C = 0.4 and p I = 0.3, a value for Z β = 1.295 is obtained. The closest value of Z β in Table 8.2 is 1.282 which corresponds to a power of 0.90. (If the exact value of N = 476 were used, the value of Z β would be 1.282.) Suppose an investigator thought he could get only 350 participants per group instead of the estimated 480. Then Z β = 0.818 which means that the power 1 − β is somewhat less than 0.80. If the value of Z β is negative, the power is less than 0.50. For more details of power calculations, a standard text in biostatistics [27–29, 31, 33–37, 51] or sample size [17–21, 65] should be consulted.

For a given 2N, α, 1 − β, and p C the reduction in event rate that can be detected can also be calculated. This function is nonlinear and, therefore, the details will not be presented here. Approximate results can be obtained by scanning Table 8.3, by using the calculations for several p I until the sample size approaches the planned number, or by using a figure where sample sizes have been plotted. In Fig. 8.2, α is 0.05 and 1 − β is 0.90. If the sample size is selected as 1000, with p C = 0.4, k is determined to be about 0.25. This means that the expected p I would be 0.3. As can be seen in Table 8.3, the actual sample size for these assumptions is 960.

The above approach yields an estimate which is more accurate as the sample size increases. Modifications [49, 51–55, 58, 59, 74] have been developed which give some improvement in accuracy to the approximate formula presented for small studies. However, the availability of computer software to perform exact computations [66–73] has reduced the need for good small sample approximations. Also, given that sample size estimation is somewhat imprecise due to assumptions of intervention effects and event rates, the formulation presented is probably adequate for most clinical trials.

Designing a trial comparing proportions using the confidence interval approach, we would need to make a series of assumptions as well [6, 42, 52]. A 100(1 − α)% confidence interval for a treatment comparison θ would be of the general form $$ \widehat{\theta}\pm {Z}_{\upalpha}\mathrm{S}\mathrm{E}\left(\widehat{\theta}\right) $$, where $$ \widehat{\theta} $$ is the estimate for θ and $$ \mathrm{S}\mathrm{E}\left(\widehat{\theta}\right) $$ is the standard error of $$ \widehat{\theta} $$. In this case, the specific form would be:

If we want the width of the confidence interval (CI) not to exceed W CI, where W CI is the difference between the upper confidence limit and the lower confidence limit, then if N = N I = N C, the width W CI can be expressed simply as:

or after solving this equation for N,

Thus, if α is 0.05 for a 95% confidence interval, p C = 0.4 and p I = 0.3 or 0.35, N = 8(1.96)2(0.35)(0.65)/(W CI)². If we desire the upper limit of the confidence interval to be not more than 0.10 from the estimate or the width to be twice that, then W CI = 0.20 and N = 175 or 2N = 350. Notice that even though we are essentially looking for differences in p C − p I to be the same as our previous calculation, the sample size is smaller. If we let p C − p I = W CI/2 and substitute this into the previous sample size formula, we obtain

This formula is very close to the confidence interval formula for two proportions. If we select 50% power, β is 0.50 and Z β is 0 which would yield the confidence interval formula. Thus, a confidence interval approach gives 50% power to detect differences of W CI/2. This may not be adequate, depending on the situation. In general, we prefer to specify greater power (e.g., 80–90%) and use the previous approach.

Analogous sample size estimation using the confidence interval approach may be used for comparing means, hazard rates, or regression slopes. We do not present details of these since we prefer to use designs which yield power greater than that obtained from a confidence interval approach.

Paired Dichotomous Response

For designing a trial where the paired outcomes are binary, the sample size estimate is based on McNemar’s test [60–64]. We want to compare the frequency of success within an individual on intervention with the frequency of success on control (i.e., p I − p C). McNemar’s test compares difference in discordant responses within an individual p I − p C, between intervention and control.

In this case, the number of paired observations, N p, may be estimated by:

where d = difference in the proportion of successes (d = p I − p C) and f is the proportion of participants whose response is discordant. An alternative approximate formula for N p is

Example: Consider an eye study where one eye is treated for loss in visual acuity by a new laser procedure and the other eye is treated by standard therapy. The failure rate on the control, p C is estimated to be 0.40 and the new procedure is projected to reduce the failure rate to 0.20. The discordant rate f is assumed to be 0.50. Using the latter sample size formula for a two-sided 5% significance level and 90% power, the number of pairs N p is estimated as 132. If the discordant rate is 0.8, then 210 pairs of eyes will be needed.

Adjusting Sample Size to Compensate for Nonadherence

During the course of a clinical trial, participants will not always adhere to their prescribed intervention schedule. The reason is often that the participant cannot tolerate the dosage of the drug or the degree of intervention prescribed in the protocol. The investigator or the participant may then decide to follow the protocol with less intensity. At all times during the conduct of a trial, the participant’s welfare must come first and meeting those needs may not allow some aspects of the protocol to be followed. Planners of clinical trials must recognize this possibility and attempt to account for it in their design. Examples of adjusting for nonadherence with dichotomous outcomes can be found in several clinical trials [75–82].

In the intervention group a participant who does not adhere to the intervention schedule is often referred to as a “drop-out.” Participants who stop the intervention regimen lose whatever potential benefit the intervention might offer. Similarly, a participant on the control regimen may at some time begin to use the intervention that is being evaluated. This participant is referred to as a “drop-in.” In the case of a drop-in a physician may decide, for example, that surgery is required for a participant assigned to medical treatment in a clinical trial of surgery versus medical care [77]. Drop-in participants from the control group who start the intervention regimen will receive whatever potential benefit or harm that the intervention might offer. Therefore, both the drop-out and drop-in participants must be acknowledged because they tend to dilute any difference between the two groups which might be produced by the intervention. This simple model does not take into account the situation in which one level of an intervention is compared to another level of the intervention. More complicated models for nonadherence adjustment can be developed. Regardless of the model, it must be emphasized that the assumed event rates in the control and intervention groups are modified by participants who do not adhere to the study protocol.

People who do not adhere should remain in the assigned study groups and be included in the analysis. The rationale for this is discussed in Chap. 18. The basic point to be made here is that eliminating participants from analysis or transferring participants to the other group could easily bias the results of the study. However, the observed δ is likely to be less than projected because of nonadherence and thus have an impact on the power of the clinical trial. A reduced δ, of course, means that either the sample size must be increased or the study will have smaller power than intended. Lachin [11] has proposed a simple formula to adjust crudely the sample size for a drop-out rate of proportion R O. This can be generalized to adjust for drop-in rates, R I, as well. The unadjusted sample size N should be multiplied by the factor {1/(1 − R O − R I)}² to get the adjusted sample size per arm, N*. Thus, if R O = 0.20 and R I = 0.05, the originally calculated sample should be multiplied by 1/(0.75)2, or 16/9, and increased by 78%. This formula gives some quantitative idea of the effect of drop-out on the sample size:

However, more refined models to adjust sample sizes for drop-outs from the intervention to the control [83–89] and for drop-ins from the control to the intervention regimen [83] have been developed. They adjust for the resulting changes in p I and p C, the adjusted rates being denoted p I* and p C*. These models also allow for another important factor, which is the time required for the intervention to achieve maximum effectiveness. For example, an anti-platelet drug may have an immediate effect; conversely, even though a cholesterol-lowering drug reduces serum levels quickly, it may require years to produce a maximum effect on coronary mortality.

Example: A drug trial [76] in post myocardial infarction participants illustrates the effect of drop-outs and drop-ins on sample size. In this trial, total mortality over a 3-year follow-up period was the primary response variable. The mortality rate in the control group was estimated to be 18% (p C = 0. 18) and the intervention was believed to have the potential for reducing p C by 28% (k = 0.28) yielding p I = 0.1296. These estimates of p C and k were derived from previous studies. Those studies also indicated that the drop-out rate might be as high as 26% over the 3 years; 12% in the first year, an additional 8% in the second year, and an additional 6% in the third year. For the control group, the drop-in rate was estimated to be 7% each year for a total drop-in rate of 21%.

Using these models for adjustment, p C* = 0.1746 and p I* = 0.1375. Therefore, instead of δ being 0.0504 (0.18 − 0.1296), the adjusted δ* is 0.0371 (0.1746 − 0.1375). For a two-sided test with α = 0.05 and 1 − β = 0.90, the adjusted sample size was 4020 participants compared to an unadjusted sample size of 2160 participants. The adjusted sample size almost doubled in this example due to the expected drop-out and drop-in experiences and the recommended policy of keeping participants in the originally assigned study groups. The remarkable increases in sample size because of drop-outs and drop-ins strongly argue for major efforts to keep nonadherence to a minimum during trials.

Sample Size Calculations for Continuous Response Variables

Similar to dichotomous outcomes, we consider two sample size cases for response variables which are continuous [9, 11, 90]. The first case is for two independent samples. The other case is for paired data.

Two Independent Samples

For a clinical trial with continuous response variables, the previous discussion is conceptually relevant, but not directly applicable to actual calculations. “Continuous” variables such as length of hospitalization, blood pressure, spirometric measures, neuropsychological scores and level of a serum component may be evaluated. Distributions of such measurements frequently can be approximated by a normal distribution. When this is not the case, a transformation of values, such as taking their logarithm, can often make the normality assumption approximately correct.

Suppose the primary response variable, denoted as x, is continuous with N I and N C participants randomized to the intervention and control groups respectively. Assume that the variable x has a normal distribution with mean μ and variance σ². The true levels of μI and μC for the intervention and control groups are not known, but it is assumed that σ² is known. (In practice, σ² is not known and must be estimated from some data. If the data set used is reasonably large, the estimate of σ² can be used in place of the true σ². If the estimate for σ² is based on a small set of data, it is necessary to be cautious in the interpretation of the sample size calculations.)

The null hypothesis is H₀: δ = μC − μI = 0 and the two-sided alternative hypothesis is HA: δ = μC − μI ≠ 0. If the variance is known, the test statistic is:

where $$ {\overline{\mathrm{x}}}_{\mathrm{I}} $$ and $$ {\overline{\mathrm{x}}}_{\mathrm{C}} $$ represent mean levels observed in the intervention and control groups respectively. For adequate sample size (e.g. 50 participants per arm) this statistic has approximately a standard normal distribution. The hypothesis-testing concepts previously discussed apply to the above statistic. If Z > Z α, then an investigator would reject H ₀ at the α level of significance. By use of the above test statistic it can be determined how large a total sample 2N would be needed to detect a true difference δ between μI and μC with power (1 − β) and significance level α by the formula:

Example: Suppose an investigator wishes to estimate the sample size necessary to detect a 10 mg/dL difference in cholesterol level in a diet intervention group compared to the control group. The variance from other data is estimated to be (50 mg/dL)². For a two-sided 5% significance level, Z α = 1.96 and for 90% power, Z β = 1.282. Substituting these values into the above formula, 2N = 4(1.96 + 1.282)²(50)²/10² or approximately 1,050 participants. As δ decreases, the value of 2N increases, and as σ² increases the value of 2N increases. This means that the smaller the difference in intervention effect an investigator is interested in detecting and the larger the variance, the larger the study must be. As with the dichotomous case, setting a smaller α and larger 1 − β also increases the sample size. Figure 8.3 shows total sample size 2 N as a function of δ/σ. As in the example, if δ = 10 and σ = 50, then δ/σ = 0.2 and the sample size 2N for 1 − β = 0.9 is approximately 1,050.

Fig. 8.3

Total sample size (2N) required to detect the difference (δ) between control group mean and intervention group mean as a function of the standardized difference (δ/σ) where σ is the common standard deviation, with two-sided significance level of 0.05 and power (1 − β) of 0.80 and 0.90

Paired Data

In some clinical trials, paired outcome data may increase power for detecting differences because individual or within participant variation is reduced. Trial participants may be assessed at baseline and at the end of follow-up. For example, instead of looking at the difference between mean levels in the groups, an investigator interested in mean levels of change might want to test whether diet intervention lowers serum cholesterol from baseline levels when compared to a control. This is essentially the same question as asked before in the two independent sample case, but each participant’s initial cholesterol level is taken into account. Because of the likelihood of reduced variability, this type of design can lead to a smaller sample size if the question is correctly posed. Assume that ΔC and ΔI represent the true, but unknown levels of change from baseline to some later point in the trial for the control and intervention groups, respectively. Estimates of ΔC and ΔI would be $$ {\overline{d}}_{\mathrm{C}}={\overline{\mathrm{x}}}_{{\mathrm{C}}_1}-{\overline{\mathrm{x}}}_{{\mathrm{C}}_2} $$ and $$ {\overline{d}}_{\mathrm{I}}={\overline{\mathrm{x}}}_{{\mathrm{I}}_1}-{\overline{\mathrm{x}}}_{{\mathrm{I}}_2} $$. These represent the differences in mean levels of the response variable at two points for each group. The investigator tests H ₀: ΔC − ΔI = 0 versus H A: ΔC − ΔI = δ ≠ 0. The variance σ² in this case reflects the variability of the change, from baseline to follow-up, and is assumed here to be the same in the control and intervention arms. This variance is likely to be smaller than the variability at a single measurement. This is the case if the correlation between the first and second measurements is greater than 0.5. Using δ and σ Δ ², as defined in this manner, the previous sample size formula for two independent samples and graph are applicable. That is, the total sample size 2 N can be estimated as

Another way to represent this is

where $$ {\upsigma}_{\varDelta}^2=2{\upsigma}^2\left(1-\uprho \right) $$ and σ² is the variance of a measurement at a single point in time, the variability is assumed to be the same at both time points (i.e. at baseline and at follow-up), and ρ is the correlation coefficient between the first and second measurement. As indicated, if the correlation coefficient is greater than 0.5, comparing the paired differences will result in a smaller sample size than just comparing the mean values at the time of follow-up.

Example: Assume that an investigator is still interested in detecting a 10 mg/dL difference in cholesterol between the two groups, but that the variance of the change is now (20 mg/dL)². The question being asked in terms of δ is approximately the same, because randomization should produce baseline mean levels in each group which are almost equal. The comparison of differences in change is essentially a comparison of the difference in mean levels of cholesterol at the second measurement. Using Fig. 8.3, where δ/σΔ = 10/20 = 0.5, the sample size is 170. This impressive reduction in sample size from 1,050 is due to a reduction in the variance from (50 mg/dL)² to (20 mg/dL)².

Another type of pairing occurs in diseases that affect paired organs such as lungs, kidneys, and eyes. In ophthalmology, for example, trials have been conducted where one eye is randomized to receive treatment and the other to receive control therapy [61–64]. Both the analysis and the sample size estimation need to take account of this special kind of stratification. For continuous outcomes, a mean difference in outcome between a treated eye and untreated eye would measure the treatment effect and could be compared using a paired t-test [9, 11], $$ Z=\overline{d}/{S}_{\mathrm{d}}\sqrt{1/N} $$, where $$ \overline{d} $$ is the average difference in response and S d is the standard deviation of the differences. The mean difference μd is equal to the mean response of the treated or intervention eye, for example, minus the mean response of the control eye; that is μd = μI − μC. Under the null hypothesis, μd equals δd. An estimate of δd, $$ \overline{d} $$, can be obtained by taking an estimate of the average differences or by calculating $$ {\overline{\mathrm{x}}}_{\mathrm{I}}-{\overline{\mathrm{x}}}_{\mathrm{C}} $$. The variance of the paired differences σ d² is estimated by S d². Thus, the formula for paired continuous outcomes within an individual is a slight modification of the formula for comparison of means in two independent samples. To compute sample size, N d, for number of pairs, we compute:

As discussed previously, participants in clinical trials do not always fully adhere with the intervention being tested. Some fraction (R O) of participants on intervention drop-out of the intervention and some other fraction (R I) drop-in and start following the intervention. If we assume that these participants who drop-out respond as if they had been on control and those who drop-in respond as if they had been on intervention, then the sample size adjustment is the same as for the case of proportions. That is, the adjusted sample size N* is a function of the drop-out rate, the drop-in rate, and the sample size N for a study with fully compliant participants:

Therefore, if the drop-out rate were 0.20 and the drop-in 0.05, then the original sample size N must be increased by 16/9 or 1.78; that is, a 78% increase in sample size.

Sample Size for Repeated Measures

The previous section briefly presented the sample size calculation for trials where only two points, say a baseline and a final visit, are used to determine the effect of intervention and these two points are the same for all study participants. Often, a continuous response variable is measured at each follow-up visit. Considering only the first and last values would give one estimate of change but would not take advantage of all the available data. Many models exist for the analysis of repeated measurements and formulae [13, 91–97] as well as computer software [66, 67, 69–73] for sample size calculation are available for most. In some cases, the response variable may be categorical. We present one of the simpler models for continuous repeated measurements. While other models are beyond the scope of this book, the basic concepts presented are still useful in thinking about how many participants, how many measurements per individual, and when they should be taken, are needed. In such a case, one possible approach is to assume that the change in response variable is approximately a linear function of time, so that the rate of change can be summarized by a slope. This model is fit to each participant’s data by the standard least squares method and the estimated slope is used to summarize the participant’s experience. In planning such a study, the investigator must be concerned about the frequency of the measurement and the duration of the observation period. As discussed by Fitzmaurice and co-authors [98], the observed measurement x can be expressed as x = a + bt + error, where a = intercept, b = slope, t = time, and error represents the deviation of the observed measurement from a regression line. This error may be due to measurement variability, biological variability or the nonlinearity of the true underlying relationship. On the average, this error is expected to be equally distributed around 0 and have a variability denoted as σ(error)². Though it is not necessary, it simplifies the calculation to assume that σ(error)² is approximately the same for each participant.

The investigator evaluates intervention effectiveness by comparing the average slope in one group with the average slope in another group. Obviously, participants in a group will not have the same slope, but the slopes will vary around some average value which reflects the effectiveness of the intervention or control. The amount of variability of slopes over participants is denoted as σ b ². If D represents the total time duration for each participant and P represents the number of equally spaced measurements, σb² can be expressed as:

where σ B ² is the component of variance attributable to differences in participants’ slope as opposed to measurement error and lack of a linear fit. The sample size required to detect difference δ between the average rates of change in the two groups is given by:

As in the previous formulas, when δ decreases, 2N increases. The factor on the right-hand side relates D and P with the variance components σ B ² and σ(error)². Obviously as σ B ² and σ(error)² increase, the total sample size increases. By increasing P and D, however, the investigator can decrease the contribution made by σ(error)². The exact choices of P and D will depend on how long the investigator can feasibly follow participants, how many times he can afford to have participants visit a clinic and other factors. By manipulating P and D, an investigator can design a study which will be the most cost effective for his specific situation.

Example: In planning for a trial, it may be assumed that a response variable declines at the rate of 80 units/year in the control group. Suppose a 25% reduction is anticipated in the intervention group. That is, the rate of change in the intervention group would be 60 units/year. Other studies provided an estimate for σ(error) of 150 units. Also, suppose data from a study of people followed every 3 months for 1 year (D = 1 and P = 5) gave a value for the standard deviation of the slopes, σb = 200. The calculated value of σB is then 63 units. Thus, for a 5% significance level and 90% power (Zα = 1.96 and Zβ = 1.282), the total sample size would be approximately 630 for a 3-year study with four visits per year (D = 3, P = 13). Increasing the follow-up time to 4 years, again with four measurements per year, would decrease the variability with a resulting sample size calculation of approximately 510. This reduction in sample size could be used to decide whether or not to plan a 4-year or a 3-year study.

Sample Size Calculations for “Time to Failure”

For many clinical trials, the primary response variable is the occurrence of an event and thus the proportion of events in each group may be compared. In these cases, the sample size methods described earlier will be appropriate. In other trials, the time to the event may be of special interest. For example, if the time to death or a nonfatal event can be increased, the intervention may be useful even though at some point the proportion of events in each group are similar. Methods for analysis of this type of outcome are generally referred to as life table or survival analysis methods (see Chap. 15). In this situation, other sample size approaches are more appropriate than that described for dichotomous outcomes [99–118]. At the end of this section, we also discuss estimating the number of events required to achieve a desired power.

The basic approach is to compare the survival curves for the groups. A survival curve may be thought of as a graph of the probability of surviving, or not having an event, up to any given point in time. The methods of analysis now widely used are non-parametric; that is, no mathematical model about the shape of the survival curve is assumed. However, for the purpose of estimating sample size, some assumptions are often useful. A common model assumes that the survival curve, S(t), follows an exponential distribution, S(t) = e−λt  = exp(−λt) where λ is called the hazard rate or force of mortality. Using this model, survival curves are totally characterized by λ. Thus, the survival curves from a control and an intervention group can be compared by testing H ₀: λ C = λ I. An estimate of λ is obtained as the inverse of the mean survival time. If the median survival time, T M, is known, the hazard rate λ may also be estimated by − ln(0.5)/T M. Sample size formulations have been considered by several investigators [103, 112, 119]. One simple formula is given by

where N is the size of the sample in each group and Z α and Z β are defined as before. As an example, suppose one assumes that the force of mortality is 0.30 in the control group and expects it to be 0.20 for the intervention being tested; that is, λ C/λ I = 1.5. If α = .05 (two-sided) and 1 − β = 0.90, then N = 128 or 2N = 256. The corresponding mortality rates for 5 years of follow-up are 0.7769 and 0.6321 respectively. Using the comparison of two proportions, the total sample size would be 412. Thus, the time to failure method may give a more efficient design, requiring a smaller number of participants.

The method just described assumes that all participants will be followed to the event. With few exceptions, clinical trials with a survival outcome are terminated at time T before all participants have had an event. For those still event-free, the time to event is said to be censored at time T. For this situation, Lachin [11] gives the approximate formula:

where φ(λ) = λ²/(1 − e−λT ) and where φ(λC) or φ(λI) are defined by replacing λ with λC or λI, respectively. If a 5 year study were being planned (T = 5) with the same design specifications as above, then the sample size, 2N is equal to 376. Thus, the loss of information due to censoring must be compensated for by increasing the sample size. If the participants are to be recruited continually during the 5 years of the trial, the formula given by Lachin is identical but with φ(λ) = λ³ T/(λT − 1 + e−λT ). Using the same design assumptions, we obtain 2N = 620, showing that not having all the participants at the start requires an additional increase in sample size.

More typically participants are recruited uniformly over a period of time, T ₀, with the trial continuing for a total of T years (T > T ₀). In this situation, the sample size can be estimated as before using:

Here, the sample size (2N) of 466 is between the previous two examples suggesting that it is preferable to get participants recruited as rapidly as possible to get more follow-up or exposure time.

One of the methods used for comparing survival curves is the proportional hazards model or the Cox regression model which is discussed briefly in Chap. 15. For this method, sample size estimates have been published [101, 115]. As it turns out, the formula by Schoenfeld for the Cox model [115] is identical to that given above for the simple exponential case, although developed from a different point of view. Further models are given by Lachin [11].

All of the above methods assume that the hazard rate remains constant during the course of the trial. This may not be the case. The Beta-Blocker Heart Attack Trial [76] compared 3-year survival in two groups of participants with intervention starting one to 3 weeks after an acute myocardial infarction. The risk of death was high initially, decreased steadily, and then became relatively constant.

For cases where the event rate is relatively small and the clinical trial will have considerable censoring, most of the statistical information will be in the number of events. Thus, the sample size estimates using simple proportions will be quite adequate. In the Beta-Blocker Heart Attack Trial, the 3 year control group event rate was assumed to be 0.18. For the intervention group, the event rate was assumed to be approximately 0.13. In the situation of ϕ(λ) = λ²(1 − e−λT ), a sample size 2N = 2,208 is obtained, before adjustment for estimated nonadherence. In contrast, the unadjusted sample size using simple proportions is 2,160. Again, it should be emphasized that all of these methods are only approximations and the estimates should be viewed as such.

As the previous example indicates, the power of a survival analysis still is a function of the number of events. The expected number of events E(D) is a function of sample size, hazard rate, recruitment rate, and censoring distribution [11, 106]. Specifically, the expected number of events in the control group can be estimated as

where ϕ(λC) is defined as before, depending on the recruitment and follow-up strategy. If we assume a uniform recruitment over the interval (0,T ₀) and follow-up over the interval (0,T), then E(D) can be written using the most general form for ϕ(λC);

This estimate of the number of events can be used to predict the number of events at various time points during the trial including the end of follow-up. This prediction can be compared to the observed number of events in the control group to determine if an adjustment needs to be made to the design. That is, if the number of events early in the trial is larger than expected, the trial may be more powerful than designed or may be stopped earlier than the planned T years of follow-up (see Chap. 16). However, more worrisome is when the observed number of events is smaller than what is expected and needed to maintain adequate power. Based on this early information, the design may be modified to attain the necessary number of events by increasing the sample size or expanding recruitment effort within the same period of time, increasing follow-up, or a combination of both.

This method can be illustrated based on a placebo-controlled trial of congestive heart failure [82]. Severe or advanced congestive heart failure has an expected 1 year event rate of 40%, where the events are all-cause mortality and nonfatal myocardial infarction. A new drug was to be tested to reduce the event rate by 25%, using a two-sided 5% significance level and 90% power. If participants are recruited over 1.5 years (T ₀ = 1.5) during a 2 year study (T = 2) and a constant hazard rate is assumed, the total sample size (2N) is estimated to be 820 participants with congestive heart failure. The formula E(D) can be used to calculate that approximately 190 events (deaths plus nonfatal myocardial infarctions) must be observed in the control group to attain 90% power. If the first year event rate turns out to be less than 40%, fewer events will be observed by 2 years than the required 190. Table 8.4 shows the expected number of control group events at 6 months and 1 year into the trial for annual event rates of 40, 35, 30, and 25%. Two years is also shown to illustrate the projected number of events at the completion of the study. These numbers are obtained by calculating the number of participants enrolled by 6 months (33% of 400) and 1 year (66% of 400) and multiplying by the bracketed term on the right hand side of the equation for E(D). If the assumed annual event rate of 40% is correct, 60 control group events should be observed at 1 year. However, if at 1 year only 44 events are observed, the annual event rate might be closer to 30% (i.e., λ = 0.357) and some design modification should be considered to assure achieving the desired 190 control group events. One year would be a sensible time to make this decision, based only on control group events, since recruitment efforts are still underway. For example, if recruitment efforts could be expanded to 1220 participants in 1.5 years, then by 2 years of follow-up the 190 events in the placebo group would be observed and the 90% power maintained. If recruitment efforts were to continue for another 6 months at a uniform rate (T ₀ = 2 years), another 135 participants would be enrolled. In this case, E(D) is 545 × 0.285 = 155 events which would not be sufficient without some additional follow-up. If recruitment and follow-up continued for 27 months (i.e., T ₀ = T = 2.25), then 605 control group participants would be recruited and E(D) would be 187, yielding the desired power.

Table 8.4

Number of expected events (in the control group) at each interim analysis given different event rates in control group

Number of expected events

Calendar time into study

Yearly event rate in control group

6 Months

1 Year

1.5 Years

2 Years

(N = 138/group)

(N = 275/group)

(N = 412/group)

(N = 412/group)

40%

16

60

124

189

35%

14

51

108

167

30%

12

44

94

146

25%

10

36

78

123

Assumptions

 1. Time to event exponentially distributed

 2. Uniform entry into the study over 1.5 years

 3. Total duration of 2 years

Sample Size for Testing “Equivalency” or Noninferiority of Interventions

In some instances, an effective intervention has already been established and is considered the standard. New interventions under consideration may be preferred because they are less expensive, have fewer side effects, or have less adverse impact on an individual’s general quality of life. This issue is common in the pharmaceutical industry where a product developed by one company may be tested against an established intervention manufactured by another company. Studies of this type are sometimes referred to as trials with positive controls or as noninferiority designs (see Chaps. 3 and 5).

Given that several trials have shown that certain beta-blockers are effective in reducing mortality in post-myocardial infarction participants [76, 120, 121], it is likely that any new beta-blockers developed will be tested against proven agents. The Nocturnal Oxygen Therapy Trial [122] tested whether the daily amount of oxygen administered to chronic obstructive pulmonary disease participants could be reduced from 24 to 12 h without impairing oxygenation. The Intermittent Positive Pressure Breathing [80] trial considered whether a simple and less expensive method for delivering a bronchodilator into the lungs would be as effective as a more expensive device. A breast cancer trial compared the tumor regression rates between subjects receiving the standard, diethylstilbestrol, or the newer agent, tamoxifen [123].

The problem in designing noninferiority trials is that there is no statistical method to demonstrate complete equivalence. That is, it is not possible to show δ = 0. Failure to reject the null hypothesis is not a sufficient reason to claim two interventions to be equal but merely that the evidence is inadequate to say they are different [124]. Assuming no difference when using the previously described formulas results in an infinite sample size.

While demonstrating perfect equivalence is an impossible task, one possible approach has been discussed for noninferiority designs [125–128]. The strategy is to specify some value, δ, such that interventions with differences which are less than this might be considered “equally effective” or “noninferior” (see Chap. 5 for discussion of noninferiority designs). Specification of δ may be difficult but it is a necessary element of the design. The null hypothesis states that p C > p I + δ while the alternative specifies p C < p I + δ. The methods developed require that if the two interventions really are equally effective or at least noninferior, the upper 100(1 − α)% confidence interval for the intervention difference will not exceed δ with the probability of 1 − β. One can alternatively approach this from a hypothesis testing point of view, stating the null hypothesis that the two interventions differ by less than δ.

For studies with a dichotomous response, one might assume the event rate for the two interventions to be equal to p (i.e., p = p C = p I). This simplifies the previously shown sample size formula to

where N, Zα and Zβ are defined as before. Makuch and Simon [127] recommend for this situation that α = 0.10 and β = 0.20. However, for many situations, β or Type II error needs to be 0.10 or smaller in order to be sure a new therapy is correctly determined to be equivalent to an older standard. We prefer an α = 0.05, but this is a matter of judgment and will depend on the situation. (This formula differs slightly from its analogue presented earlier due to the different way the hypothesis is stated.) The formula for continuous variables,

is identical to the formula for determining sample size discussed earlier. Blackwelder and Chang [126] give graphical methods for computing sample size estimates for studies of equivalency.

As mentioned above and in Chap. 5, specifying δ is a key part of the design and sample size calculations of all equivalency and noninferiority trials. Trials should be sufficiently large, with enough power, to address properly the questions about equivalence or noninferiority that are posed.

Sample Size for Cluster Randomization

So far, sample size estimates have been presented for trials where individuals are randomized. For some prevention trials or health care studies, it may not be possible to randomize individuals. For example, a trial of smoking prevention strategy for teenagers may be implemented most easily by randomizing schools, some schools to be exposed to the new prevention strategy while other schools remain with a standard approach. Individual students are grouped or clustered within each school. As Donner et al. [129] point out, “Since one cannot regard the individuals within such groups as statistically independent, standard sample size formulas underestimate the total number of subjects required for the trial.” Several authors [129–133] have suggested incorporating a single inflation factor in the usual sample size calculation to account for the cluster randomization. That is, the sample size per intervention arm N computed by previous formulas will be adjusted to N* to account for the randomization of N m clusters, each with m individuals.

A continuous response is measured for each individual within a cluster of these components. Differences of individuals within a cluster and differences of individuals between clusters contribute to the overall variability of the response. We can separate the between-cluster variance σ² b and within cluster variance σ² w. Estimates are denoted S b² and S w², respectively and can be estimated by standard analysis of variance. One measure of the relationship of these components is the intra-class correlation coefficient. The intra-class correlation coefficient ρ is $$ {\upsigma}_{\mathrm{b}}^2/\left({\upsigma}_{\mathrm{b}}^2+{\upsigma}_{\mathrm{w}}^2\right) $$ where 0 ≤ ρ ≤ 1. If ρ = 0, all clusters respond identically so all of the variability is within a cluster. If ρ = 1, all individuals in a cluster respond alike so there is no variability within a cluster. Estimates of ρ are given by $$ r={S}_{\mathrm{b}}^2/\left({S}_{\mathrm{b}}^2+{S}_{\mathrm{w}}^2\right) $$. Intra-class correlation may range from 0.1 to 0.4 in typical clinical studies. If we computed the sample size calculations assuming no clustering, the sample size per arm would be N participants per treatment arm. Now, instead of randomizing N individuals, we want to randomize N m clusters with m individuals each for a total of N* = N m × m participants per treatment arm. The inflation factor [133] is [1 + (m − 1)r] so that

Note that the inflation factor is a function of both cluster size m and intra-class correlation. If the intra-cluster correlation (ρ = 0), then each individual in one cluster responds like any individual in another cluster, and the inflation factor is unity (N* = N). That is, no penalty is paid for the convenience of cluster randomization. At the other extreme, if all individuals in a cluster respond the same (ρ = 1), there is no added information within each cluster, so only one individual per cluster is needed, and the inflation factor is m. That is, our adjusted sample N* = N × m and we pay a severe price for this type of cluster randomization. However, it is unlikely that ρ is either 0 or 1, but as indicated, is more likely to be in the range of 0.1–0.4 in clinical studies.

Example: Donner et al. [129] provide an example for a trial randomizing households to a sodium reducing diet in order to reduce blood pressure. Previous studies estimated the intra-class correlation coefficient to be 0.2; that is $$ \widehat{\uprho}=r={S}_{\mathrm{b}}^2/\left({S}_{\mathrm{b}}^2+{S}_{\mathrm{w}}^2\right)=0.2 $$. The average household size was estimated at 3.5 (m = 3.5). The sample size per arm N must be adjusted by 1 + (m − 1)ρ = 1 + (3.5 − 1)(0.2) = 1.5. Thus, the normal sample size must be inflated by 50% to account for this randomization indicating a small between cluster variability. If ρ = 0.1, then the factor is 1 + (3.5 − 1)(0.1) or 1.25. If ρ = 0.4, indicating a larger between cluster component of variability, the inflation factor is 2.0 or a doubling.

For binomial responses, a similar expression for adjusting the standard sample size can be developed. In this setting, a measure of the degree of within cluster dependency or concordancy rate in participant responses is used in place of the intra-class correlation. The commonly used measure is the kappa coefficient, denoted κ, and may be thought of as an intra-class correlation coefficient for binomial responses, analogous to ρ for continuous responses. A concordant cluster with κ = 1 is one where all responses within a cluster are identical, all successes or failures, in which a cluster contributes no more than a single individual. A simple estimate for κ is provided [129]:

Here p* is the proportion of the control group with concordant clusters, and p C is the underlying success rate in the control group. The authors then show that the inflation factor is [1 + (m − 1)κ], or that the regular sample size per treatment arm N must be multiplied by this factor to attain the adjust sample size N*:

Example: Donner et al. [129] continues the sodium diet example where couples (m = 2) are randomized to either a low sodium or a normal diet. The outcome is the hypertension rate. Other data suggest the concordancy of hypertension status among married couples is 0.85 (p* = 0.85). The control group hypertension rate is 0.15 (p C = 0.15). In this case, κ = 0.41, so that the inflation factor is 1 + (2 − 1)(0.41) = 1.41; that is, the regular sample size must be inflated by 41% to adjust for the couples being the randomization unit. If there is perfect control group concordance, p* = 1 and κ = 1, in which case, N* = 2 N.

Cornfield proposed another adjustment procedure [130]. Consider a trial where C clusters will be randomized, each cluster of size m i (i = 1, 2, …, C) and each having a different success rate of p i (i = 1, 2, …, C). Define the average cluster size $$ \overline{m}=\varSigma {m}_{\mathrm{i}}/C $$ and $$ \overline{p}=\varSigma {m}_{\mathrm{i}}{p}_{\mathrm{i}}/\varSigma\;{m}_{\mathrm{i}} $$ as the overall success rate weighted by cluster size. The variance of the overall success rate is $$ {\upsigma}_p^2=\varSigma\;{m}_{\mathrm{i}}{\left({p}_{\mathrm{i}}-\overline{p}\right)}^2/C{\overline{m}}^2 $$. In this setting, the efficiency of simple randomization to cluster randomization is $$ E=\overline{p}{\left(1-\overline{p}\right)}^2\overline{m}\;{\upsigma}_p^2 $$. The inflation factor (IF) for this design is $$ \mathrm{IF}=1/E=\overline{m}\;{\upsigma}_{\mathrm{p}}^2/\left(1-\overline{p}\right) $$. Note that if the response rate varies across clusters, the normal sample size must be increased.

While cluster randomization may be logistically required, the process of making the cluster the randomization unit has serious sample size implications. It would be unwise to ignore this consequence in the design phase. As shown, the sample size adjustments can easily be factors of 1.5 or higher. For clusters which are schools or cities, the intra-class correlation is likely to be quite small. However, the cluster size is multiplied by the intra-class correlation so the impact might still be nontrivial. Not making this adjustment would substantially reduce the study power if the analyses were done properly, taking into account the cluster effect. Ignoring the cluster effect in the analysis would be viewed critically in most cases and is not recommended.

Multiple Response Variables

We have stressed the advantages of having a single primary question and a single primary response variable, but clinical trials occasionally have more than one of each. More than one question may be asked because investigators cannot agree about which outcome is most important. As an example, one clinical trial involving two schedules of oxygen administration to participants with chronic obstructive pulmonary disease had three major questions in addition to comparing the mortality rate [122]. Measures of pulmonary function, neuro-psychological status, and quality of life were evaluated. For the participants, all three were important.

Sometimes more than one primary response variable is used to assess a single primary question. This may reflect uncertainty as to how the investigator can answer the question. A clinical trial involving participants with pulmonary embolism [134] employed three methods of determining a drug’s ability to resolve emboli. They were: lung scanning, arteriography, and hemodynamic studies. Another trial involved the use of drugs to limit myocardial infarct size [135]. Precordial electrocardiogram mapping, radionuclide studies, and enzyme levels were all used to evaluate the effectiveness of the drugs. Several approaches to the design and analysis of trials with multiple endpoints have been described [136–139].

Computing a sample size for such clinical trials is not easy. One could attempt to define a single model for the multidimensional response and use one of the previously discussed formulas. Such a method would require several assumptions about the model and its parameters and might require information about correlations between different measurements. Such information is rarely available. A more reasonable procedure would be to compute sample sizes for each individual response variable. If the results give about the same sample size for all variables, then the issue is resolved. However, more commonly, a range of sample sizes will be obtained. The most conservative strategy would be to use the largest sample size computed. The other response variables would then have even greater power to detect the hoped-for reductions or differences (since they required smaller sample sizes). Unfortunately, this approach is the most expensive and difficult to undertake. Of course, one could also choose the smallest sample size of those computed. That would probably not be desirable, because the other response variables would have less power than usually required, or only larger differences than expected would be detectable. It is possible to select a middle range sample size, but there is no assurance that this will be appropriate. An alternative approach is to look at the difference between the largest and smallest sample sizes. If this difference is very large, the assumptions that went into the calculations should be re-examined and an effort should be made to resolve the difference.

As is discussed in Chap. 18, when multiple comparisons are made, the chance of finding a significant difference in one of the comparisons (when, in fact, no real differences exist between the groups) is greater than the stated significance level. In order to maintain an appropriate significance level α for the entire study, the significance level required for each test to reject H ₀ should be adjusted [41]. The significance level required for rejection (α′) in a single test can be approximated by α/k where k is the number of multiple response variables. For several response variables this can make α′ fairly small (e.g., k = 5 implies α′ = 0.01 for each of k response variables with an overall α = 0.05). If the correlation between response variables is known, then the adjustment can be made more precisely [140, 141]. In all cases, the sample size would be much larger than if the use of multiple response variables were ignored, so that most studies have not strictly adhered to this solution of modifying the significance level. Some investigators, however, have attempted to be conservative in the analysis of results [142]. There is a reasonable limit as to how much α′ can be decreased in order to give protection against false rejection of the null hypothesis. Some investigators have chosen α′ = 0.01 regardless of the number of tests. In the end, there are no easy solutions. A somewhat conservative value of α′ needs to be set and the investigators need to be aware of the multiple testing problem during the analysis.

Estimating Sample Size Parameters

As shown in the methods presented, sample size estimation is quite dependent upon assumptions made about variability of the response, level of response in the control group, and the difference anticipated or judged to be clinically relevant [16, 143–148]. Obtaining reliable estimates of variability or levels of response can be challenging since the information is often based on very small studies or studies not exactly relevant to the trial being designed. Applying Bayesian methods to incorporate explicitly uncertainty in these estimated parameters has been attempted [149]. Sometimes, pilot or feasibility studies may be conducted to obtain these data. In such cases, the term external pilot has been used [148].

In some cases, the information may not exist prior to starting the trial, as was the case for early trials in AIDS; that is, no incidence rates were available in an evolving epidemic. Even in cases where data are available, other factors affect the variability or level of response observed in a trial. Typically, the variability observed in the planned trial is larger than expected or the level of response is lower than assumed. Numerous examples of this experience exist [143]. One is provided by the Physicians’ Health Study [150]. In this trial, 22,000 U.S. male physicians were randomized into a 2 × 2 factorial design. One factor was aspirin versus placebo in reducing cardiovascular mortality. The other factor was beta-carotene versus placebo for reducing cancer incidence. The aspirin portion of the trial was terminated early in part due to a substantially lower mortality rate than expected. In the design, the cardiovascular mortality rate was assumed to be approximately 50% of the U.S. age-adjusted rate in men. However, after 5 years of follow-up, the rate was approximately 10% of the U.S. rate in men. This substantial difference reduced the power of the trial dramatically. In order to compensate for the extremely low event rate, the trial would have had to be extended another 10 years to get the necessary number of events [150]. One can only speculate about reasons for low event rates, but screening of potential participants prior to the entry almost certainly played a part. That is, screenees had to complete a run-in period and be able to tolerate aspirin. Those at risk for other competing events were also excluded. This type of effect is referred to as a screening effect. Physicians who began to develop cardiovascular signs may have obtained care earlier than non-physicians. In general, volunteers for trials tend to be healthier than the general population, a phenomenon often referred to as the healthy volunteer effect.

Another approach to obtaining estimates for ultimate sample size determination is to design so-called internal pilot studies [148]. In this approach, a small study is initiated based the best available information. A general sample target for the full study may be proposed, but the goal of the pilot is to refine that sample size estimate based on screening and healthy volunteer effects. The pilot study uses a protocol very close if not identical to the protocol for the full study, and thus parameter estimates will reflect those effects. If the protocol for the pilot and the main study are essentially identical, then the small pilot can become an internal pilot. That is, the data from the internal pilot become part of the data for the overall study. This approach was used successfully in the Diabetes Control and Complications Trial [151]. If data from the internal pilot are used only to refine estimates of variability or control group response rates, and not changes in treatment effect, then the impact of this two-step approach on the significance level is negligible. However, the benefit is that this design will more likely have the desired power than if data from external pilots and other sources are relied on exclusively [147]. It must be emphasized that pilot studies, either external or internal, should not be viewed as providing reliable estimates of the intervention effect [152]. Because power is too small in pilot studies to be sure that no effect exists, small or no differences may erroneously be viewed as reason not to pursue the question. A positive trend may also be viewed as evidence that a large study is not necessary, or that clinical equipoise no longer exists.

Our experience indicates that both external and internal pilot studies are quite helpful. Internal pilot studies should be used if at all possible in prevention trials, when screening and healthy volunteer effects seem to cause major design problems. Design modifications based on an internal pilot are more prudent than allowing an inadequate sample size to create yield misleading results.

One approach is to specify the number of events needed for a desired power level. Obtaining the specified number of events requires a number of individuals followed for a period of time. How many participants and how long a follow-up period can be adjusted during the early part of the trial, or during an internal pilot study, but the target number of events does not change. This is also discussed in more detail in Chaps. 16 and 17.

Another approach is to use adaptive designs which modify the sample size based on an emerging trend, referred to as trend adaptive designs (see Chaps. 5 and 17). Here the sample size may be adjusted for an updated estimate of the treatment effect, δ, using the methods described in this chapter. However, an adjustment must then be made at the analysis stage which may require a substantially larger critical value than the standard one in order to maintain a prespecified α level.

References

1.

Freiman JA, Chalmers TC, Smith H, Jr., Kuebler RR. The importance of beta, the type II error and sample size in the design and interpretation of the randomized control trial. Survey of 71 “negative” trials. N Engl J Med 1978;299:690–694.

2.

Moher D, Dulberg CS, Wells GA. Statistical power, sample size, and their reporting in randomized controlled trials. JAMA 1994;272:122–124.

3.

Chan AW, Altman DG. Epidemiology and reporting of randomised trials published in PubMed journals. Lancet 2005;365:1159–1162.

4.

Halpern SD, Karlawish JH, Berlin JA. The continuing unethical conduct of underpowered clinical trials. JAMA 2002;288:358–362.

5.

Altman DG. Statistics and ethics in medical research: III How large a sample? Br Med J 1980;281:1336–1338.

6.

Brown BW. Statistical Controversies in the Design of Clinical-Trials - Some Personal Views. Control Clin Trials 1980;1:13–27.

7.

Campbell MJ, Julious SA, Altman DG. Estimating sample sizes for binary, ordered categorical, and continuous outcomes in two group comparisons. BMJ 1995;311:1145–1148.

8.

Day SJ, Graham DF. Sample size estimation for comparing two or more treatment groups in clinical trials. Stat Med 1991;10:33–43.

9.

Donner A. Approaches to sample size estimation in the design of clinical trials—a review. Stat Med 1984;3:199–214.

10.

Gore SM. Statistics in question. Assessing clinical trials—trial size. Br Med J (Clin Res Ed) 1981;282:1687–1689.

11.

Lachin JM. Introduction to sample size determination and power analysis for clinical trials. Control Clin Trials 1981;2:93–113.

12.

Phillips AN, Pocock SJ. Sample size requirements for prospective studies, with examples for coronary heart disease. J Clin Epidemiol 1989;42:639–648.

13.

Schlesselman JJ. Planning a longitudinal study. I. Sample size determination. J Chronic Dis 1973;26:553–560.

14.

Schouten HJ. Planning group sizes in clinical trials with a continuous outcome and repeated measures. Stat Med 1999;18:255–264.

15.

Streiner DL. Sample size and power in psychiatric research. Can J Psychiatry 1990;35:616–620.

16.

Whitehead J. Sample sizes for phase II and phase III clinical trials: an integrated approach. Stat Med 1986;5:459–464.

17.

Chow SC, Shao J, Wang H. Sample size calculations in clinical research, ed 2nd ed. Boca Raton, Taylor & Francis, 2008.

18.

Desu MM, Raghavarao D. Sample Size Methodology. Boston, Academic Press, 1990.

19.

Julious SA. Sample Sizes for Clinical Trials. Chapman and Hall, 2009.

20.

Machin D, Campbell MJ, Tan S-B, Tan S-H. Sample Size Tables for Clinical Studies, ed 3rd. Wiley-Blackwell, 2008.

21.

Odeh RE, Fox M. Sample Size Choice: Charts for Experiments with Linear Models, ed 2nd. New York, Marcel Dekker, 1991.

22.

Braunholtz DA, Edwards SJ, Lilford RJ. Are randomized clinical trials good for us (in the short term)? Evidence for a “trial effect”. J Clin Epidemiol 2001;54:217–224.

23.

Paynter NP, Sharrett AR, Louis TA, et al. Paired comparison of observed and expected coronary heart disease rates over 12 years from the Atherosclerosis Risk in Communities Study. Ann Epidemiol 2010;20:683–690.

24.

Brancati FL, Evans M, Furberg CD, et al. Midcourse correction to a clinical trial when the event rate is underestimated: the Look AHEAD (Action for Health in Diabetes) Study. Clin Trials 2012;9:113–124.

25.

McClure LA, Szychowski JM, Benavente O, Coffey CS. Sample size re-estimation in an on-going NIH-sponsored clinical trial: the secondary prevention of small subcortical strokes experience. Contemp Clin Trials 2012;33:1088–1093.

26.

Wittes J. On changing a long-term clinical trial midstream. Statist Med 10-15-2002;21:2789–2795.

27.

Armitage P, Berry G, Mathews J. Statistical Methods in Medical Research, ed 4th. Malden MA, Blackwell Publishing, 2002.

28.

Brown BW, Hollander M. Statistics - A Biomedical Introduction. New York, John Wiley and Sons, 1977.MATH

29.

Dixon WJ, Massey FJ Jr. Introduction to Statistical Analysis, ed 3rd. New York, McGraw-Hill, 1969.

30.

Fisher L, Van Belle G. Biostatistics - A Methodology for the Health Sciences. New York, John Wiley and Sons, 1993.MATH

31.

Fisher L, Van Belle G, Heagerty PL, Lumley TS. Biostatistics - A Methodology for the Health Sciences. New York, John Wiley and Sons, 2004.MATH

32.

Fleiss JL, Levin B, Paik MC. Statistical Methods for Rates and Proportions, ed 3rd. John Wiley & Sons, Inc., 2003.

33.

Remington RD, Schork MA. Statistics With Applications to the Biological and Health Sciences. Englewood Cliffs, Prentice-Hall, 1970.

34.

Rosner B. Fundamentals of Biostatistics, ed 3rd. Boston, PWS-Kent, 1990.

35.

Schork MA, Remington RD. Statistics With Applications to the Biological and Health Sciences, ed 3rd. Englewood Cliffs, Prentice-Hall, 2000.

36.

Snedecor GW, Cochran WG. Statistical Methods, ed 8th. Ames, Iowa State University Press, 1989.

37.

Woolson RF, Clarke WR. Statistical Methods for the Analysis of Biomedical Data. Wiley, 2011.

38.

Canner PL, Klimt CR. The Coronary Drug Project. Experimental design features. Control Clin Trials 1983;4:313–332.

39.

Davis BR, Cutler JA, Gordon DJ, et al. Rationale and design for the Antihypertensive and Lipid Lowering Treatment to Prevent Heart Attack Trial (ALLHAT). ALLHAT Research Group. Am J Hypertens 1996;9:342–360.

40.

Dunnett CW. A multiple comparison procedure for comparing several treatments with a control. J Am Statist Assoc 1955;50:1096–1121.MATH

41.

Costigan T. Bonferroni inequalities and intervals; in Armitage P, Colton T (eds): Encyclopedia of Biostatistics. John Wiley and Sons, 2007.

42.

Brittain E, Schlesselman JJ. Optimal Allocation for the Comparison of Proportions. Biometrics 1982;38:1003–1009.

43.

The Cardiac Arrhythmia Suppression Trial (CAST) Investigators. Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N Engl J Med 1989;321:406–412.

44.

Rothman KJ. A show of confidence. N Engl J Med 1978;299:1362–1363.

45.

Brown BW, Hollander M. Statistics: A Biomedical Introduction. Wiley, 2009.

46.

McHugh RB, Le CT. Confidence estimation and the size of a clinical trial. Control Clin Trials 1984;5:157–163.

47.

Armitage P, McPherson CK, Rowe BC. Repeated Significance Tests on Accumulating Data. J R Stat Soc Ser A 1969;132:235–244.MathSciNet

48.

Bristol DR. Sample sizes for constructing confidence intervals and testing hypotheses. Statist Med 1989;8:803–811.

49.

Casagrande JT, Pike MC, Smith PG. An Improved Approximate Formula for Calculating Sample Sizes for Comparing Two Binomial Distributions. Biometrics 1978;34:483–486.MATH

50.

Day SJ. Optimal placebo response rates for comparing two binomial proportions. Statist Med 1988;7:1187–1194.

51.

Fleiss JL, Tytun A, Ury HK. A Simple Approximation for Calculating Sample Sizes for Comparing Independent Proportions. Biometrics 1980;36:343–346.

52.

Fu YX, Arnold J. A Table of Exact Sample Sizes for Use with Fisher’s Exact Test for 2 × 2 Tables. Biometrics 1992;48:1103–1112.MATH

53.

Gail MH, Gart JJ. The Determination of Sample Sizes for Use with the Exact Conditional Test in 2 x 2 Comparative Trials. Biometrics 1973;29:441–448.

54.

Gail MH. The determination of sample sizes for trials involving several independent 2 × 2 tables. J Chronic Dis 1973;26:669–673.

55.

Haseman JK. Exact Sample Sizes for Use with the Fisher-Irwin Test for 2 × 2 Tables. Biometrics 1978;34:106–109.

56.

Lachenbruch PA. A note on sample size computation for testing interactions. Statist Med 1988;7:467–469.

57.

McMahon RP, Proschan M, Geller NL, et al. Sample size calculation for clinical trials in which entry criteria and outcomes are counts of events. Statist Med 1994;13:859–870.

58.

Ury HK, Fleiss JL. On Approximate Sample Sizes for Comparing Two Independent Proportions with the Use of Yates’ Correction. Biometrics 1980;36:347–351.

59.

Wacholder S, Weinberg CR. Paired versus Two-Sample Design for a Clinical Trial of Treatments with Dichotomous Outcome: Power Considerations. Biometrics 1982;38:801–812.

60.

Connor RJ. Sample Size for Testing Differences in Proportions for the Paired-Sample Design. Biometrics 1987;43:207–211.

61.

Donner A. Statistical Methods in Ophthalmology: An Adjusted Chi-Square Approach. Biometrics 1989;45:605–611.MATH

62.

Gauderman W, Barlow WE. Sample size calculations for ophthalmologic studies. Arch Ophthalmol 1992;110:690–692.

63.

Rosner B. Statistical Methods in Ophthalmology: An Adjustment for the Intraclass Correlation between Eyes. Biometrics 1982;38:105–114.

64.

Rosner B, Milton RC. Significance Testing for Correlated Binary Outcome Data. Biometrics 1988;44:505–512.MATH

65.

Hayes RL, Moulton LH. Cluster Randomised Trials. Chapman and Hall, 2009.

66.

Cytel: SiZ. Cambridge, MA, Cytel Software Corporation, 2011.

67.

Elashoff JD. nQuery Advisor Version 7.0 User’s Guide. Cork, Ireland, Statistical Solutions, 2007.

68.

Pezzullo JC. Web Pages that Perform Statistical Calculations. Computer Program 2014.

69.

R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria, R Foundation for Statistical Computing, 2013.

70.

SAS Institute: Getting Started with the SAS Power and Sample Size Application. Cary, NC, SAS Institute Inc., 2004.

71.

Shiboski S. Power and Sample Size Programs. Department of Epidemiology and Biostatistics, University of California San Francisco. Computer Program 2006.

72.

StataCorp: Stata: Release 13. College Station, TX, StataCorp, 2013.

73.

TIBCO Software I: SPLUS. TIBCO Softward Inc., 2008.

74.

Feigl P. A Graphical Aid for Determining Sample Size when Comparing Two Independent Proportions. Biometrics 1978;34:111–122.

75.

Aspirin Myocardial Infarction Study Research Group. A randomized, controlled trial of aspirin in persons recovered from myocardial infarction. JAMA 1980;243:661–669.

76.

Beta Blocker Heart Attack Trial Research Group. A randomized trial of propranolol in patients with acute myocardial infarction: I. mortality results. JAMA 1982;247:1707–1714.

77.

CASS Principle Investigators and Their Associates. Coronary artery surgery study (CASS): a randomized trial of coronary artery bypass surgery. Survival data. Circulation 1983;68:939–950.

78.

The Coronary Drug Project Research Group. The Coronary Drug Project: Design, Methods, and Baseline Results. Circulation 1973;47:I–1.

79.

Hypertension Detection and Follow-up Program Cooperative Group. Five-year findings of the hypertension detection and follow-up program: I. reduction in mortality of persons with high blood pressure, including mild hypertension. JAMA 1979;242:2562–2571.

80.

The Intermittent Positive Pressure Breathing Trial Group. Intermittent Positive Pressure Breathing Therapy of Chronic Obstructive Pulmonary Disease A Clinical Trial. Ann Intern Med 1983;99:612–620.

81.

Multiple Risk Factor Intervention Trial Research Group. Multiple risk factor intervention trial: Risk factor changes and mortality results. JAMA 1982;248:1465–1477.

82.

Packer M, Carver JR, Rodeheffer RJ, et al. Effect of Oral Milrinone on Mortality in Severe Chronic Heart Failure. N Engl J Med 1991;325:1468–1475.

83.

Barlow W, Azen S. The effect of therapeutic treatment crossovers on the power of clinical trials. Control Clin Trials 1990;11:314–326.

84.

Halperin M, Rogot E, Gurian J, Ederer F. Sample sizes for medical trials with special reference to long-term therapy. J Chronic Dis 1968;21:13–24.

85.

Lakatos E. Sample size determination in clinical trials with time-dependent rates of losses and noncompliance. Control Clin Trials 1986;7:189–199.

86.

Lavori P. Statistical Issues: Sample Size and Dropout; in Benkert O, Maier W, Rickels K (eds): Methodology of the Evaluation of Psychotropic Drugs. Springer Berlin Heidelberg, 1990, pp 91–104.

87.

Newcombe RG. Explanatory and pragmatic estimates of the treatment effect when deviations from allocated treatment occur. Statist Med 1988;7:1179–1186.

88.

Schork MA, Remington RD. The determination of sample size in treatment-control comparisons for chronic disease studies in which drop-out or non-adherence is a problem. J Chronic Dis 1967;20:233–239.

89.

Wu MC, Fisher M, DeMets D. Sample sizes for long-term medical trial with time-dependent dropout and event rates. Control Clin Trials 1980;1:111–124.

90.

Pentico DW. On the Determination and Use of Optimal Sample Sizes for Estimating the Difference in Means. Am Stat 1981;35:40–42.

91.

Dawson JD, Lagakos SW. Size and Power of Two-Sample Tests of Repeated Measures Data. Biometrics 1993;49:1022–1032.MathSciNetMATH

92.

Kirby AJ, Galai N, Munoz A. Sample size estimation using repeated measurements on biomarkers as outcomes. Control Clin Trials 1994;15:165–172.

93.

Laird NM, Wang F. Estimating rates of change in randomized clinical trials. Control Clin Trials 1990;11:405–419.

94.

Lipsitz SR, Fitzmaurice GM. Sample size for repeated measures studies with binary responses. Statist Med 1994;13:1233–1239.

95.

Nam J. A Simple Approximation for Calculating Sample Sizes for Detecting Linear Trend in Proportions. Biometrics 1987;43:701–705.

96.

Overall JE, Doyle SR. Estimating sample sizes for repeated measurement designs. Control Clin Trials 1994;15:100–123.

97.

Rochon J. Sample Size Calculations for Two-Group Repeated-Measures Experiments. Biometrics 1991;47:1383–1398.MathSciNet

98.

Fitzmaurice GM, Laird NM, Ware JH. Applied longitudinal analysis, ed 2nd. John Wiley & Sons, 2011.

99.

Cantor AB. Power estimation for rank tests using censored data: Conditional and unconditional. Control Clin Trials 1991;12:462–473.

100.

Emrich LJ. Required Duration and Power Determinations for Historically Controlled-Studies of Survival Times. Statist Med 1989;8:153–160.

101.

Freedman LS. Tables of the number of patients required in clinical trials using the logrank test. Statist Med 1982;1:121–129.

102.

Gail MH. Applicability of sample size calculations based on a comparison of proportions for use with the logrank test. Control Clin Trials 1985;6:112–119.

103.

George SL, Desu MM. Planning the size and duration of a clinical trial studying the time to some critical event. J Chronic Dis 1974;27:15–24.

104.

Halperin M, Johnson NJ. Design and Sensitivity Evaluation of Follow-Up Studies for Risk Factor Assessment. Biometrics 1981;37:805–810.MATH

105.

Hsieh FY. Sample size tables for logistic regression. Statist Med 1989;8:795–802.

106.

Lachin JM, Foulkes MA. Evaluation of sample size and power for analyses of survival with allowance for nonuniform patient entry, losses to follow-up, noncompliance, and stratification. Biometrics 1986;42:507–519.MATH

107.

Lachin JM. Biostatistical Methods: The Assessment of Relative Risks, ed 2nd. John Wiley & Sons, Inc., 2010.

108.

Lakatos E. Sample Sizes Based on the Log-Rank Statistic in Complex Clinical Trials. Biometrics 1988;44:229–241.MathSciNetMATH

109.

Lui KJ. Sample size determination under an exponential model in the presence of a confounder and type I censoring. Control Clin Trials 1992;13:446–458.

110.

Morgan TM. Nonparametric Estimation of Duration of Accrual and Total Study Length for Clinical Trials. Biometrics 1987;43:903–912.MathSciNetMATH

111.

Palta M, Amini SB. Consideration of covariates and stratification in sample size determination for survival time studies. J Chronic Dis 1985;38:801–809.

112.

Pasternack BS, Gilbert HS. Planning the duration of long-term survival time studies designed for accrual by cohorts. J Chronic Dis 1971;24:681–700.

113.

Rubinstein LV, Gail MH, Santner TJ. Planning the duration of a comparative clinical trial with loss to follow-up and a period of continued observation. J Chronic Dis 1981;34:469–479.

114.

Schoenfeld DA, Richter JR. Nomograms for Calculating the Number of Patients Needed for a Clinical Trial with Survival as an Endpoint. Biometrics 1982;38:163–170.

115.

Schoenfeld DA. Sample-Size Formula for the Proportional-Hazards Regression Model. Biometrics 1983;39:499–503.MATH

116.

Taulbee JD, Symons MJ. Sample Size and Duration for Cohort Studies of Survival Time with Covariables. Biometrics 1983;39:351–360.

117.

Wu MC. Sample size for comparison of changes in the presence of right censoring caused by death, withdrawal, and staggered entry. Control Clin Trials 1988;9:32–46.

118.

Zhen B, Murphy JR. Sample size determination for an exponential survival model with an unrestricted covariate. Statist Med 1994;13:391–397.

119.

Pasternack BS. Sample sizes for clinical trials designed for patient accrual by cohorts. J Chronic Dis 1972;25:673–681.

120.

Hjalmarson A, Herlitz J, Malek I, et al. Effect on Mortality of Metoprolol in Acute Myocardial Infarction: A Double-blind Randomised Trial. The Lancet 1981;318:823–827.

121.

The Norwegian Multicenter Study Group. Timolol-Induced Reduction in Mortality and Reinfarction in Patients Surviving Acute Myocardial Infarction. N Engl J Med 1981;304:801–807.

122.

Nocturnal Oxygen Therapy Trial Group. Continuous or Nocturnal Oxygen Therapy in Hypoxemic Chronic Obstructive Lung Disease A Clinical Trial. Ann Intern Med 1980;93:391–398.

123.

Ingle JN, Ahmann DL, Green SJ, et al. Randomized Clinical Trial of Diethylstilbestrol versus Tamoxifen in Postmenopausal Women with Advanced Breast Cancer. N Engl J Med 1981;304:16–21.

124.

Spriet A, Beiler D. When can ‘non significantly different’ treatments be considered as ‘equivalent’? Br J Clin Pharmacol 1979;7:623–624.

125.

Blackwelder WC. “Proving the null hypothesis” in clinical trials. Control Clin Trials 1982;3:345–353.

126.

Blackwelder WC, Chang MA. Sample size graphs for “proving the null hypothesis”. Control Clin Trials 1984;5:97–105.

127.

Makuch R, Simon R. Sample size requirements for evaluating a conservative therapy. Cancer Treat Rep 1978;62:1037–1040.

128.

Rothmann MD, Wiens BL, Chan ISF. Design and Analysis of Non-Inferiority Trials. Taylor & Francis, 2011.

129.

Donner A, Birkett N, Buck C. Randomization by cluster. Sample size requirements and analysis. Am J Epidemiol 1981;114:906–914.

130.

Cornfield J. Randomization by group: A formal analysis. Am J Epidemiol 1978;108:100–102.

131.

Hsieh FY. Sample size formulae for intervention studies with the cluster as unit of randomization. Statist Med 1988;7:1195–1201.

132.

Lee EW, Dubin N. Estimation and sample size considerations for clustered binary responses. Statist Med 1994;13:1241–1252.

133.

Murray DM. Design and Analysis of Group-randomized Trials. Oxford University Press, 1998.

134.

Urokinase Pulmonary Embolism Trial Study Group: Urokinase-streptokinase embolism trial: Phase 2 results. JAMA 1974;229:1606–1613.

135.

Roberts R, Croft C, Gold HK, et al. Effect of Propranolol on Myocardial-Infarct Size in a Randomized Blinded Multicenter Trial. N Engl J Med 1984;311:218–225.

136.

Follmann D. A Simple Multivariate Test for One-Sided Alternatives. J Am Stat Assoc 1996;91:854–861.MathSciNetMATH

137.

O’Brien PC. Procedures for Comparing Samples with Multiple Endpoints. Biometrics 1984;40:1079–1087.MathSciNet

138.

Tang DI, Gnecco C, Geller NL. Design of Group Sequential Clinical Trials with Multiple Endpoints. J Am Stat Assoc 1989;84:776–779.

139.

Tang DI, Geller NL, Pocock SJ. On The Design and Analysis of Randomized Clinical Trials with Multiple Endpoints. Biometrics 1993;49:23–30.MathSciNetMATH

140.

Hsu J. Multiple Comparisons: Theory and Methods. Taylor & Francis, 1996.

141.

Miller RG. Simultaneous Statistical Inference, ed 2nd. Springer New York, 2011.

142.

The Coronary Drug Project Research Group. Clofibrate and niacin in coronary heart disease. JAMA 1975;231:360–381.

143.

Church TR, Ederer F, Mandel JS, et al. Estimating the Duration of Ongoing Prevention Trials. Am J Epidemiol 1993;137:797–810.

144.

Ederer F, Church TR, Mandel JS. Sample Sizes for Prevention Trials Have Been Too Small. Am J Epidemiol 1993;137:787–796.

145.

Neaton JD, Bartsch GE. Impact of measurement error and temporal variability on the estimation of event probabilities for risk factor intervention trials. Statist Med 1992;11:1719–1729.

146.

Patterson BH. The impact of screening and eliminating preexisting cases on sample size requirements for cancer prevention trials. Control Clin Trials 1987;8:87–95.

147.

Shih WJ. Sample size reestimation in clinical trials; in Peace KE (ed): Biopharmaceutical Sequential Statistical Applications. Taylor & Francis, 1992, pp 285–301.

148.

Wittes J, Brittain E. The role of internal pilot studies in increasing the efficiency of clinical trials. Statist Med 1990;9:65–72.

149.

Ambrosius WT, Polonsky TS, Greenland P, et al. Design of the Value of Imaging in Enhancing the Wellness of Your Heart (VIEW) trial and the impact of uncertainty on power. Clin Trials 2012;9:232–246.

150.

Steering Committee of the Physicians’ Health Study. Final Report on the Aspirin Component of the Ongoing Physicians’ Health Study. N Engl J Med 1989;321:129–135.

151.

The Diabetes Control and Complications Trial Research Group: The Effect of Intensive Treatment of Diabetes on the Development and Progression of Long-Term Complications in Insulin-Dependent Diabetes Mellitus. N Engl J Med 1993;329:977–986.

152.

Davis BR, Wittes J, Pressel S, et al. Statistical considerations in monitoring the Systolic Hypertension in the Elderly Program (SHEP). Control Clin Trials 1993;14:350–361.

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_9

9. Baseline Assessment

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

In clinical trials, baseline refers to the status of a participant before the start of intervention. Baseline data may be measured by interview, questionnaire, physical examination, laboratory tests, and procedures. Measurement need not be only numerical in nature. It can also mean classification of study participants into categories based on factors such as absence or presence of some trait or condition.

There are multiple uses of the baseline data. First, there is a need to describe the trial participants so the readers of the trial publications can determine to which patient population the findings apply. Second, it is important methodologically to present relevant baseline data by study group in order to allow assessment of group comparability and answer the question whether randomization generated balanced groups. Third, baseline data are used in statistical analyses to control for any baseline imbalances. Finally, baseline data form the basis for subgroup analyses of the trial findings. This chapter is concerned with these uses of the baseline data.

Fundamental Point

Relevant baseline data should be measured in all study participants before the start of intervention.

Uses of Baseline Data

Description of Trial Participants

Because the trial findings in a strict sense only apply to the participants enrolled in the trial, it is essential that they are properly and as completely as possible described. This is done in the typical Table 1 of the results publication. The description of baseline covariates provides documentation that can guide cautious extrapolation of trial findings to other populations with the same medical condition [1]. A common limitation is that the characteristics of the excluded participants with the study condition are seldom reported. To the reader, it would also be helpful to know what proportion of the study population was excluded. In other words, what was the recruitment yield? Also of interest would be a presentation of the reasons for exclusion. Based on the published information, clinicians need to know to which of their patients the findings directly apply. They also need to know the characteristics of the excluded patients with the study condition, so they can determine whether the trial findings can reasonably be extrapolated.

The amount of data collected at baseline depends on the nature of the trial and the purpose for which the data will be used. As mentioned elsewhere, some trials have simple protocols which means that detailed documentation of many baseline variables is omitted and only a few key demographic and medical variables are ascertained. If such trials are large, it is reasonable to expect that good balance between groups will be achieved. Because the goals of these trials are restricted to answering the primary question and one or two secondary questions, the other uses for baseline data may not be necessary.

Baseline Comparability

Baseline data allow people to evaluate whether the study groups were comparable before intervention was started. The assessment of comparability typically includes pertinent demographic and socioeconomic characteristics, risk or prognostic factors, medications, and medical history. This assessment is necessary in both randomized and nonrandomized trials. In assessment of comparability in any trial, the investigator can only look at relevant factors about which she is aware and were measured. Obviously, those which are unknown cannot be compared. The baseline characteristics of each group should be presented in the main results paper of every randomized trial. Special attention should be given to factors that may influence any benefit of the study intervention and those that may predict adverse effects. Full attention to baseline comparability is not always given. In a review of 206 surgical trials, only 73% reported baseline data [2]. Moreover, more than one quarter of those trials reported fewer than five baseline factors. Altman and Doré, in a review of 80 published randomized trials, noted considerable variation in the quality of the reporting of baseline characteristics [3]. Half of those reporting continuous covariates did not use appropriate measures of variability.

While randomization on average produces balance between comparison groups, it does not guarantee balance in every trial or for any specific baseline measure. Clearly, imbalances are more common in smaller trials and they may raise questions regarding the validity of the trial outcomes. For example, a placebo-controlled, double-blind trial in 39 participants with mucopolysaccharidosis type VI reported that the intervention significantly improved exercise endurance [4]. However, at baseline the 12-min walk test, the primary outcome measure, showed the distance walked to be 227 m in the intervention group and 381 m in the placebo group, a substantial baseline imbalance. A double-blind placebo-controlled trial in 341 participants with Alzheimer’s disease evaluated three active treatments—vitamin E, a selective monoamine oxidase inhibitor, and their combination [5]. At baseline, the Mini-Mental State Examination (MMSE) score, a variable highly predictive of the primary outcome, was significantly higher in the placebo group than in the other two groups, indicating that the placebo patients were at a lower risk. Imbalances may even exist in large studies. In the Aspirin Myocardial Infarction Study [6], which had over 4,500 participants, the aspirin group was at slightly higher overall risk than the placebo group when prognostic baseline characteristics were examined.

Assessment of baseline comparability is important in all randomized trials. In studies without randomization assessing baseline comparability is more problematic. The known baseline factors may not always have been measured accurately. Moreover, there are all the other factors that were not measured or even known. For nonrandomized studies in contrast to randomized trials one cannot assume balance in the unmeasured covariates.

The investigator needs to look at baseline variables in several ways. The simplest is to compare each variable to make sure that it has reasonably similar distribution in each study group. Means, medians, and ranges are all convenient measures. The investigator can also combine the variables, giving each one an appropriate weight or coefficient, but doing this presupposes a knowledge of the relative prognostic importance of the variables. This kind of knowledge can come only from other studies with very similar populations or by looking at the control group after the present study is completed. The weighting technique has the advantage that it can take into account numerous small differences between groups. If imbalances between most of the variables are in the same direction, the overall imbalance can turn out to be large, even though differences in individual variables are small.

In the 30-center Aspirin Myocardial Infarction Study which involved over 4,500 participants, each center can be thought of as a small study with about 150 participants [6]. When the baseline comparability within each center was reviewed, substantial differences in almost half the centers were found, some favoring intervention and some control (Furberg, CD, unpublished data). The difference between intervention and control groups in predicted 3-year mortality, using the Coronary Drug Project model, exceeded 20% in 5 of the 30 clinics. This analysis illustrates that fairly large imbalances for known baseline factors can be common in smaller studies and that they may influence the trial outcome. In larger trials these study group differences balance out and the unadjusted primary analyses are reliable. In secondary analyses, adjustments can be made using regression methods with baseline covariates. Another approach is to use propensity scores which combine individual covariates (see Chap. 18).

Identified imbalances do not invalidate a randomized trial, but they may make interpretation of results more complicated. In the North American Silver-Coated Endotracheal Tube trial, a higher number of patients with chronic obstructive pulmonary disease were randomized to the uncoated tube group [7]. The accompanying editorial [8] pointed to this imbalance as one factor behind the lack of robustness of the results, which indicated a reduction in the incidence of ventilator-associated pneumonia. Chronic obstructive pulmonary disease is a recognized risk factor for ventilator-associated pneumonia.

It is important to know which baseline factors may influence the trial outcomes and to determine whether they were imbalanced and whether observed trends of imbalance favored one group or the other. The critical baseline factors to consider ought to be prespecified in the protocol. Reliance on significance testing as a measure of baseline equivalence is common [2]. Due to the often large number of statistical tests, the challenge is to understand the meaning and importance of observed differences whether or not they are statistically significant. A nonsignificant baseline group difference in the history of hemorrhagic stroke could still affect the treatment outcome in thrombolytic trials [9].

Formal statistical testing of baseline imbalances was common in the past. However, the consensus has changed and the position today is that such testing should be avoided [10–12]. When comparing baseline factors, groups can never be shown to be identical. Only absence of “significant” differences can be demonstrated. A review of 80 trials published in four leading journals, showed that hypothesis tests of baseline comparability were conducted in 46 of these trials. Of a total of 600 such tests, only 24 (4%) were significant at the 5% level [3], consistent with what would be expected by chance.

We agree that testing of baseline imbalances should not be conducted. However, in the Results section of the main article, we recommend that in addition to a description of the study population, special attention should be paid to those characteristics that are prognostically important.

Controlling for Imbalances in the Analysis

If there is concern that one or two key prognostic factors may not “balance out” during randomization, thus yielding imbalanced groups at baseline, the investigator may conduct a covariate-adjustment on the basis of these factors. In unadjusted analyses in the Alzheimer trial discussed above, there were no outcome differences among the groups. After adjustment for the baseline difference in MMSE, all actively treated groups did better than placebo by slowing the progression of disease. Chapter 18 reviews the advantages and disadvantages of covariate adjustment. The point here is that, in order to make such adjustments, the relevant characteristics of the participants at baseline must be known and measured. A survey of 50 randomized trials showed that most trials (38 of 50) emphasized the unadjusted comparisons [13], and 28 presented covariate-adjusted results as a back-up. Of the remaining 12 trials, 6 gave no unadjusted results. We recommend presentation of both unadjusted (primary) and adjusted (secondary) results.

Subgrouping

Often, investigators are interested not only in the response to intervention in the total study group, but also in the response in one or more subgroups. Particularly, in studies in which an overall intervention effect is present, analysis of results by appropriate subgroup may help to identify the specific population most likely to benefit from, or be harmed by, the intervention. Subgrouping may also help to elucidate the mechanism of action of the intervention. Definition of such subgroups should rely only on baseline data, not data measured after initiation of intervention (except for factors such as age or gender which cannot be altered by the intervention). An example of a potential problem with establishing subgroups post hoc is the Canadian Cooperative Study Group trial of aspirin and sulfinpyrazone in people with cerebral or retinal ischemic attacks [14]. After noting an overall benefit from aspirin in reducing continued ischemic attacks or stroke, the authors observed and reported that the benefit was restricted to men. In approving aspirin for the indication of transient ischemic attacks in men, the U.S. Food and Drug Administration relied on the Canadian Cooperative Study Group. A subsequent meta-analysis of platelet-active drug trials in the secondary prevention of cardiovascular disease concluded that the effect is similar in men and women [15]. However, a later placebo-controlled primary prevention trial of low-dose aspirin (100 mg on alternate days) in women reported a favorable aspirin effect on the risk of stroke, but no overall reduction in risks of myocardial infarction and cardiovascular death and perhaps benefit in those over 65 years of age [16]. Thus, this example illustrates that any conclusions drawn from subgroup hypotheses not explicitly stated in the protocol should be given less credibility than those from hypotheses stated a priori. Retrospective subgroup analysis should serve primarily to generate new hypotheses for subsequent testing (Chap. 18).

One of the large active-control trials of rosiglitazone in people with type 2 diabetes reported a surprising increase in the risk of fractures compared to metformin or glibenclamide, a risk that was, however, limited to women [17].In this case, the post hoc observation was replicated in a subsequent trial of pioglitazone which showed a similar gender-specific increase compared to placebo [18]. Additionally, a meta-analysis confirmed that this class of hypoglycemic agents doubles the risk of fractures in women without any documented increase in men [19]. Confirmation of initial results is important in science.

In their review of 50 clinical trial reports from four major medical journals, Assmann et al. [13] noted a large variability in the presentation of subgroup findings. Thirty-five reported subgroup analyses. Seventeen of these limited the number of baseline factors to one; five included seven or more factors. Eighteen of the 35 included more than one outcome to subgroup analyses; six reported on six or more outcomes. More than half of the subgroup reports did not use statistical tests for interaction. Such tests are critical, since they directly determine whether an observed treatment difference in an outcome depends on the participant’s subgroup. Additionally, it was often difficult to determine whether the subgroup analyses were prespecified or post hoc.

In a similar survey conducted in 72 randomized surgical trials, 54 subgroup analyses were conducted in 27 of the trials [20]. The majority of these were post hoc. The investigators featured these outcome differences in 31 of the 54 subgroup analyses in the Summary and Conclusions of the publication.

A rapidly emerging field in medicine is that of pharmacogenetics which holds promise for better identification of people who may benefit more from a treatment or who are more likely to develop serious adverse effects [21]. Until quite recently the focus was on a limited number of candidate genes due to the high cost of genotyping, but as technologies have improved attention has shifted to genome-wide association (GWA) studies of hundreds of thousands or millions of single-nucleotide polymorphisms (SNPs) [22]. This approach, and cost-effective whole-genome sequencing technologies, allows examination of the whole genome unconstrained by prior hypotheses on genomic structure or function influencing a given trait [23]. This has resulted in discoveries of an enormous number of genotype-phenotype relationships [24]. Collection of biologic samples at baseline in large, long-term trials has emerged as a rich source for such pharmacogenetic studies. However, the analyses of these samples is a statistical challenge due to the very large number of variants tested, which is typically dealt with by requiring very small p-values. Using strict Bonferonni correction, dividing the standard p < 0.05 by a million or more genetic variants assayed yields significance thresholds of 5 × 10(−8) which in turn require very large sample sizes to reach a similarly large replication samples [25]. Interpretation of rare variants detected by sequencing is even more challenging as the majority of such variants are present in only one subject even in large studies. In such cases functional information about the effect of the variant on the gene and its product and other forms of experimental evidence can be used to supplement the sequencing data [26].

Genetic determinants of beneficial responses to a treatment are increasingly investigated, especially in cancer. Three cancer drugs, imatinib mesylate, trastuzumab, and gefitinib, have documented efficacy in subsets of patients with specific genetic variants (particularly variants in the genomes of their tumors), while two others, irinotecan and 6-mercaptopurine, can be toxic in standard doses in other genetically defined subsets of patients [22]. Availability of clinical tests for these variants allows treatment to be cost-effective and more efficacious by limiting recommended use to those likely to benefit. The strength by which common variants can influence the risk determination ranges from a several-fold increased risk compared to those without the variant to a 1,000-fold increase [27]. Replications of the findings are especially important for these types of studies.

The identification of new genetic variants associated with serious adverse effects is also a critical area of investigation. The goal is to identify through genetic testing those high-risk patients prior to initiation of treatment. A genome-wide association study identified a SNP within the SLCO1B1 gene on chromosome 12 linked to dose-dependent, statin-induced myopathy [28]. Over 60% of all diagnosed myopathy cases could be linked to the C allele of the SNP rs4149056, which is present in 15% of the population. Identification of C allele carriers prior to initiating therapy could reduce myopathy while retaining treatment benefits by targeting this group for lower doses or more frequent monitoring of muscle-related enzymes.

The regulatory agencies are increasingly relying on subgroup analyses of pharmacogenetic markers. Presence of specific alleles, deficient gene products, inherited familial conditions and patterns of drug responses can provide important efficacy and safety information. As of 2014, the FDA has included in the labeling this type of subgroup data for approximately 140 different drugs [29].

The sample size requirements, the analytic problem of multiplicity (a genome-wide panel may have over 2.5 million SNPs after imputation) and the need for replications are discussed in Chap. 18.

What Constitutes a True Baseline Measurement?

Screening for Participants

In order to describe accurately the study participants, baseline data should ideally reflect the true condition of the participants. Certain information can be obtained accurately by means of one measurement or evaluation at a baseline interview and examination. However, for many variables, accurately determining the participant’s true state is difficult, since the mere fact of impending enrollment in a trial, random fluctuation or the baseline examination itself may alter a measurement. For example, is true blood pressure reflected by a single measurement taken at baseline? If more than one measurement is made, which one should be used as the baseline value? Is the average of repeated measurements recorded over some extended period of time more appropriate? Does the participant need to be taken off all medications or be free of other factors which might affect the determination of a true baseline level?

When resolving these questions, the screening required to identify eligible potential participants, the time and cost entailed in this identification, and the specific uses for the baseline information must be taken into account.

In almost every clinical trial, some sort of screening of potential participants for trial eligibility is necessary. This may take place over more than one visit. Screening eliminates participants who, based on the entrance criteria, are ineligible for the study. A prerequisite for inclusion is the participant’s willingness to comply with a possibly long and arduous study protocol. The participant’s commitment, coupled with the need for additional measurements of eligibility criteria, means that intervention allocation usually occurs later than the time of the investigator’s first contact with the participant. An added problem may result from the fact that discussing a study with someone or inviting him to participate in a clinical trial may alter his state of health. For instance, people asked to join a study of lipid-lowering agents because they had an elevated serum LDL cholesterol at a screening examination might change their diet on their own initiative just because of the fact they were invited to join the study. Therefore, their serum LDL cholesterol as determined at baseline, perhaps a month after the initial screen, may be somewhat lower than usual. Improvement could happen in many potential candidates for the trial and could affect the validity of the assumptions used to calculate sample size. As a result of the modification in participant behavior, there may be less room for response to the intervention. If the study calls for a special dietary regimen, this might not be so effective at the new, lowered LDL cholesterol level. Obviously, these changes occur not just in the group randomized to the active intervention, but also in the control group.

Although it may be impossible to avoid altering the behavior of potential participants, it is often possible to adjust for such anticipated changes in the study design. Special care can be taken when discussing studies with people to avoid sensitizing them. Time between invitation to join a study and baseline evaluation should be kept to a minimum. People who have greatly changed their eating habits between the initial screen and baseline, as determined by a questionnaire at baseline, can be declared ineligible to join. Alternatively, they can be enrolled and the required sample size increased. Whatever is done, these are expensive ways to compensate for the reduced expected response to the intervention.

Regression Toward the Mean

Sometimes a person’s eligibility for a study is determined by measuring continuous variables, such as blood sugar or cholesterol level. If the entrance criterion is a high or low value, a phenomenon referred to as “regression toward the mean” is encountered [30]. Regression toward the mean occurs because measurable characteristics of an individual do not have constant values but vary. Thus, individuals have days when the measurements are on the high side and other days when they are on the low side within their ranges of variability. Because of this variability, although the population mean for a characteristic may be relatively constant over time, the locations of individuals within the population change. If two sets of measurements are made on individuals within the population, therefore, when the first value is substantially larger (smaller) than the population mean, the second is likely to be lower (higher) than the first.

Therefore, whenever participants are selected from a population on the basis of the cutoff of some measured characteristic, the mean of a subsequent measurement will be closer to the population mean than is the first measurement mean. Furthermore, the more extreme the initial selection criterion (that is, the further from the population mean), the greater will be the regression toward the mean at the time of the next measurement. The “floor-and-ceiling effect” used as an illustration by Schor [31] is helpful in understanding this concept. If all the flies in a closed room near the ceiling in the morning are monitored, then at any subsequent time during the day more flies will be below where they started than above. Similarly, if the flies start close to the floor, the more probable it is for them to be higher, rather than lower, at any subsequent time.

Cutter [32] gives some nonbiological examples of regression toward the mean. He presents the case of a series of three successive tosses of two dice. The average of the first two tosses is compared with the average of the second and third tosses. If no selection or cut-off criterion is used, the average of the first two tosses would, in the long run, be close to the average of the second and third tosses. However, if a cut-off point is selected which restricts the third toss to only those instances where the average of the first and second tosses is nine or greater, regression toward the mean will occur. The average of the second and third tosses for this selected group will be less than the average of the first two tosses for this group.

As with the example of the participant changing his diet between screening and baseline, this phenomenon of regression toward the mean can complicate the assessment of intervention. In another case, an investigator may wish to evaluate the effects of an antihypertensive agent. She measures blood pressure once at the baseline examination and enters into the study only those people with systolic pressures over 150 mmHg. She then gives a drug and finds on rechecking that most people have responded with lowered blood pressures. However, when she re-examines the control group, she finds that most of those people also have lower pressures. Regression to the mean is the major explanation for the frequently seen marked mean blood pressure reduction observed early in the control group. The importance of a control group is obvious in such situations. An investigator cannot simply compare preintervention and postintervention values in the intervention group. She must compare postintervention values in the intervention group with values obtained at similar times in the control group.

This regression toward the mean phenomenon can also lead to a problem discussed previously. Because of regression, the true values at baseline are less extreme than the investigator had planned on, and there is less room for improvement from the intervention. In the blood pressure example, after randomization, many of the participants may have systolic blood pressures in the low 140’s or even below 140 rather than above 150 mmHg. There may be some reluctance to use antihypertensive agents in people with systolic pressures in the 130s, for example, than in those with higher pressures, and certainly, the opportunity to demonstrate full effectiveness of the agent may be lost.

Two approaches to reducing the impact of regression toward the mean have been used by trials relying on measurements with large variability, such as blood pressure and some chemical determinations. One approach is to use a more extreme value than the entrance criterion when people are initially screened. Secondly, mean values of multiple measurements at the same visit or from more than one screening visit have been used to achieve more stable measurements. In hypertensive trials with a cutoff of systolic blood pressure of 140 mmHg, only those whose second and third measure averaged 150 mmHg or greater would be invited at the first screening visit to the clinic for further evaluation. The average of two recordings at the second visit would constitute the baseline value for comparison with subsequent determinations.

Interim Events

When baseline data are measured too far in advance of intervention assignment, a study event may occur in the interim. The participants having events in the interval between allocation and the actual initiation of intervention would dilute the results and decrease the chances of finding a significant difference. In the European Coronary Surgery Study, coronary artery bypass surgery should have taken place within 3 months of intervention allocation [33]. However, the mean time from randomization to surgery was 3.9 months. Consequently, of the 21 deaths in the surgical group in the first 2 years, six occurred before surgery could be performed. If the response, such as death, is nonrecurring and this occurs between baseline and the start of intervention, the number of participants at risk of having the event later is reduced. Therefore, the investigator needs to be alert to any event occurring after baseline but before intervention is instituted. When such an event occurs before randomization, i.e. allocation to intervention or control, she can exclude the participant from the study. When the event occurs after allocation, but before start of intervention, participants should nevertheless be kept in the study and the event counted in the analysis. Removal of such participants from the study may bias the outcome. For this reason, the European Coronary Surgery Study Group kept such participants in the trial for purposes of analysis. The inappropriateness of withdrawing participants from data analysis is discussed more fully in Chap. 18.

Uncertainty About Qualifying Diagnosis

A growing problem in many disease areas such as arthritis, diabetes and hypertension is finding potential participants who are not receiving competing treatments prior to randomization. So-called washout phases are often relied on in order to determine “true” baseline values.

Particularly difficult are those studies where baseline factors cannot be completely ascertained until after intervention has begun. For optimal benefit of thrombolytic therapy in patients with a suspected acute myocardial infarction, treatment has to be given within hours. This means that there is not time to wait for confirmation of the diagnosis with development of Q-wave abnormalities on the ECG and marked increases in serum levels of cardiac enzymes. In the Global Utilization of Streptokinase and Tissue Plasminogen Activator of Occluded Coronary Arteries (GUSTO) trial treatment had to be given within 6 h [34]. To confirm the diagnosis, the investigators had to settle for two less definitive criteria; chest pain lasting at least 20 min and ST-segment elevations on the ECG.

The challenge in the National Institute of Neurological Disorders and Stroke t-PA stroke trial was to obtain a brain imaging study and to initiate treatment within 180 min of stroke onset. This time was difficult to meet and participant enrollment lagged. As a result of a comprehensive process improvement program at the participating hospitals, the time between hospital admission and initiation of treatment was substantially reduced with increased recruitment yield. Almost half of eligible patients admitted within 125 min of stroke onset were enrolled [35].

Even if an investigator can get baseline information just before initiating intervention, she may need to compromise. For instance, being an important prognostic factor, serum cholesterol level is obtained in most studies of heart disease. Serum cholesterol levels, however, are temporarily lowered during the acute phase of a myocardial infarction and additionally a large number of participants may be on lipid-lowering therapy. Therefore, in any trial enrolling people who have just had a myocardial infarction, baseline serum cholesterol data relate poorly to their usual levels. Only if the investigator has data on participants from a time before the myocardial infarction and prior to any initiation of lipid-lowering therapy would usual cholesterol levels be known. On the other hand, because she has no reason to expect that one group would have greater lowering of cholesterol at baseline than the other group, such levels can certainly tell her whether the study groups are initially comparable.

Contamination of the Intervention

For many trials of chronic conditions, it can be difficult to find and enroll newly diagnosed patients. To meet enrollment goals, investigators often take advantage of available pools of treated patients. In order to qualify such patients, they often have to be withdrawn from their treatment. The advantage of treatment withdrawal is that a true baseline can be obtained. However, there are ethical issues involved with withdrawing active treatments (Chap. 2).

An alternative may be to lower the eligibility criteria for this group of treated patients. In the Antihypertensive and Lipid Lowering treatment to prevent Heart Attack Trial (ALLHAT), treated hypertensive patients were enrolled even if their screening blood pressures were below the treatment goal blood pressures [36]. It was assumed that these individuals were truly hypertensive and, thus, had elevated blood pressures prior to being given antihypertensive medications. The disadvantage of this approach is that the true untreated baseline values for blood pressure were unknown.

Medications that participants are taking may also complicate the interpretation of the baseline data and restrict the uses to which an investigator can put baseline data. Determining the proportion of diabetic participants in a clinical trial based on the number with elevated fasting blood sugar or HbA1C levels at a baseline examination will underestimate the true prevalence. People treated with oral hypoglycemic agents or insulin may have their laboratory values controlled. Thus, the true prevalence of diabetics would be untreated participants with elevated blood sugar or HbA1C and those being treated for their diabetes regardless of their laboratory values. Similarly, a more accurate estimate of the prevalence of hypertension would be based on the number of untreated hypertensive subjects at baseline plus those receiving antihypertensive treatment.

Withdrawing treatment prior to enrollment could introduce other potential problems. Study participants with a supply of the discontinued medications left in their medicine cabinet may use them during the trial and, thus, contaminate the findings. Similarly, if they have used other medications prescribed for the condition under study, they may also resort to these, whether or not their use is allowed according to the study protocol. The result may be discordant use in the study groups. Assessing and adjusting for the concomitant drug use during a trial can be complex. The use and frequency of use need to be considered. All of these potential problems are much smaller in trials of newly diagnosed patients.

Appreciating that, for many measurements, baseline data may not reflect the participant’s true condition at the time of baseline, investigators perform the examination as close to the time of intervention allocation as possible. Baseline assessment may, in fact, occur shortly after allocation but prior to the actual start of intervention. The advantage of such timing is that the investigator does not spend extra time and money performing baseline tests on participants who may turn out to be ineligible. The baseline examination then occurs immediately after randomization and is performed not to exclude participants, but solely as a baseline reference point. Since allocation has already occurred, all participants remain in the trial regardless of the findings at baseline. This reversal of the usual order is not recommended in single-blind or unblinded studies, because it raises the possibility of bias during the examination. If the investigator knows to which group the participant belongs, she may subconsciously measure characteristics differently, depending on the group assignment. Furthermore, the order reversal may unnecessarily prolong the interval between intervention allocation and its actual start.

Changes of Baseline Measurement

Making use of baseline data will usually add sensitivity to a study. For example, an investigator may want to evaluate a new hypoglycemic agent. She can either compare the mean change in HbA1C from baseline to some subsequent time in the intervention group against the mean change in the control group, or simply compare the mean HbA1C of the two groups at the end of the study. The former method usually is a more powerful statistical technique because it can reduce the variability of the response variables. As a consequence, it may permit either fewer participants to be studied or a smaller difference between groups to be detected (see Chap. 18).

Evaluation of possible unwanted effects requires knowledge—or at least tentative ideas—about what effects might occur. The investigator should record at baseline those clinical or laboratory features which are likely to be adversely affected by the intervention. Unexpected adverse effects might be missed, but the hope is that animal studies or earlier clinical work will have identified the important factors to be measured.

References

1.

Pocock SJ, Assmann SE, Enos LE, Kasten LE. Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Stat Med 2002;21:2917–2930.CrossRef

2.

Hall JC, Hall JL. Baseline comparisons in surgical trials. ANZ J Surg 2002;72:567–569.CrossRef

3.

Altman DG, Doré CJ. Randomization and baseline comparisons in clinical trials. Lancet 1990;335:149–153.CrossRef

4.

Harmatz P, Giugliani R, Schwartz I, et al. Enzyme replacement therapy for mucopolysaccharidosis VI: A phase 3, randomized, double-blind, placebo-controlled, multinational study of recombinant human N-acetylgalactosamine 4-sulfatase (recombinant human arylsulfatase B or RHASB) and follow-on, open-label extension study. J Pediatr 2006;148:533–539.CrossRef

5.

Sano M, Ernesto C, Thomas RG, et al. for the Members of the Alzheimer’s Disease Cooperative Study. A controlled trial of selegiline, alpha-tocopherol, or both as treatment for Alzheimer’s Disease. N Engl J Med 1997;336:1216–1222.CrossRef

6.

Aspirin Myocardial Infarction Study Research Group. A randomized, controlled trial of aspirin in persons recovered from myocardial infarction. JAMA 1980;243:661–669.CrossRef

7.

Kollef MH, Afessa B, Anzueto A, et al. for the NASCENT Investigation Group. Silver-coated endotracheal tubes and incidence of ventilator-associated pneumonia. The NASCENT randomized trial. JAMA 2008;300:805–813.CrossRef

8.

Chastre J. Preventing ventilator-associated pneumonia. Could silver-coated endotracheal tubes be the answer? (Editorial) JAMA 2008;300:842–844.CrossRef

9.

Burgess DC, Gebski VJ, Keech AC. Baseline data in clinical trials. MJA 2003;179:105–107.

10.

Senn S: Testing for baseline balance in clinical trials. Stat Med 1994;13:1715–1726.CrossRef

11.

Steyerberg EW, Bossuyt PMM, Lee KL. Clinical trials in acute myocardial infarction: Should we adjust for baseline characteristics? Am Heart J 2000;139:745–751.CrossRef

12.

Roberts C, Torgerson DJ. Understanding controlled trials. Baseline imbalance in randomised controlled trials. Br Med J 1999;319:185.CrossRef

13.

Assmann SF, Pocock SJ, Enos LE, Kasten LE. Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet 2000;355:1064–1069.CrossRef

14.

The Canadian Cooperative Study Group. A randomized trial of aspirin and sulfinpyrazone in threatened stroke. N Engl J Med 1978;299:53–59.CrossRef

15.

Antiplatelet Trialists’ Collaboration. Collaborative overview of randomised trials of antiplatelet therapy - I. Prevention of death, myocardial infarction, and stroke by prolonged antiplatelet therapy in various categories of patients. Br Med J 1994;308:81–106.

16.

Ridker PM, Cook NR, Lee I-M, et al. A randomized trial of low-dose aspirin in the primary prevention of cardiovascular disease in women. N Engl J Med 2005;352:1293–1304.CrossRef

17.

Kahn SE, Haffner SM, Heise MA, et al. for the ADOPT Study Group. Glycemic durability of rosiglitazone, metformin, or glyburide monotherapy. N Engl J Med 2006;355:2427–2443.

18.

Dormandy JA, Charbonnel B, Eckland DJA, et al. Secondary prevention of macrovascular events in patients with type 2 diabetes in the PROactive study (PROspective pioglitAzone Clinical Trial In macroVascular Events): A randomised controlled trial. Lancet 2005;366:1279–1289.CrossRef

19.

Loke YK, Singh S, Furberg CD. Long-term use of thiazolidinediones and fractures in type 2 diabetes: a meta-analysis. CMAJ 2009;180:32–39.CrossRef

20.

Bhandari M, Devereaux PJ, Li P, et al. Misuse of baseline comparison tests and subgroup analyses in surgical trials. Clin Orthop Relat Res. 2006;447:247–251.CrossRef

21.

Johnson JA, Boerwinkle E, Zineh I, et al. Pharmacogenomics of antihypertensive drugs: Rationale and design of the Pharmacogenomic Evaluation of Antihypertensive Responses (PEAR) study. Am Heart J 2009;157:442–449.CrossRef

22.

Grant SF, Hakonarson H. Recent development in pharmacogenomics: from candidate genes to genome-wide association studies. Expert Rev Mol Diagn 2007;7:371–393.CrossRef

23.

Donnelly P. Progress and challenges in genome-wide association studies in humans. Nature 2008;456:728–731.CrossRef

24.

Lu JT, Campeau PM, Lee BH. Genotype-phenotype correlation—promiscuity in the era of next-generation sequencing. N Engl J Med 2014;371:593–596.CrossRef

25.

Chanock SJ, Manolio T, Boehnke M, et al. NCI-NHGRI Working Group on Replication in Association Studies. Replicating genotype-phenotype associations. Nature 2007;447:655–660.CrossRef

26.

MacArthur DG, Manolio TA, Dimmock DP, et al. Guidelines for investigating causality of sequence variants in human disease. Nature 2014;508:469–476.CrossRef

27.

Nelson MR, Bacanu S-A, Mosteller M, et al. Genome-wide approaches to identify pharmacogenetic contributions to adverse drug reactions. Pharmacogenomics J 2009;9:23–33.CrossRef

28.

The SEARCH Collaborative Group. SLCO1B1 variants and statin-induced myopathy—A genomewide study. N Engl J Med 2008;359:789–799.CrossRef

29.

The U.S. Food and Drug Administration. Drugs. Table of pharmacogenomic biomarkers in drug labeling. Updated 08/18/2014. www.​fda.​gov/​drugs/​scienceresearch/​researchareas/​pharmacogenetics​/​ucm083378.​htm

30.

James KE. Regression toward the mean in uncontrolled clinical studies. Biometrics 1973;29:121–130.CrossRef

31.

Schor SS. The floor-and-ceiling effect. JAMA 1969;207:120.CrossRef

32.

Cutter GR. Some examples for teaching regression toward the mean from a sampling viewpoint. Am Stat 1976;30:194–197.

33.

European Coronary Surgery Study Group. Coronary-artery bypass surgery in stable angina pectoris: survival at two years. Lancet 1979;i:889–893.

34.

The GUSTO Investigators. An international randomized trial comparing four thrombolytic strategies for acute myocardial infarction. N Engl J Med 1993;329:673–682. (Correction 1994;331:277).

35.

Tilley BC, Lyden PD, Brott TG, et al. for the National Institute of Neurological Disorders and Stroke rt-PA Stroke Study Group. Total quality improvement method for reduction of delays between emergency department admission and treatment of acute ischemic stroke. Arch Neurol 1997;54:1466–1474.CrossRef

36.

Davis BR, Cutler JA, Gordon DJ, et al. for the ALLHAT Research Group. Rationale and design of the Antihypertensive and Lipid Lowering treatment to prevent Heart Attack Trial (ALLHAT). Am J Hypertens 1996;9:342–360.

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_10

10. Recruitment of Study Participants

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

Often the most difficult task in a clinical trial involves obtaining sufficient study participants within a reasonable time. Time is a critical factor for both scientific and logistical reasons. From a scientific viewpoint, there is an optimal window of time within which a clinical trial can and should be completed. Changes in medical practice, including introduction of new treatment options, may make the trial outdated before it is completed. Other investigators may answer the questions sooner. In terms of logistics, the longer recruitment extends beyond the initially allotted recruitment periods, the greater the pressure becomes to meet the goal. Lagging recruitment will also reduce the statistical power of the trial. Selective recruitment of a lower proportion of eligible participants may increase the non-representative nature of the sample. Costs increase, frustration and discouragement often follow. The primary reasons for recruitment failure include overly optimistic expectations, failure to start on time, inadequate planning, and insufficient effort.

Approaches to recruitment of participants will vary depending on the type and size of the trial, the length of time available, the setting (hospital, physician’s office, community), whether the trial is single- or multicenter, and many other factors. Because of the broad spectrum of possibilities, this chapter summarizes concepts and general methods rather than elaborating on specific techniques. Emphasis is placed on anticipating and preventing problems. This chapter addresses plans for the recruitment effort, common recruitment problems, major recruitment strategies and sources, use of electronic health records for screening, and monitoring of recruitment.

Fundamental Point

Successful recruitment depends on developing a careful plan with multiple strategies, maintaining flexibility, establishing interim goals, preparing to devote the necessary effort, and obtaining the sample size in a timely fashion.

Considerations Before Participant Enrollment

Selection of Study Sample

In Chap. 4, we define the study population as “the subset of the population with the condition or characteristics of interest defined by the eligibility criteria.” The group of participants actually recruited into the trial, i.e., the study sample, is a selection from this study population. Those enrolled into a trial do not represent a random sample of those eligible for enrollment. Eligible individuals who volunteer to participate in a randomized trial may be different from eligible non-participants (see below). The impact of this potential selection bias on the results of a trial is not well understood. A better understanding of the factors that influence either willingness or unwillingness to participate in a research project can be very helpful in the planning of recruitment efforts.

The public is generally willing to participate in clinical trials [1]. A survey of around 1,000 Americans conducted in May 2013 showed that 64% would “take part in a clinical trial if I was asked by someone I trust.” Yet only about 15% of Americans report that they have participated in a clinical trial. In this survey, lack of trust was the major barrier to participation in trials. When asked to rate factors involved with decision to volunteer, nearly 70% reported reputation of the people or institution conducting the research and whether medical bills resulting from injury from the study would be covered as very important factors. Opportunity to possibly improve one’s own health was noted as very important for 61%, privacy and confidentiality issues for 53%, and opportunity to improve health of others for 50%. Complementary information was reported in a review of 14 studies through 2001 that had addressed the question—What reasons do people give for participating and not participating in clinical trials [2]? The answers came from 2,189 participants and 6,498 who declined. The variability was large, but trial participants gave as their major reason for participating potential health benefit (45%), physician influence (27%), and potential benefit to others (18%). Less commonly mentioned reasons given by participants in other studies included a desire to learn more about their condition, get free and better care, encouragement by family members and friends, favorable impression of and trust in clinical staff, and even to help promote the investigators’ careers [3–6].

Several reasons for declining participation in research projects have also been reported. In the Emergency Care Research Institute (ECRI) survey [2], the major general reasons for not participating were inconvenience (25%), concern over experimentation (20%), potential lack of health benefit (19%), and physician influence (14%). Many patients also lacked interest and preferred to stay with their own physicians. In another survey, fear was given as a major reason by half of those declining participation and the use of a placebo by almost one-quarter [6].

Logistical issues are sometimes given—demands on time, conflicts with other commitments, and problems with travel/transportation and parking. Barriers to participation in cancer trials include concerns with the trial setting, a dislike of randomization, presence of a placebo or no-treatment group, and potential adverse effects [7].

Common Recruitment Problems

The published experience from recruitment of participants into clinical trials through 1995 is nicely summarized in a literature review and annotated bibliography [8]. Over 4,000 titles were identified and 91 articles considered useful for formulation of recruitment strategies in clinical trials are annotated. The review focuses on experiences recruiting diverse populations such as ethnic minorities, women, and the elderly. Also discussed are successful recruitment approaches, which include use of registries, occupational sites, direct mailing, and use of media. The article highlights the value of pilot studies, projecting and monitoring recruitment, and the use of data tracking systems. Many of these issues are covered in more detail later in this chapter.

A review from the United Kingdom of 114 clinical trials that recruited participants between 1994 and 2002 explored the factors related to good and poor recruitment [9]. Approximately one-third of all trials met their original recruitment goal within the proposed time frame while approximately half had to be extended. Among those failing to make the original target, one half revised the goals. About 40% of all trials did not initiate recruitment as planned, mostly due to staffing and logistical issues. Almost two-thirds of the trials acknowledged early recruitment problems. More than half of the reviewed trials, a remarkably high number, had a formal pilot study that led to changes in the recruitment approach for the main trial. The written trial materials were revised, the trial design altered, the recruitment target changed, the number of sites increased, and/or the inclusion criteria broadened.

A systematic review of recruitment methods identified 37 trials describing four broad categories of recruitment strategies: novel trial designs (including different consent strategies), recruiter interventions (including training), incentives, and provision of trial information to potential participants [10]. Strategies that increased recruitment rates included increasing awareness of the health problem being studied, educational sessions, health questionnaires, and monetary incentives. A study using semistructured investigator interviews in the Palliative Care Research Cooperative Group identified five effective recruitment strategies: systematic screening of patient lists or records, messaging to patients to make research relevant, flexible protocols to account for patient needs, clinical champion support, and involvement in the cooperative group [11]. There is little published information on cost-effectiveness of interventions to improve recruitment, although one review of ten studies found that directly contacting potential participants seemed to be an efficient strategy [12].

Electronic health records, combined with programming to systematically screen for eligible participants, provide an important tool for certain trials. A single-center study showed that when an automated electronic health record alert system was put into place in clinics to identify patients for a trial of type II diabetes, referral rates increased by tenfold and enrollment rate by twofold [13]. An electronic screening tool was effective, and it performed particularly well in excluding ineligible patients at Columbia University for the National Institutes of Health (NIH)-sponsored Action to Control Cardiovascular Risk in Diabetes (ACCORD) Trial [14]. A hybrid approach of electronic screening coordinated with paper screening forms was successful in the Veterans Affairs VA-STRIDE trial to recruit patients for a physical activity intervention [15].

A review of the challenges and opportunities of use of electronic health records to support clinical trial enrollment identifies regulatory issues such as use of screening information preparatory to research and barriers related to desire to approach a patient’s treating doctor before approaching the patient directly [16]. An overarching goal is to use electronic systems to better integrate clinical trials into clinical practice. An example of success is the Thrombus Aspiration in ST-Elevation myocardial infarction in Scandinavia (TASTE) trial, where all interventional hospitals (and cardiologists) in Sweden agreed to approach all eligible patients for enrollment using the national registry platform for randomization and data collection [17]. During the recruitment period (see Fig. 5.​1), 59.7% of all the patients presenting with ST segment elevation myocardial infarction and referred for percutaneous coronary intervention and 76.9% of all the patients potentially eligible for enrollment in Sweden and Iceland were included in the trial, at a low per-patient cost. Not only did this remarkably high participant capture allow the trial enrollment to be completed in 2 years and 9 months, but it enhanced generalizability of the results.

But it is more typical that even when carefully planned and perfectly executed, recruitment may still proceed slowly. Investigators should always expect problems to occur, despite their best efforts. Most of the problems are predictable but a few may be completely unforeseen. In one multicenter study, there were reports of murders of inpatients at the hospital adjacent to the study clinic. It is hardly surprising that attendance at the clinic fell sharply.

Overestimation of eligible participants is a common reason for recruitment difficulties. A group of Finnish investigators [18] conducted a retrospective chart review. The typical eligibility criteria for clinical trials of patients with gastric ulcer were applied to 400 patients hospitalized with that diagnosis. Only 29% met the eligibility criteria but almost all deaths and serious complications such as gastric bleeding, perforation, and stenosis during the first 5–7 years occurred among those who would have been ineligible. Clearly, the testing of H₂–blockers or other compounds for the prevention of long-term complication of gastric ulcer in low-risk participants should not be generalized to the entire ulcer population. Troubling in this report is the evidence that the eligibility criteria can have such a dramatic effect on the event rates in those qualifying for participation.

Reliance on physician referrals is common and often problematic. Usually this technique results in very few eligible participants. In 2005, a survey of 7,000 physicians reported that only 31% of them had ever referred a patient to a clinical trial [6]. In one multicenter trial an investigator invited internists and cardiologists from a large metropolitan area to a meeting. He described the study, its importance, and his need to recruit men who had had a myocardial infarction. Each of the physicians stood up and promised to contribute one or more participants. One hundred fifty participants were pledged; only five were ultimately referred. Despite this, such pleas may be worthwhile because they make the professional community aware of a study and its purpose. Investigators who stay in close contact with physicians in a community and form a referral network have more success in obtaining cooperation and support.

When recruitment becomes difficult, one possible outcome is that an investigator will begin to loosely interpret entry criteria or will deliberately change data to enroll otherwise ineligible participants or even “enroll” fictitious subjects. Unfortunately, this issue is not merely theoretical. Such practices have occurred, to a limited extent, in more than one trial [19–21]. The best way to avoid the problem is to make it clear that this type of infraction harms both the study and the participants, and that neither science nor the investigators are served well by such practices. An announced program of random record audits by an independent person or group during the trial may serve as a deterrent.

Planning

In the planning stage of a trial, an investigator needs to evaluate the likelihood of obtaining sufficient study participants within the allotted time. This planning effort entails obtaining realistic estimates of the number of available potential participants meeting the study entry criteria. However, in the United States, access to available patient data from paper and electronic medical records requires compliance with the Health Insurance Portability and Accountability Act (HIPAA) and similar regulations apply in many other countries. Access can be granted but many community practices do not have such a mechanism in place and tend to be reluctant to release patient information. Even if those restrictions are overcome, census tract data or hospital and physician records may be out of date, incomplete, or incorrect. Information about current use of drugs or frequency of surgical procedures may not reflect what will occur in the future, when the trial is actually conducted. Records may not give sufficient—or even accurate—details about potential participants to determine the impact of all exclusion criteria. Clearly, available data certainly do not reflect the willingness of people to enroll in the trial or comply with the intervention.

After initial record review, an investigator may find it necessary to expand the population base by increasing the geographical catchment area, canvassing additional hospitals, relaxing one or more of the study entrance criteria, increasing the planned recruitment time, or any combination of these. The preliminary survey of participant sources should be as thorough as possible, since these determinations are better made before, rather than after, a study begins.

Investigator and study coordinator commitment is key to success. Since lack of trust is a major barrier to patient agreement to participate, a strategy for effective communication from the treating physician to the patient about the relevance and importance of the study is critical. A concern is that investigators keep adding new trials to those they already have committed to. Trials with higher payments seem to get more attention. The investigator also needs strong support from his institution and colleagues. Other investigators in the same institution or at nearby institutions may compete for similar participants. Since participants should generally not be in more than one trial at a time, competing studies may decrease the likelihood that the investigator will meet his recruitment goal. Competition for participants may necessitate reappraising the feasibility of conducting the study at a particular site.

Announcements of the trial should precede initiation of recruitment. The courtesy of informing area health professionals about the trial in advance can facilitate cooperation, reduce opposition, and avoid local physicians’ surprise at first hearing about the study from their patients rather than from the investigator. Talks to local professional groups are critical, but these and any notices regarding a trial should indicate whether the investigator is simply notifying physicians about the study or is actively seeking their assistance in recruiting participants.

Planning also involves setting up a clinic structure for recruitment with interested and involved co-investigators, an experienced and organized coordinator in charge of recruitment, and other staff required for and dedicated to the operations. A close working relationship between the clinic staff and investigators, with regular clinic meetings, is crucial from the very beginning to enrollment of the last participant. Careful planning and clear delineation of staff responsibilities are essential features of well-performing recruitment units.

Recruitment in most trials is curvilinear, particularly in multicenter trials, with a gradual acceleration of enrollment as centers start up and refine successful strategies for enrollment. But the calculation of a sample size estimate typically assumes a constant rate of enrollment. A slow start can reduce the statistical power of the trial by reducing the average participant follow-up time. Thus, ideally, recruitment should begin no later than the first day of the designated recruitment period. As important as the best planning is, commitment and willingness by everyone to spend a considerable amount of time in the recruitment effort are equally important. Just as investigators usually overestimate the number of participants available, they often underestimate the time and effort needed to recruit. Investigators must accommodate themselves to the schedules of potential participants, many of whom work. Thus, recruitment is often done on weekends and evenings, as well as during usual working hours.

The need for multiple recruitment strategies has been well documented [22, 23]. The first randomization should take place on the first day of the identified recruitment period. Therefore, if there is a lengthy prerandomization screening period, adjustments in the timing of the first randomization should be made. Because it is difficult to know which strategies will be productive, it is important to monitor effort and yield of the various strategies. A successful strategy in one setting does not guarantee success in another. The value of multiple approaches is illustrated by one large study in which the investigator identified possible participants and wrote letters to them, inviting them to participate. He received a poor response until his study was featured on local radio and television news. The media coverage had apparently “legitimized” the study, as well as primed the community for acceptance of the trial.

Contingency plans must be available in case recruitment lags. Experience has shown that recruitment yields, in general, are much lower than anticipated. Thus, the identified sources needed to be much larger than the recruitment goals. Hence, additional sources of potential study participants should be kept in reserve. Approval from hospital staff, large group practices, managed care organizations, corporation directors, or others controlling large numbers of potential participants often takes considerable time. Waiting until recruitment problems appear before initiating such approval can lead to weeks or months of inaction and delay. Therefore, it is advisable to make plans to use other sources before the study gets underway. If they are not needed, little is lost except for additional time used in planning. Most of the time these reserves will prove useful.

If data concerning recruitment of potential participants to a particular type of trial are scanty, a pilot or feasibility study may be worthwhile. Pilot studies can provide valuable information on optimal participant sources, recruitment techniques, and estimates of yield. In a trial of elderly people, the question arose whether those in their 70s or 80s would volunteer and actively participate in a long-term, placebo-controlled trial. Before implementing a costly full-scale trial, a pilot study was conducted to answer these and other questions [24]. The study not only showed that the elderly were willing participants, but also provided information on recruitment techniques. The success of the pilot led to a full-scale trial.

Recruitment Sources

The sources for recruitment depend on the features of the study population; sick people versus well, hospitalized versus not, or acute versus chronic illness. For example, enrollment of acutely ill hospitalized patients can only be conducted in an acute care setting, while enrollment of healthy asymptomatic individuals with certain characteristics or risk factors requires a community-based screening program. Following the introduction of the HIPAA and other privacy regulations, readily available sources for recruitment have changed. Identification of potential participants through review of electronic health records requires active involvement of those patients’ own physicians. Thus, focus has shifted to direct participant appeal.

Direct invitation to study participants is an attractive approach, since it avoids many confidentiality issues. Solicitation may be done through mass media, wide dissemination of leaflets advertising the trial, or participation by the investigator in health fairs. None of these methods is foolproof. The yield is often unpredictable and seems to depend predominantly on the skill with which the approach is made and the size and kind of audience it reaches. One success story featured a distinguished investigator in a large city who managed to appear on a local television station’s evening news show. Following this single 5-min appeal, thousands of people volunteered for the screening program. Experience, however, has shown that most individuals who respond to a media campaign are not eligible for the trial.

The recruitment into the Systolic Hypertension in the Elderly Program (SHEP) was a major undertaking [25]. A total of almost 450,000 potential participants were contacted in order to enroll 4,736 (1.1%). One of the major recruitment approaches in SHEP was mass mailings. A total of 3.4 million letters were sent by 14 of the SHEP clinics and the overall response rate was 4.3%. Names were obtained from the Departments of Motor Vehicles, voter registration lists, health maintenance organizations, health insurance companies, AARP, and others. Endorsement was obtained from these organizations and groups; many of them issued the invitations on their own letterheads. Each mailing included a letter of invitation, a standard brochure describing SHEP, and a self-addressed stamped return postcard. Experience showed that the response rates varied by mailing list source. It was also clear that clinics with experienced recruitment staff did better than the others.

A U.S. survey of 620 previous trial participants asked where they first learned about the trials [6]. Media the most common answer, was given by 30%, but 26% said the internet. Web-based strategies seem to be growing in importance, although the yield appears to vary by type of trial. Only 14% in the survey first learned of the trial via physician referral.

Participants may also be approached through a third party. For instance, local chapters of patient organizations may be willing to refer members. Another approach is through physician referrals. To draw physicians’ attention to a particular study, an investigator may send letters, make telephone calls, present at professional society meetings, publish notices in professional journals, or exhibit at scientific conferences. The hope is that these physicians will identify potential participants and either notify the investigator or have the potential participant contact the investigator. As noted earlier, this usually yields few participants. To overcome the problem with physician referral, sponsors are offering financial incentives to referring physicians. The value of this practice has not been properly evaluated, but it has raised ethical issues concerning conflict of interest, disclosure to potential participants, and implications for the informed consent process [26].

The recruitment targets have to be adjusted if special subgroups of the population are being recruited. In response to a relative paucity of clinical trial data on women and minorities, the U.S. Congress in 1995 directed the NIH to establish guidelines for inclusion of these groups in clinical research. The charge to the Director of the NIH to “ensure that the trial is designed and carried out in a manner sufficient to provide valid analysis of whether the variables being studied in the trial affect women and members of minority groups, as the case may be, differently than other subjects in the trial” has major implications depending on the interpretation of the term “valid analysis” [27].

To document a similar effect, beneficial or harmful, separately for both men and women and separately for various racial/ethnic groups could increase the sample size by a factor ranging from 4 to 16. The sample size will grow considerably more if the investigator seeks to detect differences in response among the subgroups. We support adequate representation of women and minorities in clinical trials, but suggest that the primary scientific question being posed be the main determinant of the composition of the study population and the sample size. When the effort is made, successful enrollment of women and minorities can be accomplished. An example is the Selenium and Vitamin E Cancer Prevention Trial [28].

An increasingly common approach to meeting the need for large sample sizes in multicenter trials with mortality and major event response variables has been to establish clinical centers internationally [29]. This experience has been positive and the number of participants enrolled by such centers often exceeds those in the country of the study’s origin. The success in recruitment may, however, come at a cost. Trial findings may differ among countries (see also Chap. 21). Possible reasons include differences in the baseline characteristics of the study population, in the practice of medicine as a reflection of the quality of care, research traditions, and socioeconomic and other factors [30, 31]. O’Shea and Califf analyzed the international differences in cardiovascular trials and reported important differences in participant characteristics, concurrent therapies, coronary revascularizations, length of hospital stay, and clinical outcomes in the U.S. and elsewhere [32]. Importantly, they pointed out that, in general, the differing event rates would not be expected to affect the relative effects of a treatment. But there are examples of possible differences in treatment effects according to enrollment within versus outside the U.S., including less benefit of beta blockers in heart failure [33] and of ticagrelor following acute coronary syndromes [34]. The authors of a review of 657 abstracts from trials of acupuncture and other interventions concluded that some countries published unusually high proportions of positive results [35]. Possible explanations include publication biases, level of care, and differences in study populations.

Can findings from low and middle income countries be extrapolated to high income countries and regions and vice versa? It is important that the publications from large international studies address this question by presenting findings by geographic region.

Conduct

Successful recruitment of participants depends not only on proper planning but also on the successful implementation of the plan. Systems should be in place to identify all potential participants from the identified recruitment pool and to screen them for eligibility. For hospital-based studies, logging all admissions to special units, wards, or clinics is invaluable. However, keeping such logs complete can be difficult, especially during evenings or weekends. During such hours, those most dedicated to the study are often not available to ensure accuracy and completeness. Vacation times and illness may also present difficulties in keeping the log up to date. Therefore, frequent quality checks should be made. Participant privacy is also important and is guided by ethics committees, and in the U.S., HIPAA regulations. At what point do the investigators obtain consent? For those who refuse to participate, what happens to the data that had been collected and used to identify them? The answers to this will vary from institution to institution and depend on who is keeping the log and for what reason. Information recorded by code numbers can facilitate privacy. Electronic health records can be used. Electronic medical records permit software algorithms to search for patient profiles that match a particular protocol and automatically identify for the health care team those eligible for a specific trial. A group at Cincinnati Children’s Hospital reported a retrospective study estimating that the electronic health records in the emergency department could be searched with natural language processing, information extraction, and machine learning techniques to reduce screening workload by over 90% for 13 randomly selected, disease-specific trials [36]. However, the broad experience of electronic systems for clinical trial recruitment shows inconsistent evidence of value. The integration of the systems with human workflow may be more important than sophisticated algorithms [37].

For community-based studies, screening large numbers of people is typically a major undertaking especially if the yield is low. Prescreening potential participants by telephone to identify those with major exclusion criteria (e.g., using demographics, medical history) has been employed in many projects. In the Lung Health Study, investigators used prescreening to reduce the number of screening visits to approximately half of those projected [38, 39]. Investigators need to identify the best times to reach the maximum number of potential participants. If they intend to make home visits or hope to contact people by telephone, they should count on working evenings or weekends. Unless potential participants are retired, or investigators plan on contacting people at their jobs (which, depending on the nature of the job, may be difficult), normal working hours may not be productive times. Vacation periods and summers are additional slow periods for recruitment.

The logistics of recruitment may become more difficult when follow-up of enrolled participants occurs while investigators are still recruiting. In long-term studies, the most difficult time is usually towards the end of the recruitment phase when the same staff, space, and equipment may be used simultaneously for participants seen for screening, baseline, and follow-up examinations. Resources can be stretched to the limit and beyond if appropriate planning has not occurred.

The actual mechanics of recruiting participants needs to be established in advance. A smooth clinic operation is beneficial to all parties. Investigators must be certain that necessary staff, facilities, and equipment are available at appropriate times in the proper places. Keeping potential participants waiting is a poor way to earn their confidence.

Investigators and staff need to keep abreast of recruitment efforts. Conducting regular staff meetings and generating regular reports may serve as forums for discussion of yields from various strategies, percent of recruitment goal attained, as well as brainstorming and morale-boosting. These meetings, useful for both single- and multicenter trials, also provide the opportunity to remind everyone about the importance of following the study protocol, including paying careful attention to collection of valid data.

Record keeping of recruitment activities is essential to allow analyses of recruitment yields and costs from the various recruitment strategies. Recruiting large numbers of potential participants requires the creation of timetables, flow charts, and databases to ensure that screening and recruitment proceed smoothly. Such charts should include the number of people to be seen at each step in the process at a given time, the number and type of personnel and amount of time required to process each participant at each step, and the amount of equipment needed (with an allowance for “down” time). A planned pilot phase is helpful in making these assessments. One positive aspect of slow early recruitment is that the “bugs” in the start-up process can be worked out and necessary modifications made.

Several additional points regarding the conduct of recruitment are worth emphasizing:

First, the success of a technique is unpredictable. What works in one city at one time may not work at the same place at another time or in another city. Therefore, the investigator needs to be flexible and to leave room for modifications.

Second, investigators must maintain good relationships with participants’ personal physicians. Physicians disapproving of the study or of the way it is conducted are more likely to urge their patients not to participate.

Third, investigators must respect the families of potential participants. Most participants like to discuss research participation with their family and friends. Investigators should be prepared to spend time reviewing the study with them. If the study requires long-term cooperation from the participant, we encourage such discussions. Anything that increases family support is likely to lead to better recruitment and protocol adherence.

Fourth, recruiting should not be overly aggressive. While encouragement is necessary, excessive efforts to convince, or “arm twist” people to participate could prove harmful in the long run, in addition to raising ethical concerns. One might argue that excessive salesmanship is unethical. Those reluctant to join may be more likely to abandon the study later or be poor adherers to study interventions after randomization. Effective work on adherence begins during the recruitment phase.

Fifth, the recruitment success is closely associated with the level of commitment and effectiveness of communication of the investigator and the study coordinator.

Sixth, electronic health records and social media provide important opportunities to use electronic data and communication for certain types of trials.

Monitoring

Successful trial recruitment often depends on establishing short-term and long-term recruitment goals. The investigator should record these goals and make every effort to achieve them. Since lagging recruitment commonly results from a slow start, timely establishment of initial goals is crucial. The investigator should be ready to randomize participants on the first official day of study opening.

The use of weekly and/or monthly interim goals in a long-term study orients the investigator and staff to the short-term recruitment needs of the study. These goals can serve as indicators for lagging recruitment and may help avoid a grossly uneven recruitment pace. Inasmuch as participant follow-up is usually done at regular intervals, uneven recruitment results in periods of peak and slack during the follow-up phase. This threatens effective use of staff time and equipment. Of course, establishing a goal in itself does not guarantee timely participant recruitment. The goals need to be realistic and the investigator must make the commitment to meet each interim goal.

The reasons for falling behind the recruitment goal(s) should be determined. In a multicenter clinical trial, valuable insight can be obtained by comparing results and experiences from different centers. Those clinical sites with the best recruitment performance can serve as role models for other sites, which should be encouraged to incorporate other successful techniques into their recruitment schemes. Multicenter studies require a central office to oversee recruitment, compare enrollment results, facilitate communication among sites, and lend support and encouragement. Frequent feedback to the centers by means of tables and graphs, which show the actual recruitment compared with originally projected goals, are useful tools. Examples are shown in the following figures and table. Figure 10.1 shows the progress of an investigator who started participant recruitment on schedule and maintained a good pace during the recruitment period. The investigator and clinic staff accurately assessed participant sources and demonstrated a commitment to enrolling participants in a relatively even fashion. Figure 10.2 shows the record of an investigator who started slowly, but later improved. However, considerable effort was required to compensate for the poor start. Clinic efforts included expanding the base from which participants were recruited and increasing the time spent in enrollment. Even if the clinic eventually catches up, the person-years of exposure to the intervention has been reduced which may affect event rates and trial power. In contrast, as seen in Fig. 10.3, the investigator started slowly and was never able to improve his performance. This center was dropped from a multicenter study because it could not contribute enough participants to the study to make its continued participation efficient. Figure 10.4 shows enrollment in the TASTE trial [17], an example of a trial that enrolled the majority of eligible patients in an entire country, and even in this highly organized trial, enrollment started gradually and the rate increased over the first few months.

Fig. 10.1

Participant recruitment in clinic that consistently performed at goal rate

Fig. 10.2

Participant recruitment in a clinic that started slowly and then performed at greater than goal rate

Fig. 10.3

Participant recruitment in a clinic that performed poorly

Fig. 10.4

Participant recruitment in the TASTE trial [17] using a national registry for participant identification. PCI = percutaneous coronary intervention

Table 10.1 shows goals, actual recruitment, and projected final totals (assuming no change in enrollment pattern) for three centers of a multicenter trial. Such tables are useful to gauge recruitment efforts short-term as well as to project final numbers of participants. The tables and figures should be updated as often as necessary.

Table 10.1

Weekly recruitment status report by center

Center

(1)

(2)

(3)

(4)

(5)

(6)

(7)

(8)

Contracted goal

Enrollment this week

Actual enrollment to date

Goal enrollment to date

Actual minus goal

Success rates (3)/(4)

Final projected intake

Final deficit or excess (7)-(1)

A

150

1

50

53.4

−3.4

0.94

140

−10

B

135

1

37

48.0

−11.0

0.77

104

−31

C

150

2

56

53.4

2.6

1.06

157

7

Table used in the Beta-Blocker Heart Attack Trial: Coordinating Center, University of Texas, Houston

In single-center trials, the investigator should also monitor recruitment status at regular and frequent intervals. Review of these data with staff keeps everyone aware of recruitment progress. If recruitment lags, the delay can be noted early, the reasons identified, and appropriate action taken.

Approaches to Lagging Recruitment

We have identified five possible approaches to deal with lagging recruitment, in addition to the strategies to enhance enrollment reviewed above.

The first is to accept a smaller number of participants than originally planned. Doing this is far from ideal as the power of the study will be reduced. In accepting a smaller number of participants than estimated, the investigator must either alter design features such as the primary response variable, or change assumptions about intervention effectiveness and participant adherence. As indicated elsewhere, such changes midway in a trial may be liable to legitimate criticism. Only if the investigator is lucky and discovers that some of the assumptions used in estimating sample size were too pessimistic would this “solution” provide comparable power. There are rare examples of this happening. In a trial of aspirin in people with transient ischemic attacks, aspirin produced a greater effect than initially postulated [40]. Therefore, the less-than-hoped-for number of participants turned out to be adequate.

A second approach is to relax the inclusion criteria. This should be done only if little expectation exists that the study design will suffer. The design can be marred when, as a result of the new type of participants, the control group event rate is altered to such an extent that the estimated sample size is no longer appropriate. Also, the expected response to intervention in the new participants may not be as great as in the original participants. Furthermore, the intervention might have a different effect or have a greater likelihood of being harmful in the new participants than in those originally recruited. The difference in additional participants would not matter if the proportion of participants randomized to each group stayed the same throughout recruitment. However, as indicated in Chap. 6, certain randomization schemes alter that proportion, depending on baseline criteria or study results. Under these circumstances, changing entrance criteria may create imbalances among study arms.

The Coronary Drug Project provides a classic example [41]. Only people with documented Q-wave myocardial infarctions were originally eligible. With enrollment falling behind, the investigators decided to admit participants with non-Q-wave infarctions. Since there was no reason to expect that the action of lipid-lowering agents being studied would be any different in the new group than in the original group and since the lipid-lowering agents were not contraindicated in the new participants, the modification seemed reasonable. However, there was some concern that overall mortality rate would be changed because mortality in people with non-Q-wave infarctions may be less than mortality in people with Q-wave infarctions. Nevertheless, the pressure of recruitment overrode that concern. Possible baseline imbalances did not turn out to be a problem. In this particular study, where the total number of participants was so large (8,341), there was every expectation that randomization would yield comparable groups. If there had been uncertainty regarding this, stratified randomization could have been employed (Chap. 6). Including people with non-Q-wave infarctions may have reduced the power of the study because this group had a lower mortality rate than those with Q-wave infarctions in each of the treatment groups, including the placebo group. However, the treatments were equally ineffective when people with Q-wave infarctions were analyzed separately from people with non-Q-wave infarctions [42].

The third and probably most common approach to recruitment problems is to extend the time for recruitment or, in the case of multicenter studies, to add recruiting centers. Both are the preferred solutions, requiring neither modification of admission criteria nor diminution of power. However, they are also the most costly. Whether the solution of additional time or additional centers is adopted depends on cost, on the logistics of finding and training other high quality centers, and on the need to obtain study results quickly.

A fourth approach to lagging recruitment is “recycling” of potential participants. When a prospective participant just misses meeting the eligibility criteria, the temptation is natural to try to enroll them by repeating a measurement, perhaps under slightly different conditions. Due to variability in a screening test, many investigators argue that it is reasonable to allow one repeat test and give a person interested in the trial a “second chance.” In general, this practice should be discouraged. A study is harmed by enrolling persons for whom the intervention might be ineffective or inappropriate. However, in some progressive diseases, waiting a year to recycle a potential participant may prove to be useful.

Instances exist where, in order to enter a drug study, the participant needs to be off all other medication with similar actions. At baseline, participants may be asked whether they have adhered to this requirement. If they have not, the investigator may repeat the instructions and have the participants return in a week for repeat baseline measurements. The entrance criterion checks on a participant’s ability to adhere to a protocol and their understanding of instructions. This “second chance” is different from recycling and it is legitimate from a design point of view. However, the second-chance participant, even if he or she passes the repeat baseline measurement, may not be as good a candidate for the study as someone who adhered on the first occasion [43].

The fifth approach of broadening or changing the pre-specified primary response variable is very common, and is discussed in more detail in Chap. 3. Enrollment was slower than expected in the Prevention of Events with Angiotensin Converting Enzyme Inhibition (PEACE) trial [44]. This, combined with a narrowing of entry criteria to exclude patients with diabetes and either proteinuria or hypertension and microalbuminuria for whom benefit was clearly established from other trials, prompted a change in the primary outcome (of death from cardiovascular causes or nonfatal myocardial infarction) to include coronary revascularization, reducing the sample size from 14,100 patients to 8,100.

References

1.

America speaks, polling data reflecting the views of Americans on medical, health and scientific research. Available at http://​www.​researchamerica.​org/​uploads/​AmericaSpeaksV14​.​pdf. Access January 21, 2015.

2.

ECRI Health Technology Assessment Information Service. Patients’ reasons for participation in clinical trials and effect of trial participation on patient outcomes. ECRI Evidence Report. April 2002, Issue 74.

3.

Wright JR, Crooks D, Ellis PM, et al. Factors that influence the recruitment of patients to phase III studies in oncology. The perspective of the clinical research assistant. Cancer 2002;95:1584–1591.CrossRef

4.

Cox K, McGarry J. Why patients don’t take part in cancer clinical trials: an overview of the literature. Eur J Cancer Care 2003;12:114–122.CrossRef

5.

Sharp L, Cotton SC, Alexander L, et al. on behalf of the TOMBOLA group. Reasons for participation and non-participation in a randomized controlled trial: postal questionnaire surveys of women eligible for TOMBOLA (Trial of Management of Borderline and Other Low-grade Abnormal smears). Clin Trials 2006;3:431–442.

6.

Barnes K. Patients provide insight into trial participation. Outsourcing-Pharma.com, July 4, 2007. Available at www.​outsourcing-pharma.​com/​content/​view/​print/​135930. Access January 21, 2015.

7.

Mills EJ, Seely D, Rachlis B, et al. Barriers to participation in clinical trials of cancer: a meta-analysis and systematic review of patient-reported factors. Lancet Oncol 2006;7:141–148.CrossRef

8.

Lovato LC, Hill K, Hertert S, et al. Recruitment for controlled clinical trials: Literature summary and annotated bibliography. Control Clin Trials 1997;18:328–357.CrossRef

9.

McDonald AM, Knight RC, Campbell MK, et al. What influences recruitment to randomised controlled trials? A review of trials funded by two UK funding agencies. Trials 2006;7:9.

10.

Caldwell PH, Hamilton S, Tan A, Craig JC. Strategies for increasing recruitment to randomised controlled trials: systematic review. PLoS Med 2010;7:e1000368.CrossRef

11.

Hanson LC, Bull J, Wessell K, et al. Strategies to support recruitment of patients with life-limiting illness for research: the palliative care research cooperative group. J Pain Symptom Manage 2014;48:1021–1030.CrossRef

12.

Huynh L, Johns B, Liu SH, et al. Cost-effectiveness of health research study participant recruitment strategies: a systematic review. Clin Trials 2014;11:576–583.CrossRef

13.

Embi PJ, Jain A, Clark J, et al. Effect of a clinical trial alert system on physician participation in trial recruitment. Arch Intern Med 2005;165:2272–2277.CrossRef

14.

Thadani SR, Weng C, Bigger JT, et al. Electronic screening improves efficiency in clinical trial recruitment. J Am Med Inform Assoc 2009;16:869–873.CrossRef

15.

Hawkins MS, Hough LJ, Berger MA, et al. Recruitment of veterans from primary care into a physical activity randomized controlled trial: the experience of the VA-STRIDE study. Trials 2014;15:11.CrossRef

16.

Weng C, Appelbaum P, Hripcsak G, et al. Using EHRs to integrate research with patient care: promises and challenges. J Am Med Inform Assoc 2012;19:684–687.CrossRef

17.

Fröbert O, Lagerqvist B, Olivecrona GK, et al. Thrombus aspiration during ST-segment elevation myocardial infarction. N Engl J Med 2013;369:1587–1597.CrossRef

18.

Kääriäinen I, Sipponen P, Siurala M. What fraction of hospital ulcer patients is eligible for prospective drug trials? Scand J Gastroenterol 1991;186:73–76.CrossRef

19.

Sheldon T. Dutch neurologist found guilty of fraud after falsifying 438 case records. Br Med J 2002;325:734.CrossRef

20.

Ross DB. The FDA and the case of Ketek. N Engl J Med 2007;356:1601–1604.

21.

POISE Study Group. Effects of extended-release metoprolol succinate in patients undergoing non-cardiac surgery (POISE trial): a randomised controlled trial. Lancet 2008;371:1839–1847.CrossRef

22.

Hunninghake DB. Summary conclusions. Control Clin Trials 1987;8:1S–5S.CrossRef

23.

Kingry C, Bastien A, Booth G, et al for the ACCORD Study Group. Recruitment strategies in the Action to Control Cardiovascular Risk in Diabetes (ACCORD) Trial. Am J Cardiol 2007;99 (Suppl):68i–79i.

24.

Hulley SB, Furberg CD, Gurland B, et al. Systolic Hypertension in the Elderly Program (SHEP): antihypertensive efficacy of chlorthalidone. Am J Cardiol 1985;56:913–920.CrossRef

25.

Cosgrove N, Borhani NO, Bailey G, et al. Mass mailing and staff experience in a total recruitment program for a clinical trial: The SHEP Experience. Control Clin Trials 1999;19:133–148.CrossRef

26.

Bryant J, Powell J. Payment to healthcare professionals for patient recruitment to trials: a systematic review. Br Med J 2005;331:1377–1378.CrossRef

27.

Freedman LS, Simon R, Foulkes MA, et al. Inclusion of women and minorities in clinical trials and the NIH Revitalization Act of 1993—the perspective of NIH clinical trialists. Control Clin Trials 1995;16:277–285.CrossRef

28.

Cook ED, Moody-Thomas S, Anderson KB, et al. Minority recruitment to the Selenium and Vitamin E Cancer Prevention Trial (SELECT). Clin Trials 2005;2:436–442.CrossRef

29.

Glickman SW, McHutchison JG, Peterson ED, et al. Ethical and scientific implications of the globalization of clinical research. N Engl J Med 2009;360:816–823.CrossRef

30.

Shibata M, Flather M, de Arenaza DP, et al. Potential impact of socioeconomic differences on clinical outcomes in international clinical trials. Am Heart J 2001;141:1019–1024.CrossRef

31.

Orlandini A, Diaz R, Wojdyla D, et al. Outcomes of patients in clinical trials with ST-segment elevation myocardial infarction among countries with different gross national incomes. Eur Heart J 2006;27:527–533.CrossRef

32.

O’Shea JC, Califf RM. International differences in cardiovascular clinical trials. Am Heart J 2001;141:866–874.CrossRef

33.

O'Connor CM, Fiuzat M, Swedberg K, et al. Influence of global region on outcomes in heart failure β-blocker trials. J Am Coll Cardiol 2011;58:915–922.CrossRef

34.

Mahaffey KW, Wojdyla DM, Carroll K, et al. Ticagrelor compared with clopidogrel by geographic region in the Platelet Inhibition and Patient Outcomes (PLATO) trial. Circulation 2011;124:544–554.CrossRef

35.

Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials 1998;19:159–166.CrossRef

36.

Ni Y, Kennebeck S, Dexheimer JW, et al. Automated clinical trial eligibility prescreening: increasing the efficiency of patient identification for clinical trials in the emergency department. J Am Med Inform Assoc 2014; epub ahead of print: doi: 10.​1136/​amiajnl-2014-002887.

37.

Köpcke F, Prokosch HU. Employing computers for the recruitment into clinical trials: a comprehensive systematic review. J Med Internet Res 2014;16:e161.CrossRef

38.

Durkin DA, Kjelsberg MO, Buist AS, et al. Recruitment of participants in the Lung Health Study, I: description of methods. Control Clin Trials 1993;14:20S–37S.CrossRef

39.

Daly M, Seay J, Balshem A, et al. Feasibility of a telephone survey to recruit health maintenance organization members into a tamoxifen chemoprevention trial. Cancer Epidemiol Biomarkers Prev 1992;1:413–416.

40.

Fields WS, Lemak NA Frankowski RF, et al. Controlled trial of aspirin in cerebral ischemia. Stroke 1977;8:301–314.CrossRef

41.

The Coronary Drug Project Research Group. The Coronary Drug Project: design, methods, and baseline results. Circulation 1973;47:I-1-I-50.

42.

The Coronary Drug Project Research Group. Clofibrate and niacin in coronary heart disease. JAMA 1975;231:360–381.CrossRef

43.

Sackett DL. A compliance practicum for the busy practitioner. In Haynes RB, Taylor DW, Sackett DL (eds.). Compliance in Health Care. Baltimore: Johns Hopkins University Press, 1979.

44.

The PEACE Trial Investigators. Angiotensin-converting-enzyme inhibition in stable coronary artery disease. N Engl J Med 2004;351:2058–2068.

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_11

11. Data Collection and Quality Control

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

Valid and informative results from clinical trials depend on data that are of high enough quality and sufficiently robust to address the question posed. Such data in clinical trials are collected from several sources—medical records (electronic and paper), interviews, questionnaires, participant examinations, laboratory determinations, or public sources like national death registries. Data elements vary in their importance, but having valid data regarding key descriptors of the population, the intervention, and primary outcome measures is essential to the success of a trial. Equally important, and sometimes a trade-off given limited resources, is having a large enough sample size and number of outcome events to obtain a sufficiently narrow estimate of the intervention effect. Modest amounts of random errors in data will not usually affect the interpretability of the results, as long as there are sufficient numbers of outcome events. However, systematic errors can invalidate a trial’s results.

Avoiding problems in the data collection represents a challenge. There are many reasons for poor quality data and avoiding them completely is difficult, so the goal is to limit their amount and, thus, their impact on the trial findings. Many steps can be taken during the planning phase to optimize collection of high quality data. The problems encompass missing data, erroneous (including falsified and fabricated) data, large variability, and long delays in data submission. Even with the best planning, data quality needs to be monitored throughout the trial and corrective actions taken to deal with unacceptable problems. This chapter addresses the problems in data collection, how to minimize collection of poor quality data, and the need for quality monitoring, which includes audits.

Concerted efforts to improve data quality in clinical trials, and to focus on important aspects of quality in large trials, have increased markedly. The International Conference on Harmonisation Good Clinical Practice (ICH-GCP [E6]) guidelines, crafted in the 1990s by a selected group of regulators and industry representatives, defined international ethical and scientific standards for clinical trials [1]. The guidelines cover all the phases of clinical trials from design and conduct to recording and reporting. However, these guidelines are focused on earlier phase pharmaceutical trials as they are overly complex for many large outcome trials [2, 3]. The roadmap of responsibilities in the ICH-GCP E6 guidance document was most recently revised in 2007 [4], and another revision is in process. Other organizations have issued their own versions of quality assurance guidelines. In 1998, the Society for Clinical Trials issued guidelines for multicenter trials [5]. The oncology community has guidelines issued by the American Society of Clinical Oncology [6] and special standards for pediatric oncology [7]. Others have addressed the specific needs of large trials, including assuring quality without undue regulatory burden. Reports have been published from the 2007 [8] and 2012 [3] Conferences on Sensible Guidelines. A summary of a 2013 meeting of the Clinical Trials Transformation Initiative (CTTI), a public-private partnership founded by the U.S. Food and Drug Administration (FDA) and Duke University, addressed specific issues related to large, simple trials [9]. An article by Acosta et al. [10] discussed the implementation of GCP guidelines in developing countries. The texts by McFadden [11] and Meinert [12] contain detailed descriptions of data collection. Finally, guidance concerning the use of electronic source data in clinical trials has been published by the FDA in 2013 [13] and European Medicines Agency in 2010 [14].

Fundamental Point

During all phases of a study, sufficient effort should be spent to ensure that all data critical to the interpretation of the trial, i.e., those relevant to the main questions posed in the protocol, are of high quality.

The definition of key data depends on trial type and objectives. Baseline characteristics of the enrolled participants, particularly those related to major eligibility measures are clearly key as are primary and important secondary outcome measures, and adverse effects. The effort expended on assuring minimal error for key data is considerable. It is essential that conclusions from the trial be based on accurate and valid data. But fastidious attention to all data is not possible, and in fact can be counterproductive. One approach is to decide in advance the degree of error one is willing to tolerate for each type of data. The key data, as well as certain process information such as informed consent, should be as close to error free as possible. A greater error rate should be tolerated for other data. The confirmation, duplicate testing, and auditing that is done on data of secondary importance should not be as extensive. Perhaps only a sampling of audits is necessary.

In addition to collecting the right data, the method used to collect the data is critical. For some variables, it will be simple collection of numeric information. For other data, the quality depends on carefully constructed questions to assure accurate capture. A well-designed case report form that clearly guides investigators to enter accurate and complete information is critical to the success of the trial.

The data collected should focus on the answers to the questions posed in the protocol. Essential data vary by the type of trial, and they include:

baseline information, such as inclusion and exclusion criteria that define the population;

measures of adherence to the study intervention;

important concomitant interventions;

primary response variable(s);

important secondary response variables;

adverse effects with emphasis on predefined serious events;

other prespecified response variables.

Data are collected to answer questions about benefits, risks, and ability to adhere to the intervention being tested. Trials must collect data on baseline covariates or risk factors for at least three purposes: (1) to verify eligibility and describe the population studied; (2) to verify that randomization did balance the important known risk factors; and (3) to allow for limited subgroup analyses. Obviously, data must be collected on the primary and secondary response variables specified in the protocol and in some cases tertiary level variables. Some measures of adherence to the interventions specified in the protocol are necessary as well as important concomitant medications used during the trial. That is, to validly test the intervention, the trial must describe how much of the intervention the participant was exposed to and what other interventions were used. Collection of adverse events is challenging for many reasons (see Chap. 12).

Each data element considered should be examined as to its importance in answering the questions. Trialists cannot include every outcome that might be “nice to know.” Each data element requires collection, processing, and quality control, as discussed below, and adds to the cost and overall burden of the trial. We think that far too much data are generally collected [15]. Only a small portion is actually used in trial monitoring and publications. Excessive data collection is not only costly but can indirectly affect the quality of the more critical data elements.

Problems in Data Collection

Major Types

There are four major types of data problems discussed here: (1) missing data, (2) incorrect data, (3) excess variability, and (4) delayed submission.

First, incomplete and irretrievably missing data can arise, for example, from the inability of participants to provide necessary information, from inadequate assessment like physical examinations, from laboratory mishaps, from carelessness in completion of data entry, or from inadequate quality control within electronic data management systems. Missing outcome data, for example due to withdrawal of participant consent or loss to follow-up, can result in unreliable results. When the results of the Anti-Xa Therapy to Lower Cardiovascular Events in Addition to Standard Therapy in Subjects with Acute Coronary Syndrome (ATLAS-ACS 2) trial testing rivaroxaban following acute coronary syndromes were reviewed by an FDA Advisory Committee, drug approval was not recommended in large part due to over 10% of the participants having incomplete follow-up [16]. The percent of missing critical data in a study is considered as one indicator of the quality of the data and, therefore, the quality of the trial.

Second, erroneous data may not be recognized and, therefore, can be even more troublesome than incomplete data. For study purposes, a specified condition may be defined in a particular manner. A clinic staff member may unwittingly use a clinically acceptable definition, but one that is different from the study definition. Specimens may be mislabeled. In one clinical trial, the investigators appropriately suspected mislabeling errors when, in a glucose tolerance test, the fasting levels were higher than the 1-h levels in some participants. Badly calibrated equipment can be a source of error. In addition, the incorrect data may be entered on a form. A blood pressure of 84/142 mmHg, rather than 142/84 mmHg, is easy to identify as wrong. However, while 124/84 mmHg may be incorrect, it is a perfectly reasonable measurement, and the error would not necessarily be recognized. The use of electronic data capture allows automatic checks for data being “out of range” or inconsistent with other data in the participant’s record (like diastolic higher than systolic blood pressure). An immediate query can lead to correction right away. The most troublesome types of erroneous data are those that are falsified or entirely fabricated. The pressure to recruit participants may result in alterations of laboratory values, blood pressure measurements, and critical dates in order to qualify otherwise ineligible participants for enrollment [17, 18].

The third problem is variability in the observed characteristics. Variability reduces the opportunity to detect any real changes. The variability between repeated assessments can be unsystematic (or random), systematic, or a combination of both. Variability can be intrinsic to the characteristic being measured, the instrument used for the measurement, or the observer responsible for obtaining the data. People can show substantial day-to-day variations in a variety of physiologic measures. Learning effects associated with many performance tests also contribute to variability. The problem of variability, recognized many decades ago, is not unique to any specific field of investigation [19, 20]. Reports of studies of repeat chemical determinations, determinations of blood pressure, physical examinations, and interpretations of X-rays, electrocardiograms and histological slides indicate the difficulty in obtaining highly reproducible data. People perform tasks differently and may vary in knowledge and experience. These factors can lead to interobserver variability. In addition, inconsistent behavior of the same observer between repeated measurements may also be much greater than expected, though intraobserver inconsistency is generally less than interobserver variability.

Reports from studies of laboratory determinations illustrate that the problem of variability has persisted for almost seven decades. In 1947, Belk and Sunderman [21] reviewed the performance of 59 hospital laboratories on several common chemical determinations. Using prepared samples, they found that unsatisfactory results outnumbered the satisfactory. Regular evaluation of method performance, often referred to as proficiency testing, is now routinely conducted and required by laboratories in many countries [22, 23]. All laboratories performing measurements for clinical trials should be certified by the Clinical Laboratory Improvement Amendments (CLIA) or a similar agency [24].

Diagnostic procedures that rely on subjective interpretations are not surprisingly more susceptible to variability. One example is radiologists’ interpretation of screening mammograms [25]. Nine radiologists read cases with verified cancers, benign, and negative findings in the clinic. Approximately 92% of the mammograms of verified cases were, on average, read as positive. The interradiologist variability was modest. The reading of the negative mammograms showed a substantial interreader variability. In a trial of acute ST segment elevation myocardial infarction, over one-quarter of participants enrolled (and for whom the investigator indicated criteria were met) did not meet inclusion criteria when the electrocardiograms were interpreted by a core laboratory [26]. Since electrocardiogram interpretation in an emergency clinical setting may be less rigorous than in a core laboratory, some degree of disagreement is not surprising.

In another study, the intra- and interreader variability in QT interval measurement on electrocardiograms was estimated by 2 different methods [27]. Eight readers analyzed the same set of 100 electrocardiograms twice 4 weeks apart. Five consecutive complexes were measured. For the more commonly used threshold method, the intrareader standard deviation was 7.5 ms and the interreader standard deviation 11.9 ms. Due to the association between QT prolongation and malignant arrhythmias, the FDA is concerned about drugs that prolong the QT interval by a mean of about 5 ms. Thus, the usual variability in measurement is greater than what is considered a clinically important difference.

Another type of variability is the use of nonstandardized terms. As a result, the ability to exchange, share, analyze, and integrate clinical trial data is limited by this lack of coordination in terms of semantics. Increased attention has been devoted to so-called harmonized semantics [28, 29]. A “universal definition” of myocardial infarction is an attempt to standardize definitions of this event, including harmonizing definitions in clinical trials [30]. In response to the confusion and inconsistency resulting from more than 10 definitions of bleeding used in trials of antithrombotic therapy for coronary intervention, a group of academic leaders and FDA representatives developed a standardized classification of bleeding that has been widely adopted in such trials [31]. Professional societies are becoming engaged in proposing clinical data standards, in large part to establish standard definitions of clinical conditions and outcomes for clinical research [32].

The fourth problem, delayed submission of participant data from the clinical site in multicenter trials, used to be a major issue. However, it has decreased markedly with the onset of electronic data entry (see below).

Minimizing Poor Quality Data

General approaches for minimizing potential problems in data collection are summarized below. Most of these should be considered during the planning phase of the trial. Examples in the cardiovascular field are provided by Luepker et al. [33]. In this section, we discuss design of the protocol and manual, development of data entry tools, training and certification, pretesting, techniques to reduce variability, and data entry.

Design of Protocol and Manual

The same question can often be interpreted in many ways. Clear definitions of entry and diagnostic criteria and methodology are therefore essential. These should be included in the protocol and written so that all investigators and staff can apply them in a consistent manner throughout the trial. Accessibility of these definitions is also important. Even the same investigator may forget how he previously interpreted a question unless he can readily refer to instructions and definitions. A manual of procedures, or the equivalent using an electronic format, should be prepared in every clinical trial. Although it may contain information about study background, design, and organization, the manual is not simply an expanded protocol. In addition to listing eligibility criteria and response variable definitions, it should indicate how the criteria and variables are determined. The manual provides detailed answers to all conceivable “how to” questions, and answers to questions that arise during the trial so they can be documented, shared, and harmonized. Most importantly, the manual should describe the participant visits—their scheduling and content—in detail. Instructions for filling out forms; performing tasks such as laboratory determinations; drug ordering, storing and dispensing; and adherence monitoring must be clear and complete. Finally, recruitment techniques, informed consent, participant safety, emergency unblinding, use of concomitant therapy, and other issues need to be addressed. Updates and clarifications usually occur during the course of a study. These revisions should be made available to every staff person involved in data collection.

Descriptions of laboratory methods or imaging techniques and the ways the results are to be reported also need to be stated in advance. In one study, plasma levels of the drug propranolol were determined by using standardized methods. Only after the study ended was it discovered that two laboratories routinely were measuring free propranolol and two other laboratories were measuring propranolol hydrochloride. A conversion factor allowed investigators to make simple adjustments and arrive at legitimate comparisons. Such adjustments are not always possible.

Development of Forms and Data Entry Tools

Ideally, the study forms, which are increasingly electronic or web-based, should contain all necessary information [12]. If that is not possible, the forms or electronic data entry tools should outline the key information and refer the investigator to the appropriate detailed information. Well-designed tools will minimize errors and variability. Data entry questions and fields should be as clear and well organized as possible, with a logical sequence to the questions. Entry tools should be designed to minimize missing data, for example with inability to proceed until something is entered. To know whether or not a condition is present, one should ask for the answer as “yes” or “no,” rather than as a single checkbox if present. This is because the lack of a check mark could mean the condition is not present, it is unknown if it is present, or the question was simply skipped. If it may not be known, then include an “unknown” choice. Questions should be clear, with few “write-in” answers since unstructured text fields will rarely provide helpful information in the typical clinical trial. As little as possible should be left to the imagination of the person completing the form. The questions should elicit the necessary information and little else. Questions that are included because the answers would be “nice to know” are rarely analyzed and may distract attention from the pertinent questions. In several studies where death is the primary response variable, investigators may have an interest in learning about the circumstances surrounding the death. In particular, the occurrence of symptoms before death, the time lapse from the occurrence of such symptoms until death, and the activity and location of the participant at the time of death have been considered important and may help in classifying the cause of death. While this may be true, focusing on these details has led to the creation of extraordinarily complex forms which take considerable time to complete. Moreover, questions arise concerning the accuracy of the information, because much of it is obtained from proxy sources who may not have been with the participants when they died. Unless investigators clearly understand how these data will be used, simpler forms are preferable.

A comprehensive review of the multitude of issues in the design of study forms is presented by Cook and DeMets [34]. They describe the categories of data typically collected in randomized clinical trials: participant identification and treatment assignment; screening and baseline information; follow-up visits, tests, and procedures; adherence to study treatment; adverse experiences; concomitant medication and interventions; clinical outcomes and participant treatment; and follow-up and survival status. Also discussed are mechanisms for data collection and design and review of case report forms.

Training and Certification

There are two types of training for research staff: generic training covering research in general, and training specific to an individual trial. General training includes topics of regulatory requirements, ethics, and basic principles of research and randomized clinical trials, and this is particularly important for junior investigators and study coordinators (see Chap. 2 for discussion of ethics training). For an individual trial, the training is focused on assuring understanding of the protocol and the ability to faithfully execute it.

It has long been recognized that training sessions for investigators and staff to promote standardization of procedures are crucial to the success of any large study. Whenever more than one person is performing data entry or examining participants, training sessions help to minimize errors. There may be more than one correct way of doing something in clinical practice, but for study purposes, there is only one way. Similarly, the questions on a form should always be asked in the same way. The answer to, “Have you had any stomach pain in the last 3 months?” may be different from the answer to, “You haven’t had any stomach pain in the last 3 months, have you?” Even differences in tone or the emphasis placed on various parts of a question can alter or affect the response. Kahn et al. [35] reviewed the favorable impact of training procedures instituted in the Framingham Eye Study. The 2 days of formal training included duplicate examinations, discussions about differences, and the use of a reference set of fundus photographs. Neaton et al. [36] concluded that initial training is useful and should cover the areas of clinic operation, technical measurements, and delivery of intervention. Centralized interim training of new staff is less efficient and can be substituted by regional training, teleconferencing, or web-based approaches.

Mechanisms to verify that clinic staff perform trial procedures and tests the same way, when that may affect trial validity, should be developed. For certain tests, the most reliable interpretation will be using a core laboratory, but even then, standard acquisition of the information at the site must be assured. Mechanisms may include instituting certification procedures for specified types of data collection. If blood pressure is an important outcome in a trial, then there should be standardized procedures for measurement since the approach may have a major impact on the measurement [37]. For certain tests, the people performing these determinations should not only be trained, but also be tested and certified as competent. Periodic retraining and certification are especially useful in long-term studies since people tend to forget, and personnel turnover is common. For situations in which staff must conduct clinical interviews, special training procedures to standardize the approach have been used. In a study of B-mode ultrasonography of the carotid arteries, marked differences in intimal-medial thickness measurements were found between the 13 readers at the reading center [38]. During the 5-year study, the relative biases of readers over time varied, in some cases changing from low to high and vice versa. A sharp increase in average intimal-medial thickness measurements observed toward the end of the study was explained by readers reading relatively high having an increased workload, the hire of a new reader also reading high, and a reader changing from reading low to high.

Pretesting

Pretesting of data entry and procedures is almost always helpful, particularly for variables and formats that have not been used before. Several people similar to the intended participants may participate in simulated interviews and examinations to make sure procedures are properly performed and questions on the forms or screens flow well and provide the desired information. Furthermore, by pretesting, the investigator and staff grow familiar and comfortable with the data entry process. Fictional case histories can be used to check data entry design and the care with which forms are completed. When developing data entry screens, most investigators cannot even begin to imagine the numerous ways questions can be misinterpreted until several people have been given the same information and asked to complete the same form. One explanation for different answers may be due to carelessness on the part of the person completing the data entry. The use of “de-briefing” in the pilot test may bring to light misinterpretations that would not be detected when real participants enter the data. Inadequacies in data entry structure and logic can also be uncovered by use of pretesting. Thus, pretesting reveals areas in which forms might be improved and where additional training might be worthwhile. It also allows one to estimate the time needed to complete data entry, which may be useful for planning, staffing, and budgeting.

De-briefing is an essential part of the training process. This helps people completing data entry understand how the forms are meant to be completed and what interpretations are wanted. Discussion also alerts them to carelessness. When done before the start of the study, this sort of discussion allows the investigator to modify inadequate items on the data entry screens. These case history exercises might be profitably repeated several times during the course of a long-term study to indicate when education and retraining are needed. Ideally, data entry screens should not be changed after the study has started. Inevitably, though, modifications are made, and the earlier the better. Pretesting can help to minimize the need for such modifications.

Techniques to Reduce Variability Including Central Adjudication of Events

Both variability and bias in the assessment of response variables should be minimized through repeat assessment, blinded assessment, or (ideally) both. At the time of the examination of a participant, for example, an investigator may determine blood pressure two or more times and record the average. Performing the measurement without knowing the group assignment helps to minimize bias. A trial of the evaluation of the effect of renal artery denervation on blood pressure illustrates this point. Open-label and less rigorously conducted trials showed a 20 mmHg reduction in systolic blood pressure with the denervation procedure, whereas a larger and more rigorous sham controlled trial found only a 2 mmHg non-significant effect of the procedure compared with control [39]. In unblinded or single-blinded studies, the examination might be performed by someone other than the investigator, who is blinded to the assignment. In assessing slides, X-rays, images or electrocardiograms, two individuals can make independent, blinded evaluations, and the results can be averaged or adjudicated in cases of disagreement. Independent evaluations are particularly important when the assessment requires an element of judgment and when there is subjectivity in the assessment.

Centralized classification of major health outcomes by blinded reviewers is common in large clinical trials. There are three related objectives: to improve accuracy of event rates and of intervention effect, to reduce bias related to knowledge of intervention assignment (especially in open-label trials), and to improve credibility of results. The common focus of central adjudication is on removing events that are not true events that may create background noise and thus improve the estimate of the true intervention effect. However, it is also possible to identify outcome events that are otherwise missed with centralized clinical events review. For example, myocardial infarction may be difficult to detect around the time of coronary procedures. A systematic central screening for elevated blood cardiac biomarkers substantially increased the number of outcome events detected in the Second Platelet IIb/IIIa Antagonist for the Reduction of Acute Coronary Syndrome Events in a Global Organization Network (PARAGON-B) trial [40].

In an open-label trial, blinding of treatment assignment to adjudicators, as in the case of PROBE (Prospective, Randomized, Open-label, Blinded-Endpoint) design, may reduce bias. It will not eliminate bias, however, since complete blinding is difficult [41] and ascertainment of possible events by the investigator may differ with knowledge of the treatment assignment. It is important to consider aspects other than participant or investigator bias that could impact on event rates in an open-label trial. For example, it is possible that the monthly visits to test for level of anticoagulation (only conducted in the warfarin arm) of the Randomized Evaluation of Long-Term Anticoagulation Therapy (RE-LY) trial [42] could have led to the identification of more events in the warfarin arm.

The central adjudication process may also reduce the variability induced by having a large number of local investigators classifying certain types of events. A critical factor is how well the diagnostic criteria in a trial are specified and communicated to local investigators responsible for the initial classification. Reviews [43, 44] of cardiovascular trials have shown that the event rates and the effects of interventions are only modestly changed when using adjudicated (versus investigator defined) outcomes. And while one might expect the adjudicated results to more clearly demonstrate a treatment benefit of an effective therapy, this was not the case in five of six trials reviewed [43]. It is unclear whether these observations also apply to other disease areas. The FDA encourages the use of standard definitions and of centralized review and classification of critical outcomes [43].

Data Entry

The shift in medical information to an electronic format, both in clinical medicine and in clinical research, has markedly improved data quality and timeliness of data management in clinical trials. Systems are routinely used for data entry as well as for validation of forms and data, document management, and queries and their resolution [45, 46]. Litchfeld et al. [47] compared the efficiency and ease of use of internet data capture with the conventional paper-based data collection system. They reported substantial reductions with the internet-driven approach in terms of time from visit to data entry, time to database release after the last participant visit, and time from a visit to a query being resolved. Seventy-one percent of the sites preferred the web-based approach. Different web-based systems have been developed. Examples include the Validation Studies Information Management System (VSIMS) [48], a system developed for the Childhood Asthma Research (CARE) Network [49], and the Query and Notification System [50].

A variety of approaches are possible to capture and transfer data into the electronic system. The worst case is to first write the data onto paper forms and then transcribe these to the electronic system, since this increases opportunity for transcription error and saves little time. Directly entering the data onto the electronic case report form, or better yet having the data flow directly from the electronic health record, is the goal.

Electronic Source Data

There is a growing opportunity to directly transfer electronic health information into clinical trial databases. Defining when and how this can be done to support pragmatic (and other) clinical trials is a focus of the National Institutes of Health (NIH) Health Systems Collaboratory [51]. Important work is being done to define when and how clinical outcomes can be accurately identified by electronic health records. For example, the FDA’s Mini-Sentinel project has developed and validated programs to identify hospitalization for acute myocardial infarction using an algorithm based on the International Classification of Diseases [52]. The FDA has provided guidance for use of electronic source data, emphasizing the same principles that have existed for any source data. This includes that it be “attributable, legible, contemporaneous, original, and accurate (ALCOA)” and also that it meet the regulatory requirements for recordkeeping [13]. Development of integrated electronic systems to direct and manage data flow from various sources is essential for larger trials [45] (Fig. 11.1). These systems can also be leveraged for more efficient direct collection of patient-reported outcomes.

Fig. 11.1

Source dataflow. From Society for Clinical Data Management. eSource Implementation in Clinical Research: A Data Management Perspective: A White Paper [45]

Quality Monitoring

Even though every effort is made to obtain high quality data, a formal monitoring or surveillance system is crucial. When errors are found, this system enables the investigator to take corrective action. Monitoring is most effective when it is current so that when deficiencies are identified, measures can be instituted to fix the problem as early as possible. Additionally, monitoring allows an assessment of data quality when interpreting study results. Numerous procedures, including drug handling and the process of informed consent, can and should be monitored, but monitoring all procedures and study variables will divert resources from more important uses to improve trial quality. Modest amounts of (sometimes unavoidable) random error can be overcome by assuring that there is a robust sample size and number of outcome events. Minimizing missing data, particularly of the primary outcome and major safety outcomes, is crucially important. Monitoring those areas most important to the trial is recommended. This can be done by data quality and consistency checks in the electronic database as well as by on-site review of trial procedures and data.

Monitoring of data quality proves most valuable when there is feedback to the clinic staff and technicians. Once weaknesses and errors have been identified, this should prompt action to improve the trial through additional training and/or through improving data collection of the problematic variables. Chapter 20 contains several tables illustrating quality control reports. With careful planning, reports can be provided and improvement can be accomplished without unblinding the staff. Investigators need to focus their efforts on those procedures that yield key data, i.e., those on which the conclusions of the study critically depend.

For clinical trials that will form the basis for regulatory decisions, the volume of data is typically very high and the data monitoring very elaborate. Sophisticated clinical trial management systems are used that can integrate electronic data capture, data tracking and management, and other aspects of trial conduct like site performance, pharmacy tracking, adverse event reporting, and payments.

Eisenstein et al. [53, 54] examined ways of reducing the cost of large, phase III trials. The major contributors to the expense are the number of case report form pages, the number of monitoring visits (for comparison of data in source records to the data on trial forms), and the administrative workload. Verification of critical information is important. Limiting the data verification of noncritical data may increase the error rate, but this may have no impact on the overall trial quality as these data are not important to the main findings. There may even be negative impact since limited resources should be focused on where they will make a difference (as outlined in Table 11.1), as opposed to verifying noncritical data. Electronic data entry allows data checks and quality assurance at the time of initial data entry. This can reduce the cost related to traditional “queries” to resolve discrepancies that can be very costly with estimates of more than $100 each. In sensitivity analyses, the authors have shown that the total trial cost could be cut by more than 40% by reducing excessive data collection and verification. Regular site visits to confirm that all case report forms are consistent with patient records is usually excessive. As discussed below, sampling or selective site monitoring would be more appropriate in most situations. Programs and initiatives like the CTTI [55], as well as FDA guidance on risk-based monitoring [56] and on adverse event reporting [57] are addressing these issues.

Table 11.1

Key elements of high quality, randomized clinical outcome trials

Relevant question being addressed

Protocol

–Clear, practical, focused

–Written to avoid “deviations” that are not relevant to “quality”

Adequate number of events to answer question with confidence

Conducted in a general practice setting to make results generalizable

Proper randomization

Reasonable assurance that participants receive (and stay on) assigned intervention

Reasonably complete follow-up and accurate ascertainment of primary outcome (and other key outcomes like death)

Plan for ongoing measurement, feedback, and improvement of quality measures during trial conduct

Safeguards against bias in determining clinically relevant outcomes

Protection of rights of research participants

Monitoring of Data

During the study, data entered into the system should be centrally checked electronically for completeness, internal consistency, and consistency with other data fields. There should be a system to assure that important source data matches what is in the database, although this can be focused on certain variables and can be supported by selected and focused source-data verification [56]. When the data fields disagree, the group responsible for ensuring consistent and accurate data should assure that a system is in place to correct the discrepancy. Dates and times are particularly prone to error, and systems to minimize missing data are important. Electronic source data systems, especially if they can directly transfer clinical data to an electronic database, can reduce certain types of errors [13].

It may be important to examine consistency of data over time. A participant with a missing leg on one examination was reported to have palpable pedal pulses on a subsequent examination. Cataracts which did not allow for a valid eye examination at one visit were not present at the next visit, without an interval surgery having been performed. The data forms may indicate extreme changes in body weight from one visit to the next. In such cases, changing the data after the fact is likely to be inappropriate because the correct weight may be unknown. The observed differences in measurements may be less dramatic and not obvious. A quality control program based on randomly selected duplicate assessments has been advocated by Lachin [58]. However, the investigator can take corrective action for future visits by more carefully training staff. Sometimes, mistakes can be corrected. In one trial, comparison of successive electrocardiographic readings disclosed gross discrepancies in the coding of abnormalities. The investigator discovered that one of the technicians responsible for coding the electrocardiograms was fabricating his readings. In this instance, correcting the data was possible.

A system should be in place to constantly monitor data completeness and currency to find evidence of missing participant visits or visits that are off schedule, in order to correct any problems. Frequency of missing or late visits may be associated with the intervention. Differences between groups in missed visits may bias the study results. To improve data quality, it may be necessary to observe actual clinic procedures.

Monitoring of Procedures

Extreme laboratory values should be checked. Values incompatible with life such as potassium of 10 mEq/L are obviously incorrect. Other less extreme values (i.e., total cholesterol of 125 mg/dL in male adults in the United States who are not taking lipid-lowering agents) should be questioned. They may be correct, but it is unlikely. Finally, values should be compared with previous ones from the same participant. Certain levels of variability are expected, but when these levels are exceeded, the value should be flagged as a potential outlier. For example, unless the study involves administering a lipid-lowering therapy, any determination which shows a change in serum cholesterol of 20% or more from one visit to the next should be repeated. Repetition would require saving samples of serum until the analysis has been checked. In addition to checking results, a helpful procedure is to monitor submission of laboratory specimens to ensure that missing data are kept to a minimum.

Investigators doing special procedures (laboratory work, electrocardiogram reading) need to have an internal quality control system. Such a system should include re-analysis of duplicate specimens or materials at different times in a blinded fashion. A system of resubmitting specimens from outside the laboratory or reading center might also be instituted. These specimens need to be indistinguishable from actual study specimens. An external laboratory quality control program established in the planning phase of a trial can detect errors at many stages (specimen collection, preparation, transportation, and reporting of results), not just at the analysis stage. Thus, it provides an overall estimate of quality. Unfortunately, the system most often cannot indicate at which step in the process errors may have occurred.

Recording equipment specific to a trial should be checked periodically. Even though initially calibrated, machines can break down or require adjustment. Scales can be checked by means of standard weights. Factors such as linearity, frequency response, paper speed, and time constant should be checked on electrocardiographic machines. In one long-term trial, the prevalence of specific electrocardiographic abnormalities was monitored. The sudden appearance of a threefold increase in one abnormality, without any obvious medical cause, led the investigator correctly to suspect electrocardiographic machine malfunction.

Monitoring of Drug Handling

In a drug study, the quality of the drug preparations should be monitored throughout the trial. This includes periodically examining containers for possible mislabeling and for proper contents (both quality and quantity). It has been reported that in one trial, “half of the study group received the wrong medication” due to errors at the pharmacy. In another trial, there were concerns about asymmetric mis-allocation of control and experimental drug that turned out to be primarily due to transcription error [59]. Investigators should carefully look for discoloration and breaking or crumbling of capsules or tablets. When the agents are being prepared in several batches, samples from each batch should be examined and analyzed. The actual bottle content of pills should not vary by more than 1% or 2%. The number of pills in a bottle is important to know if pill count will be used to measure participant adherence.

Another aspect to consider is the storage shelf life of the preparations and whether they deteriorate over time. Even if they retain their potency, do changes in odor (as with aspirin) or color occur? If shelf life is long, preparing all agents at one time will minimize variability. Products having a short shelf life require frequent production of small batches. Records should be maintained for study drugs prepared, examined, and used. Ideally, a sample from each batch should be saved. After the study is over, questions about drug identity or purity may arise and samples will be useful.

The dispensing of medication should also be monitored. Checking has two aspects. First, were the proper drugs sent from the pharmacy or pharmaceutical company to the clinic? If the study is double-blind, the clinic staff will be unable to check this. They must assume that the medication has been properly coded. However, in unblinded studies, staff should check to assure that the proper drugs and dosage strengths have been received. In one case, the wrong strength of potassium chloride was sent to the clinic. The clinic personnel failed to notice the error. An alert participant to whom the drug was issued brought the mistake to the attention of the investigator. Had the participant been less alert, serious consequences could have arisen. An investigator has the obligation to be as careful about dispensing drugs as a licensed pharmacist. Close reading of labels is essential, and bar coding can be helpful, as well as documentation of all drugs that are distributed to participants.

Second, when the study is blinded, the clinic personnel need to be absolutely sure that the code number on the container is the proper one. Labels and drugs should be identical except for the code; therefore, extra care is essential. If bottles of coded medication are lined up on a shelf, it is relatively easy to pick up the wrong bottle accidentally. Unless the participant notices the different code, such errors may not be recognized. Even if the participant is observant, he may assume that he was meant to receive a different code number. The clinic staff should be asked to note on a study form the code number of the bottle dispensed and the code number of bottles that are returned by the participants. Theoretically, that should enable investigators to spot errors. In the end, however, investigators must rely on the care and diligence of the staff person dispensing the drugs.

The drug manufacturer assigns lot, or batch, numbers to each batch of drugs prepared. If contamination or problems in preparation are detected, then only those drugs from the problem batch need to be recalled. The use of batch numbers is especially important in clinical trials, since the recall of all drugs can severely delay, or even ruin, the study. When only some drugs are recalled, the study can usually manage to continue. Therefore, the lot number of the drug as well as the name or code number should be listed in the participant’s study record.

Audits

There are three general types of audits: routine audits of a random sample of records, structured audits, and audits for cause. Site visits are commonly conducted in long-term multicenter trials. In many non–industry-sponsored trials, a 5–10% random sample of study forms may be audited for the purpose of verifying accurate transfer of data from hospital source records. This becomes less important if electronic source data can be directly transferred to the database. More complete audits are usually performed in industry-sponsored trials, although there is a movement towards “risk-based monitoring” to focus on critical variables and to customize the intensity of monitoring to the likely benefit of such monitoring [56]. While the traditional model has been for study monitors (or clinical research associates) to visit the sites in order to verify that the entered data are correct, a more appropriate role may be to perform selected source-data verification for critical variables and to spend more time assuring that appropriate systems are in place and training has been performed.

Some investigators have objections to random external data audits, especially in the absence of evidence of scientific misconduct. However, the magnitude of the problems detected when audits occur makes it difficult to take a position against them. Of interest, the FDA has not performed audits of trials sponsored by the National Cancer Institute (NCI), according to a long-standing agreement. It relies on a NCI-sponsored audit program that has been in place since 1982, now known as the Clinical Trials Monitoring Branch of the Cancer Therapy Evaluation Program [60]. A review of 4 cycles of internal audits conducted over an 11-year period by the investigators of the Cancer and Leukemia Group B (CLGB) showed similarities with FDA audits [61]. The deficiency rate (among main institutions) of 28% in the first cycle dropped to 13% in the fourth cycle. Only two cases of major scientific impropriety were uncovered during these on-site peer reviews. Compliance with institutional review board requirements improved over time, as did compliance with having properly signed consent forms. The consent form deficiencies dropped from 18.5% in the first cycle to 4% in the fourth. Although compliance with eligibility improved from 90 to 94%, no changes were noted for disagreement with auditors for treatment responses (5%) and deviations from the treatment protocol (11%). The authors concluded that the audit program had been successful in “pressuring group members to improve adherence to administrative requirements, protocol compliance, and data submission. It has also served to weed out poorly performing institutions.”

The NCI National Clinical Trials Network program now has replaced the Cooperative Group Program for overseeing clinical trial activity including quality assurance [60]. Another cooperative group, the National Surgical Adjuvant Breast and Bowel Project, conducted a review of almost 6,000 participant records [62]. The objective was to confirm participant eligibility, disease, and vital status. No additional treatment failures or deaths and only seven cases of ineligible participants were found. The audit was time-consuming and costly and since few discrepancies were found, the authors concluded that routine use of cooperative chart reviews cannot be supported. A similar conclusion was reached in the Global Utilization of Streptokinase and TPA for Occluded Coronary Arteries (GUSTO) trial [63]. Following an audit of all case report forms, the auditors reported only a small percentage of errors and determined that these errors did not change the trial conclusions.

The third type of audit is for cause, i.e., to respond to allegations of possible scientific misconduct. This could be expanded to include any unusual performance pattern such as enrolling participants well in excess of the number contracted for or anticipated. The Office of Research Integrity in the U.S. Department of Health and Human Services promotes integrity in biomedical and behavioral research sponsored by the U.S. Public Health Service at over 7,000 institutions worldwide. It monitors institutional investigations of research misconduct which includes fabrication, falsification or plagiarism in proposing, performing, or reviewing research or in reporting research findings. In a review of 136 investigations resulting in scientific misconduct between 1992 and 2002, only 17 involved clinical trials. The most severe penalty, debarment from U.S. government funding, was applied in six of the cases. Junior employees were often cited and the applied sanction was often a requirement to implement a supervision plan [64, 65].

The FDA conducts periodic audits as well as investigations into allegations of violations of the Federal Food, Drug, and Cosmetic Act through its Office of Criminal Investigations. These may include clinical investigator fraud such as falsifying documentation and enrolling ineligible patients. There were 4,059 FDA inspections in 2013. Most did not justify regulatory action and any corrective action was left to the investigator, with a total of 79 having official action indicated [66].

The quality of any trial is dependent on the quality of its data. Experience has shown that too much data are being collected, much of which are never used for publication or review. As emphasized above, the data collection should be closely linked to the trial objectives and the questions posed in the protocol. The case report form must be carefully constructed to accurately and completely collect the necessary data. Over-collection of data adds to the cost and effort of conducting the trial. Overemphasis on detailed audits of case report forms has similar effects. Moreover, the error rates are often so low that the value of most audits has been questioned, particularly when the errors are “random” in nature. Rather, we should focus our quality control and auditing efforts on key variables. For other variables, samples should be audited with more reliance on statistical quality control procedures. Data collection in clinical trials should be streamlined whenever possible.

References

1.

ICH E6. Good clinical practice: Consolidated guideline, Step 5 as of May 1996. Available at http://​www.​ich.​org/​products/​guidelines/​efficacy/​efficacy-single/​article/​good-clinical-practice.​html. Accessed January 16, 2015.

2.

Yusuf S, Bosch J, Devereaux PJ, et al. Sensible guidelines for the conduct of large randomized trials. Clin Trials 2008;5:38-39.CrossRef

3.

Reith C, Landray M, Devereaux PJ, et al. Randomized clinical trials—removing unnecessary obstacles. N Engl J Med 2013;369:1061-1065.

4.

International Conference on Harmonisation. September 2007. Available at http://​www.​ich.​org. Accessed January 16, 2015.

5.

Knatterud GL, Rockhold FW, George SL, et al. Guidelines for quality assurance in multicenter trials: A position paper. Control Clin Trials 1998;19:477-493.CrossRef

6.

Zon R, Meropol NJ, Catalano RB, Schilsky RL. American Society of Clinical Oncology statement on minimum standards and exemplary attributes of clinical trial sites. J Clin Oncol 2008;26:2562-2567.CrossRef

7.

Devine S, Dagher RN, Weiss KD, Santana VM. Good clinical practice and the conduct of clinical studies in pediatric oncology. Pediatr Clin North Am 2008;55:187-208.CrossRef

8.

Baigent C, Harrell FE, Buyse M, et al. Ensuring trial validity by data quality assurance and diversification of monitoring methods. Clin Trials 2008;5:49-55.CrossRef

9.

Eapen ZJ, Lauer MS, Temple RJ. The imperative of overcoming barriers to the conduct of large, simple trials. JAMA 2014;311:1397-1398.CrossRef

10.

Acosta CJ, Galindo CM, Ochiai RL, et al. Implementation of good clinical practice guidelines in vaccine trials in developing countries. Vaccine 2007;25:2852-2857.CrossRef

11.

McFadden E. Management of Data in Clinical Trials. Hoboken, New Jersey: John Wiley & Sons, 2007.CrossRef

12.

Meinert CL. Clinical Trials: Design, Conduct, and Analysis. Second Edition. New York: Oxford University Press, 2012.

13.

FDA Guidance for Industry. Electronic Source Data in Clinical Investigations. Available at http://​www.​fda.​gov/​downloads/​Drugs/​GuidanceComplian​ceRegulatoryInfo​rmation/​Guidances/​UCM328691.​pdf. Accessed January 16, 2015.

14.

European Medicines Agency. Reflection paper on expectations for electronic source data and data transcribed to electronic data collection tools in clinical trials. Available at http://​www.​ema.​europa.​eu/​docs/​en_​GB/​document_​library/​Regulatory_​and_​procedural_​guideline/​2010/​08/​WC500095754.​pdf. Accessed January 16, 2015.

15.

O’Leary E, Seow H, Julian J, Levine M, Pond GR. Data collection in cancer clinical trials: too much of a good thing, Clin Trials 2013;10:624-632.CrossRef

16.

Briefing Information for the May 23, 2012 Meeting of the Cardiovascular and Renal Drugs Advisory Committee. Available at http://​www.​fda.​gov/​AdvisoryCommitte​es/​CommitteesMeetin​gMaterials/​Drugs/​Cardiovascularan​dRenalDrugsAdvis​oryCommittee/​ucm304754.​htm. Accessed January 16, 2015.

17.

Neaton JD, Bartsch GE, Broste SK, et al. A case of data alteration in the Multiple Risk Factor Intervention Trial (MRFIT). Control Clin Trials 1991;12:731-740.CrossRef

18.

Fisher B, Redmond CK. Fraud in breast-cancer trials. N Engl J Med 1994;330:1458-1462.CrossRef

19.

Koran LM. The reliability of clinical methods, data and judgments. Part 1. N Engl J Med. 1975;293:642-646.CrossRef

20.

Koran LM. The reliability of clinical methods, data and judgments. Part 2. N Engl J Med 1975;293:695-701.CrossRef

21.

Belk WP, Sunderman FW. A survey of the accuracy of chemical analyses in clinical laboratories. Am J Clin Pathol 1947;17:853-861.CrossRef

22.

Westgard JO. Basic Method Validation. Madison, Wisconsin: Westgard QC, Inc., 2003, pp. 102-103.

23.

McPherson RA, Pincus MR (eds.). Henry’s Clinical Diagnosis and Management by Laboratory Methods. (21st Edition), Philadelphia: Elsevier Saunders, Inc. 2007, pages 4-5.

24.

Clinical Laboratory Improvement Amendments (CLIA). Available at http://​www.​cms.​gov/​Regulations-and-Guidance/​Legislation/​CLIA/​index.​html. Accessed January 16, 2015.

25.

Gur D, Bandos AI, Cohen CS, et al. The “laboratory” effect: Comparing radiologists’ performance and variability during prospective clinical and laboratory mammography interpretations. Radiology 2008;249:47-53.CrossRef

26.

Tjandrawidjaja MC, Fu Y, Al-Khalidi H, et al. Failure of investigator adherence to electrocardiographic entry criteria is frequent and influences clinical outcomes: lessons from APEX-AMI. Eur Heart J 2007;28:2850-2857.CrossRef

27.

Panicker GK, Karnad DR, Natekar M, et al. Intra- and interreader variability in QT interval measurement by tangent and threshold methods in central electrocardiogram laboratory. J Electrocardiol 2009;42:348-352.CrossRef

28.

Fridsma DB, Evans J, Hastak S, Mead CN. The BRIDG project: A technical report. J Am Med Inform Assoc 2008;15:130-137.CrossRef

29.

Weng C, Gennari JH, Fridsma DB. User-centered semantic harmonization: A case study. J Biomed Inform 2007;40:353-364.CrossRef

30.

Thygesen K, Alpert JS, Jaffe AS et al. Third universal definition of myocardial infarction. Eur Heart J 2012;33:2551-2567.CrossRef

31.

Mehran R, Rao SV, Bhatt DL, et al. Standardized bleeding definitions for cardiovascular clinical trials: a consensus report from the Bleeding Academic Research Consortium. Circulation 2011;123:2736-2747.CrossRef

32.

Radford MJ, Heidenreich PA, Bailey SR, et al. American College of Cardiology; American Heart Association Task Force on Clinical Data Standards. ACC/AHA 2007 methodology for the development of clinical data standards: a report of the American College of Cardiology/American Heart Association Task Force on Clinical Data Standards. J Am Coll Cardiol 2007;49:830-837.

33.

Luepker RV, Evans A, McKeigue P, Reddy KS. Cardiovascular Survey Methods (Third Edition). Geneva, Switzerland, World Health Organization, 2004.

34.

Cook TD, DeMets DL. Data collection and quality control. In Cook TD, DeMets DL (eds.). Introduction to Statistical Methods for Clinical Trials. Boca Raton, Florida: Chapman & Hall/CRC, 2007, pp 171-200.

35.

Kahn HA, Leibowitz H, Gauley JP, et al. Standardizing diagnostic procedures. Am J Ophthalmol 1975;79:768-775.CrossRef

36.

Neaton JD, Duchene AG, Svendson KH, Wentworth D. An examination of the efficacy of some quality assurance methods commonly employed in clinical trials. Stat Med 1990;9:115-124.CrossRef

37.

Kandzari DE, Bhatt DL, Sobotka PA, et al. Catheter-based renal denervation for resistant hypertension: rationale and design of the SYMPLICITY HTN-3 Trial. Clin Cardiol 2012;35:528-535.CrossRef

38.

Furberg CD, Byington RP, Craven TE. Lessons learned from clinical trials with ultrasound endpoints. J Intern Med 1994;236:575-580.CrossRef

39.

Bhatt DL, Kandzari DE, O’Neill WW, et al. A controlled trial of renal denervation for resistant hypertension. N Engl J Med 2014;370:1393-1401.CrossRef

40.

Mahaffey KW, Roe MT, Dyke CK, et al. Misreporting of myocardial infarction end points: results of adjudication by a central clinical events committee in the PARAGON-B trial. Second Platelet IIb/IIIa Antagonist for the Reduction of Acute Coronary Syndrome Events in a Global Organization Network Trial. Am Heart J 2002;143:242-248.CrossRef

41.

Cohen D. Concerns over data in key dabigatran trial. BMJ 2014;349:g4747.CrossRef

42.

Connolly SJ, Ezekowitz MD, Yusuf S, et al. Dabigatran versus warfarin in patients with atrial fibrillation. N Engl J Med 2009;361:1139-1151.CrossRef

43.

Granger CB, Vogel V, Cummings SR, et al. Do we need to adjudicate major clinical events? Clin Trials 2008;5:56-60.CrossRef

44.

Pogue J, Walter SD, Yusuf S. Evaluating the benefit of event adjudication of cardiovascular outcomes in large simple RCTs. Clin Trials 2009;6:239-251.CrossRef

45.

Society for Clinical Data Management. eSource Implementation in Clinical Research: A Data Management Perspective. A White Paper. Available at http://​www.​scdm.​org/​sitecore/​content/​be-bruga/​scdm/​Publications/​white%20​paper.​aspx#sthash.​SUHoIyYl.​dpuf http://​www.​scdm.​org/​sitecore/​content/​be-bruga/​scdm/​Publications/​white%20​paper.​aspx. Accessed January 16, 2015.

46.

Reboussin D, Espeland MA. The science of web-based clinical trial management. Clin Trials 2005;2:1-2.CrossRef

47.

Litchfield J, Freeman J, Schou H, et al. Is the future for clinical trials internet-based? A cluster randomized clinical trial. Clin Trials 2005;2:72-79.CrossRef

48.

Winget M, Kincaid H, Lin P, et al. A web-based system for managing and co-ordinating multiple multisite studies. Clin Trials 2005;2:42-49.CrossRef

49.

Schmidt JR, Vignati AJ, Pogash RM, et al. Web-based distributed data management in the Childhood Asthma Research and Education (CARE) Network. Clin Trials 2005;2:50-60.CrossRef

50.

Mitchell R, Shah M, Ahmad S, et al. for the Adolescent Medicine Trials Network for HIV/AIDS interventions. A unified web-based query and notification system (QNS) for subject management, adverse events, regulatory, and IRB components of clinical trials. Clin Trials 2005;2:61-71.CrossRef

51.

Richesson RL, Hammond WE, Nahm M, et al. Electronic health records based phenotyping in next-generation clinical trials: a perspective from the NIH Health Care Systems Collaboratory. J Am Med Inform Assoc 2013;20(e2):e226-31.CrossRef

52.

Cutrona SL, Toh S, Iyer A, et al. Validation of acute myocardial infarction in the Food and Drug Administration’s Mini-Sentinel program. Pharmacoepidemiol Drug Saf 2012;22:40-54.CrossRef

53.

Eisenstein EL, Collins R, Cracknell BS, et al. Sensible approaches for reducing clinical trial costs. Clin Trials 2008;5:75-84.CrossRef

54.

Eisenstein EL, Lemons II PW, Tardiff BE, et al. Reducing the costs of phase III cardiovascular clinical trials. Am Heart J 2005;149;482-488.CrossRef

55.

Clinical Trials Transformation Initiative. Available at https://​www.​ctti-clinicaltrials.​org. Accessed January 16, 2015.

56.

Guidance for Industry. Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring. Available at http://​www.​fda.​gov/​downloads/​Drugs/​…/​Guidances/​UCM269919.​pdf. Accessed January 16, 2015.

57.

Guidance for Clinical Investigators, Sponsors, and IRBs. Adverse Event Reporting to IRBs—Improving Human Subject Protection. Available at http://​www.​fda.​gov/​downloads/​RegulatoryInform​ation/​Guidances/​UCM126572.​pdf. Accessed January 16, 2015.

58.

Lachin JM. The role of measurement reliability in clinical trials. Clin Trials 2004;1:553-566.CrossRef

59.

Alexander JH, Levy E, Lawrence J, et al. Documentation of study medication dispensing in a prospective large randomized clinical trial: experiences from the ARISTOTLE Trial. Am Heart J 2013;166:559-65.CrossRef

60.

National Cancer Institute Clinical Trials Monitoring Branch. Available at http://​ctep.​cancer.​gov/​branches/​ctmb/​clinicalTrials/​monitoring_​coop_​ccop_​ctsu.​htm. Accessed January 16, 2015.

61.

Weiss RB, Vogelzang NJ, Peterson BA, et al. A successful system of scientific data audits for clinical trials. JAMA 1993;270:459-464.CrossRef

62.

Soran A, Nesbitt L, Mamounas EP, et al. Centralized medical monitoring in phase III clinical trials: the National Surgical Adjuvant Breast and Bowel Project (NSABP) experience. Clin Trials 2006;3:478-485.CrossRef

63.

Califf RM, Karnash SL, Woodlief LH. Developing systems for cost-effective auditing of clinical trials. Control Clin Trials 1997;18:651-660.CrossRef

64.

Reynolds SM. ORI findings of scientific misconduct in clinical trials and publicly funded research, 1992-2002. Clin Trials 2004;1:509-516.CrossRef

65.

Department of Health and Human Services. Public Health Service policies on research misconduct. Final rule. Federal Register 2005;70;28370-28400.

66.

FDA Inspections Data Dashboard. Available at http://​govdashboard.​fda.​gov/​public/​dashboards?​id=​143. Accessed January 16, 2015.

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_12

12. Assessment and Reporting of Harm

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴, Christopher B. Granger⁵ and Thomas J. Moore⁶

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

(6)

Institute for Safe Medication Practices, Alexandria, Virginia, USA

Assessment of harm is more complex than the assessment of benefit of an intervention. The measures of favorable effects are or should be prespecified in the protocol and they are limited in number. In contrast, the number of adverse events is typically very large and they are rarely prespecified in the protocol. Some may not even be known at the time of trial initiation. These facts introduce analytic challenges.

Most intervention effects have three dimensions. The major objective of any trial is to measure change in the incidence or rate of a clinical event, symptom, laboratory test, or other measure. For benefit, the hoped-for difference in rate between the two study groups is prespecified and forms the basis for the sample size calculation. Such prespecifications rarely exist for the adverse events and clinical trials are often underpowered statistically for the documentation or dismissal of evidence of harm. There are two other dimensions of interest—severity of the event and recurrence or duration of its occurrence. In terms of severity, a clinical event can be uncomplicated or complicated, including being fatal, while a symptom can vary from mild to severe. There are few good objective scales for quantifying symptoms; their severity is based on the participants’ perceptions. In arthritis trials, a pain scale is commonly used to determine treatment benefit although it could also be used to determine adverse effects. Their recurrence can vary substantially from occasional to being constant. As a result of these methodological limitations the reporting of harm is often limited to whether adverse events occurred or not; rarely are severity and recurrence reported.

Contributing to the complexity of assessment and reporting of harm is a common confusion about the terminology. An adverse event is “any untoward event that occurs during a drug or medical treatment whether or not a causal relationship with the treatment is suspected or proven” [1]. Thus, adverse events might be experienced by treated as well as untreated patients. The incidence of adverse events is assessed in and reported for both study groups. One objective of trials is to compare the adverse event experiences in participants receiving active intervention or control. An adverse effect has been described as “a noxious or unintended response to a medical product in which a causal relationship is at least a reasonable possibility” [1]. In this text we will use these definitions of adverse events and adverse effects, except that they are broadened to include not just medical treatment, but any intervention.

Harm is the sum of all adverse effects and is used to determine the benefit-harm balance of an intervention. Risk is the probability of developing an adverse effect. Severe is a measure of intensity. Serious is an assessment of the medical consequence (see below). Expected adverse events or effects are those that are anticipated based on prior knowledge. Unexpected are findings not previously identified in nature, severity, or degree in incidence.

Fundamental Point

Careful attention needs to be paid to the assessment, analysis, and reporting of adverse effects to permit valid assessment of harm from interventions.

Assessment of Harm

There are three categories of adverse events—serious adverse events, general adverse events and adverse events of special interest. Serious adverse events are defined by the U.S. Food and Drug Administration (FDA) as those events that are (a) life threatening (b) result in initial or prolonged hospitalization, (c) cause irreversible, persistent or significant disability/incapacity, (d) are a congenital anomaly/birth defect, (e) require intervention to prevent harm, or (f) have other medically serious consequences [2]. General adverse events are those which patients or trial participants have complained about or clinicians have observed. These may range in intensity from very mild and not of much consequence to severe. Adverse events of special interest are typically derived from studies of mechanisms of action of the intervention (for example immunosuppression), animal studies, or observations from chemically similar drugs or related interventions. Assessment of adverse events of special interest requires prospective definition, specific ascertainment, and plans for reporting. Another area of importance is the evaluation of adverse drug interactions.

Strengths

There are four distinct advantages to assessment of harm in clinical trials, as opposed to other kinds of clinical research. First, adverse events can be defined prospectively, which allows proper hypothesis testing and adds substantial credibility. Post hoc observations, common in the area of harm, are often difficult to interpret in terms of causation and therefore often lead to controversy.

Second, randomized clinical trials by definition have a proper and balanced control group which allows for comparisons between the study groups. Randomization assures that intervention and control groups have similar characteristics—even those unknown to science at the time the trial was conceived. Other study designs have a dilemma when comparing users of a particular intervention to non-users. In observational studies, there is no guarantee that the user and non-user groups are comparable. There are clinical reasons why some people are prescribed a particular intervention while others are not. Observed group differences can be intervention-induced, due to differences in the composition and characteristics of the groups, or a combination thereof. Statistical adjustments can help but will never be able to control fully for unmeasured differences between users and non-users.

Third, clinical trials with a blinded design reduce potential biases in the collection, assessment and reporting of data on harm (Chap. 7).

Fourth, participants in clinical trials are closely and systematically assessed, including physical examinations, regular blood work, weekly or monthly clinic visits, vital signs, clinical events, and detailed assessment of concomitant medications.

Limitations

There are also four potential limitations in relying on clinical trials for evaluation of harm. First, the trial participants are a selected non-random sample of people with a given condition who volunteered for the trial. The selectivity is defined by the scope of the trial inclusion and exclusion criteria and the effects of enrolling only volunteers. In general, trial participants are healthier than non-participants with the same disease. In addition, certain population groups may be excluded, for example, women who are pregnant or breastfeeding. Trials conducted prior to regulatory agency approval of a product are typically designed to document clear findings of benefit and, therefore, often exclude from participation those who are old, have complicating medical conditions and/or are taking other medications which may affect the outcome. Trial sponsors also exclude participants at higher risk of suffering an adverse event. This reduces the incidence of such events and contributes to the likelihood of not documenting harm. The absence of serious adverse effects observed in low-risk participants in pre-approval trials is no assurance that a drug lacks harmful effects when it reaches the marketplace. Another limitation is that the ascertainment of adverse events often relies on volunteered information by the participant rather than specific, solicited information (see below). An early survey showed that most FDA-approved drugs have one serious adverse effect detected after approval when there is more exposure to higher-risk patients and longer treatment duration [3]. More recent high-profile cases of serious adverse effects not detected pre-approval are the treatments of osteoarthritis with COX-2 inhibitors [4–7], of type II diabetes with rosiglitazone [8–10], and prevention of thromboembolic events with oral anticoagulants [11–13]. The reported high rates of new Boxed Warnings and drug withdrawals over the past two decades illustrate a limitation of FDA’s current process for documenting real and potential harm pre-approval [14].

A second limitation relates to the statistical power of finding a harm, if it exists. Small sample sizes and short trial durations, as well as the focus on low-risk populations, reduce the likelihood of detecting serious adverse effects. Drug manufacturers often conduct a large number of small, short-term trials, and their large trials are often not of long duration. Due to limited statistical power, clinical trials are often unreliable for attributing causality to rare serious adverse events. Approximately 3,000 participants are required to detect a single case with 95% probability, if the true incidence is one in 1,000; a total of 6,500 participants are needed to detect three cases [15]. When a new drug is approved for marketing, approximately 500–2,000 participants have typically been exposed to it in both controlled and uncontrolled settings. More commonly, rare serious adverse effects are initially discovered through case reports, other observational studies or reports of adverse events filed with regulatory agencies after approval [16, 17]. However, clinical trials can detect precursors of serious adverse effects through measurements such as elevated ALT levels (acute liver failure) or prolonged QT interval on the electrocardiogram (sudden cardiac death). Vandenbroucke and Psaty [18] properly concluded that “the benefit side [of drugs] rests on data from randomized trials and the harms side on a mixture of randomized trials and observational evidence, often mainly the latter.”

Third, inability to detect late serious adverse effects is another potential limitation of clinical trials. When a new compound is introduced for long-term treatment of a non-life threatening disease, the minimum regulatory standard is only several hundred participants exposed for 1 year or longer [19]. This is obviously inadequate for evaluation of drugs intended for chronic or long-term use. Moreover, a long lag time to harm must be considered for drugs that may be carcinogenic or have adverse metabolic effects. For example, the lag time for carcinogens to cause cancer may often be longer than most long-term trials. We support the view that evaluation of harm should continue the entire time a drug intended for chronic use is on the market [20].

Fourth, the investigators or sponsors may be unaware of some adverse effects because they are unexpected, or, in the case of known adverse effects, not ascertained. Potentially lethal cardiac rhythm disturbances may not be identified because electrocardiographic studies are not performed. Diabetes risk may be overlooked because laboratory testing does not include periodic assessment of HbA1c. Adverse effects related to sexual function or suicidal ideation may be underestimated because participants rarely volunteer information about sexual problems or suicidal ideation in response to general questions about changes in their health status. Ascertaining withdrawal and rebound effects require a special protocol to monitor discontinuation symptoms. Drug interactions may be overlooked because of rigid exclusion criteria in the protocol and failure to analyze concomitant medication data in relation to adverse events. Additionally, it is very challenging to be rigorous in these analyses.

The methods for collecting information on harm should take advantage of the strengths of clinical trials and to supplement them with properly designed and conducted observational studies post-trial, especially if issues or signals of harm emerge. Establishment of such long-term safety registries as one tool for post-marketing surveillance is becoming more common [21].

Identification of Harm in Clinical Trials

As pointed out earlier in this chapter, randomized clinical trials are not optimal for the detection of rare, late and unexpected serious adverse events. Experience has shown that critical information on serious reactions comes from multiple sources.

The role of clinical trials in identifying serious adverse reactions was investigated in an early study by Venning [16], who reviewed the identification and report of 18 adverse reactions in a variety of drugs. Clinical trials played a key role in identifying only three of the 18 adverse effects discussed. Another comparison of evidence of harm of various interventions in 15 large randomized and observational studies showed that the non-randomized studies often were more likely to find adverse effects [22].

A clinical trial may, however, suggest that further research on adverse reactions would be worthwhile. As a result of implications from the Multiple Risk Factor Intervention Trial [23] that high doses of thiazide diuretics might increase the incidence of sudden cardiac death, Siscovick and colleagues conducted a population-based case-control study [24]. This study confirmed that high doses of thiazide diuretics, as opposed to low doses, were associated with a higher rate of cardiac arrest.

Drugs of the same class generally are expected to have a similar effect on the primary clinical outcome of interest. However, they may differ in degree if not in kind of adverse effects. One illustration is cerivastatin which was much more likely to cause rhabdomyolysis than the other marketed statins [25]. Longer acting preparations, or preparations that are absorbed or metabolized differently, may be administered in different doses and have greater or lesser adverse effects. It cannot be assumed in the absence of appropriate comparisons that the adverse effects from similar drugs are or are not alike. As noted, however, a clinical trial may not be the best vehicle for detecting these differences, unless it is sufficiently large and of long duration.

Genomic biomarkers have assumed an increasing and important role in identifying people at an increased risk of adverse effects from medications. A large number of FDA approved drugs have pharmacogenomics information in different sections of the labeling [26]. Thus, adverse drugs effects observed in genetically defined subgroups of people are reflected in label additions of Boxed Warnings, Contraindications, Warnings, Precautions and Drug Interactions.

Classification of Adverse Events

Since the late 1990s, adverse drug events in clinical trials and many other clinical studies around the world are classified and described with a common terminology, the Medical Dictionary for Regulatory Activities (MedDRA) [27]. It was established by the International Conference on Harmonisation, a global organization created by the pharmaceutical industry to coordinate requirements among the world’s regulatory agencies.

The structure and characteristics of the MedDRA terminology have an effect on how adverse events are collected, coded, assessed, and reported in clinical trials. The most important feature is its pyramidal, hierarchical structure with highly granular terms at the bottom and 26 System Organ Classes at the top. The structure is shown in Table 12.1.

Table 12.1

MedDRA terminology hierarchya

Term

Abbrev.

Number of terms

System Organ Class

SOC

26

High Level Group Term

HLGT

334

High Level Term

HLT

1,717

Preferred Term

PT

20,307

Low Level Term

LLT

72,072

aMedDRA version 16.1

The number of Low Level Terms is very large and intended to facilitate human MedDRA coders using auto-coding software in assigning MedDRA terms to adverse event narratives by including a large number of phrases that might appear. These terms are aggregated at the Preferred Term level, the most granular level normally used in study reports. A key feature of Preferred Terms is that they do not necessarily describe an adverse event. A term could be a sign, symptom, diagnosis, surgical treatment, outcome (such as death), or person characteristic (such as bed sharing, aged parent, or surrogate mother). They are often coded based on a participant complaint noted in the medical record or data collection form.

The terminology designers sought to overcome some of the limitations of the hierarchical structure by allowing links across categories (a multi-axial structure) and the creation of Standardized MedDRA Queries (SMQs). For example an “Air embolism” has a primary link to the Vascular Disorder System Organ Class and a secondary link to Injury and Poisoning. SMQs, on the other hand, are designed to capture specifically adverse events independent of the hierarchical structure. Version 16.1 of MedDRA included 211 SMQs organized on four hierarchical levels [28].

The methodological strengths of the MedDRA terminology include the following: It is an accepted global standard with multiple language translations, which facilitates comparisons among trials. As a granular terminology it provides for detailed and accurate coding of narratives without requiring complex medical judgment in each case. The hierarchical structure and SMQs provide alternative tools for identifying adverse events.

While the MedDRA terminology design provides for simple and accurate coding of narratives, the terms thus selected do not necessarily describe adverse events. If analysis is limited to the approximately 20,000 Preferred Terms, the result is so granular that the number of participants for each listed event often becomes too few to evaluate meaningfully. See Table 12.2 for the large number of synonyms for depression. The SMQs in particular vary widely in design, specificity, sensitivity and other features and need to be assessed specifically in each case. By design, the terminology is continuously revised, with new versions appearing twice a year. This complicates replication of previous study results and comparisons among studies, and may even require special procedures to update an ongoing clinical trial that lasts for longer than 6 months. A participant may express the same complaint differently on two clinical visits. As a result, they are likely to be recorded differently, and thus coded differently, which makes it impossible to track a particular adverse event across visits.

Table 12.2

MedDRA preferred terms describing depression in a clinical trial

Agitated depression

Anhedonia

Childhood depression

Decreased interest

Depressed mood

Depression

Depression postoperative

Depression suicidal

Depressive symptom

Dysthymic disorder

Feeling guilty

Feeling of despair

Feelings of worthlessness

Major depression

Menopausal depression

Morose

Negative thoughts

Post stroke depression

Postictal depression

Postpartum depression

Psychomotor retardation

Tearfulness

Source: MedDRA version 16.1

Data monitoring based on the MedDRA terminology has turned out to be a challenge. The small numbers of events for each term due to the granular terminology are very difficult to interpret, and the aggregation of individual granular terms into a category with more events requires judgment in order to be clinically meaningful.

The National Cancer Institute (NCI) Common Terminology Criteria for Adverse Events v3.0 is another advanced system for reporting adverse events [29]. One strength is the 5-step severity scale for each adverse event ranging from mild to any fatal adverse event. It is available without charge.

Ascertainment

The issue often arises whether one should elicit adverse events by means of a checklist or rely on the participant to volunteer complaints. Eliciting adverse events has the advantage of allowing a standard way of obtaining information on a preselected list of symptoms. Thus, both within and among trials, the same series of events can be ascertained in the same way, with assurance that a “yes” or “no” answer will be present for each. This presupposes, of course, adequate training in the administration of the questions. Volunteered responses to a question such as “Have you had any health problems since your last visit?” have the possible advantage of tending to yield only the more serious episodes, while others are likely to be ignored or forgotten. In addition, only volunteered responses will give information on truly unexpected adverse events.

The difference in the yield between elicited and volunteered ascertainment has been investigated. In the Aspirin Myocardial Infarction Study [30] investigators first asked a general question about adverse events, followed by questions about specific complaints. The results for three adverse events are presented in Table 12.3. Two points might be noted. First, for each adverse event, eliciting gave a higher percent of participants with complaints in both intervention and placebo groups than did asking for volunteered problems. Second, similar aspirin-placebo differences were noted, regardless of the method. Thus, in this case the investigators could detect the adverse effect with both techniques. Volunteered events may be of greater severity but fewer in number, reducing the statistical power of the comparison.

Table 12.3

Percent of participants ever reporting (volunteered and solicited) selected adverse events, by study group, in the Aspirin Myocardial Infarction Study

Hematemesis

Tarry stools

Bloody stools

Volunteered

 Aspirin

0.27

1.34a

1.29b

 Placebo

0.09

0.67

0.45

Elicited

 Aspirin

0.62

2.81a

4.86a

 Placebo

0.27

1.74

2.99

aAspirin-placebo difference >2 S.E

bAspirin-placebo difference >3 S.E

Aspirin group: N = 2,267

Placebo group: N = 2,257

Spontaneously volunteered events also may substantially undercount some types of adverse effects, notably psychiatric symptoms. For example, when specifically queried using the Arizona Sexual Experiences Scale, 46.5% reported sexual dysfunction in one study, compared to 1–2% as spontaneously ascertained in clinical trials of fluoxetine [31]. Spontaneous reports could also underestimate new onset diabetes occurring in an unrelated treatment, as well as effects that are not typically characterized medically, such as falls, anger, or tremor.

Prespecified Adverse Events

The rationale for defining adverse events in the protocol is similar to that for defining any important benefit variable; it enables investigators to record something in a consistent manner. Further, it allows someone reviewing a trial to assess it more accurately, and possibly to compare the results with those of other trials of similar interventions.

Because adverse events are typically viewed as secondary or tertiary response variables, they are not often systematically and prospectively evaluated and given the same degree of attention as the primary and secondary benefit endpoints. They usually are not defined, except by the way investigators apply them in their daily practice. A useful source is the Investigator’s Brochure for the study drug. The diagnosis of acute myocardial infarction may be based on non-standardized hospital records. Depression may rely on a patient-reported symptom of non-specified severity and duration rather than a careful evaluation by a psychiatrist or responses to a standardized depression questionnaire. Thus, study protocols seldom contain written definitions of adverse events, except for those that are recognized clinical conditions. Multicenter trials open the door to even greater levels of variability in event definitions. In those cases, an adverse event may be simply what each investigator declares it to be. Thus, intrastudy consistency may be as poor as interstudy consistency.

However, given the large number of possible adverse events, it is not feasible to define all of them in advance and, in addition, many do not lend themselves to satisfactory definition. Some adverse events cannot be defined because they are not listed in advance, but are spontaneously mentioned by the participants. Though it is not always easy, important adverse events which are associated with individual signs or laboratory findings, or a constellation of related signs, symptoms, and laboratory results can and should be well-defined. These include the events known to be associated with the intervention and which are clinically important, i.e. adverse events of special interest. Other adverse events that are purely based on a participant’s report of symptoms may be important, but are more difficult to define. These may include nausea, fatigue, or headache. Changes in the degree of severity of any symptom should be part of the definition of an adverse event. The methods by which adverse events were ascertained should be stated in any trial publication.

Characteristics of Adverse Events

The simplest way of recording presence of an adverse event is with a yes/no answer. This information is likely to be adequate if the adverse event is a serious clinical event such as a stroke, a hospitalization or a significant laboratory abnormality. However, symptoms have other important dimensions such as severity, duration and frequency of recurrence.

The severity of subjective symptoms is typically rated as mild, moderate or severe. However, the clinical relevance of this rating is unclear. Participants have different thresholds for perceiving and reporting their symptoms. In addition, staff’s recorded rating of the reported symptom may also vary. One way of dealing with this dilemma is to consider the number of participants who were taken off the study medication due to an adverse event, the number who had their dose of the study medication reduced and those who continued treatment according to protocol in spite of a reported adverse event. This classification of severity makes clinical sense and is generally accepted. A challenge may be to decide how to classify participants who temporarily are withdrawn from study medication or have their doses temporarily reduced.

The duration or frequency with which a particular adverse event occurs in a participant can be viewed as another measure of severity. For example, episodes of nausea sustained for weeks rather than occasionally is a greater safety concern. Investigators should plan in advance how to assess and present all severity results.

Length of Follow-up

The duration of a trial has a substantial impact on adverse event assessment. The longer the trial, the more opportunity one has to discover adverse events, especially those with low frequency. Also, the cumulative number of participants in the intervention group complaining will increase, giving a better estimate of the incidence of the adverse event. Of course, eventually, most participants will report some general complaint, such as headache or fatigue. However, this will occur in the control group as well. Therefore, if a trial lasts for several years, and an adverse event is analyzed simply on the basis of cumulative number of participants suffering from it, the results may not be very informative, unless controlled for severity and recurrences. For example, the incidence could be annualized in long-term trials.

Duration of follow-up is also important in that exposure time may be critical. Some drugs may not cause certain adverse effects until a person has been taking them for a minimum period. An example is the lupus syndrome with procainamide [32]. Given enough time, a large proportion of participants will develop this syndrome, but very few will do so if treated for only several weeks. Other sorts of time patterns may be important as well. Many adverse effects even occur soon after initiation of treatment. In such circumstance, it is useful, and indeed prudent, to monitor carefully participants for the first few hours or days. If no effects occur, the participant may be presumed to be at a low risk of developing these effects subsequently.

In the Diabetes Control and Complications Trial (DCCT) [33], cotton exudates were noted in the eyes early after onset of the intervention of the participants receiving tight control of the glucose level. Subsequently, the progression of retinopathy in the regular control group surpassed that in the tight control group, and tight control was shown to reduce this retinal complication in insulin-dependent diabetes. Focus on only this short-term adverse effect might have led to early trial termination. Fortunately, DCCT continued and reported a favorable long-term benefit-harm balance.

Figure 12.1 illustrates the first occurrence of ulcer symptoms and complaints of stomach pain, over time, in the Aspirin Myocardial Infarction Study [30]. Ulcer symptoms rose fairly steadily in both the aspirin and placebo groups, peaking at 36 months. In contrast, complaints of stomach pain were maximal early in the aspirin group, then decreased. Participants on placebo had a constant, low level of stomach pain complaints. If a researcher tried to compare adverse effects in two studies of aspirin, one lasting weeks and the other several months, the findings would be different. To add to the complexity, the aspirin data in a study of longer duration may be confounded by changes in aspirin dosage and concomitant therapy.

Fig. 12.1

Percent of participants reporting selected adverse events, over time, by study group, in the Aspirin Myocardial Infarction Study

An intervention may cause continued discomfort throughout a trial, and its persistence may be an important feature. Yet, unless the discomfort is considerable, such that the intervention is stopped, the participant may eventually stop complaining about it. Unless the investigator is alert to this possibility, the proportion of participants with symptoms at the final assessment in a long-term trial may be misleadingly low.

Analyzing Adverse Events

Analysis of adverse events in clinical trial results depends in part on the intended use of the analysis. On one hand, drug regulators may provide detailed specifications for both required format and content of information of harm. On the other, peer reviewed journals typically provide space limited to a single table and a paragraph or two in the Results section (although electronic publication can allow considerably more space). Analysis will also depend on specifics of the participant population and intervention under study. Collection, analysis and reporting for prevention in a largely healthy population may differ substantially from an intervention in hospitalized patients with pre-existing heart failure. Nevertheless many trials provide important opportunities unavailable outside a clinical study setting to evaluate potential harm of interventions and public health is served by thorough analysis, even if results are reported in appendixes or on-line supplements.

This section will review four basic types of analysis: standard reporting of adverse events occurring in the trial, prespecified analysis of adverse events of interest, post hoc data-mining, including other exploratory analysis and meta-analysis.

Standard Reporting

The most basic form of assessment of harm is a complete accounting for all participants including those who did not complete the trial. Overall dropout rates are a useful measure of the tolerability of the drug or other interventions, and can be compared across many interventions. Dropout reporting is typically divided into at least three subcategories: dropout due to adverse events, dropouts for lack of efficacy and dropouts for administrative reasons. Assignment of a case to these subcategories may be more subjective than it appears. Lack of efficacy dropouts may rise because symptomatic adverse events might persuade some participants that they are not getting enough benefit to continue. Withdrawals of consent or other administrative departures may conceal problems with the drug, or the conduct of the trial. The overall dropout rate across all categories should be presented. If the dropouts have characteristics over time (such as dropouts related to short-term, early onset adverse events), some form of survival analysis of dropout rate over time may provide useful insights for managing treatment or suggest a need for dose titration.

Another standard analysis consists of a table of reported adverse events at the MedDRA level of Preferred Terms, with control and each intervention arm forming a column for easy comparison across groups. To make the list manageable in length, investigators typically set a threshold value for a subset of adverse events that total more than 1%, 5%, or 10% of patients. This has the major drawback of excluding less common adverse events which may be the more serious ones. Tests of statistical significance may be presented, but must be interpreted cautiously. Longer tables are usually organized by body system using the MedDRA System Organ Class. These standard event tables do not distinguish the severity and frequency of adverse events and are typically dominated by frequently occurring symptoms such as headache, nausea or dizziness.

Standard safety analysis may also include a listing of deaths, serious adverse events, clinically significant laboratory abnormalities, and changes in vital signs.

Prespecified Analysis

Possible adverse effects that could be reasonably expected from the known mechanism of action of the evaluated intervention, prior studies, or underlying participant conditions could be defined and analyzed from the perspectives of ascertainment, classification, and in particular statistical power, but these are rarely done. An investigator needs to consider prospectively and in the analysis the possibility of Type I or Type II error in the context of all three.

Adjudication is a tool frequently used when adverse events are of particular importance or are difficult to define. Adjudicated events are typically assessed by expert panels blinded to study group and following written protocols. While adjudicated results are typically seen as increasing the credibility and objectivity of the findings they may also reduce already limited statistical power by discarding cases with incomplete information. Adjudication can also be abused to suppress adverse event counts though unreasonably specific and restrictive case definitions. In addition, bias may be introduced if the adjudicators are not fully blinded. In the Randomized Evaluation of Long-Term Anticoagulation Therapy (RE-LY) trial, a team of adjudicators reviewed the outcome documents after reported to have been blinded [11]. Subsequently, the FDA took a closer look at the documents and concluded that information on intervention group assignment was available in 17% of the cases [34]. The credibility of adjudication results can be enhanced by accounting for possible but excluded cases.

Post Hoc Analysis

All post hoc analyses of adverse events may be subject to the criticism that it introduces bias because the analyses were not prospectively defined. Bias may also be introduced by problems of ascertainment and classification. These concerns are valid, but must be considered in light of two factors. First, analyses of prespecified events may themselves have biases and additional, even post hoc, analyses may provide further insight. Second, good clinical research is expensive, difficult to conduct, and seldom repeated without addressing new scientific issues. Therefore, post hoc analysis may yield important information and clues not otherwise obtainable.

One straightforward post hoc analysis addresses limitations of adverse event classification that occur due to the underlying MedDRA terminology. With approximately 20,000 Preferred Terms to describe an adverse event, this terminology permits substantial precision at the cost of disaggregating adverse events and is raising issues about accuracy. For example, at the Preferred Term level, a case of depression could be coded into any of 22 different terms (Table 12.2). Problems of gastrointestinal tolerability might be divided into nausea, vomiting, dyspepsia, and various forms of abdominal pains. Adverse event tables can be examined at all three key levels of the MedDRA hierarchy (Preferred, High Level and High Level Group Terms) as well through other categories created or Standardized MedDRA Queries. Additional understanding of adverse events could be expanded through examining time to reaction, effect duration, or severity. While these post hoc analyses may provide valuable insights into the harm of drugs and medical interventions, they should be specifically identified as separate from prospectively defined analyses.

Statistical techniques for data mining may provide additional opportunities to detect new signals of harm overlooked by clinical investigators in blinded trials. These techniques can be applied initially to the analysis of spontaneous adverse event reports but can be used both for signal detection in individual clinical trials and pooled data sets. With large populations, repeated visits, multiple outcome measures, many concomitant medications, and measures of underlying disease severity, the accumulated data are often too massive to exploit effectively with a prospective data analysis plan. However, the results of data mining analysis should be regarded as hypothesis generating that, after evaluation, would require additional investigation. Such signals may provide a useful basis for additional post hoc studies of existing data or enable prespecified analysis in future clinical trials. Data mining results may also provide context and focus to interpret particular results that were prespecified. Statistical tools such as the false discovery rate estimation [35] can help identify reliable associations in larger spontaneous reporting databases; other analysis might point to the need to explore associations that appeared borderline initially.

Meta-analysis

When individual trials are inconclusive, one approach is the combination of data on harm from multiple trials in a meta-analysis or systematic review (see Chap. 18).

Meta-analyses or pooled analyses conducted by manufacturers are commonly included in New Drug Applications submitted to regulatory agencies. Meta-analyses of treatment harm are now being published in leading medical journals. Singh and colleagues published three meta-analyses showing that rosiglitazone and pioglitazone double the risk of heart failure and fractures (in women) in type 2 diabetes [36, 37] and that rosiglitazone, in contrast to pioglitazone, also increases the risk of heart attacks [38]. None of these adverse effects was recognized at the time of regulatory approval of these drugs. Singh and colleagues concluded that cumulative clinical trial data revealed increased cardiovascular harm associated with rofecoxib a couple of years before the drug was withdrawn from the U.S. market. It has been recommended that cumulative meta-analysis be conducted to explore whether and when pooled adverse effect data reveal increased harm [39].

It is important to keep in mind that meta-analyses of harm have many limitations. Adverse event data in published studies are usually limited and event ascertainment seldom disclosed. Individual trials revealing unfavorable results may never be reported or published leading to publication bias and underestimation of the true rate of adverse effects. Experience has shown that conclusions from meta-analyses of a large number of small trials are not always confirmed in subsequent large trials.

Even though the clinical trials were prospectively designed, meta-analysis for harm is vulnerable to all the biases of a post hoc study design about a controversial safety issue when both the relevant trials and the number of events in each trial are already known by the study investigators. Small differences in inclusion or exclusion criteria can have large effects on the relative risk calculation, but are not evident in published results.

A substantial problem arises when investigators report that a meta-analysis of numerous trials detected no evidence of an adverse drug event reported using other methods. The failure to disprove the null hypothesis (no difference observed) is then claimed to be an assurance of safety. In this setting, additional evidence is required to rule out a simple Type II statistical error—that a difference existed but could not be detected in this study. In comparative clinical trials with an active drug control this problem is managed with relatively rigorous statistical standards for demonstrating non-inferiority. No such standards exist for meta-analysis of drug adverse events. Finally, when the magnitude of reported harm is small (for example a relative risk <2) all these imperfections in this technique mandate caution in interpreting the results.

Reporting of Harm

Selecting the appropriate and relevant data about harm from the large amount of data collected is a substantial challenge and may vary by the type and duration of the clinical study.

The usual measures of harm include:

(a)

Participants taken off study medication or device removed;

(b)

Participants on reduced dosage of study medication or on lower intensity of intervention;

(c)

Type, severity and recurrence of participant symptoms or complaints;

(d)

Abnormal laboratory measurements, including X-rays and imaging;

(e)

Clinical complications

(f)

In long-term studies, possible intervention-related reasons participants are hospitalized;

(g)

Combinations or variations of any of the above.

All of these measures can be reported as the number of participants with the occurrence at any point during the trial. Presenting data about how frequently these occurred in the same participant requires more detailed data and may consume considerable space in tables (again, electronic publication may allow considerably more space). Another method is to select a frequency threshold and assume that adverse events which recur less often in a given time period are less important. As an example, of ten participants having nausea, three might have it at least twice a week, three at least once a week, but less than twice, and four less than once a week. Only those six having nausea at least once a week might be included in a table, with the criteria fully disclosed.

Severity indices may be used. It can be assumed that a participant who was taken off study drug because of an adverse event had a more serious episode than one who merely had his dosage reduced. Someone who required dose reduction probably had a more serious event than one who complained, but continued to take the dose required by the study protocol. Data from the Aspirin Myocardial Infarction Study [30], using the same adverse events as in the previous example, are shown in Table 12.4. In the aspirin and placebo groups, the percent of participants complaining about hematemesis, tarry stools, and bloody stools are compared with the percent having their medication dosage reduced for those adverse events. As expected, numbers of participants complaining were many times greater than those prescribed reduced dosages. Thus, the implication is that most of the complaints were for relatively minor occurrences or were transient in nature.

Table 12.4

Percent of participants with drug dosage reduced or complaining of selected adverse events, by study group, in the Aspirin Myocardial Infarction Study

Aspirin (N = 2,267)

Placebo (N = 2,257)

Hematemesis

 Dose reduced

0.00

0.00

 Complaints

0.27

0.09

Tarry stools

 Dose reduced

0.09

0.04

 Complaints

1.34

0.67

Bloody stools

 Dose reduced

0.22

0.04

 Complaints

1.29

0.45

As mentioned above, another way of reporting severity is to establish a hierarchy of consequences of adverse events, such as permanently off study drug, which is more severe than permanently on reduced dosage, which is more severe than ever on reduced dosage, which is more severe than ever complaining about the effect. Unfortunately, few published clinical trial reports present such severity data.

Scientific Journal Publication

Published reports of clinical trials typically emphasize the favorable results; the harmful effects attributed to a new intervention are often incompletely reported. This discordance undermines an assessment of the benefit-harm balance. A review of randomized clinical trials published in 1997 and 1998 showed that reporting of harm varied widely and, in general, was inadequate [40]. Adverse effect reporting was considered adequate in only 39% of 192 clinical trial articles from seven therapeutic areas. The 2001 CONSORT statement included a checklist of 22 items that investigators ought to address in the reporting of randomized clinical trials. However, it only included one item related to adverse events which recommended that every report presents “All important adverse events or side effects in each intervention group” [41].

In 2004 [42], the checklist was extended to include ten new recommendations related to the reporting of harm-related issues and accompanying explanations (Table 12.5). The authors encouraged the investigators to use the term “harm” instead of “safety”, which is a reassuring term. In the first two years after the publication of the 2004 CONSORT guidelines the impact was negligible. Pitrou et al. [43] analyzed 133 reports of randomized clinical trials published in six general medical journals in 2006. No adverse events were reported in 11% of the reports. Eighteen percent did not provide numerical data by treatment group and 32% restricted the reporting to the most common events. The data on severity of adverse events were missing in 27% of the publications and almost half failed to report the proportion of participants withdrawn from study medication due to adverse events.

Table 12.5

Endorsed recommendations regarding better reporting of harms in randomized trials [42]

Recommendation

Description

1

If the study collected data on harms and benefits, the title or abstract should so state

2

If the trial addresses both harms and benefits, the introduction should so state

3

List adverse events with definitions for each (with attention, when relevant, to grading, expected vs. unexpected reactions, reference to standardized and validated definitions, and description of new definitions)

4

Clarify how harms-related information was collected (mode of data collection, timing, attribution methods, intensity of ascertainment, and harms-related monitoring and stopping rules, if pertinent)

5

Describe plans for presenting and analyzing information on harms (including coding, handling of recurrent reactions, specification of timing issues, handling of continuous measures, and any statistical analyses)

6

Describe for each arm the participant withdrawals that are due to harms and the experience with the allocated treatment

7

Provide the denominators for analyses on harms

8

Present the absolute risk of each adverse event (specifying type, grade, and seriousness per arm), and present appropriate metrics for recurrent reactions, continuous variables and scale variables, whenever pertinent

9

Describe any subgroup analyses and exploratory analyses for harms

10

Provide a balanced discussion of benefits and harms with emphasis on study limitations, generalizability, and other sources of information on harms

Ioannidis [44] proposed six explanations for inadequate reporting of adverse events that reflects diverse motives. (1) the study design ignored or undervalued adverse events, (2) collection of adverse events during the trial was neglected, (3) reporting of adverse events was lacking, (4) reporting of adverse events was restricted, (5) reporting of adverse events was distorted, and (6) the evidence of harm was silenced. The same recommendations are included in the 2010 CONSORT statement [45].

This is clearly an area in reporting of trial results that is not handled well. It is imperative that investigators devote more attention to reporting the key data on harm from their clinical trials. If not in the main results article, additional data on harm could be included in appendices to this paper or, if possible, covered in separate articles.

Regulatory Considerations

The regulatory issues related to the reporting of harm and efficacy in clinical trials are discussed in more detail in Chap. 22 (Regulatory Issues). Guidance for safety evaluation can be found in documents issued by the US Department of Health and Human Services [46–51].

The purpose of premarketing assessment of harm is to identify adverse effects prior to regulatory approval for marketing. This assessment is typically incomplete for several reasons. Very few early phase studies are designed to test specified hypotheses about harm. They are often too small to detect less common serious adverse events or adverse events of special interest. Additionally, the assessment of multiple adverse events raises analytic questions regarding multiplicity and thus proper significance levels. Moreover, the premarketing trials tend to focus on low-risk participants by excluding elderly persons, those with other medical conditions, and those on concomitant medications, which also reduces the statistical power.

The major drug regulatory agencies in the world have requirements for expedited reporting of adverse events in clinical trials. These requirements apply to serious, unexpected, and drug-related events. As described earlier, a serious adverse event is defined as death, life-threatening event, hospitalization initial or prolonged, persistent or significant disability, congenital anomaly/birth defect, or required intervention to prevent harm or other medically serious event. Unexpected means an effect is not listed in the Investigator’s Brochure or product label at the severity observed. The unexpected events in trials registered with the FDA must be reported by the trial sponsor in writing within 15 calendar days of being informed. For an unexpected death or life-threatening reaction, the report should be made within 7 days of notification. The regulations do not specify deadlines for sites to report these reactions to the study sponsor, although sponsors typically establish their own deadlines.

To deal with often limited information on harm, special regulatory attention is given to adverse trends in the data. The regulatory term safety signal [49] is defined as “a concern about an excess of adverse events compared to what would be expected to be associated with a product’s use.” These signals generally indicate a need for further investigation in order to determine whether they are drug-induced or chance findings. As part of the approval decision, the sponsor may be required to conduct post-approval phase IV studies.

Rules for reporting adverse events to the local ethics review committees vary. Many require that investigators report all events meeting regulatory agency definitions. These committees have, based on the safety report, several options. These include making no change, requiring changes to the informed consent and the trial protocol, placing the trial on hold, or terminating approval of the trial. However, the committees seldom have the adequate expertise or infrastructure to deal with serious adverse event reports from multicenter trials, or even local trials. When the trial is multicenter, different rules and possible actions from different ethics committees can cause considerable complications. These complications can be reduced when the ethics review committees agree to rely on safety review by a study-wide data monitoring committee.

Recommendations for Assessing and Reporting Harm

Substantial improvements are needed in the ascertainment, analysis, and reporting of harm in clinical trials. One advance would be to match better sample size, patient population, and trial duration to clinical use, especially when long-term treatment is intended.

Second, to meet higher standards in the evaluation of harm, efforts should be made in pre-approval trials to prespecify and collect data on known or potential intervention-induced adverse effects. The data ought to be solicited with special questions asked of the participants rather than left completely open-ended and be based on a volunteered response. Asking participants whether they had any general problem since the last contact will underestimate the true rate of reported adverse events, especially those that are sensitive. Collection of known adverse effects is also important in trials of new populations or when new indications are investigated in order to permit determination of the benefit-harm balance. If groups of participants are believed to be susceptible to adverse events or effects, prespecified subgroup analyses ought to be identified in the protocol. As stated above, subgrouping based on genetic variations has been very informative.

Third, limiting the assessment of harm to the simple frequency of adverse events is a crude approach. As stated above, many adverse events have additional dimensions—severity, time of onset, and duration. By ignoring these, one episode of a mild adverse symptom is given equal weight to a severe, constant symptom leading to discontinuation of the intervention. As a minimum, the number of participants taken off the study intervention due to an adverse event, the number who had their dose reduced and those who continued treatment according to protocol in spite of an adverse event, ought to be assessed and reported in publications.

Fourth, all serious events should be fully disclosed, by study group. There is no reason to omit, restrict or suppress these events especially if they are of a serious nature. Even non-significant imbalances are important. In the disclosure, it is also essential to account for all randomized participants.

Fifth, we endorse the ten CONSORT recommendations regarding better reporting in the literature of harms in randomized trials (Table 12.5). There should be a full and open accounting of all important adverse effects in the main trial publications.

Sixth, we support cooperation with investigators who are pooling and analyzing adverse effect data from multiple clinical trials. This type of data sharing has strong support in the academic community [52–58]. Support for data sharing has also been given by industry [59–61], funders of research [62], major organizations [63] and medical journals [64]. A 2015 report from the Institute of Medicine recommends responsible data sharing for completed trials, with focus on data used in trial publications as well as data used in the complete study report submitted for regulatory review [65]. More details of this report are presented in Chap. 20.

Seventh, we have limited sympathy for investigators who question the existence of adverse effects unless clearly documented in randomized clinical trials. Other types of studies, systematically analyzed case reports, and use of registries have a role in the identification of serious adverse effects. A detailed discussion of these falls outside the scope of this book. Very large observational studies have been successfully used in the past [22]. Spontaneous adverse event reporting continues to be a critical and primary source for identifying new serious adverse drug reactions that were not fully evident in clinical trials. One study of all new major safety warnings from the FDA in 2009 showed that 76% of new Boxed Warnings in the drug label were based on spontaneous reports [17]. A subsequent paper from the FDA confirmed that spontaneous reports accounted for over half of all safety-related drug label changes [66]. Thus, these data can establish associations, but the incidence of such adverse effects needs to be determined through additional studies.

References

1.

International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. Clinical Safety Data Management: Definition and Standards for Expedited Reporting E2A, October 27, 1994.

2.

United States Code, Code of Federal Regulations 21 CFR 314.80(a) Postmarketing reporting of adverse drug experiences. Definitions.

3.

US General Accounting Office. FDA Drug Review: Postapproval Risks, 1976-85. Washington, DC: US General Accounting Office; April 26, 1990. GAO/PEMD-90-15.

4.

Bombardier C, Laine L, Reicin A, et al. for the VIGOR Study Group. Comparison of upper gastrointestinal toxicity of rofecoxib and naproxen in patients with rheumatoid arthritis. N Engl J Med 2000;343:1520-1528.CrossRef

5.

Bresalier RS, Sandler RS, Quan H, et al. for the Adenomatous Polyp Prevention on Vioxx (APPROVe) Trial Investigators. Cardiovascular events associated with rofecoxib in a colorectal adenoma chemoprevention trial. N Engl J Med 2005;352:1092-1102.CrossRef

6.

Solomon SD, McMurray JJV, Pfeffer MA, et al. for the Adenoma Prevention with Celecoxib (APC) Study Investigators. Cardiovascular risk associated with celecoxib in a clinical trial for colorectal adenoma prevention. N Engl J Med 2005;352:1071-1080.CrossRef

7.

Psaty BM, Furberg CD. COX-2 inhibitors – Lessons in drug safety. N Engl J Med 2005;352:1133-1135.CrossRef

8.

Nissen SE, Wolski K. Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med 2007;356:2457-2471.CrossRef

9.

Food and Drug Administration. FDA briefing document: advisory committee meeting for NDA 21071 Avandia (rosiglitazone maleate) tablet July 13 and 14, 2010. www.​fda.​gov/​downloads/​AdvisoryCommitte​es/​CommitteesMeetin​gMaterials/​Drugs/​Endocrinologican​dMetEndocrinolog​icandMetabolicDr​/​UCM218493.​pdf.

10.

Nissen SE. Rosiglitazone: a case of regulatory hubris. The FDA’s defensiveness over its decisions means further drug safety disasters may occur. BMJ 2013;347:f7428.

11.

Connolly SJ, Ezekowitz MD, Yusuf S, et al. Dabigatran versus warfarin in patients with atrial fibrillation. N Engl J Med 2009;361:1139-1151.CrossRef

12.

Moore TJ, Cohen MR, Furberg CD. Quarterwatch 2012 Quarter 2. www.​ismp.​org/​quarterwatch/​pdfs/​2012Q2.​pdf

13.

Eikelboom JW, Connolly SJ, Brueckmann M, et al. Dabigatran versus warfarin in patients with mechanical heart valves. N Engl J Med 2013;369:1206-1214. CrossRef

14.

Frank C, Himmelstein DU, Woolhandler S, et al. Era of faster FDA drug approval has also been increased black-box warnings and market withdrawal. Health Affair 2014;33:1453-1459.CrossRef

15.

Furberg BD, Furberg CD. Evaluating Clinical Research. All that Glitters is Not Gold (2nd edition). New York, NY: Springer, 2007, pp. 17-18.

16.

Venning GR. Identification of adverse reactions to new drugs. II: How were 18 important adverse reactions discovered and with what delays? Br Med J 1983;286:289-292 and 365-368.

17.

Moore TJ, Singh S, Furberg CD. The FDA and new safety warnings. Arch Intern Med 2012;172:78-80.CrossRef

18.

Vandenbroucke JP, Psaty BP. Benefits and risks of drug treatments. How to combine the best evidence on benefits with the best data about adverse effects. JAMA 2008;300:2417-2419.CrossRef

19.

Guideline for Industry: The extent of population exposure to assess clinical safety for drugs intended for long-term treatment of non-life-threatening conditions. International Conference on Harmonization, Geneva, March 1995.

20.

Committee on the Assessment of the US Drug Safety System. Baciu A, Stratton K, Burke SP (eds.). The Future of Drug Safety: Promoting and Protecting the Health of the Public. Washington, DC: The National Academies Press, 2006.

21.

Furberg CD, Levin AA, Gross PA, et al. The FDA and drug safety. A proposal for sweeping changes. Arch Intern Med 2006;166:1938-1942.CrossRef

22.

Papanikolaou PN, Christidi GD, Ioannidis JPA. Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies. CMAJ 2006;174:635-641.CrossRef

23.

Multiple Risk Factor Intervention Trial Research Group. Baseline rest electrocardiographic abnormalities, antihypertensive treatment, and mortality in the Multiple Risk Factor Intervention Trial. Am J Cardiol 1985;55:1-15.CrossRef

24.

Siscovick DS, Raghunathan TE, Psaty BM, et al. Diuretic therapy for hypertension and the risk of primary cardiac arrest. N Engl J Med 1994;330:1852-1857.CrossRef

25.

Psaty BM, Furberg CD, Ray WA, Weiss NS. Potential for conflict of interest in the evaluation of suspected adverse drug reactions: use of cerivastatin and risk of rhabdomyolysis. JAMA 2004;292:2622-2631.CrossRef

26.

www.​fda.​gov/​drugs/​scienceresearch/​researchareas/​pharmacogenetics​/​ucm083378.​htm

27.

Introductory Guide MedDRA Version 16.1. Maintenance and Support Services Organization, Chantilly (MSSO), VA, 2013.

28.

Introductory Guide for Standardised MedDRA Queries (SMQs) Version 16.1. MedDRA Maintenance and Support Services Organization (MMSSO), Chantilly, VA, 2013.

29.

NCI Guidelines for Investigators. http://​ctep.​cancer.​gov.

30.

Aspirin Myocardial Infarction Study Research Group. A randomized, controlled trial of aspirin in persons recovered from myocardial infarction. JAMA 1980;243:661-669.CrossRef

31.

Lee K, Lee Y, Nam J, et al. Antidepressant-induced sexual dysfunction among newer antidepressants in a naturalistic setting. Psychiatry Investig 2010;7:55-59.CrossRef

32.

Dalle Vedove C, Simon JC, Girolomoni G. Drug-induced lupus erythematosus with emphasis on skin manifestations and the role of anti-TNFα agents. J Dtsch Dermatol Ges 2012;10:889-897.

33.

The Diabetes Control and Complications Trial Research Group. The effect of intensive treatment of diabetes on the development and progression of long-term complications in insulin-dependent diabetes mellitus. N Engl J Med 1993;329:977-986.CrossRef

34.

Beasley N, Thompson, A. Clinical Review of NDA 022-512 Dabigatran (Pradaxa), August 24, 2010. (Amended October 17, 2010). US Food and Drug Administration, Center for Drug Evaluation and Research, Silver Spring, MD, page 42.

35.

Ahmed I, Dalmasso C, Haramburu F, et al. False discovery rate estimation for frequentist pharmacovigilance signal detection methods. Biometrics 2010;66:301-309.MathSciNetCrossRefMATH

36.

Singh S, Loke YK, Furberg C. Thiazolidinediones and heart failure: A teleo-analysis. Diabetes Care 2007;30:2148-2153.CrossRef

37.

Loke YK, Singh S, Furberg CD. Long-term use of thiazolidinediones and fractures in type 2 diabetes: a systematic review and meta-analysis. CMAJ 2009;180:32-29CrossRef

38.

Singh S, Loke YK, Furberg CD. Long-term risk of cardiovascular events with rosiglitazone. JAMA 2007;298:1189-1195.CrossRef

39.

Ross JS, Madigan D, Hill KP, et al. Pooled analysis of rofecoxib placebo-controlled clinical trial data. Lessons for postmarket pharmaceutical safety surveillance. Arch Intern Med 2009;169:1976-1984.CrossRef

40.

Ioannidis JPA, Lau J. Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas. JAMA 2001;285:437-443.CrossRef

41.

Moher D, Schulz KF, Altman DG, for the CONSORT Group. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 2001;357:1191-1194.

42.

Ioannidis JPA, Evans SJ, Gøtzsche PC, et al. for the CONSORT Group. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med 2004;141:781-788.

43.

Pitrou I, Boutron I, Ahmad N, Ravaud P. Reporting of safety results in published reports of randomized controlled trials. Arch Intern Med 2009;169:1756-1761.CrossRef

44.

Ioannidis JPA. Adverse events in randomized trials. Neglected, restricted, distorted, and silenced. Arch Intern Med 2009;169:1737-1739.CrossRef

45.

Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ 2010;340:c332.CrossRef

46.

Department of Health and Human Services, Food and Drug Administration: International Conference on Harmonisation; Guideline on clinical safety data management: Definitions and standards for expedited reporting, Notice. Federal Register 60 (1 March 1995): 11284-11287.

47.

Department of Health and Human Services, Food and Drug Administration. International Conference on Harmonisation; Draft guidance on E2D postapproval safety data management: Definitions and standards for expedited reporting, Notice. Federal Register 68 (15 September 2003): 53983-53984.

48.

U.S. Department of Health and Human Services. Food and Drug Administration. Guidance for Industry. Premarketing risk assessment. March 2005. www.​fda.​gov/​downloads/​RegulatoryInform​ation/​Guidances/​ucm126958.​pdf.

49.

U.S. Department of Health and Human Services. Food and Drug Administration. Guidance for Industry. Good pharmacovigilance practices and pharmacoepidemiologic assessment. March 2005. http://​www.​fda.​gov/​downloads/​RegulatoryInform​ation/​Guidances/​UCM126834.​pdf.

50.

U.S. Department of Health and Human Services. Food and Drug Administration. Reviewer Guidance. Conducting a clinical safety review of a new product application and preparing a report on the review. March 2005. http://​www.​fda.​gov/​downloads/​Drugs/​GuidanceComplian​ceRegulatoryInfo​rmation/​Guidances/​UCM072974.​pdf.

51.

European Medicines Agency, ICH Topic E 2 A: Clinical Safety Data Management: Definitions and Standards for Expedited Reporting. European Medicines Agency, London, UK. June 1995.

52.

Gøtzsche PC. Why we need easy access to all data from all clinical trials and how to accomplish it. Trials 2011;12:249.CrossRef

53.

Boulton G, Rawlins M, Vallance P, Walport M. Science as a public enterprise: the case for open data. Lancet 2011;377:1633-1635.CrossRef

54.

Loder E. Sharing data from clinical trials. Where we are and what lies ahead. BMJ 2013;347:f4794.

55.

Mello MM, Francer JK, Wilenzick M, et al. Preparing for responsible sharing of clinical trial data. N Engl J Med 2013;369:1651-1658.CrossRef

56.

Zarin DA. Participant-level data and the new frontier in trial transparency. N Engl J Med 2013;369:468-469.CrossRef

57.

Eichler H-G, Pétavy F, Pignatti F, Rasi G. Access to patient-level data—a boon to drug developers. N Engl J Med 2013;369:1577-1579CrossRef

58.

Krumholz HM, Peterson ED. Open access to clinical trials data. JAMA 2014;312:1002-1003.CrossRef

59.

Wellcome Trust. Sharing research data to improve public health: full joint statement by funders of health research. 2011. http://​www.​wellcome.​ac.​uk/​About-us/​Policy/​Spotlight-issues/​Data-sharing/​Public-health-and-epidemiology/​WTDV030690.​htm.

60.

PhRMA (Pharmaceutical Research and Manufacturers of America) and EFPIA (European Federation of Pharmaceutical Industries and Associations). Principles for responsible clinical trial data sharing: Our commitment to patients and researchers. 2013. http://​phrma.​org/​sites/​default/​files/​pdf/​PhRMA PrinciplesForRes​ponsibleClinical​TrialDataSharing​.​pdf.

61.

Nisen P, Rockhold F. Access to patient-level data from GlaxoSmithKline clinical trials. N Engl J Med 2013;369:475-478.CrossRef

62.

NIH (National Institutes of Health). Final NIH statement on sharing research data. 2003. http://​grants.​nih.​gov/​grants/​policy/​data_​sharing.

63.

IOM. Sharing clinical research data: Workshop summary. 2013. Washington, DC: The National Academies Press.

64.

Godlee, F, Groves T. The new BMJ policy on sharing data from drug and device trials. BMJ 2012;345:1-3.

65.

Institute of Medicine Committee on Strategies for Responsible Sharing of Clinical Trial Data. Sharing clinical trial data: maximizing benefits, minimizing risk. Washington, D.C.: The National Academies Press, 2015.

66.

Lester J, Neyarapally GA, Lipowski E, et al. Evaluation of FDA safety-related drug label changes in 2010. Pharmacoepidemiol Drug Saf 2013;22:302-305.CrossRef

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_13

13. Assessment of Health Related Quality of Life

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴, Christopher B. Granger⁵, Michelle J. Naughton⁶ and Sally A. Shumaker⁷

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

(6)

The Ohio State University, Columbus, OH, USA

(7)

Wake Forest School of Medicine, Winston-Salem, NC, USA

The major goal of randomized clinical trials is to determine the potential benefits and harms of an intervention. The benefits of most available interventions in medicine are symptom improvements. Thus, relief or reduction of symptoms is a common primary outcome in clinical trials (Chap. 3). Most of the adverse effects of interventions are also symptom-related (Chap. 12). Most changes in symptomatology are subjective and reported by trial participants, with a special form of outcomes related to various types of functioning, traditionally covered by the term health-related quality of life (HRQL) [1–4].

A person’s perspectives and experiences have recently been integrated in a new term—“Patient-Reported Outcomes” [5–7], defined by the FDA as “…any report of the status of a patient’s health condition that comes directly from the patient, without interpretation of the patient’s response by a clinician or anyone else” [8].

In this chapter we focus on the traditional outcome of HRQL and discuss types of measures, their uses, methodological issues and their selection.

Fundamental Point

Assessments of the effects of interventions on participants’ daily functioning and health-related quality of life are critical components of many clinical trials, especially ones that involve interventions directed to the primary or secondary prevention of chronic diseases.

Types of HRQL Measures

Primary Measures

What is meant by quality of life varies greatly depending on the context. In some settings, it may include such components as employment status, income, housing, material possessions, environment, working conditions, or the availability of public services. The kinds of indices that reflect quality of life from a medical or health viewpoint are very different, and would include those aspects that might be influenced not only by conditions or diseases, but also by medical treatment or other types of interventions. Thus, HRQL is commonly used to mean the measurement of one’s life quality from a health or medical perspective.

In general, HRQL measures are multi-dimensional to reflect different components of people’s lives. Although there are some variations, there is general agreement on the primary dimensions of HRQL that are essential to most HRQL assessments [9]. These include: physical, social and psychological functioning, and participants’ overall assessment of their life quality and/or perceptions of their health status.

Physical functioning refers to an individual’s ability to perform daily life activities. These types of activities are often classified as either ‘activities of daily living,’ which include basic self-care activities, such as bathing and dressing, or ‘intermediate activities of daily living,’ which refer to a higher level of usual activities, such as cooking, and performing household tasks.

Social functioning is defined as a person’s ability to interact with family, friends and the community. Instruments measuring social functioning may include such components as the person’s participation in activities with family, friends, and in the community, and the number of individuals in his or her social network. A key aspect of social functioning is the person’s ability to maintain social roles and obligations at desired levels. An illness or intervention may be perceived by people as having less of a negative impact on their daily lives if they are able to maintain role functions that are important to them, such as caring for children or grandchildren or engaging in social activities with friends. In contrast, anything that reduces one’s ability to participate in desired social activities, even though it may improve clinical status, may reduce the person’s general sense of social functioning.

Psychological functioning refers to the individual’s emotional well-being. It has been common to assess the negative effects of an illness or intervention, such as levels of anxiety, depression, guilt and worry. However, the positive emotional states of individuals should not be neglected. Interventions may produce improvements in a person’s emotional functioning, and therefore such aspects as vigor, hopefulness for the future, and resiliency are also important to assess.

Global Quality of Life represents a person’s perception of his or her overall sense of well-being and quality of life. For example, participants may be asked to indicate a number between 0 (worst possible quality of life) and 10 (best possible quality of life) which indicates their overall quality of life for a defined time period (for example, in the last month).

Perceptions of health status need to be distinguished from actual health. Individuals who are ill and perceive themselves as such, may, after a period of adjustment, reset their expectations and adapt to their life situation, resulting in a positive sense of well-being. In contrast, persons in good health may be dissatisfied with their life situation, and rate their overall quality of life as poor. Participants may be asked to rate their overall health in the past month, their health compared to others their own age, or their health now compared to 1 year ago. It is interesting to note that perceived health ratings are strongly and independently associated with an increased risk of morbidity and mortality [10–12], indicating that health perceptions may be important predictors of health outcomes and HRQL, independent of clinical health status.

The dimensions of HRQL assessed in a trial should match the aims of the study. Some trials will necessitate the measurement of multiple dimensions, whereas others may suffice with the inclusion of one or two dimensions. For example, it is unlikely that in the examination of the short-term effects of hormone therapy on peri-menopausal symptoms, general physical functioning of the study participants (women in their mid-forties to early fifties) will be influenced. The inclusion of this dimension of HRQL in the trial may simply increase participant burden without benefit. It is important for investigators to indicate clearly the dimensions of HRQL used in a trial and provide a rationale for their inclusion (or exclusion), for example, deleting HRQL dimensions that might make the treatment under study “look bad”.

Additional Measures

Sleep disturbance has been related to depression and anxiety, as well as diminished levels of energy and vitality. Instruments assessing sleep habits may examine such factors as sleep patterns (e.g., ability to fall asleep at night, number of times awakened during the night, waking up too early in the morning or difficulty in waking up in the morning, number of hours slept during a typical night); and, the restorativeness of sleep.

Neuropsychological functioning refers to the cognitive abilities of a person, such as memory, executive functioning, spatial and psychomotor skills. This dimension is being more commonly assessed for a wide range of health conditions or procedures, such as the effects of a stroke, cardiac surgery, chemotherapy or multiple medications on cognitive functioning, as well as in studies of older people.

Sexual functioning measures include items regarding a person’s ability to perform and/or participate in sexual activities, the types of sexual activities in which one engages, the frequency with which such activities occur, and persons’ satisfaction with their sexual functioning or level of activity. These assessments are particularly important in studies in which the disease’s or condition’s natural history or its treatment, can influence sexual functioning (for example, antihypertensive therapy, prostate cancer surgery, or sequelae of a stroke).

Work-related impacts encompass a wide range of both paid and unpaid activities in which individuals engage. Measures of this dimension might include paid employment (for instance, time of return to work, hours worked per week); household tasks; and volunteer or community activities. Also, among employed individuals, the impact of the inability to work or fully return to employment, as well as health and life insurance issues are being increasingly assessed.

Although the above symptoms are some of the more commonly assessed in clinical research, other symptoms may be important to measure. Again, the specific symptoms relevant for a given clinical trial will depend upon the intervention under investigation, the disease or condition being studied, the aims of the trial, and the study population [13].

Uses of HRQL Measures

For many participants, there are two primary outcomes that are important when assessing the efficacy of a particular intervention: changes in their life expectancies and the quality of their lives. HRQL measures provide a method of measuring intervention effects, as well as the effects of the untreated course of diseases/health conditions, in a manner that makes sense to both the participant and the investigator. In countries where chronic rather than acute conditions dominate the health care system, the major goals of interventions include the reduction of symptoms, and maintenance or improvement in functional status. Increasing costs of health care and prescription medications also necessitate the thorough evaluation of competing treatments for optimal health and quality of life outcomes. Thus, it is important to determine how the person’s life is influenced by both the disease and its intervention, and whether the effects are better or worse than the effects of the untreated course of the underlying disease.

There are now many published studies assessing the HRQL and symptoms of participants in clinical trials. One classic clinical trial by Sugarbaker and colleagues examined 26 patients with soft tissue sarcoma to compare the impact of two treatments on physical functioning and symptoms [14]. Patients were randomized to amputation plus chemotherapy or limb-sparing surgery plus radiation therapy and chemotherapy. After all treatments had been completed and the participants’ physical status had stabilized, assessments were completed to measure HRQL, economic impact, mobility, pain, sexual relationships and treatment trauma. Contrary to expectations, participants receiving amputation plus chemotherapy reported better mobility and sexual functioning than those receiving limb-sparing surgery plus irradiation and chemotherapy. Based on the results of this study, practices in limb-sparing surgery, radiation and physical therapy were modified to improve patient care and functioning.

An example of a clinical trial examining findings first noted in observational studies, which has had widespread impact on clinical care, is the Women’s Health Initiative (WHI) hormone therapy trials. During the 1980s and early 1990s, observational and case-control studies suggested that the use of estrogen would decrease the incidence of cardiovascular events among post-menopausal women. In order to determine if this observation would be replicated in a large, randomized controlled trial, the WHI was initiated in 1993 [15]. Post-menopausal women ages 50–79 at baseline were randomized to either conjugated equine estrogens plus medroxyprogesterone acetate (CEE + MPA) versus placebo if they had not had a hysterectomy, or conjugated equine estrogens (CEE-alone) versus placebo among participants who had had a hysterectomy. The trial was expected to last an average of 8.5 years. Health-related quality of life was assessed annually after trial initiation. In 2002, the trial component testing CEE + MPA was stopped early, due to higher rates of cardiovascular events and breast cancers among women in the CEE + MPA arm versus the placebo group [16]. A year and a half later, the CEE-alone component was also stopped due to adverse outcomes among women randomized to the hormone therapy group [17]. The results of these two trials have had a major impact on the care recommendations of post-menopausal women, and spurred a debate among primary care practitioners, cardiologists, and gynecologists about the validity of the WHI results [18]. One argument that was made was that although estrogen therapy may not be indicated for cardiovascular disease protection, women still reported better HRQL when taking estrogen therapy. However, the quality of life results from the WHI did not support this argument [19]. Among women randomized to CEE + MPA versus placebo, the use of active treatment was associated with a statistically significant, but small and not clinically meaningful benefit in terms of sleep disturbance, physical functioning, and bodily pain 1 year after the initiation of the study. At 3 years, however, there were no significant benefits in terms of any HRQL outcomes. Among women aged 50–54 with moderate-to-severe vasomotor symptoms at baseline, active therapy improved vasomotor symptoms and sleep quality, but had no benefit on other quality of life outcomes. Similar results were found in the CEE-alone trial of the WHI among women with hysterectomy. At both 1 and 3 years after the initiation of the trial, CEE had no significant and clinically meaningful effects on HRQL [20]. Thus, the potential harmful effects of estrogen therapy among post-menopausal women were not outweighed by any significant gains in quality of life.

More recent trials have utilized HRQL as both primary and secondary outcomes. Richardson and colleagues conducted a randomized trial to assess a collaborative care intervention versus usual care for adolescents with depression [21]. Youth between the ages of 13–17, who screened positive for depression using the Patient Health Questionnaire (PHQ) [22] on two separate occasions and met criteria for major depression, were recruited. Adolescents randomized to the intervention arm had an in-person clinic visit with subsequent regular follow-up sessions with a master’s level clinician. The control group participants received their screening results and were referred to mental health services in the health care plan. The primary outcome was a change in depressive symptoms as measured by the Children’s Depression Rating Scale-Revised (CDRS-R) [23] from baseline to 12 months. Secondary outcomes included the change in Columbia Impairment Score (CIS) [24], depression response (>50% decrease on the CDRS-R) and a PHQ-9 score <5, signifying depression remission. The results indicated that the adolescents in the intervention group had statistically significantly greater decreases in the CDRS-R scores than the usual care group. Both groups experienced improvement on the CIS, with no significant differences between the groups. However, the intervention youth were more likely to achieve depression response and remission than the control group of adolescents. The results suggested that mental health treatment can be integrated into primary care services.

The Comparison of Laser, Surgery and Foam Sclerotherapy (CLASS) clinical trial examined the impact of treatment for primary varicose veins on HRQL [25]. This was a multicenter study of 11 vascular surgery departments in the United Kingdom involving 798 participants. Participants were randomized to ablation therapy, surgery, or foam sclerotherapy. For the primary outcomes, the investigators used the disease-specific Aberdeen Varicose Veins Questionnaire [26] and the generic SF-36 [27] and the Euroqol Group 5-Dimension [28] measures. Secondary outcomes were complication rates and measures of clinical success. Outcomes were assessed at baseline and 6 weeks and 6 months after treatment. The results indicated similar HRQL outcomes across the three groups, although slightly worse disease-specific quality of life was observed in the foam group as compared with the surgery group. All treatments had similar rates of clinical success, but complications were lower in the laser treatment group, and the foam group had less successful ablation of the main trunk of the saphenous vein than the surgery group. Thus, all of these examples indicate that HRQL can be used as both primary and secondary outcomes, and can have substantial impact on clinical care practices and treatment options.

Methodological Issues

The rationale and execution of a well-designed and conducted randomized clinical trial assessing HRQL is the same as for other study outcomes. The reasons for its inclusion must be specified with supporting scientific literature and the HRQL measures selected should match the specific aims and have sound psychometric properties. If HRQL measures are secondary outcomes, it is also important to have sufficient study power to detect changes in these outcomes. The double-blind design minimizes the risk of bias.

The basic principles of data collection (Chap. 11) which ensure that the data are of the highest quality are also applicable to HRQL assessments. The methods must be feasible and designed to limit missing data. Training sessions of investigators and staff should be conducted for all trials, as well as pretesting of study data collection procedures and study measures, including HRQL assessments. An ongoing monitoring or surveillance system enables prompt corrective action when errors and other problems are found.

Design Issues

Several issues must be considered when using HRQL measures in clinical trials [3, 4]. These include the characteristics of the participants, type of treatment or intervention being studied, and protocol considerations.

Study Population

It is critical to specify key population demographics that could influence the choice of HRQL measures and the mode of administration. Education level, gender, age range, literacy levels, the primary language(s) spoken, and cultural diversity should be carefully considered prior to selecting any measures. Functional limitations should also be assessed. Elderly people may have more vision or hearing problems than middle-aged persons, making accommodations to self- or interviewer-administered questionnaires necessary. Ethnically diverse groups also require measures that have been validated across several different cultures and/or languages [29]. Children generally need instruments specifically for their age group, as well as assessments from parents regarding their perceptions of their child’s symptoms and physical and psychological health status.

The health status of the participant at baseline must also be taken into account in the development of the protocol and data collection procedures, including the severity of the illness, the effects of the participants’ illness or health condition on daily life, symptom levels or whether symptoms are acute or chronic. Healthy or mildly ill individuals will likely be able to participate more in a trial than those with debilitating chronic health conditions. These considerations have ramifications for the burden placed on participants (and staff) in completing study requirements and data collection, or those in acute phases of an illness. Participants who are children and/or are unable to complete HRQL assessments themselves may require the use of family proxy and/or investigator or staff assessments to collect HRQL data.

It is also important to be sensitive to how the underlying condition will progress and affect the HRQL of participants in the control group, as it is to understand the effects of the study intervention on those in the intervention arm(s). The point is to select dimensions and measures of HRQL that are sufficiently sensitive to detect changes in both the intervention and the control group participants. Using the same instruments for both groups will ensure an unbiased and comparable assessment.

Type of Intervention

Three major intervention-related factors are relevant to the assessment of HRQL—the favorable and adverse effects of intervention, the time course of the effects, and the possible synergism of the intervention with existing medications and pre-existing health conditions. It is important to understand how a proposed intervention could affect various aspects of an individual’s life in both positive and negative ways. What effects may the participant experience as a result of intervention? Some oral contraceptives, for instance, may be very effective in preventing pregnancy, while producing cyclical symptoms like bloating and breast tenderness, and in severe cases, blood clots. Dietary interventions designed to increase fruit and vegetable intake and lower dietary saturated fats may cause mild gastrointestinal effects, which may dissipate over time. Thus, the time course of an intervention’s effects is important both in terms of the selection of measures and the timing of when HRQL measures are administered to study participants. Furthermore, it is important to know the medications the participants are likely to be on prior to randomization and how these medications might interact with the trial intervention, (either a pharmacological or behavioral intervention), to influence the dimensions of HRQL.

The frequency of HRQL assessments will depend on the nature of the condition being investigated (acute versus chronic), the expected effects of the intervention, and the specific aims of the trial. Ideally, a baseline assessment should be completed prior to randomization and the initiation of the intervention. Follow-up assessments should be timed to match expected changes in functioning due to either the intervention or the condition itself. In a trial comparing a new acne skin serum with a placebo oil-free lotion for the treatment of severe acne in adolescents, assessing skin redness, sensitivity and acne reduction at only 1 and 3 weeks after baseline might not be sufficient to accurately measure the effectiveness of the intervention vs. placebo, given that severe acne may take longer than 3 weeks to show noticeable skin improvements even with known effective treatments. If the HRQL assessments are instead completed at baseline and 2 week intervals through 8 weeks, treatment effects (or non-effects) might be more accurately assessed. Thus, the timing of the HRQL assessment will affect the interpretation of the benefits (or negative effects) of the interventions.

Frequency of Assessment (Acute Versus Chronic)

In general, acute conditions resolve themselves in one of four ways: a rapid resolution without a return of the condition or symptoms; a rapid resolution with a subsequent return of the conditions after some period of relief (relapse); conversion of the acute condition to a chronic problem; or death [30]. In the case of rapid resolution, HRQL assessments would likely focus on the relative effect of the condition’s symptoms on the participant’s daily life. When there is a risk of relapse, a longer duration of follow-up is necessary, because relapses may have a broad impact on the participant’s general functioning and well-being. If the acute problem converts to a chronic condition, the evaluation is complicated by the duration of time and the problem of how to balance participant functioning in making treatment decisions.

Interventions that have little or no adverse effects on participant function are best evaluated on the basis of their impact on survival or change in disease severity or risk. In these situations, HRQL assessments will be of lesser importance. However, when a disease or condition affects functional capacity, interventions should to be evaluated for their effects upon the participants’ level of functioning and well-being. Again, in these situations, the type of HRQL instruments used and the timing of the assessments will depend on the nature of the condition, the intervention, and the expected time course of effects on the participants.

Protocol Considerations

After consideration has been given to the study population, the nature of the condition being studied and characteristics of the proposed intervention, there are additional protocol-related factors that need to be taken into account when developing HRQL collection procedures. Factors such as the venue for the proposed intervention (e.g., in clinic or hospital, community site, home, or school) and whether the intervention is done by trained staff, via computer, or using some other method, will influence the methods used to collect data. In addition, the number of participants being recruited to the trial, the number of follow-up assessment points, and the overall length of the trial (e.g., 8 weeks vs. 4 years) will have ramifications for the study design. Participants seen in clinics at regular intervals may afford easy access to completing assessments. Other modes of data collection, such as telephone, mail, or computer all have strengths and weaknesses. Telephoning participants to complete symptom or HRQL measures takes up staff time, but may involve less staffing, expense and missing data than preparing and sending a mailing to participants, tracking the responses and perhaps doing a second mailing or telephone call to increase the response rate. Interviewer administered instruments also generally provide more complete data and allow for probes and clarification. However, there may be a reluctance on the part of some participants to openly discuss some issues (for example, depression, sexuality), whereas they may be willing to respond to questions about these issues in a self-administered format. For populations with a relatively high proportion of functional illiteracy, in-person interviewer administration may be required. Interviewer administration may also be the best way to obtain information for culturally diverse populations. Interviewer-administered instruments, however, are subject to interviewer bias and require intensive interviewer training, certification, and repeat training, especially within the context of multi-site clinical trials that may be of a long duration. Thus, they can be more expensive than self-administered instruments and serious thought must be given at the planning phases of a trial regarding the trade-offs between these strategies.

On-line ascertainment has become more feasible and popular. They may not be an optimal choice, however, for those without ready access to on-line resources. Hand held devices and tablets for the tracking of symptoms are becoming more widely used, but take time to train staff and participants in their use. In addition, depending on the number of participants in a trial, obtaining these devices may be cost-prohibitive. If participants are only being assessed at 6 month intervals vs. weekly, for example, the use of mailed or on-line ascertainment may be more cost-effective.

All methods of data collections have their pluses and minuses and need to be considered in devising the most optimal methods for completing HRQL assessments economically with as little participant and staff burden as possible, while minimizing missing data. Options for data collection need to be assessed during the development of the protocol, and not as an afterthought. If HRQL assessments are secondary outcomes, data collections procedures will need to accommodate the needs of the primary aims, but should still be approached with the same rigor and planning as the collection of primary outcomes data.

Modifying and Mediating Factors

HRQL measures may be influenced by both modifying and mediating factors. Modifying factors are those variables that can modify the effect of an intervention on an outcome. They can be divided into three categories: contextual, interpersonal, and intrapersonal [31]. Contextual factors include such variables as study setting or the living environment of a participant (for example, urban vs. rural, single dwelling house vs. multiunit building, clinic vs. home intervention); economic structure (e.g., national health insurance); and sociocultural variations (e.g., customs, social norms). Interpersonal factors include variables such as the social support available to individuals, stress, economic pressures, and the occurrence of major life events, such as bereavement and the loss of a job. Intrapersonal factors are associated with the individual, such as coping skills, personality traits, or physical health. Mediating factors are any changes, improvements or impairments to a participant’s well-being that are induced by the study intervention. These are the changes that are most often assessed in trials with HRQL or symptom outcomes. For example, in a trial studying the effectiveness of aromatase inhibitors in preventing cancer recurrence among breast cancer survivors, these drugs may cause moderate to severe joint and muscle pain, which could lead to reduced HRQL and treatment adherence, although the study drug is effective in increasing overall cancer-free survival.

In addition, changes in the natural course of the disease or condition (i.e., whether the condition improves or deteriorates) must be considered in HRQL assessments, especially in trials of relatively long duration. Investigators should consider what effects the intervention or the health condition itself will have on participants’ well-being, and any factors that might moderate these relationships, in order to better select and measure pertinent HRQL variables. Consideration of these factors will aid in the interpretation of study findings, and may enable investigators to explain better the results of a specific intervention.

Selection of HRQL Instruments

All HRQL outcomes must be participant-centered, and the instruments used must match the specific aims of each particular clinical trial. For example, in a study examining the impact of post-surgery swelling on physical and social activities, one would not only need to determine whether and where swelling occurs, but how much it interferes with the ability to carry out physical and social activities. Simply measuring the occurrence and frequency of swelling, for example, would not answer the question of the effect on daily life.

Recently, there have been several reviews that have identified minimum quality standards for HRQL and other patient-reported outcomes [32, 33]. These attributes include measures with 1) a conceptual model; 2) established reliability; 3) established validity; 4) responsiveness to changes in clinical status and/or as a result of one intervention; 5) interpretability of scores; 6) cultural and language translations or adaptations; 7) feasibility in the desired setting; and 8) participant and staff/investigator burden. It is beyond the scope of this chapter to review techniques and practices used to develop HRQL measures, but references regarding scaling procedures and psychometric considerations of instruments (reliability, validity, and the responsiveness of instruments to change) may be consulted [3, 4, 34].

Types of Measures

HRQL measures can be classified as either generic (that is, instruments designed to assess outcomes in a broad range of populations), condition/disease specific (e.g., congestive heart failure, cancer) or symptom-specific (e.g., pain, anxiety) [13]. Within these categories of measures are single or multiple questionnaire items. Single questionnaire items that ask participants to rate their current severity of a symptom on a scale from 0 to 10, have the advantage of limiting participant burden and can generally be completed and understood by most people. Multiple questionnaire items provide greater information and have higher content validity and reliability (by reducing measurement error). Multiple questionnaire measures, though, can increase participant and staff burden and may increase study costs.

Some of the more commonly used generic HRQL instruments are the SF-36 [27], the EQ-5D [28], the Rotterdam Symptom Checklist [35], and the Memorial Symptom Assessment Scale [36]. The National Institutes of Health sponsored Patient-Reported Outcomes Measurement Information System (PROMIS) is also a good resource for HRQL and symptom assessment measures [37], and offers options to tailor the measures to meet specific investigator and study needs. Generic pediatric measures include the PedsQL [38], the KidsSCREEN [39], and PROMIS [37].

Frequently used condition-specific instruments include the Functional Assessment of Cancer Therapy (FACT) [40] and the European Organization for Research and Treatment of Cancer Quality of Life (EORTC QLQ) [41], both of which are multidimensional measures assessing the HRQL of individuals with cancer. Other condition specific instruments include the Centers for Epidemiological Studies—Depression (CES-D) [42], the Profile of Mood States (POMS) [43], and the Patient Health Questionnaire (PHQ) [22], all of which assess psychological distress and well-being; and the Barthel Index, which measures physical functioning and independence [44]. There are several good reviews of HRQL measures in the literature [45, 46], as well as on select websites [47].

Within a specific symptom or dimension of HRQL, like physical functioning, one can assess the degree to which an individual is able to perform a particular task, his or her satisfaction with the level of performance, the importance to him or her of performing the task, or the frequency with which the task is performed. Thus, the aspects of HRQL or symptoms measured in clinical trials vary depending on the specific research questions of the trial. When selecting appropriate HRQL instruments, one should consider the specific aspects of the disease/condition or symptom.

Some professional societies advocate the use of certain assessment tools or the measurement of specific sets of symptoms, so study results can be compared across trials using the same measures [48]. Consulting professional societies affiliated with certain conditions or diseases is advisable. For example, the American Society of Clinical Oncology has guidelines for the screening, assessment and care of anxiety and depressive symptoms in adults with cancer [49]. It recommended several screening instruments for ongoing use, with the hope that more uniformity in tracking these symptoms would be established in the cancer area.

Scoring of HRQL Measures

Instruments may be used to assess changes in specific dimensions or symptoms, describe the intervention and control groups at specific times, and examine the correspondence between HRQL measures and clinical or physiological measures. Plans for data analysis are tailored to the specific goals and research questions of the clinical trial.

Most established instruments have standard scoring algorithms. Adhering to these scoring methods is critical in order to interpret scores accurately and compare trial results with those from other studies. In many clinical trials, several measures are used, such that several distinct scores will be calculated (e.g., depression or pain). Some HRQL instruments may also produce an overall HRQL score in addition to separate scores for each HRQL dimension [40].

Determining the Significance of HRQL Measures

An important issue in evaluating HRQL measures is determining how to interpret score changes on a given scale. For example, how many points must one increase or decrease on a scale for that change to be considered clinically meaningful? Does the change in score reflect a small, moderate, or large improvement or deterioration in a participant’s health status? Recent years have seen an increase in research examining the question of the clinical significance of HRQL and symptom scores. Demonstrating clinical significance is also important for achieving successful product claims through regulatory agencies [50].

Information on how to interpret changes in HRQL is based on the minimal important difference [51, 52]. When the change in score is connected to clinical measures, the difference is sometimes referred to as the minimal clinically important difference. This difference is defined as the smallest score or change in scores that is perceived by participants as improving or decreasing their HRQL and which would lead a clinician to consider a change in treatment or follow-up [52, 53]. The responsiveness of a HRQL instrument (i.e., the instrument’s ability to measure change) and the minimal important difference can vary based on population and contextual characteristics. Thus, there will not be a single value for a HRQL instrument across all uses and populations, but rather a range in estimates that vary across patient populations and observational and clinical trial applications [51]. A variety of methods have been used to determine the minimal important difference. However, there is currently no consensus on which method is best and therefore multiple approaches are used [51, 53, 54]. More in-depth discussion of issues regarding the minimal important difference and HRQL and other PRO measures can be found elsewhere [51].

Utility Measures/Preference Scaling and Comparative Effectiveness Research

The types of HRQL instruments discussed in this chapter have been limited to measures that were derived using psychometric methods. These methods examine the reliability, validity, and responsiveness of instruments. Other approaches to measuring quality of life and health states are used, however, and include utility measures and preference scaling [55, 56]. Utility measures are derived from economic and decision theory, and incorporate the preferences of individuals for particular interventions and health outcomes. Utility scores reflect a person’s preferences and values for specific health states and allow morbidity and mortality changes to be combined into a single weighted measure, called quality-adjusted life years (QALYs). These measures provide a single summary score representing the net change in quality of life (the gains from the intervention minus adverse effects and burden). Utility scores are most often used in cost-effectiveness analyses that combine quality of life and duration of life [57–59]. Ratios of cost per QALY can be used to decide among competing interventions.

In utility approaches, one or more scaling methods are used to assign a numerical value from 0.0 (death) to 1.0 (full health) to indicate an individual’s quality of life. Procedures commonly used to generate utilities are lottery or standard gamble (most usually the risk of death one would be willing to take to improve a state of health) [56]. Preferences for health states are generated from the general population, clinicians, or patients using multi-attribute scales, visual analogue rating scales, time trade-off (how many months or years of life one would be willing to give up in exchange for a better health state), or other scaling methods [55, 60]. Utility measures are useful in decision-making regarding competing treatments and/or for the allocation of limited resources. They also can be used as a predictor of future health events. For example, Clarke and colleagues examined the use of index scores based on the EQ-5D, a 5-item generic health status measure, as an independent predictor of vascular events, other major complications and mortality in people with type 2 diabetes, as well as to quantify the relationship between these scores and future survival [61]. The investigators enrolled 7,348 people from Australia and New Zealand, aged 50–75, to the Fenofibrate Intervention and Event Lowering in Diabetes (FIELD) study. After adjusting for standard risk factors, a 0.1 higher index score derived from the EQ-5D was associated with an additional 7% lower risk of vascular events, a 13% lower risk of complications, and a 14% lower rate of all-cause mortality. Thus, the EQ-5D was an independent marker for mortality, future vascular events, and other complications in participants with type 2 diabetes.

In general, psychometric and utility-based methods measure different components of health. The two approaches result in different yet related, and complementary assessments of health outcomes, and both are useful in clinical research. Issues regarding the use of utility methods include the methodologies used to derive the valuation of health states; the cognitive complexity of the measurement task; potential population and contextual effects on utility values; and analysis and interpretation of utility data [55, 56]. For a further review of issues related to utility analyses/preference scaling, and the relationship between psychometric and utility-based approaches to the measurement of life quality, additional references may be consulted [55–60, 62].

References

1.

Quality of Life Assessment in Cancer Clinical Trials. Report of the Workshop on Quality of Life Research in Cancer Clinical Trials. USDHHS, Bethesda, Maryland, 1991.

2.

Spilker B (ed.). Quality of Life and Pharmacoeconomics in Clinical Trials. Philadelphia: Lippincott-Raven Publishers, 1996.

3.

Fairclough DL. Design and Analysis of Quality of Life Studies in Clinical Trials (Interdisciplinary Statistics). Boca Raton, Florida: Chapter & Hall/CRC, 2002.

4.

Fayers P, Machin D. Quality of Life: The Assessment, Analysis and Interpretation of Patient-Reported Outcomes. Chichester: John Wiley & Sons, Ltd., 2007.CrossRef

5.

Snyder CF, Jensen RE, Segal JB, Wu AW. Patient-Reported Outcomes (PROs): Putting the patient perspective in patient-centered outcomes research. Med Care 2013;51:S73-S79.CrossRef

6.

Calvert M, Blazeby J, Altman DG, Revicki DA, et al. for the CONSORT PRO Group. Reporting of Patient-Reported Outcomes in randomized trials. The CONSORT PRO extension. JAMA 2013;309:814-822.

7.

Calvert M, Brundage M, Jacobsen PB, et al. The CONSORT Patient-Reported Outcome (PRO) extension: Implications for clinical trials and practice. Health Qual Life Outcomes 2013;11:184-190.CrossRef

8.

Food and Drug Administration. Guidance for Industry, Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Silver Spring, MD: Office of Communications, Division of Drug Information Center for Drug Evaluation and Research Food and Drug Administration, 2009.

9.

Berzon R, Hays RD, Shumaker SA. International use, application and performance of health-related quality of life instruments. Qual Life Res 1993;2:367-368.CrossRef

10.

Mossey JM, Shapiro E. Self-rated health: A predictor of mortality among the elderly. Am J Public Health 1982;72:800-808.CrossRef

11.

Kaplan GA, Camacho T. Perceived health and mortality: A nine-year follow-up of the human population laboratory cohort. Am J Epidemiol 1983;117:292-304.

12.

Oei TP, McAlinden NM, Cruwys T. Exploring mechanisms of change: The relationships between cognitions, symptoms, and quality of life over the course of group cognitive-behaviour therapy. J Affect Disord 2014;168:C72-C77.CrossRef

13.

Schron EB, Shumaker SA. The integration of health quality of life in clinical research: Experiences from cardiovascular clinical trials. Prog Cardiovasc Nurs 1992;7:21-28.

14.

Sugarbaker PH, Barofsky I, Rosenberg SA, Gianola FJ. Quality of life assessment of patients in extremity sarcoma clinical trials. Surgery 1982;91:17-23.

15.

The Women’s Health Initiative Study Group. Design of the Women’s Health Initiative Clinical Trial and Observational Study. Control Clin Trials 1998;19:61-109.CrossRef

16.

Writing Group for the Women’s Health Initiative Investigators. Risks and benefits of estrogen plus progestin in healthy postmenopausal women. JAMA 2002;288:321-333.CrossRef

17.

The Women’s Health Initiative Steering Committee. Effects of conjugated equine estrogen in postmenopausal women with hysterectomy. JAMA 2004;291:1701-1712.CrossRef

18.

Naughton MJ, Jones AS, Shumaker SA. When practices, promises, profits, and policies outpace hard evidence: The post-menopausal hormone debate. J Soc Issues 2005;61:159-179.CrossRef

19.

Hays J, Ockene JK, Brunner RL, et al for the Women’s Health Initiative Investigators. Effects of estrogen plus progestin on health-related quality of life. N Engl J Med 2003;348:1839-1854.

20.

Brunner RL, Gass M, Aragaki A, et al for the Women’s Health Initiative Investigators. Effects of conjugated equine estrogen on health-related quality of life in postmenopausal women with hysterectomy: results from the Women’s Health Initiative Randomized Clinical Trial. Arch Intern Med 2005;165:1976-1986.

21.

Richardson LP, Ludman E, McCauley E, et al. Collaborative care for adolescents with depression in primary care: a randomized clinical trial. JAMA 2014;312:809-816.CrossRef

22.

Kroenke K, Spitzer RL, Williams JB. The PHQ-9: Validity of a brief depression severity measure. J Gen Intern Med 2001;16:606–613.CrossRef

23.

Poznanski E, Mokros H. Children’s Depression Rating Scale-Revised (CDRS-R). Los Angeles, CA: WPS, 1996.

24.

Bird HR, Andrews H, Schwab-Stone M, et al. Global measures of impairment for epidemiologic and clinical use with children and adolescents. Int J Methods Psychiatr Res 1996;6:295-307.CrossRef

25.

Brittenden J, Cotton SE, Elders A, et al. A randomized trial comparing treatments for varicose veins. N Engl J Med 2014;371:1218-1227.CrossRef

26.

Garratt AM, Macdonald LM, Ruta DA, et al. Towards measurement of outcome for patients with varicose veins. Qual Health Care 1993;2:5-10.CrossRef

27.

Ware JE Jr, Sherbourne CD. The MOS 36-item short-form health survey (SF-36). 1. Conceptual framework and item selection. Med Care 1992;30:473-483.CrossRef

28.

Rabin R, de Charro F. EQ-5D: a measure of health status from the EuroQol Group. Ann Med 2001;33:337-343.CrossRef

29.

Shumaker SA, Anderson R, Berzon R, Hayes R (eds.) International use, application and performance of health-related quality of life measures. Qual Life Res 1993;2:367-368.

30.

Cella DF, Wiklund I, Shumaker SA, et al. Integrating health-related quality of life into cross-national clinical trials. Qual Life Res 1993;2:433-440.CrossRef

31.

Naughton MJ, Shumaker SA, Anderson R, Czajkowski S. Psychological Aspects of Health-Related Quality of Life Measurement: Tests and Scales. In Spilker B (ed.), Quality of Life and Pharmacoeconomics in Clinical Trials. Philadelphia: Lippincott-Raven Publishers, 1996.

32.

Reeve BB, Wyrwich KW, Wu AW, et al. ISOQOL recommends minimum standards for patient-reported outcome measures used in patient-centered outcomes and comparative effectiveness research. Qual Life Res 2013;22:1889-1905.CrossRef

33.

Wu AW, Bradford AN, Velanovich V, et al. Clinician’s checklist for reading and using an article about patient-reported outcomes. Mayo Clin Proc 2014;89:653-661.CrossRef

34.

Hays RD, Revicki DA. Reliability and validity (including responsiveness). In: Fayers P, Hays R (eds.). Assessing quality of life in clinical trials (2 nd edition). New York: Oxford University Press, 2005.

35.

Hardy JR, Edmonds P, Turner R, et al. The use of the Rotterdam Symptom Checklist in palliative care. J Pain Symptom Manage 1999;18:79–84. CrossRef

36.

Portenoy RK, Thaler HT, Kornblith AB, et al. The Memorial Symptom Assessment Scale: An instrument for the evaluation of symptom prevalence, characteristics and distress. Eur J Cancer 1994;30:1326-1336.CrossRef

37.

National Institutes of Health, Patient-Reported Outcomes Measurement Information Systems website: www.​nihPROMIS.​org.

38.

Varni JW, Seid M, Kurtin PS. PedsQLTM 4.0: Reliability and validity of The Pediatric Quality of Life Inventory™ version 4.0 Generic Core Scales in healthy and patient populations. Med Care 2001;39:800–812.CrossRef

39.

Ravens-Sieberer U, Gosch A, Rajmil L, et al. KIDSCREEN-52 quality-of-life measure for children and adolescents. Expert Rev Pharmacoecon Outcomes Res 2005;5:353–364.CrossRef

40.

Cella DF, Tulsky DS, Gray G, et al. The functional assessment of cancer therapy scale: development and validation of the general measure. J Clin Oncol 1993;11:570-579.

41.

Aaronson NK, Ahmedzai S, Bergman B, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: A quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 1993;85:365-376.CrossRef

42.

Radloff LS. The CES-D Scale: A self-report depression scale for research in the general population. Appl Psych Meas 1977;1:385-401.CrossRef

43.

McNair DM, Loor M, Droppleman LF. Profile of Mood States. San Diego, CA: Educational and Industrial Testing Service, 1981.

44.

Collin C, Wade DT, Davies S, Horne V. The Barthel ADL Index: A reliability study. Int Disabil Stud 1988;10:61-63.CrossRef

45.

Wilkin D, Hallam L, Doggett M. Measures of Need and Outcome for Primary Health Care. New York, NY: Oxford Medical Publications, 1992.

46.

McDowell I. Measuring Health: A Guide to Rating Scales and Questionnaires. New York: Oxford University Press, 2006.CrossRef

47.

Website for the International Society for Quality of Life Research (ISOQOL): www.​ISOQOL.​org

48.

Reeve BB, Mitchell SA, Dueck A, et al. Recommended patient-reported core set of symptoms to measure in adult cancer treatment trials. J Natl Cancer Inst 2014;106.

49.

Anderson BL, DeRubeis RJ, Berman BS. Screening, assessment, and care of anxiety and depressive symptoms in adults with cancer: An American Society of Clinical Oncology guideline adaptation. J Clin Oncol 2014;32:1605-1619.CrossRef

50.

Revicki DA, Osoba D, Fairclough D, et al. Recommendations on health-related quality of life research to support labeling and promotional claims in the United States. Qual Life Res 2000;9:887-900.CrossRef

51.

Revicki D, Hays RD, Cella D, Sloan J. Recommended methods for determining responsiveness and minimally important differences for patient-reported outcomes. J Clin Epidemiol 2008;61:102-109.CrossRef

52.

Jaeschke R, Singer J, Guyatt G. Measurement of health status. Ascertaining the minimal clinically important difference. Control Clin Trials 1991;12:S266-S269.CrossRef

53.

Guyatt G, Walter S, Norman G. Measuring change over time: Assessing the usefulness of evaluative instruments. J Chronic Dis 1987;40:171-178.CrossRef

54.

Guyatt G, Osoba D, Wu AW. Methods to explain the clinical significance of health status measures. Mayo Clin Proc 2002;77:371-383.CrossRef

55.

Weinstein MC, Torrance G, McGuire A. QALYs: The basics. Value Health 2009;12:S5-S9.CrossRef

56.

Revicki DA, Kaplan RM. Relationship between psychometric and utility-based approaches to the measurement of health-related quality of life. Qual Life Res 1993;2;477-487.CrossRef

57.

Neumann PJ, Auerbach HR, Cohen JT, Greenberg D. “Low-value” services in value-based insurance design. Am J Manag Care 2010;16:280-286.

58.

Greenberg D, Rosen AB, Wacht O, et al. A bibliometric review of cost-effectiveness analysis in the economic and medical literature, 1976-2007. Medical Decis Making 2010;30:320-327.CrossRef

59.

Greenberg D, Earle CC, Fang CH, et al. When is cancer care cost-effective? A systematic overview of cost-utility studies in oncology. J Natl Cancer Inst 2010;102:82-88.CrossRef

60.

Kaplan RM, Feeny D, Revicki DA. Methods for assessing relative importance in preference based outcome measures. Qual Life Res 1993;2:467-475.CrossRef

61.

Clark PM, Hayes AJ, Glasziou PG, et al. Using the EQ-5D index score as a predictor of outcomes in patients with type 2 diabetes. Med Care 2009;47:61-68.CrossRef

62.

Moving the QALY forward: Building a pragmatic road. Value Health 2009;12:S1-S39

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_14

14. Participant Adherence

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

The terms compliance and adherence are often used interchangeably. In 1979, Haynes et al., defined compliance as “the extent to which a person’s behavior (in terms of taking medications, following diets or executing lifestyle changes) coincides with medical or health advice” [1]. More recently, an international consensus statement crafted by the World Health Organization and the International Society of Pharmacoeconomics and Outcomes Research defined medication adherence as “the extent to which a patient acts in accordance with the prescribed interval and dose of a dosing regime” [2]. Patient adherence is also recently reviewed in additional articles [3–5]. The term adherence implies active participant involvement in the decision to take a medication, use a device or engage in a behavior change, and is the term used in this book. In this chapter, we primarily refer to drug adherence but the concepts apply generally. In a drug trial, adherence typically refers to ingestion of predetermined amount of drug such as 80% of the protocol dose. This dose will depend on the nature and half-life of the drug being evaluated. Persistence is a related term that refers to remaining on a medical treatment for a specified period of time, regardless of the proportion of the doses taken. Distinguishing adherence vs. persistence is important since the metrics are different as well as the implications for trial interpretation [6, 7].

Medication adherence is a major challenge for patients, the consequences of which affect clinical practitioners and investigators alike. As many as one-third of all prescriptions are reportedly never filled and, among those filled, a large proportion is associated with incorrect administration [8]. Even among patients who receive medication at no cost from their health plans, rates of nonadherence reach nearly 40% [9]. Nonadherence has been estimated to cause nearly 125,000 deaths per year in the U.S. and has been linked to 10% of hospital admissions and 23% of nursing home admissions [8]. Poor medication adherence in the U.S. has a resultant cost of approximately $100 billion a year [10].

This chapter discusses what can be done before enrollment to reduce future adherence problems, how to maintain good adherence during a trial, how to monitor adherence, and how to address low adherence. In the monitoring section, we also discuss visit adherence. Readers interested in a more detailed discussion of various adherence issues are referred to an excellent text [11] and a review of the literature [10].

Fundamental Point

Many potential adherence problems can be prevented or minimized before participant enrollment. Once a participant is enrolled, measures to monitor and enhance participant adherence are essential.

Definitions

Since reduced adherence with the intervention has a major impact on the power of a trial, realistic estimates of cross-overs, drop-ins and drop-outs must be used in calculating the sample size. Underestimates are common and lead to underpowered trials that fail to test the trial hypotheses properly. See Chap. 8 for further discussion of the sample size implications of low adherence.

A cross-over is a participant who, although assigned to the control group, follows the intervention regimen; or a participant who, assigned to an intervention group, follows either the control regimen or the regimen of another intervention group when more than one intervention is being evaluated. A drop-in is a special kind of cross-over. In particular, the drop-in is unidirectional, referring to a person who was assigned to the control group but begins following the intervention regimen. A drop-out is also unidirectional and refers to a person assigned to an intervention group who fails to adhere to the intervention regimen. If the control group is either on placebo or on no standard intervention or therapy, as is the case in many superiority trials, then the drop-out is equivalent to a cross-over. However, if the control group is assigned to an alternative therapy, as is the case in noninferiority or comparative effectiveness trials, then a drop-out from an intervention group does not necessarily begin following the control regimen. Moreover, in this circumstance, there may also be a drop-out from the control group. Participants who are unwilling or unable to return for follow-up visits represent another type of low adherence, sometimes also referred to as drop-outs. Because of the possible confusion in meanings, this text will limit the term drop-out to mean the previously defined adherence-related behavior. Those who stop participating in a trial and have no further follow-up will be referred to as withdrawals. Importantly, participants who stop taking their study medication but continue their scheduled follow-up are not withdrawals.

Medication Adherence

The optimal trial from an adherence point of view is one in which the investigator has total control over the participant, the administration of the intervention regimen, which may be a drug, diet, exercise, or other intervention, and follow-up. That situation can only realistically be achieved in animal experiments. Any clinical trial, which, according to the definition in this text, must involve human beings, will have variability in adherence with the intervention and the study procedures. There are several reasons for low adherence. Life events such as illnesses, loss of employment, or divorce are factors associated with reduced adherence. In addition, participants may not perceive any treatment benefit, they may be unwilling to change their behaviors, they are forgetful, may lack family support, or ultimately they may change their minds regarding trial participation. Another reason for low adherence is adverse effects to the medication or intervention. Therefore, even studies of a one-time intervention such as surgery or a single medication dose can suffer from nonadherence. In fact, some surgical procedures can be declined or even be reversed. In addition, the participant’s condition may deteriorate, and thus require termination of the study treatment or a switch from control to intervention. In a clinical trial in stable coronary disease, participants were randomized to percutaneous coronary intervention (PCI) plus optimal medical therapy compared to optimal medical therapy alone [12]. Among the 1,149 participants in the PCI group, 46 never underwent the procedure and another 27 had lesions that could not be opened. During a median follow-up of 4.6 years, 32.6% of the 1,138 participants in the optimal medical therapy alone group had revascularization. The trial showed no difference for the primary outcome of all-cause mortality or non-fatal myocardial infarction. However, it is difficult to determine how much the cross-overs influenced the overall finding. Moreover, such a trial can be considered to be testing the initial intervention strategy, with recognition that those who fail medical therapy will often have revascularization.

Most of the available information on adherence is obtained from the clinical therapeutic encounter rather than from the clinical trial setting. Although the differences between patients and volunteer clinical trial participants are important, and agreement to participate tends to minimize low adherence rates in trials, the basic principles observed in practice settings apply to research as well. In clinical trial databases, it has been shown that adherence to intervention, and even adherence to placebo, is independently associated with improved survival [13]. This observation suggests that adherent behavior may have benefits, or at least that adherence is associated with unmeasured factors related to better outcome. The results of a trial can be affected by low adherence to the intervention leading to an underestimation of possible therapeutic as well as potential toxic effects, and can undermine even a properly designed study. Data from a meta-analysis suggest that the difference in health benefits between high and low adherence has been shown to reach 26% [14]. Given the intention-to-treat principle of analysis (see Chap. 18), in order to maintain equivalent power, a 20% reduction in drug adherence may result in the need for a greater than 50% increase in sample size and 30% reduction will require doubling of the study cohort (see Chap. 8). Poor adherence is especially problematic in non-inferiority trials, where it will bias the results toward no difference between the intervention and control groups and decrease the reliability of the observed results.

Considerations Before Participant Enrollment

There are three major considerations affecting adherence to the study medications that investigators and sponsors ought to consider during the planning phase. First, efforts should be made to limit the impact of design features that may adversely influence the level of adherence. Second, steps should be taken to avoid enrollment of study participants who are likely to have low adherence while not being so restrictive as to decrease the generalizability of the results. Third, the research setting influences participant adherence over the long term. It is important to have realistic estimates of the adherence level during a trial, so that proper upward adjustments of the sample size can be made during the planning phase. Even practice-based trials that attempt to mimic real-life situations need to consider adherence in their designs.

Design Factors

Four study design factors can influence adherence—study duration, setting, simplicity of the regimen and the use of a run-in period.

Study duration influences adherence. The shorter the trial, the more likely participants are to adhere with the intervention regimen. A study in which intervention is started and completed in 1 day (such as fibrinolytic therapy for acute myocardial infarction or stroke) or during a hospital stay has great advantages over longer trials with regards to adherence. Trials in which the participants are under supervision, such as hospital-based ones, tend to have fewer problems with low adherence [15]. It is important to be mindful of the fact that there is a difference between special hospital wards and clinics with trained staff who are familiar with research requirements and general medical or surgical wards and clinics, where research experience might not be common or protocol requirements might not be appreciated. Regular hospital staff have many other duties which compete for their attention, and they perhaps have little understanding of the need for precisely following a study protocol and the importance of good adherence. On the other hand, if the intent is to assess how an intervention may perform in general practice, the regular clinical setting may have advantages.

The setting of the trial is also important. Whenever the study involves participants who will be living at home, the chances for low adherence increase. Studies of interventions that require changing a habit are particularly susceptible to this hazard. A challenge is dietary studies. A participant may need special meals, which are different from those consumed by other family members. It may be difficult to adhere when having meals outside the home. Multiple educational sessions and preparation of meals by the investigator team may be necessary. Family involvement is essential, especially if the participant is not the usual meal preparer [16, 17]. In studies, when the participants’ sources of food come only from the hospital kitchen or are supplied by the trial through a special commissary [18], participants are more likely to adhere with the study regimen than when they buy and cook their own food. This may also allow for blinded design.

The treatment regimen is an important factor and simplicity facilitates adherence. Single daily dose drug regimens are preferable to multiple daily dose regimens. Despite a simple regimen, 10–40% of participants have imperfect dosing [10]. A review of 76 trials, in which electronic monitors were used, showed that adherence is inversely proportional to the frequency of dosing [19]. Patients on a four-times-a-day regimen achieved on-schedule average adherence rates of about 50%. Adhering to multiple study interventions simultaneously poses special difficulties. For example, behavior changes such as quitting smoking, losing weight and reducing the intake of saturated fat at the same time requires highly motivated participants. Unlike on-going interventions such as drugs, diet, or exercise, trials of surgery and vaccination generally have the design advantage, with few exceptions, of enforcing adherence with the intervention.

Where feasible, a run-in period before actual randomization may be considered to identify those potential participants who are likely to become poor adherers and thereby exclude them from long-term trials. During the run-in, potential participants may be given either active medication or placebo over several weeks or months. An active run-in also allows identification of potential participants who do not have a favorable response to treatment on a biomarker or who develop side effects prior to randomization [20]. However, this design may be less informative about the effects of a treatment in practice, where the question for the clinician is whether or not to use it, not whether to use it after determining tolerability. A placebo run-in allows a determination of the potential participant’s willingness to comply with the study intervention. Run-in phases were common already in 2001 when a literature search resulted in more than 1,100 examples of trials in which run-in phases were used [21]. This approach was successfully employed in a trial of aspirin and beta-carotene in US physicians [22]. By excluding physicians who reported taking less than 50% of the study pills, the investigators were able to randomize excellent adherers. After 5 years of follow-up, over 90% of those allocated to aspirin reported still taking the pills. An additional goal of the run-in is to stabilize the potential trial participants on specific treatment regimens or to wash-out the effects of discontinued medications. Though the number of participants eliminated by the run-in period is usually small (5–10%), it can be important as even this level of low adherence affects study power. A potential disadvantage of a run-in is that participants may notice a change in their medication following randomization thereby influencing the blinding of the assignment. It also delays entry of participants into a trial, perhaps by a few weeks.

Berger et al. [21] raised the issue of external validity of the findings of trials that excluded potential poor adherers during a run-in phase. Can the results from trials with run-in selection of participants reasonably be fully extrapolated to all those patients meeting the trial eligibility criteria? The question about the generalizability of trial findings can be raised regarding the PARADIGM HF trial of patients with heart failure [23]. The trial had two consecutive run-in phases—the first over 2 weeks with enalapril and a second over 4 weeks with a valsartan-neprilysin inhibitor. A large number (20%) of eligible participants were excluded mostly due to adverse events (see Chap. 4). As always, whether to use a run-in depends on the question being posed. Does the trial have many exclusion criteria (a so-called efficacy trial) or few exclusions (a pragmatic or effectiveness trial)? Stated differently: What is the effect of the intervention in optimal circumstances? Or, what is the effect when, as is common in clinical settings, a large number of people fail to adhere to prescribed medication? Both are valid questions, but in the latter situation, as noted earlier, a larger sample size will be required. Lee et al. [24] compared the effect size in 43 clinical trials of selective serotonin uptake inhibitors in patients with depression that included a placebo run-in and those that did not. They found no statistically significant difference in the results.

In another approach, the investigator may instruct prospective participants to refrain from taking the active agent and then evaluate how well his request was followed. In the Aspirin Myocardial Infarction Study, for instance, urinary salicylates were monitored before enrollment, and very few participants were excluded because of a positive urine test.

Participant Factors

An important factor in preventing adherence problems is the selection of appropriate participants. Ideally, only those people likely to follow the study protocol should be enrolled. In the ACCORD trial, the screenees’ willingness to test blood sugars frequently was taken as a measure of commitment to participate [25]. This may, however, influence the ability to generalize the findings (see Chap. 4 for a discussion of generalization). Several articles have reported that there is convincing evidence that nonadherers are substantially different from adherers in ways that are quite independent of the effects of the treatment prescribed [10, 26].

Exclusion of individuals who are unlikely to be good participants is usually advisable unless the trial is aimed at those individuals. A number of participant-related factors have been shown to negatively affect adherence [11]. People with cognitive impairment or low literacy are likely to have more problems with adherence [27]. It is obviously important that participants understand instructions and follow through on these. A related issue is low self-efficacy, which relates to a person’s ability to follow through with recommendations or make behavior change a permanent feature of his/her life [28]. It is important that participants believe in their own ability to do so. Positive health beliefs and attitudes (i.e., less fear of adverse effects) are also helpful. Mental health issues represent other predictors of poor adherence. Meta-analyses have shown that depressed patients have a 2 to 3-fold higher rate of nonadherence compared to those who were not depressed [26]. However, a successful behavioral weight-loss intervention in persons with serious mental illness was recently reported [29]. A combination of group and individual weight-management sessions and group exercise sessions over 18 months led to a statistically significant weight reduction in the intervention group compared to the control group. The connection with anxiety is less clear. A person’s personality or characteristic traits may also be a factor to consider. Conscientiousness predicts good adherence and hostility poor adherence [26]. Similarly, those with a known history of missed appointments or adherence problems might be considered for exclusion. Logistic factors may also influence adherence, for example, persons who live too far away, or those who are likely to move before the scheduled termination of the trial. Traveling long distances may be an undue burden on disabled people. Those with concomitant disease may be less adherent because they have other medicines to take or are participating in other trials. Furthermore, it is important to be aware of the potential for contamination of the study results by these other medicines or trials. When applicable, the factors discussed above should be incorporated in the study exclusion criteria. These factors are difficult to define, so the final decision often is left to the discretion of the study investigator.

Financial and other incentives to motivate adherence are sometimes offered. These have been reported to improve adherence [30–32]. A concern is that financial incentives, if excessive, may lead to enrollment of participants more interested in the payment than in supporting science. As discussed in Chap. 2, Institutional Review Boards and others would view this practice as unethical.

An informed participant appears to be a better adherer. Proper education of the participant and the participant’s family or caregiver is thought to be the most positive factor to high adherence, but the scientific evidence is not conclusive [33]. However, for ethical concerns, the participant (or, in special circumstances, his guardian) in any trial should be clearly instructed about the study and told what is expected from him. He should have proper insight into his illness and be given a full disclosure of the potential effects—good and bad—of the study medication. Sufficient time should be spent with a candidate and he should be encouraged to consult with his family or private physician. A brochure with information concerning the study is often helpful. As an example, the pamphlet used in the NIH-sponsored Women’s Health Initiative trial is shown in Box 14.1. Many clinical trials develop websites with educational material directed at physicians and potential participants.

Box 14.1: Women’s Health Initiative Brochure

What is the Women’s Health Initiative?

The Women’s Health Initiative (WHI) is a major research study of women and their health. It will help decide how diet, hormone therapy, and calcium and vitamin D might prevent heart disease, cancer, and bone fractures. This is the first such study to examine the health of a very large number of women over a long period of time. About 160,000 women of various racial and ethnic backgrounds from 45 communities across the United States will take part in the study.

Who can join the WHI?

You may be able to join if you are:

a woman 50–79 years old

past menopause or the “change of life”

planning to live in the same area for at least 3 years

Why is this study important?

Few studies have focused on health concerns unique to women. Being a part of this important project will help you learn more about your own health. You will also help doctors develop better ways to treat all women. This study may help us learn how to prevent the major causes of death and poor health in women: heart disease, cancer, and bone fractures.

What will I be asked to do?

If you agree to join us, you will be scheduled for several study visits. These visits will include questions on your medical history and general health habits, a brief physical exam, and some blood tests. Based on your result, you may be able to join at least one of the following programs.

Dietary: In this program you are asked to follow either your usual eating pattern or a low-fat eating program.

Hormone: In this program you are asked to either take hormone pills or inactive pills (placebos). If you are on hormones now, you would need to talk with your doctor about joining this program.

Calcium and Vitamin D: In this program you are asked to either take calcium and vitamin D or inactive pills. Only women in the Dietary or Hormone programs may join this program.

Health Tracking: If you are not able to join the other programs, your medical history and health habits will be followed during the study.

How long will the study last?

You will be in the study for a total of 8–12 years, depending on what year you enter the study. This period of time is necessary to study the long-term effects of the programs.

How will I benefit?

If you join the study, your health will be followed by the staff at our center. Certain routine tests will be provided, although these are not meant to replace your usual health care. Depending on which program you join, you may receive other health-care services, such as study pills and dietary sessions. You will not have to pay for any study visits, tests, or pills.

You will also have the personal satisfaction of knowing that results from the WHI may help improve your health and the health of women for generations to come.

Social support and involvement have emerged as major determinants of adherence [34]. Thus, it is recommended that family members, significant others or friends be informed about the trial and its expectations at the same time as the potential participant. After all, a large proportion of participants join trials at the support of family and friends [35]. The support they can offer in terms of assistance, encouragement and supervision can be very valuable. Practical support is most consistently associated with greater medication adherence [34]. Support is especially important in trials of lifestyle interventions. For example, cooking classes for spouses as well as participants have been very effective in dietary intervention trials [16, 17].

Major factors associated with low adherence are summarized in Table 14.1, listed in alphabetical order. Most of them are, as would be expected, the opposite of factors associated with high adherence. The consensus is that older persons generally show higher rates of adherence.

Table 14.1

Factors associated with low adherence

Cognitive impairments

Complexity of drug regimen

Concomitant diseases

Hostile personality

Lack of information and inadequate instructions

Lack of social support

Logistic factors

Low self-efficacy

Low literacy

Mental health issues, primarily depression

Negative health beliefs

Unsatisfying participant-investigator relationship

Adapted from Williams, Haskard-Zolnierik & DeMatteo [26]

Studies have shown that patients’ recall of medical topics discussed with providers is poor and between 40–80% is forgotten immediately [36] while up to half of the information retained by patients is incorrect [37]. The “teach-back” method can be used to improve knowledge retention among patients [38] and confirm that patients understand what they have been told. If the investigator says to the study participant that he has high blood pressure that needs treatment, the participant would say, “I have high blood pressure that needs treatment.” When told to take one pill every morning until the next clinic visit, the participant would repeat, “I should take one pill every morning until I return for my next clinic visit.” When the participants accurately explain in their own words what they have been told their understanding is confirmed. A recent study of hospitalized patients with heart failure showed a trend toward lower readmissions for heart failure among those with more correct answers to teach-back questions [39].

Maintaining Good Participant Adherence

The foundation for high adherence during a trial is a well-functioning setting with committed clinic staff (Table 14.2). Establishing a positive research setting at the first contact with future participants is a worthwhile investment for the simple reason that satisfied participants are better adherers. A warm and friendly relationship between participants and staff established during the recruitment phase should be nurtured. This approach covers the spectrum from trusting interactions, adequate time to discuss complaints, demonstrating sincere concern and empathy, when appropriate, convenient clinic environment, short waiting times, etc. “Bonding” between the participant and clinical trials staff members is a recognized and powerful force in maintaining good adherence. The clinic visits should be pleasant and participants should be encouraged to contact staff between scheduled visits if they have questions or concern. Close personal contact is key. Clinic staff may employ various means of engagement, including phone calls, mail and e-mail. Sending cards on special occasions such as birthdays and holidays is a helpful gesture. Visiting the participant if he is hospitalized demonstrates concern. It is helpful to investigators and staff to make notes of what participants tell them about their families, hobbies and work so that in subsequent visits they can follow-up and show interest and involvement. Other valued factors are free parking and, for working participants, opportunities for evening or weekend visits. For participants with difficulties attending clinic visits, home visits by staff could be attempted. Continuity of care is ranked as a high priority by participants. Continued family involvement is especially important during the follow-up phase.

Table 14.2

Factors in improving likelihood of medication adherence in clinical trials

Approach

Activity

Trial design

Simple schedule (once or twice daily dosing) that fits into daily routine [40]

Relationships and communication

Enhanced relationship of study coordinator with participant with regular communication [41, 42]

Passive monitoring

Electronic monitoring tools

Education

Medication usage skills [33]

Reinforce beliefs

Association activities using medication-outcome relationships

Reminders

Alarms (e.g., set watch or cell phone reminders to medication schedule) and associations (e.g., put medication beside toothbrush or use a behavior trigger)

Incentives

Monetary or other rewards

During a study, it is important to keep the participants informed about relevant published findings from related trials. They should also be reminded, when applicable, that a data monitoring committee is reviewing the trial data for safety and efficacy throughout the duration of the trial which should be described to them. Brief communications from this committee assuring the participants that no safety concerns have been noted, can also be helpful.

The use of various types of general reminders can also reduce the risk of low adherence. Clinic staff should typically remind the participant of upcoming clinic visits or study procedures. Sending out postcards, calling, e-mailing or text messaging a few days before a scheduled visit can help. Paper-based reminders seem to be most effective [43]. A telephone call though has the obvious advantage that immediate feedback is obtained and a visit can be rescheduled if necessary—a process that reduces the number of participants who fail to keep appointments. Telephoning also helps to identify a participant who is ambivalent regarding his continued participation or who has suffered a study event. To preclude the clinic staff’s imposing on a participant, it helps to ask in advance if the participant objects to being called frequently. Asking a participant about the best time to contact him is usually appreciated. Reminders can then be adjusted to his particular situation. In cases where participants are reluctant to come to clinics, more than one staff person might contact the participant. For example, the physician investigator could have more influence with the participant than the staff member who usually schedules visits. In summary, the quantity and quality of interaction between an investigator and the participant can positively influence adherence.

For drug studies, special pill organizers help the participant keep track of when to take the medication. These organizers allow participants to divide, by day and time of day, all medications prescribed during a 7-day period. If the participant cannot remember whether he took the morning dose, he can easily find out by checking the compartment of the pill box for that day. Special reminders such as noticeable stickers in the bathroom or on the refrigerator door or on watches have been used. Placing the pill bottles (child proof as appropriate) on the kitchen table or nightstand with the tooth brushes are other suggestions for participants.

The effectiveness of electronic reminders to improve medication and visit adherence in clinical trials has received much attention recently [43–45]. The rationale for their use is that one of the most commonly reported reasons for not being adherent is forgetfulness. Additionally, these simple interventions are less expensive and time-consuming than personal attention by investigators.

Vervloet et al. [46] conducted a comprehensive literature review and identified 13 studies meeting their inclusion criteria. Three types of automatic electronic reminders were considered—1) short reminder messages sent to the participant’s mobile phone, 2) audiovisual reminders were sent through a specific electronic reminder device at predetermined times, and 3) text messages sent to a participant’s pager to alert them to take the study medication. The main conditions studied were HIV, glaucoma, hypertension, and asthma. The review showed evidence for short-term (<6 months) effectiveness in 8 of the 13 studies, especially those of short messages sent through mobile phones. The effectiveness beyond 6 months was noted in only one of those studies. A potential weakness of these studies was that reminders were sent to all participants regardless of whether they took their study medication. This could have a negative impact. One of the studies reported that weekly reminders were more effective than daily reminders. Tailored messages may be more effective than standard text. This evolving technology has also been evaluated in clinical practice with mixed results [47].

Interventions to maintain good adherence for lifestyle changes can be very challenging. Most people have good intentions that can wane with time unless there is reinforcement. A special brochure, which contains essential information and reminders, may be helpful in maintaining good participant adherence (Box 14.2). The telephone number where the investigator or staff can be reached should be included in the brochure.

Box 14.2: Aspirin Myocardial Infarction Study Brochure

Text of brochure used to promote participant adherence in the Aspirin Myocardial Infarction Study. DHEW Publication No. (NIH) 76-1080.

1.

Your Participation in the Aspirin Myocardial Infarction Study (AMIS) is Appreciated! AMIS, a collaborative study supported by the National Heart and Lung Institute, is being undertaken at 30 clinics throughout the United States and involves over 4,000 volunteers. As you know, this study is trying to determine whether aspirin will decrease the risk of recurrent heart attacks. It is hoped that you will personally benefit from your participation in the study and that many other people with coronary heart disease may also greatly benefit from your contribution.

2.

Your Full Cooperation is Very Important to the Study. We hope that you will follow all clinic recommendations contained in this brochure, so that working together, we may obtain the most accurate results. If anything is not clear, please ask your AMIS Clinic Physician or Coordinator to clarify it for you. Do not hesitate to ask questions.

3.

Keep Appointments. The periodic follow-up examinations are very important. If you are not able to keep a scheduled appointment, call the Clinic Coordinator as soon as possible and make a new appointment. It is also important that the dietary instructions you have received be followed carefully on the day the blood samples are drawn. At the annual visit, you must be fasting. At the non-annual visits you are allowed to have a fat-free diet. Follow the directions on your Dietary Instruction Sheet. Don’t forget to take your study medication as usual on the day of your visit.

4.

Change in Residence. If you are moving within the Clinic area, please let the Clinic Coordinator know of your change of address and telephone number as soon as possible. If you are moving away from the Clinic area, every effort will be made to arrange for continued follow-up here or at another participating AMIS clinic.

5.

Long Vacations. If you are planning to leave your Clinic area for an extended period of time, let the Clinic Coordinator know so that you can be provided with sufficient study medication. Also give the Clinic Coordinator your address and telephone number so that you can be reached if necessary.

6.

New Drugs. During your participation in AMIS you have agreed not to use non-study prescribed aspirin or aspirin-containing drugs. Therefore, please call the Clinic Coordinator before starting any new drug as it might interfere with study results. At least 400 drugs contain aspirin, among them cold and cough medicines, pain relievers, ointments and salves, as well as many prescribed drugs. Many of these medications may not be labeled as to whether or not they contain aspirin or aspirin-related components. To be sure, give the Clinic Coordinator a call.

7.

Aspirin-Free Medication. Your Clinic will give you aspirin-free medication for headaches, other pains and fever at no cost. The following two types may be provided.

Acetaminophen. The effects of this drug on headaches, pain and fever resemble those of aspirin. The recommended dose is 1–2 tablets every 6 h as needed or as recommended by your Clinic Physician.

Propoxyphene hydrochloride. The drug has an aspirin-like effect on pain only and cannot be used for the control of fever. The recommended dose is 1–2 capsules every 6 h as needed or as recommended by your Clinic Physician.

8.

Study Medication. You will be receiving study medication from your Clinic. You are to take two capsules each day unless prescribed otherwise. Should you forget to take your morning capsule, take it later during the day. Should you forget the evening dose, you can take it at bedtime with a glass of water or milk. The general rule is: Do not take more than 2 capsules a day.

9.

Under Certain Circumstances It Will Be Necessary to Stop Taking the Study Medication:

If you are hospitalized, stop taking the medication for the period of time you are in the hospital. Let the Clinic Coordinator know. After you leave the hospital, a schedule will be established for resuming medication, if it is appropriate to do so.

If you are scheduled for surgery, we recommend that you stop taking your study medication 7 days prior to the day of the operation. This is because aspirin may, on rare occasions, lead to increased bleeding during surgery. In case you learn of the plans for surgery less than 7 days before it is scheduled, we recommend that you stop the study medication as soon as possible. And again, please let the Clinic Coordinator know. After you leave the hospital, a schedule will be established for resuming medication, if it is appropriate to do so.

If you are prescribed non-study aspirin or drugs containing aspirin by your private physician, stop taking the study medication. Study medication will be resumed when these drugs are discontinued. Let the Clinic Coordinator know.

If you are prescribed anti-coagulants (blood thinners), discontinue study medication and let your Clinic Coordinator know.

If you have any adverse side effects which you think might be due to the study medication, stop taking it and call the Clinic Coordinator immediately.

10.

Study-Related Problems or Questions. Should you, your spouse, or anyone in your family have any questions about your participation in AMIS, your Clinic will be happy to answer them. The clinic would like for you or anyone in your family to call if you have any side effects that you suspect are caused by your study medication and also if there is any change in your medical status, for example, should you be hospitalized.

11.

Your Clinic Phone Number Is on the Back of This Brochure. Please Keep This Brochure as a Reference Until the End of the Study.

A commonly asked question is whether a low adherence rate should be discussed directly with study participants. There is a consensus that any discussion should not be confrontational. The preferred approach is to open any discussion by saying that adherence to medications can be very difficult for many people. After being given examples of common reasons for low adherence, many participants seem to be more willing to discuss their own situations and adherence problems. Thus, sympathy and understanding may be helpful if followed by specific recommendations regarding ways to improve adherence. A large number of interviewing techniques of patients in the clinical setting are discussed by Shea [48]. Tools like the Morisky Scale [49] could be used to identify participants at high risk for non-adherence on whom to focus preventive efforts.

A remarkable recovery program was developed and implemented by Probstfield et al. [50]. Through participant counseling, the investigators succeeded in about 90% of the 36 drop-outs in approximately 6 months to return for clinic visits. Even more notable was the virtual absence of recidivism over the remaining 5 years of intervention. Approximately 70% of the drop-outs resumed taking their study medication, though typically at a lower dose than specified in the protocol.

Adherence Monitoring

Monitoring adherence is important in a clinical trial for two reasons: first, to identify any problems so steps can be taken to enchance adherence; second, to be able to relate the trial findings to the level of adherence. In general, analysis of trial outcomes by level of adherence is strongly discouraged as it can in fact lead to serious bias, the direction of which cannot always be predicted (see Chap. 18). However, in so far as the control group is not truly a control and the intervention group is not being treated as intended, group differences are diluted, and generally lead to an underestimate of both the therapeutic benefit and the adverse effects. Differential adherence to two equally effective regimens can also lead to possibly erroneous conclusions about the effects of the intervention. The level of adherence that occurred can also be compared with what was expected when the trial was designed.

In some studies, measuring adherence is relatively easy. This is true for trials in which one group receives surgery and the other group does not, or for trials which require only a one-time intervention such as a vaccine. Most of the time, however, assessment of adherence is more complex. No single measure of adherence gives a complete picture, and all are subject to possible inaccuracies and varying interpretations. Furthermore, there is no widely accepted definition or criterion for either high or low adherence. A review of 192 publications showed that only 36% assessed and adequately reported medication adherence [51]. The level of adherence that occurred can also be compared to what was expected when the trial was designed.

In monitoring adherence for a long-term trial, the investigator may also be interested in changes over time. When reductions in adherence are noted, corrective action can possibly be taken. This monitoring could be by calendar time (e.g., current 6 months versus previous 6 months) or by clinic visit (e.g., follow-up visit number four versus previous visits). In multicenter trials, adherence to the intervention also ought to be examined by clinic or by region in multinational trials. In all studies, it is important for clinic staff to receive feedback about level of adherence. In double-blind trials where data by study group generally should not be disclosed, the adherence data can be combined for the study groups. In trials that are not double-blind, all adherence tables can be reviewed with the clinic staff. Frequent determinations obviously have more value than infrequent ones. A better indication of true adherence can be obtained. Moreover, when the participant is aware that he is being monitored, frequent measures may encourage adherence.

There are several indirect methods of assessing adherence. In drug trials, pill or capsule count, is the easiest and most commonly used way of evaluating participant adherence. Since this assumes that the participant has ingested all medication not returned to the clinic, the validity of pill count is debated. For example, if the participant returns the appropriate number of leftover pills at a follow-up visit, did he in fact take what he was supposed to, or take only some and throw the rest out? Pill count is possible only as long as the pills are available to be counted. Participants sometimes forget or neglect to bring their pills to the clinic to be counted. In such circumstances, the investigator may ask the participant to count the pills himself at home and to notify the investigator of the result by telephone. Obviously, these data may be less reliable. The frequency with which data on pill counts are missing gives an estimate of the reliability of pill count as an adherence measure.

In monitoring pill count, the investigators ought to anticipate questions of interest to readers of the trial report when published. What was the overall adherence to the protocol prescription? If the overall adherence with the intervention was reduced, what was the main reason for the reduction? Were the participants prescribed a reduced dose of the study medication, or did they not follow the investigator’s prescription? Were there differences between the study groups with regard to protocol dosages, investigator prescriptions, or participant adherence to the prescribed dosages? What were the reasons for reduced participant adherence? Was it because of intervening life events, specific adverse effects or was it simply forgetfulness? The answers to these questions may increase the understanding and interpretation of the results of the trial.

When discussing adherence assessed by pill count, the investigator has to keep in mind that these data may be inflated and misleading. Additionally, these data do not include information from participants who omit a visit. Most participants tend to overestimate their adherence either in an effort to please the investigator or because of faulty memory. Those who miss one or more visits typically have low adherence. Therefore, the adherence data should be viewed within the framework of all participants who are scheduled to be seen at a particular visit. There is general agreement on one point—the participant who says he did not take his study medication can be trusted.

Electronic monitoring of adherence has been used [52, 53]. A device electronically records drug package opening times and duration, thus, describes dosing histories. The correlation between package openings and measured drug concentrations in serum is very high. The obvious advantage of electronic monitoring is that the dose-timing can be assessed to see if it is punctual and regular. In an HIV trial, overall adherence was 95%, but only 81% of the doses were taken within the prescribed dosing interval (±3 h) [52]. In a study of hypertensive participants, about 10% of the scheduled doses were omitted on any day [53]. Drug holidays, defined as omissions of all doses during 3 or more days, were recorded in 43% of the participants. An interesting observation was that participants with dosing problems were more likely later to become permanent drop-outs. It is not known whether or to what extent low adherence to dose-timing influences the trial findings.

A recent development is an FDA-approved device which has a body-worn sensor or patch that collects physiological and behavioral metrics generated by an ingestible sensor. The system can be used to monitor when the patient takes his medication. This sensor is embedded inside an inactive tablet and it activates and communicates its presence and unique identifier to the patch [54].

Indirect information on adherence can also be obtained through interviews or record keeping by the participant. A diet study might use a 24-h recall or a 7-day food record. Exercise studies may use diaries to record frequency and kind of exercise. Trials of people with angina might record frequency of attacks or pain and nitroglycerin consumption.

There are two major direct methods for measuring adherence. Biochemical analyses are sometimes made on either blood or urine in order to detect the presence of the active drug or metabolites. A limitation in measuring substances in urine or blood is the short half-life of most drugs. Therefore, laboratory determinations usually indicate only what has happened in the preceding day or two. A control participant who takes the active drug (obtained from a source outside the trial) until the day prior to a clinic visit, or a participant in the intervention group who takes the active drug only on the day of the visit might not always be detected as being a poor adherer. Moreover, drug adherence in participants taking an inert placebo tablet cannot be assessed by any laboratory determination. Adding a specific chemical substance such as riboflavin can serve as a marker in cases where the placebo, the drug or its metabolites are difficult to measure. However, the same drawbacks apply to markers as to masking substances—the risk of toxicity in long-term use may outweigh benefits.

Laboratory tests obtained on occasions not associated with clinic visits may give a better picture of regular or true adherence. Thus, the participant may be instructed, at certain intervals, to send a vial of urine to the clinic. Such a technique is of value only so long as the participant does not associate this request with an adherence monitoring procedure. In at least one study, information obtained in this manner contributed no additional information to laboratory results done at scheduled visits, except perhaps as a confirmation of such results.

Measurement of physiological response variables can be helpful in assessing adherence. Cholesterol reduction by drug or diet is unlikely to occur in 1 or 2 days. Therefore, a participant in the intervention group cannot suddenly adhere with the regimen the day before a clinic visit and expect to go undetected. Similarly, the serum cholesterol level of a nonadherent control participant is unlikely to rise in the 1 day before a visit if he skips the non-study lipid-lowering drug. Other physiological response variables that might be monitored are blood pressure in an antihypertensive study, carbon monoxide in a smoking study, platelet aggregation in an aspirin study, and graded exercise in an exercise study. In all these cases, the indicated response variable would not be the primary response variable but merely an intermediate indicator of adherence to the intervention regimen. Unfortunately, not every person responds in the same way to medication, and some measures, such as triglyceride levels, are highly variable. Therefore, indications of low adherence of individual participants using these measures are not easily interpreted. Group data, however, may be useful.

Another aspect of monitoring deals with participant adherence to study procedures such as attendance at scheduled visits or visit adherence. One of the major purposes of these visits is to collect response variable data. The data will be better if they are more complete. Thus, completeness of data in itself can be a measure of the quality of a clinical trial. Studies with even a moderate amount of missing data or participants lost to follow-up could give misleading results and should be interpreted with caution. By reviewing the reasons why participants missed scheduled clinic visits, the investigator can identify factors that can be corrected or improved. Having the participants come in for study visits facilitates and encourages adherence to study medication. Study drugs are dispensed at these visits and the dose is adjusted when necessary.

From a statistical viewpoint, every randomized participant should be included in the primary analysis (Chaps. 8 and 18). Consequently, the investigator must keep trying to get all participants back for scheduled visits until the trial is over. Even if a participant is taken off the study medication by an investigator or stops taking it, he should be encouraged to come in for regular study visits or at least be followed by telephone. Complete follow-up data on the response variables are critical so visit adherence is important. In addition, participants do change their minds. For a long time, they may want to have nothing to do with the trial and later may agree to come back for visits and even resume taking their assigned intervention regimen. Special attention to the specific problems of each participant withdrawn from the trial and an emphasis on potential contribution to the trial can lead to successful retrieval of a large proportion of withdrawn participants. Inasmuch as the participant will be counted in the analysis, leaving open the option for the participant to return to active participation in the study or at least agree to a visit or phone contact at the end of the trial is worthwhile.

The purpose of adherence monitoring is to acquire a general understanding of the level of adherence, so steps can be taken to improve it if necessary. Thus, there is limited value in obtaining precise assessments since we don’t favor data analysis by adherence.

Dealing with Low Adherence

If low adherence is related to difficulties making appointments, it may be useful to offer more convenient clinic hours, such as evenings and weekends as mentioned above. Home visits are another option for participants with disabilities who have difficulties making it to the clinic. For participants who have moved, the investigator might be able to arrange for follow-up in other cities.

One of the challenges in clinical trials is the complete ascertainment of response variables in participants who are no longer actively involved in the trial. The Internet provides opportunities to track participants lost to follow-up. There are both fee-for-service and free search engines. The basic information required for a search is complete name, birth date and Social Security Number or some other specific identification number. These searches are more effective if several and if different search engines are employed.

Steps should be taken to prevent situations in which participants request that they never be contacted. These are sometimes referred to as complete withdrawal. Participants who end their active participation in a clinical trial often agree to be contacted at the end of the trial for ascertainment of key response variables. For those who are lost to follow-up, but have not withdrawn their consent, alternative sources of information are family members and medical providers. The goal is to limit the amount of missing information.

Special Populations

Although the approaches to dealing with prevention of low adherence and maintenance of high adherence are generally applicable, there are factors that need consideration when dealing with special populations. Older adults represent a growing number of participants in clinical trials. They typically have more health complaints than their younger counterparts. There is a rich literature on factors that may influence adherence and on strategies to increase adherence in the clinical setting among older people. Many of these are highly relevant for clinical trials.

There are special challenges of maintaining adherence in patients with chronic health illnesses. Specific management interventions for several prevalent conditions are highlighted in the Handbook of Health Behavior Change [11]. These include cardiovascular diseases [55], diabetes [56], chronic respiratory diseases [57], chronic infectious diseases [58], cancer [59] and obesity [60].

References

1.

Haynes RB, Taylor DW, Sackett DL (eds.). Compliance in Health Care. Baltimore: Johns Hopkins University Press, 1979.

2.

Cramer JA, Roy A, Burrell A, et al. Medication compliance and persistence: terminology and definitions. Value Health 2008;11:44-47.CrossRef

3.

Vrijens B, De Geest S, Hughes DA, et al. A new taxonomy for describing and defining adherence to medications. Br J Clin Pharmacol 2012;73:691-705.

4.

Zeber JE, Manias E, Williams AF, et al. A systematic literature review of psychosocial and behavioral factors associated with initial medication adherence: a report of the ISPOR medication adherence & persistence special interest group. Value Health 2013;16:891-900.CrossRef

5.

Raebel MA, Schmittdiel J, Karter AJ, et al. Standardizing terminology and definitions of medication adherence and persistence in research employing electronic databases. Med Care 2013;51:S11-S21.CrossRef

6.

Eapen Z, Mi X, Qualls LG, et al. Adherence and persistence in the use of warfarin after hospital discharge among patients with heart failure and atrial fibrillation. J Card Fail 2014;20:23-30.CrossRef

7.

Granger BB, Rusincovitch SA, Avery S, et al. Missing signposts on the roadmap to quality: a call to improve medication adherence indicators in data collection for population research. Front Pharmacol 2013;4:139.CrossRef

8.

Peterson AM, Takiya L, Finley R. Meta-analysis of trials of interventions to improve medication adherence. Am J Health-Syst Pharm 2003;60:657-665.

9.

Cutler DM, Everett W. Thinking outside the pillbox—medication adherence as a priority for health care reform. N Engl J Med 2010;362:1553-1555.CrossRef

10.

Osterberg L, Blaschke T. Adherence to medication. N Engl J Med 2005;353:487-497.CrossRef

11.

Riekert KA, Ockene JK, Pbert L (eds.). The Handbook of Health Behavior Change (4th edition). New York: Springer Publishing Company, 2014.

12.

Boden WE, O’Rourke RA, Teo KK, et al for the COURAGE Trial Research Group. Optimal medical therapy with or without PCI for stable coronary disease. N Engl J Med 2007;356:1503-1516.CrossRef

13.

Granger BB, Swedberg K, Ekman I, et al. Adherence to candesartan and placebo and outcomes in chronic heart failure in the CHARM programme: double-blind, randomized, controlled clinical trial. Lancet 2005;366:2005-2011.CrossRef

14.

DiMatteo MR. Variations in patients’ adherence to medical recommendations. A quantitative review of 50 years of research. Med Care 2004;42:200-209.CrossRef

15.

Härkäpää K, Järvikoski A, Mellin G, Hurri H. A controlled study of the outcome of inpatient and outpatient treatment of low back pain. Part I. Pain, disability, compliance, and reported treatment benefits three months after treatment. Scand J Rehabil Med 1989;21:81-89.

16.

McLean N, Griffin S, Toney K, Hardeman W. Family involvement in weight control, weight maintenance and weight-loss interventions: a systematic review of randomised trials. Intern J Obesity 2003;27:987-1005.CrossRef

17.

Voils CI, Yancy Jr WS, Kovac S, et al. Study protocol: Couples Partnering for Lipid Enhancing Strategies (CouPLES)—a randomized, controlled trial. Trials 2009;10:10-19.CrossRef

18.

Sacks FM, Svetkey LP, Vollmer WM, et al. Effects on blood pressure of reduced dietary sodium and the Dietary Approaches to Stop Hypertension (DASH) diet. N Engl J Med 2001;344:3-10.CrossRef

19.

Claxton AJ, Cramer J, Pierce C. A systematic review of the associations between dose regimens and medication compliance. Clin Ther 2001;23:1296-1310.CrossRef

20.

Probstfield JL. The clinical trial prerandomization compliance (adherence) screen. In Cramer JA, Spilker B (eds.). Patient Compliance in Medical Practice and Clinical Trials. New York: Raven Press, 1991.

21.

Berger VW, Rezvani A, Makarewicz VA. Direct effect on validity of response run-in selection in clinical trials. Control Clin Trials 2003;24;156-166.CrossRef

22.

Lang JM, Buring JE, Rosner B, et al. Estimating the effect of the run-in on the power of the Physicians' Health Study. Stat Med 1991;10:1585-1593.CrossRef

23.

McMurray JJV, Packer M, Desai AS, et al. Angiotensin-neprilysin inhibition versus enalapril in heart failure. N Engl J Med 2014;371:993-1004.CrossRef

24.

Lee S, Walker JR, Jakul L, Sexton K. Does elimination of placebo responders in a placebo run-in increase the treatment effect in randomized clinical trials? A meta-analytic evaluation. Depress Anxiety 2004;19:10-19.CrossRef

25.

Kingry C, Bastien A, Booth G, et al. for the ACCORD Study Group. Recruitment strategies in the Action to Control Cardiovascular Risk of Diabetes (ACCORD) Trial. Am J Cardiol 2007;99[suppl]:68i-79i.

26.

Williams SL, Haskard-Zolnierik KB, DiMatteo MR. Psychosocial predictors of behavioral change. In Riekert KA, Ockene JK, Pbert L (eds.). The Handbook of Health Behavior Change (4th edition). New York: Springer Publishing Company, 2014, pp. 69-86.

27.

Youmans SL, Schillinger D. Functional health literacy and medication use: the pharmacist’s role. Ann Pharmacother 2003;37:1726-1729.CrossRef

28.

Wolf MS, Davis TC, Osborn CY, et al. Literacy, self-efficacy, and HIV medication adherence. Pat Educ Counsel 2007;65:253-260.CrossRef

29.

Daumit GL, Dickerson FB, Wang NY, et al. A behavioral weight-loss intervention in persons with serious mental illness. N Engl J Med 2013;368:1594-1602.CrossRef

30.

Kimmel SE, Troxel AB, Loewenstein G, et al. Randomized trial of lottery-based incentives to improve warfarin adherence. Am Heart J 2012;164:268-274.CrossRef

31.

Ryan R, Santesso N, Lowe D. et al. Interventions to improve safe and effective medicines use by consumers: an overview of systematic reviews. Cochrane Database Syst Rev 2014;4:CD007768.

32.

DeFulio A, Silverman K. The use of incentives to reinforce medication adherence. Prev Med 2012;55:S86-S94.CrossRef

33.

Hiligsmann M, Salas M, Hughes DA, et al. Interventions to improve osteoporosis medication adherence and persistence: a systematic review and literature appraisal by the ISPOR medication adherence & persistence special interest group. Osteoporos Int 2013;24:2907-2918.CrossRef

34.

Scheurer D, Choudhry N, Swanton KA, et al. Association between different types of social support and medication adherence. Am J Manag Care 2012;18:e461-e467.

35.

Barnes K. Patients provide insight into trial participation. Outsourcing-Pharma.com, July 4, 2007. www.​outsourcing-pharma.​com/​content/​view/​print/​135930.

36.

Kessels RPC. Patients’ memory for medical information. J R Soc Med 2003;96:219-222.CrossRef

37.

Anderson JL, Dodman S, Kopelman M, et al. Patient information recall in a rheumatology clinic. Rheumatology 1979;18:18-22.CrossRef

38.

Tamur-Lis W. Teach-back for quality education and patient safety. Urol Nurs 2013;33:267-271.

39.

White M, Garbez R, Carroll M, et al. Is “teach-back” associated with knowledge retention and hospital readmission in hospitalized heart failure patients? J Cardiovasc Nurs 2013;28:137-146.CrossRef

40.

Xie L, Frech-Tamas F, Marrett E, Baser O. A medication adherence and persistence comparison of hypertensive patients treated with single-, double- and triple-pill combination therapy. Curr Med Res Opin 2014;29:1-8.

41.

Demonceau J, Ruppar T, Kristanto P, et al. Identification and assessment of adherence-enhancing interventions in studies assessing medication adherence through electronically compiled drug dosing histories: a systematic literature review and meta-analysis. Drugs 2013;73:545-562.CrossRef

42.

Zullig LL, Peterson ED, Bosworth HB. Ingredients of successful interventions to improve medication adherence. JAMA 2013;310:2611-2612.CrossRef

43.

Dexheimer JW, Sanders DL, Rosenbloom ST, et al. Prompting clinicians: A systematic review of prevention care reminders. AMIA Annu Symp Proc. 2005; 2005;938.

44.

Hasvold PE, Wootton R. Use of telephone and SMS reminders to improve attendance at hospital appointments: a systematic review. J Telemed Telecare 2011;17:358-364.CrossRef

45.

Gurol-Urganci I, de Jongh T, Vodopivec-Jamsek V, et al. Mobile phone messaging reminders for attendance at healthcare appointments. Cochrane Database of Systematic Reviews 2013, Issue 12

46.

Vervloet M, Linn AJ, vanWeert, JCM, et al. The effectiveness of interventions using electronic reminders to improve adherence to chronic medication: a systematic review of the literature. J Am Med Inform Assoc, 2012; 19:696-704.CrossRef

47.

Granger BB, Bosworth H. Medication adherence: emerging use of technology. Curr Opin Cardiol. 2011; 26:279-287.CrossRef

48.

Shea SC. Improving medication adherence: How to talk with patients about their medications. Philadelphia: Lippincott, Williams & Wilkins, 2006.

49.

Morisky DE, Ang A, Krousel-Wood M, Ward HJ. Predictive validity of a medication adherence measure in an outpatient setting. J Clin Hypertens 2008;10:348-354.CrossRef

50.

Probstfield JL, Russell ML, Henske JC, et al. Successful program for recovery of dropouts to a clinical trial. Am J Med 1986;80:777-784.CrossRef

51.

Gossec L, Tubach F, Dougados M, Ravaud P. Reporting of adherence to medication in recent randomized controlled trials of 6 chronic diseases: A systematic literature review. Am J Med Sci 2007;334:248-254.CrossRef

52.

Vrijens B, Rousset E, Rode R, et al. Successful projection of the time course of drug concentration in plasma during a 1-year period from electronically compiled dosing-time data used as input to individually parameterized pharmacokinetic models. J Clin Pharmacol 2005;45:461-467.CrossRef

53.

Vrijens B, Vincze G, Kristanto P, et al. Adherence to prescribed antihypertensive drug treatments: longitudinal study of electronically compiled dosing histories. Br Med J 2008;336:1114-1117.CrossRef

54.

Proteus Digital Health, Inc. www.​proteusdigitalhe​alth.​com/​technology/​proteus-publications/​

55.

Hayman LL, Mruk MM. Chronic disease management interventions: cardiovascular disease. In Riekert KA, Ockene JK, Pbert L (eds). The Handbook of Health Behavior Change (4th edition). New York, Springer Publishing Company, 2014, pp 273-289.

56.

Hood KK, Raymond JK, Harris MA. Diabetes management behaviors: The key to optimal health and quality of life outcomes. In Riekert KA, Ockene JK, Pbert L (eds). The Handbook of Health Behavior Change (4th edition). New York, Springer Publishing Company, 2014, pp 291-308.

57.

Welkom JS, Riekert KA, Eakin, MN, et al. Behavioral management of chronic respiratory diseases: examples from asthma and chronic obstructive pulmonary disease. In Riekert KA, Ockene JK, Pbert L (eds). The Handbook of Health Behavior Change (4th edition). New York, Springer Publishing Company, 2014, pp 309-329.

58.

Rhodes SD, Wilkin AM, Abraham C, Bachmann LH. Chronic infectious disease management interventions. In Riekert KA, Ockene JK, Pbert L (eds). The Handbook of Health Behavior Change (4th edition). New York, Springer Publishing Company, 2014, pp 331-345.

59.

Peterman AH, Victorson D, Cella D. Adherence to treatment and lifestyle changes among people with cancer. In Riekert KA, Ockene JK, Pbert L (eds). The Handbook of Health Behavior Change (4th edition). New York, Springer Publishing Company, 2014, pp 347-362.

60.

Burke LE, Turk MW. Obesity. In Riekert KA, Ockene JK, Pbert L (eds). The Handbook of Health Behavior Change (4th edition). New York, Springer Publishing Company, 2014, pp 363-378.

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_15

15. Survival Analysis

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

This chapter reviews some of the fundamental concepts and basic methods in survival analysis. Frequently, event rates such as mortality or occurrence of nonfatal myocardial infarction are selected as primary response variables. The analysis of such event rates in two groups could employ the chi-square statistic or the equivalent normal statistic for the comparison of two proportions. However, when the length of observation is different for each participant, estimating an event rate is more complicated. Furthermore, simple comparison of event rates between two groups is not necessarily the most informative type of analysis. For example, the 5-year survival for two groups may be nearly identical, but the survival rates may be quite different at various times during the 5 years. This is illustrated by the survival curves in Fig. 15.1. This figure shows survival probability on the vertical axis and time on the horizontal axis. For Group A, the survival rate (or one minus the mortality rate) declines steadily over the 5 years of observation. For Group B, however, the decline in the survival rate is rapid during the first year and then levels off. Obviously, the survival experience of the two groups is not the same, although the mortality rate at 5 years is nearly the same. If only the 5-year survival rate is considered, Group A and Group B appear equivalent. Curves such as these might reasonably be expected in a trial of surgical versus medical intervention, where surgery might carry a high initial operative mortality.

Fig. 15.1

Survival experience for two groups (A and B)

Fundamental Point

Survival analysis methods are important in trials where participants are entered over a period of time and have various lengths of follow-up. These methods permit the comparison of the entire survival experience during the follow-up and may be used for the analysis of time to any dichotomous response variable such as a nonfatal event or an adverse event.

A review of the basic techniques of survival analysis can be found in elementary statistical textbooks [1–6] as well as in overview papers [7]. A more complete and technical review is in other texts [8–11]. Many methodological advances in the field have occurred and this book will not be able to cover all developments. The following discussion will concern two basic aspects: first, estimation of the survival experience or survival curve for a group of participants in a clinical trial and second, comparison of two survival curves to test whether the survival experience is significantly different. Although the term survival analysis is used, the methods can be applied to any dichotomous response variable when the time from enrollment to the time of the event, not just the fact of its occurrence, is an important consideration. For ease of communication, we shall use the term event, unless death is specifically the event.

Estimation of the Survival Curve

The graphical presentation of the total survival experience during the period of observation is called the survival curve, and the tabular presentation is called the lifetable. In the sample size discussion (Chap. 8), we utilized a parametric model to represent a survival curve, denoted S(t), where t is the time of follow-up. A classic parametric form for S(t) is to assume an exponential distribution S(t) = e −λt  = exp(−λt), where λ is the hazard rate [11]. If we estimate λ, we have an estimate for S(t). One possible estimate for the hazard ratio is the number of observed events divided by the total exposure time of the person at risk of the event. Other estimates are also available and are described later. While this estimate is not difficult to obtain, the hazard rate may not be constant during the trial. If λ is not constant, but rather a function of time, we can define a hazard rate λ(t), but now the definition of S(t) is more complicated. Specifically, $$ S(t)= \exp \left[{\displaystyle {\int}_o^t\lambda (s)ds}\right] $$, that is, the exponential of the area under the hazard function curve from time 0 to time t. Furthermore, we cannot always be guaranteed that the observed survival data will be described well by the exponential model, even though we often make this assumption for computing sample size. Thus, biostatisticians have relied on parameter-free or non-parametric ways to estimate the survival curve.

This chapter will cover two similar non-parametric methods, the Cutler-Ederer method [12] and the Kaplan-Meier method [13] for estimating the true survival curve or the corresponding lifetable. We use the Cutler-Ederer method to motivate the more flexible Kaplan-Meier method which is the current standard. Before a review of these specific methods, however, it is necessary to explain how the survival experience is typically obtained in a clinical trial and to define some of the associated terminology.

The clinical trial design may, in a simple case, require that all participants be observed for T years. This is referred to as the follow-up or exposure time. If all participants are entered as a single cohort at the same time, the actual period of follow-up is the same for all participants. If, however, as in most clinical trials, the entry of participants is staggered over some recruitment period, then equal periods of follow-up may occur at different calendar times for each participant, as illustrated in Fig. 15.2.

Fig. 15.2

T year follow-up time for four participants with staggered entry

A participant may have a study event during the course of follow-up. The event time is the accumulated time from entry into the study to the event. The interest is not in the actual calendar date when the event took place but rather the interval of time from entry into the trial until the event. Figures 15.3 and 15.4 illustrate the way the actual survival experience for staggered entry of participants is translated for the analysis. In Fig. 15.3, participants 2 and 4 had an event while participants 1 and 3 did not during the follow-up time. Since, for each participant, only the time interval from entry to the end of the scheduled follow-up period or until an event is of interest, the time of entry can be considered as time zero for each participant. Figure 15.4 illustrates the same survival experience as Fig. 15.3, but the time of entry is considered as time zero.

Fig. 15.3

Follow-up experience of four participants with staggered entry: two participants with observed events (asterisk) and two participants followed for time T without events (open circle).

Fig. 15.4

Follow-up experience of four participants with staggered entry converted to a common starting time: two participants with observed events (asterisk) and two participants followed for time T without events (open circle).

Some participants may not experience an event before the end of observation. The follow-up time or exposure time for these participants is said to be censored; that is, the investigator does not know what happened to these participants after they stopped participating in the trial. Another example of censoring is when participants are entered in a staggered fashion, and the study is terminated at a common date before all participants have had at least their complete T years of follow-up. Later post-trial events from these participants are also unobserved, but the reason for censoring is administrative. Administrative censoring could also occur if a trial is terminated prior to the scheduled time because of early benefits or harmful effects of the intervention. In these cases, censoring is assumed to be independent of occurrence of events.

Figure 15.5 illustrates several of the possibilities for observations during follow-up. Note that in this example the investigator has planned to follow all participants to a common termination time, with each participant being followed for at least T years. The first three participants were randomized at the start of the study. The first participant was observed for the entire duration of the trial with no event, and her survival time was censored because of study termination. The second participant had an event before the end of follow-up. The third participant was lost to follow-up. The second group of three participants was randomized later during the course of the trial with experiences similar to the first group of three. Participants 7 through 11 were randomized late in the study and were not able to be followed for at least T years because the study was terminated early. Participant 7 was lost to follow-up and participant 8 had an event before T years of follow-up time had elapsed and before the study was terminated. Participant 9 was administratively censored but theoretically would have been lost to follow-up had the trial continued. Participant 10 was also censored because of early study termination, although she had an event afterwards which would have been observed had the trial continued to its scheduled end. Finally, the last participant who was censored would have survived for at least T years had the study lasted as long as first planned. The survival experiences illustrated in Fig. 15.5 would all be shifted to have a common starting time equal to zero as in Fig. 15.4. The follow-up time, or the time elapsed from calendar time of entry to calendar time of an event or to censoring could then be analyzed.

Fig. 15.5

Follow-up experience of 11 participants for staggered entry and a common termination time, with observed events (asterisk) and censoring (open circle). Follow-up experience beyond the termination time is shown for participants 9 through 11

In summary then, the investigator needs to record for each participant the time of entry and the time of an event, the time of loss to follow-up, or whether the participant was still being followed without having had an event when the study is terminated. These data will allow the investigator to compute the survival curve.

Cutler-Ederer Estimate

Though the Cutler-Ederer estimate is still in use [14–18], it has been largely replaced as a method for estimation of survival curves by the Kaplan-Meier estimate. Nonetheless it is useful as an introduction to survival curve estimation.

In the Cutler-Ederer or actuarial estimate [12], the assumption is that the deaths and losses are uniformly distributed over a set of fixed-length intervals. On the average, this means that one half the losses will occur during the first half of each interval. The estimate for the probability of surviving the j th interval, given that the previous intervals were survived, is $$ {\widehat{p}}_j $$, where

The λ j losses are assumed to be at risk, on the average, one half the time and thus should be counted as such. These conditional probabilities $$ {\widehat{p}}_j $$ are then multiplied together to obtain an estimate, Ŝ(t), of the survival function at time t.

Kaplan-Meier Estimate

The Kaplan-Meier Estimate relaxes the assumption of events distributed uniformly across fixed length intervals. Using the time of death, observations can be ranked. This is a useful improvement, since in a clinical trial with staggered entry of participants and censored observations, survival data will be of varying degrees of completeness.

As a very simple example, suppose that 100 participants were entered into a study and followed for 2 years. One year after the first group was started, a second group of 100 participants was entered and followed for the remaining year of the trial. Assuming no losses to follow-up, the results might be as shown in Table 15.1. For Group I, 20 participants died during the first year and of the 80 survivors, 20 more died during the second year. For Group II, which was followed for only 1 year, 25 participants died. Now suppose the investigator wants to estimate the 2-year survival rate. The only group of participants followed for 2 years was Group I. One estimate of 2-year survival, S(2), would be $$ \widehat{S}(2)=60/100 $$ or 0.60. Note that the first-year survival experience of Group II is ignored in this estimate. If the investigator wants to estimate 1 year survival rate, S(1), she would observe that a total of 200 participants were followed for at least 1 year. Of those, 155 (80 + 75) survived the first year. Thus, $$ \widehat{S}(1)=155/200 $$ or 0.775. If each group were evaluated separately, the survival rates would be 0.80 and 0.75. In estimating the 1-year survival rate, all the available information was used, but for the 2-year survival rate the 1-year survival experience of Group II was ignored.

Table 15.1

Participants entered at two points in time (Group I and Group II) and followed to a common termination timea

Years of follow-up

Group

I

II

1

Participants entered

100

100

First year deaths

20

25

First year survivors

80

75

2

Participants entered

80

Second year deaths

20

Second year survivors

60

aAfter Kaplan and Meier [13]

Another procedure for estimating survival rates is to use a conditional probability. For this example, the probability of 2-year survival, S(2), is equal to the probability of 1-year survival, S(1), times the probability of surviving the second year, given that the participant survived the first year, pr(2|1). That is, $$ S(2)=S(1)pr\left(2\Big|1\right) $$. In this example, $$ \widehat{S}(1)=0.775 $$. The estimate for pr(2|1) is 60/80 = 0.75 since 60 of the 80 participants who survived the first year also survived the second year. Thus, the estimate for $$ \widehat{S}(2)=0.775\times 0.75 $$ or 0.58, which is slightly different from the previously calculated estimate of 0.60.

Kaplan and Meier [13] described how this conditional probability strategy could be used to estimate survival curves in clinical trials with censored observations. Their procedure is usually referred to as the Kaplan-Meier estimate, or sometimes the product-limit estimate, since the product of conditional probabilities leads to the survival estimate. This procedure assumes that the exact time of entry into the trial is known and that the exact time of the event or loss of follow-up is also known. For some applications, time to the nearest month may be sufficient, while for other applications the nearest day or hour may be necessary. Kaplan and Meier assumed that a death and loss of follow-up would not occur at the same time. If a death and a loss to follow-up are recorded as having occurred at the same time, this tie is broken on the assumption that the death occurred slightly before the loss to follow-up.

In this method, the follow-up period is divided into intervals of time so that no interval contains both deaths and losses. Let p j be equal to the probability of surviving the j th interval, given that the participant has survived the previous interval. For intervals labeled j with deaths only, the estimate for p j , which is $$ {\widehat{p}}_j $$, is equal to the number of participants alive at the beginning of the j th interval, n j , minus those who died during the interval, δ j , with this difference being divided by the number alive at the beginning of the interval, i.e. $$ {\widehat{p}}_j=\left({n}_j-{\delta}_j\right)/{n}_j $$. For an interval j with only l j losses, the estimate $$ {\widehat{p}}_j $$ is one. Such conditional probabilities for an interval with only losses would not alter the product. This means that an interval with only losses and no deaths may be combined with the previous interval.

Example

Suppose 20 participants are followed for a period of 1 year, and to the nearest tenth of a month, deaths were observed at the following times: 0.5, 1.5, 1.5, 3.0, 4.8, 6.2, 10.5 months. In addition, losses to follow-up were recorded at: 0.6, 2.0, 3.5, 4.0, 8.5, 9.0 months. It is convenient for illustrative purposes to list the deaths and losses together in ascending time with the losses indicated in parentheses. Thus, the following sequence is obtained: 0.5, (0.6), 1.5, 1.5, (2.0), 3.0, (3.5), (4.0), 4.8, 6.2, (8.5), (9.0), 10.5. The remaining seven participants were all censored at 12 months due to termination of the study.

Table 15.2 presents the survival experience for this example as a lifetable. Each row in the lifetable indicates the time at which a death or an event occurred. One or more deaths may have occurred at the same time and they are included in the same row in the lifetable. In the interval between two consecutive times of death, losses to follow-up may have occurred. Hence, a row in the table actually represents an interval of time, beginning with the time of a death, up to but not including the time of the next death. In this case, the first interval is defined by the death at 0.5 months up to the time of the next death at 1.5 months. The columns labeled n j , δ j , and l j correspond to the definitions given above and contain the information from the example. In the first interval, all 20 participants were initially at risk, one died at 0.5 months, and later in the interval (at 0.6 months) one participant was lost to follow-up. In the second interval, from 1.5 months up to 3.0 months, 18 participants were still at risk initially, two deaths were recorded at 1.5 months and one participant was lost at 2.0 months. The remaining intervals are defined similarly. The column labeled $$ {\widehat{p}}_j $$ is the conditional probability of surviving the interval j and is computed as (n j  − δ j )/n j or (20 − 1)/20 = 0.95, (18 − 2)/18 = 0.89, etc. The column labeled Ŝ(t) is the estimated survival curve and is computed as the accumulated product of the $$ {\widehat{p}}_j $$ (0.85 = 0.95 × 0.89, 0.79 = 0.95 × 0.89 × 0.93, etc).

Table 15.2

Kaplan-Meier lifetable for 20 participants followed for 1 year

Interval

Interval number

Time of death

n j

δ j

l j

S(t)

Var Ŝ(t)

[0.5, 1, 5)

1

0.5

20

1

1

0.95

0.95

0.0024

[1.5, 3.0)

2

1.5

18

2

1

0.89

0.85

0.0068

[3.0, 4.8)

3

3.0

15

1

2

0.93

0.79

0.0089

[4.8, 6.2)

4

4.8

12

1

0

0.92

0.72

0.0114

[6.2, 10.5)

5

6.2

11

1

2

0.91

0.66

0.0133

[10.5, ∞)

6

10.5

8

1

7a

0.88

0.58

0.0161

n j : number of participants alive at the beginning of the j th interval

δ j : number of participants who died during the j th interval

l j : number of participants who were lost or censored during the j th interval

$$ {\widehat{p}}_j $$: estimate for p j , the probability of surviving the j th interval given that the participant has survived the previous intervals

S(t): estimated survival curve

Var Ŝ(t): variance of Ŝ(t)

aCensored due to termination of study

The graphical display of the next to last column of Table 15.2, Ŝ(t), is given in Fig. 15.6. The step function appearance of the graph is because the estimate of S(t), Ŝ(t) is constant during an interval and changes only at the time of a death. With very large sample sizes and more observed deaths, the step function has smaller steps and looks more like the usually visualized smooth survival curve. If no censoring occurs, this method simplifies to the number of survivors divided by the total number of participants who entered the trial.

Fig. 15.6

Kaplan-Meier estimate of a survival curve, Ŝ(t), from a 1-year study of 20 participants, with observed events (asterisk) and censoring (open circle).

Because Ŝ(t) is an estimate of S(t), the true survival curve, the estimate will have some variation due to the sample selected. Greenwood [19] derived a formula for estimating the variance of an estimated survival function which is applicable to the Kaplan-Meier method. The formula for the variance of Ŝ(t), denoted V[Ŝ(t)] is given by

where n j and δ j are defined as before, and K is the number of intervals. In Table 15.2, the last column labeled V[Ŝ(t)] represents the estimated variances for the estimates of S(t) during the six intervals. Note that the variance increases as one moves down the column. When fewer participants are at risk, the ability to estimate the survival experience is diminished.

Other examples of this procedure, as well as a more detailed discussion of some of the statistical properties of this estimate, are provided by Kaplan and Meier [13]. Computer programs are available [20–23] so that survival curves can be obtained quickly, even for very large sets of data.

The Kaplan-Meier curve can also be used to estimate the hazard rate, λ, if the survival curve is exponential. For example, if the median survival time is estimated as TM, then 0.5 = S(TM) = exp(−λTM) and thus $$ \widehat{\lambda}= \ln (0.5)/{\mathrm{T}}_{\mathrm{M}} $$ as an estimate of λ. Then the estimate for S(t) would be exp (−$$ \widehat{\lambda} $$t) In comparison to the Kaplan-Meier, another parametric estimate for S(t) at time t j , described by Nelson [24], is

where δ i is the number of events in the i th interval and n i is the number at risk for the event. While this is a straightforward estimate, the Kaplan-Meier does not assume an underlying exponential distribution and thus is used more than this type of estimator.

Comparison of Two Survival Curves

We have just discussed how to estimate the survival curve in a clinical trial for a single group. For two groups, the survival curve would be estimated for each group separately. The question is whether the two survival curves S C (t) and S I (t), for the control and intervention groups respectively, are different based on the estimates Ŝ C (t) and Ŝ I (t).

Point-by-Point Comparison

One possible comparison between groups is to specify a time t* for which survival estimates have been computed using the Kaplan-Meier [13] method. At time t*, one can compare the survival estimates Ŝ C (t *) and Ŝ I (t *) using the statistic

where V[Ŝ C (t *)] and V[Ŝ I (t *)] are the Greenwood estimates of variance [19]. The statistic Z(t*) has approximately a normal distribution with mean zero and variance one under the null hypothesis that $$ {\widehat{S}}_C\left(t*\right)={\widehat{S}}_I\left(t*\right) $$. The problem with this approach is the multiple looks issue described in Chap. 16. Another problem exists in interpretation. For example, what conclusions should be drawn if two survival curves are judged significantly different at time t* but not at any other points? The issue then becomes, what point in the survival curve is most important.

For some studies with a T year follow-up, the T year mortality rates are considered important and should be tested in the manner just suggested. Annual rates might also be considered important and, therefore, compared. One criticism of this suggestion is that the specific points may have been selected post hoc to yield the largest difference based on the observed data. One can easily visualize two survival curves for which significant differences are found at a few points. However, when survival curves are compared, the large differences indicated by these few points are not supported by the overall survival experience. Therefore, point-by-point comparisons are not recommended unless a few points can be justified and specified in the protocol prior to data analysis.

Comparison of Median Survival Times

One summary measure of survival experience is the time at which 50% of the cohort has had the event. One common and easy way to estimate the median survival time is from the Kaplan-Meier curve. (See for example, Altman [1].) This assumes that the cohort has been followed long enough so that over one-half of the individuals have had the event. Confidence intervals may be computed for the median survival times [25]. If this is the case, we can compare the median survival times for intervention and control M I and M C , respectively. This is most easily done by estimating the ratio of the estimates M I /M C . A ratio larger than unity implies that the intervention group has a longer median survival and thus a better survival experience. A ratio less than unity would indicate the opposite.

We can estimate 95% confidence intervals for M I /M C by

where the standard deviation, SD, of M I /M C is computed as

for cases where the survival curves are approximately exponential, and O I  = the total number of events in the intervention group (i.e., ∑δ i ) and O C  = the total number of events in the control group.

Total Curve Comparison

Because of the limitations of comparison of point-by-point estimates, Gehan [26] and Mantel [27] originally proposed statistical methods to assess the overall survival experience. These two methods were important steps in the development of analytical methods for survival data. They both assume that the hypothesis being tested is whether two survival curves are equal, or whether one is consistently different from the other. If the two survival curves cross, these methods should be interpreted cautiously. Since these two original methods, an enormous literature has developed on comparison of survival curves and is summarized in several texts [8–11]. The basic methods described here provide the fundamental concepts used in survival analysis.

Mantel [27] proposed the use of the procedure described by Cochran [28] and Mantel and Haenszel [29] for combining a series of 2 × 2 tables. In this procedure, each time, t j , a death occurs in either group, a 2 × 2 table is formed as follows:

Death at time t j

Survivors at time t j

At risk prior to time t j

Intervention

a j

b j

a j + b j

Control

c j

d j

c j + d j

a j  + c j

b j  + d j

n j

The entry a j represents the observed number of deaths at time t j in the intervention group and c j represents the observed number of deaths at time t, in the control group. At least a j or c j must be non-zero. One could create a table at other time periods (that is, when a j and c j are zero), but this table would not make any contribution to the statistic. Of the n j participants at risk just prior to time t j , a j  + b j were in the intervention group and c j  + d j were in the control group. The expected number of deaths in the intervention group, denoted E(a j ), can be shown to be

and the variance of the observed number of deaths in the intervention group, denoted as V(a j ) is given by

These expressions are the same as those given for combining 2 × 2 tables in the Appendix of Chap. 17. The Mantel-Haenszel (MH) statistic is given by

and has approximately a chi-square distribution with one degree of freedom, where K is the number of distinct event times in the combined intervention and control groups. As an asymptotic approximation,

the (signed) square root of MH, can be compared to a standard normal distribution [30, 31].

Application of this procedure is straightforward. First, the times of events and losses in both groups are ranked in ascending order. Second, the time of each event, and the total number of participants in each group who were at risk just before the death (a j  + b j , c j  + d j ) as well as the number of events in each group (a j , c j ) are determined. With this information, the appropriate 2 × 2 tables can be formed.

Example

Assume that the data in the example shown in Table 15.2 represent the data from the control group. Among the 20 participants in the intervention group, two deaths occurred at 1.0 and 4.5 months with losses at 1.6, 2.4, 4.2, 5.8, 7.0, and 11.0 months. The observations, with parentheses indicating losses, can be summarized as follows:

Intervention: 1.0, (1.6), (2.4), (4.2), 4.5, (5.8), (7.0), (11.0)

Control: 0.5, (0.6), 1.5, 1.5, (2.0), 3.0, (3.5), (4.0), 4.8, 6.2, (8.5), (9.0), 10.5.

Using the data described above, with remaining observations being censored at 12 months, Table 15.3 shows the eight distinct times of death, (t j ), the number in each group at risk prior to the death, (a j  + b j , c j  + d j ), the number of deaths at time t j , (a j , c j ), and the number of participants lost to follow-up in the subsequent interval (l j ). The entries in this table are similar to those given for the Kaplan-Meier lifetable shown in Table 15.2. Note in Table 15.3, however, that the observations from two groups have been combined with the net result being more intervals. The entries in Table 15.3 labeled a j  + b j , c j  + d j , a j  + c j , and b j  + d j become the entries in the eight 2 × 2 tables shown in Table 15.4.

Table 15.3

Comparison of survival data for a control group and an intervention group using the Mantel-Haenszel procedures

Rank

Event times

Intervention

Control

Total

j

t j

a j  + b j

a j

l j

c j  + d j

c j

l j

a j  + c j

b j  + d j

1

0.5

20

0

0

20

1

1

1

39

2

1.0

20

1

0

18

0

0

1

37

3

4.5

19

0

2

18

1

1

2

35

4

3.0

14

0

1

15

2

2

1

31

5

4.5

16

1

0

12

0

0

1

27

6

4.8

15

0

1

12

0

0

1

26

7

6.2

14

0

1

11

2

2

1

24

8

10.5

13

0

13

8

7

7

1

20

a j  + b j  = number of participants at risk in the intervention group prior to the death at time t j

c j  + d j  = number of participants at risk in the control group prior to the death at time t j

a j  = number of participants in the intervention group who died at time t j

c j  = number of participants in the control group who died at time t j

l j  = number of participants who were lost or censored between time t j and t j+1

a j  + c j  = number of participants in both groups who died at time t j

b j  + d j  = number of participants in both group who are alive minus the number who died at time t j

Table 15.4

Eight 2 × 2 tables corresponding to the event times used in the Mantel-Haenszel statistic in survival comparison of intervention (I) and control (C) groups

1. (0.5 mo)a

D †

A ‡

R §

5. (4.5 mo)

D

A

R

I

0

20

20

I

1

15

16

C

1

19

20

C

0

12

12

1

39

40

1

27

28

2. (1 mo)

D

A

R

6. (4.8 mo)

D

A

R

I

1

19

20

I

0

15

15

C

0

18

18

C

1

11

12

1

37

38

1

26

27

3. (1.5 mo)

D

A

R

7. (6.2 mo)

D

A

R

I

0

19

19

I

0

14

14

C

2

16

18

C

1

10

11

2

35

37

1

24

25

4. (3 mo)

D

A

R

8. (10.5 mo)

D

A

R

I

0

17

17

I

0

13

13

C

1

14

15

C

1

7

8

1

31

32

1

20

21

aNumber in parenthesis indicates time, t j , of a death in either group

D †  = number of participant who died at time t j

A ‡  = number of participants who are alive between time t j and time t j+1

R §  = number of participants who were at risk before death at time t j (R = D + A)

The Mantel-Haenszel statistic can be computed from these eight 2 × 2 tables (Table 15.4) or directly from Table 15.3. The term $$ {\displaystyle {\sum}_{j=1}^8{a}_j=2} $$ since there are only two deaths in the intervention group. Evaluation of the term $$ {\displaystyle {\sum}_{j=1}^8E\left({a}_j\right)}=20/40+20/38+2\times 19/37+17/32+16/28+15/27+14/25+13/21 $$ or $$ {\displaystyle {\sum}_{j=1}^8E\left({a}_j\right)=4.89} $$. The value for $$ {\displaystyle {\sum}_{j=1}^8V\left({a}_j\right)} $$ is computed as

This term is equal to 2.21. The computed statistic is MH = (2 − 4.89)²/2.21 = 3.78. This is not significant at the 0.05 significance level for a chi-square statistic with one degree of freedom. The MH statistic can also be used when the precise time of death is unknown. If death is known to have occurred within an interval, 2 × 2 tables can be created for each interval and the method applied. For small samples, a continuity correction is sometimes used. The modified numerator is

where the vertical bars denote the absolute value. For the example, applying the continuity correction reduces the MH statistic from 3.76 to 2.59.

Gehan [26] developed another procedure for comparing the survival experience of two groups of participants by generalizing the Wilcoxon rank statistic. The Gehan statistic is based on the ranks of the observed survival times. The null hypothesis, S I (t) = S C (t), is tested. The procedure, as originally developed, involved a complicated calculation to obtain the variance of the test statistic. Mantel [32] proposed a simpler version of the variance calculation, which is most often used.

The N I observations from the intervention group and the N C observations from the control group must be combined into a sequence of N C  + N I observations and ranked in ascending order. Each observation is compared to the remaining N C  + N I  − 1 observation and given a score U i which is defined as follows:

The survival outcome for the i th participant will certainly be larger than that for participants who died earlier. For censored participants, it cannot be determined whether survival time would have been less or greater than the i th observation. This is true whether the i th observation is a death or a loss. Thus, the first part of the score U i assesses how many deaths definitely preceded the i th observation. The second part of the U i score considers whether the current, i th , observation is a death or a loss. If it is a death, it definitely precedes all later ranked observations regardless of whether the observations correspond to a death or a loss. If the i th observation is a loss, it cannot be determined whether the actual survival time will be less than or greater than any succeeding ranked observation, since there was no opportunity to observe the i th participant completely.

Table 15.5 ranks the 40 combined observations (N C  = 20, N I  = 20) from the example used in the discussion of the Mantel-Haenszel statistic. The last 19 observations were all censored at 12 months of follow-up, 7 in the control group and 12 in the intervention group. The score U ₁ is equal to the zero observations that were definitely less than 0.5 months, minus the 39 observations that were definitely greater than 0.5 months, or U ₁ = −39. The score U ₂ is equal to the one observation definitely less than the loss at 0.6 months, minus none of the observations that will be definitely greater, since at 0.6 months the observation was a loss, or U ₂ = 1. U ₃ is equal to the one observation (0.5 months) definitely less than 1.0 month minus the 37 observations definitely greater than 1.0 month giving U ₃ = 36. The last 19 observations will have scores of 9 reflecting the nine deaths which definitely precede censored observations at 12.0 months.

Table 15.5

Example of Gehan statistics scores U i for intervention (I) and control (C) groups

Observation I

Ranked observed time

Group

Definitely less

Definitely more

U i

1

0.5

C

0

39

−39

2

(0.6)a

C

1

0

1

3

1.0

I

1

37

−36

4

1.5

C

2

35

−33

5

1.5

C

2

35

−33

6

(1.6)

I

4

0

4

7

(2.0)

C

4

0

4

8

(2.4)

I

4

0

4

9

3.0

C

4

31

−27

10

(3.5)

C

5

0

5

11

(4.0)

C

5

0

5

12

(4.2)

I

5

0

5

13

4.5

I

5

27

−22

14

4.8

C

6

26

−20

15

(5.8)

I

7

0

7

16

6.2

C

7

24

−17

17

(7.0)

I

8

0

8

18

(8.5)

C

8

0

8

19

(9.0)

C

8

0

8

20

10.5

C

8

20

−12

21

(11.0)

I

9

0

9

22–40

(12.0)

12I,7C

9

0

9

aParentheses indicate censored observations

The Gehan statistic, G, involves the scores U i and is defined as

where W = Σ U i , (for U i ’s in control group only) and

The G statistic has approximately a chi-square distribution with one degree of freedom [26, 32]. Therefore, the critical value is 3.84 at the 5% significance level and 6.63 at the 1% level. In the example, W = −87 and the variance V(W) = 2,314.35. Thus, G = (−87)²/2,314.35 = 3.27 for which the p-value is equal to 0.071. This is compared with the p-value of 0.052 obtained using the Mantel-Haenszel statistic.

The Gehan statistic assumes the censoring pattern to be equal in the two groups. Breslow [33] considered the case in which censoring patterns are not equal and used the same statistic G with a modified variance. This modified version should be used if the censoring patterns are radically different in the two groups. Peto and Peto [34] also proposed a version of a censored Wilcoxon test. The concepts are similar to what has been described for Gehan’s approach. However, most software packages now use the Breslow or Peto and Peto versions.

Generalizations

The general methodology of comparing two survival curves using this methodology has been further evaluated [35–40]. These two tests by Mantel-Haenzel and Gehan, can be viewed as a weighted sum of the difference between observed number of events and the expected number at each unique event time [7, 40]. Consider the previous equation for the logrank test and rewrite the numerator as

where

and w j is a weighting factor. The test statistic W ² /V(W) has approximately a chi-square distribution with one degree of freedom or equivalently $$ W/\sqrt{V(W)} $$ has approximately a standard normal distribution. If w i  = 1, we obtain the Mantel-Haenszel or logrank test. If w i  = n j /(N + 1), where N = N C  + N I or the combined sample size, we obtain the Gehan version of the Wilcoxon test. Tarone and Ware [40] pointed out that the Mantel-Haenszel and Gehan are only two possible statistical tests. They suggested a general weight function w i  = [n j /(N + 1)] θ where 0 < θ < 1. In particular, they suggested that θ = 0.5. Prentice [38] suggested a weight $$ {w}_j=I{I}_{i=1}^j{n}_i/\left({n}_i+{d}_i\right) $$ where d i  = (a i  + c i ) which is related to the product limit estimator at t j as suggested by Peto and Peto [34]. Harrington and Fleming [35] generalize this further by suggesting weights $$ {w}_j={\left\{{\prod}_{i=1}^j{n}_i/\left({n}_i+{d}_i\right)\right\}}^{\rho } $$ for ρ > 0.

All of these methods give different weights to the various parts of the survival curve. The Mantel-Haenszel or logrank statistic is more powerful for survival distributions of the exponential form where λ I (t) = θ λ C (t) or S I (t) = {S C (t)} θ where θ ≠ 1 [32]. The Gehan type statistic [26], on the other hand, is more powerful for survival distributions of the logistic form S(t,θ) = e t + θ /(1 + e t + θ ). In actual practice, however, the distribution of the survival curve of the study population is not known. When the null hypothesis is not true, the Gehan type statistic gives more weight to the early survival experience, whereas the Mantel-Haenszel weights the later experience more. Tarone and Ware [40] indicate other possible weighting schemes could be proposed which are intermediate to these two statistics [35, 40]. Thus, when survival analysis is done, it is certainly possible to obtain different results using different weighting schemes depending on where the survival curves separate, if they indeed do so. The logrank test is the standard in many fields such as cancer and heart disease. The condition λ I (t) = θ λ C (t) says that risk of the event being studied in the intervention is a constant multiple of the hazard λ C (t). That is, the hazard rate in one arm is proportional to the other and so the logrank test is best for testing proportional hazards. This idea is appealing and is approximately true for many studies.

There has been considerable interest in asymptotic (large sample) properties of rank tests [37, 39] as well as comparisons of the various analytic methods [36]. While there exists an enormous literature on survival analysis, the basic concepts of rank tests can still be appreciated by the methods described above.

Earlier, we discussed using an exponential model to summarize a survival curve where the hazard rate λ determines the survival curve. If we can assume that the hazard rate is reasonably constant during the period of follow-up for the intervention and the control group, then comparison of hazard rates is a comparison of survival curves [1]. The most commonly used comparison is the ratio of the hazards, R = λ I /λ C . If the ratio is unity, the survival curves are identical. If R is greater than one, the intervention hazard is greater than control so the intervention survival curve falls below the standard curve. That is, the intervention is worse. On the other hand, if R is less than one, the control group hazard is larger, the control group survival curve falls below the intervention curve, and intervention is better.

We can estimate the hazard ratio by comparing the ratio of total observed events (O) divided by expected number of events (E) in each group; that is, the estimate of R can be expressed as

That is, O I = Σa i , O C = Σ b i , E I = ΣE(a i ), and E C =ΣE(b i ). Confidence intervals for the odds ratio R are most easily determined by constructing confidence intervals for the log of the odds ratio ln R [41]. The 95% confidence interval for ln R is $$ K-1.96/\sqrt{V} $$ to $$ K+1.96/\sqrt{V} $$ where K = (O I  − E I )/V and V is the variance as defined in the logrank or Mantel-Haenszel statistics. (That is, V equals V(a i ).) We then connect confidence intervals for ln R to confidence intervals for R by taking antilogs of the upper and lower limit. If the confidence interval excludes unity, we could claim superiority of either intervention or control depending on the direction. Hazard ratios not included in the interval can be excluded as likely outcome summaries of the intervention. If the survival curves have relatively constant hazard rates, this method provides a nice summary and complements the Kaplan-Meier estimates of the survival curves.

Covariate Adjusted Analysis

Previous chapters have discussed the rationale for taking stratification into account. If differences in important covariates or prognostic variables exist at entry between the intervention and control groups, an investigator might be concerned that the analysis of the survival experience is influenced by that difference. In order to adjust for these differences in prognostic variables, she could conduct a stratified analysis or a covariance type of survival analysis. If these differences are not important in the analysis, the adjusted analysis will give approximately the same results as the unadjusted.

Three basic techniques for stratified survival analysis are of interest. The first compares the survival experience between the study groups within each stratum, using the methods described in the previous section. By comparing the results from each stratum, the investigator can get some indication of the consistency of results across strata and the possible interaction between strata and intervention.

The second and third methods are basically adaptations of the Mantel-Haenszel and Gehan statistics, respectively, and allow the results to be accumulated over the strata. The Mantel-Haenszel stratified analysis involves dividing the population into S strata and within each stratum j, forming a series of 2 × 2 tables for each K j event, where K j is the number of events in stratum j. The table for the i th event in the j th stratum would be as follows:

Event

Alive

Intervention

a ij

b ij

a ij  + b ij

Control

c ij

d ij

c ij  + d ij

a ij  + c ij

b ij  + d ij

n ij

The entries a ij , b ij , c ij , and d ij are defined as before and

Similar to the non-stratified case, the Mantel-Haenszel statistic is

which has a chi-square distribution with one degree of freedom. Analogous to the Mantel-Haenszel statistic for stratified analysis, one could compute a Gehan statistic W j and V(W j ) within each stratum. Then an overall stratified Gehan statistic is computed as

which also has chi-square statistic with one degree of freedom.

If there are many covariates, each with several levels, the number of strata can quickly become large, with few participants in each. Moreover, if a covariate is continuous, it must be divided into intervals and each interval assigned a score or rank before it can be used in a stratified analysis. Cox [42] proposed a regression model which allows for analysis of censored survival data adjusting for continuous as well as discrete covariates, thus avoiding these two problems.

One way to understand the Cox regression model is to again consider a simpler parametric model. If one expresses the probability of survival to time t, denoted S(t), as an exponential model, then S(t) = e −λt where the parameter, λ, is called the force of mortality or the hazard rate as described earlier. The larger the value of λ, the faster the survival curve decreases. Some models allow the hazard rate to change with time, that is λ = λ(t). Models have been proposed [43–45] which attempt to incorporate the hazard rate as a linear function of several baseline covariates, x ₁, x ₂, …, x p that is, λ(x ₁, x ₂, …, x p ) = b ₁ x ₁ + b ₂ x ₂ + … + b p x p . One of the covariates, say x ₁, might represent the intervention and the others, for example, might represent age, sex, performance status, or prior medical history. The coefficient, b ₁, then would indicate whether intervention is a significant prognostic factor, i.e., remains effective after adjustment for the other factors. Cox [42] suggested that the hazard rate could be modeled as a function of both time and covariates, denoted λ(t, x ₁, x ₂, …, x p ). Moreover, this hazard rate could be represented as the product of two terms, the first representing an unadjusted force of mortality λ ₀(t) and the second the adjustment for the linear combination of a particular covariate profile. More specifically, the Cox proportional hazard model assumes that

That is, the hazard λ(t, x ₁, x ₂, …, x n ) is proportional to an underlying hazard function λ ₀(t) by the specific factor exp (b ₁ x ₁ + b₂ x ₂ …). From this model, we can estimate an underlying survival curve S ₀(t) as a function of λ ₀(t). The survival curve for participants with a particular set of covariates X, S(t,x) can be obtained as $$ S\left(t,x\right)={\left[{S}_0(t)\right]}^{exp\left({b}_1{x}_1+{b}_2{x}_2+\dots \right)} $$. Other summary test statistics from this model are also used. The estimation of the regression coefficients b ₁,b ₂, …, b p is complex, requiring non-linear numerical methods, and goes beyond the scope of this text. Many elementary texts on biostatistics [1, 3, 5, 46] or review articles [7] present further details. A more advanced discussion may be found in Kalbfleish and Prentice [10] or Fleming and Harrington [9]. Programs exist in many statistical computing packages which provide these estimates and summary statistics to evaluate survival curve comparisons [20–23]. Despite the complexity of the parameter estimation, this method is widely applied and has been studied extensively [47–55]. Pocock, Gore, and Kerr [52] demonstrate the value of some of these methods with cancer data. For the special case where group assignment is the only covariate, the Cox model is essentially equivalent to the Mantel-Haenszel statistic.

One issue that is sometimes raised is whether the hazard rates are proportional over time. Methods such as the Mantel-Haenszel logrank test or the Cox Proportional Hazards model are optimal when the hazards are proportional [9]. However, though there is some loss of power, these methods perform well as long as the hazard curves do not cross, even if proportionality does not hold [56]. When the hazards are not proportional, which intervention is better depends on what time point is being referenced. If a significant difference is found between two survival curves using the Mantel-Haenszel logrank test or the Cox Proportional Hazards model when the hazards are not proportional, the two curves are still significantly different. For example, time to event curves are shown in Chap. 18. Figure 18.​2a shows three curves for comparison of two medical devices with best medical or pharmacologic care. These three curves do not have proportional hazards but the comparisons are still valid and in fact the two devices demonstrate statistically significant superiority over the best medical care arm. The survival curves do not cross although are close together in the early months of follow-up.

The techniques described in this chapter as well as the extensions or generalizations referenced are powerful tools in the analysis of survival data. Perhaps none is exactly correct for any given set of data but experience indicates they are fairly robust and quite useful.

References

1.

Altman DG. Practical Statistics for Medical Research. Chapman and Hall, 2015.

2.

Armitage P, Berry G, Mathews J. Statistical Methods in Medical Research, ed 4th. Malden MA, Blackwell Publishing, 2002.CrossRef

3.

Breslow NE. Comparison of survival curves; in Buyse B, Staquet M, Sylvester R (eds): The Practice of Clinical Trials in Cancer. Oxford, Oxford University Press, 1982.

4.

Brown BW, Hollander M. Statistics: A Biomedical Introduction. Wiley, 2009.

5.

Fisher L, Van Belle G, Heagerty PL, Lumley TS. Biostatistics—A Methodology for the Health Sciences. New York, John Wiley and Sons, 2004.MATH

6.

Woolson RF, Clarke WR. Statistical Methods for the Analysis of Biomedical Data. Wiley, 2011.

7.

Crowley J, Breslow N. Statistical Analysis of Survival Data. Annu Rev Public Health 1984;5:385–411.CrossRef

8.

Cox DR, Oakes D. Analysis of Survival Data. Taylor & Francis, 1984.

9.

Fleming TR, Harrington DP. Counting Processes and Survival Analysis. Wiley, 2011.

10.

Kalbfleisch JD, Prentice RL. The Statistical Analysis of Failure Time Data. Wiley, 2011.

11.

Miller RG. Survival Analysis. Wiley, 2011.

12.

Cutler SJ, Ederer F. Maximum utilization of the life table method in analyzing survival. J Chronic Dis 1958;8:699–712.CrossRef

13.

Kaplan EL, Meier P. Nonparametric Estimation from Incomplete Observations. J Am Stat Assoc 1958;53:457–481.MathSciNetCrossRefMATH

14.

Chan MCY, Giannetti N, Kato T, et al. Severe tricuspid regurgitation after heart transplantation. J Heart Lung Transplant 2001;20:709–717.CrossRef

15.

Kumagai R, Kubokura M, Sano A, et al. Clinical evaluation of percutaneous endoscopic gastrostomy tube feeding in Japanese patients with dementia. Psychiatry Clin Neurosci 2012;66:418–422.CrossRef

16.

Lara-Gonzalez JH, Gomez-Flores R, Tamez-Guerra P, et al. In Vivo Antitumor Activity of Metal Silver and Silver Nanoparticles in the L5178Y-R Murine Lymphoma Model. Br J Med Med Res 2013;3:1308–1316.CrossRef

17.

Miyamoto K, Aida A, Nishimura M, et al. Gender effect on prognosis of patients receiving long-term home oxygen therapy. The Respiratory Failure Research Group in Japan. Am J Respir Crit Care Med 1995;152:972–976.CrossRef

18.

Sarris GE, Robbins RC, Miller DC, et al. Randomized, prospective assessment of bioprosthetic valve durability. Hancock versus Carpentier-Edwards valves. Circulation 1993;88:II55-II64.CrossRef

19.

Greenwood M. The natural duration of cancer. Reports on Public Health and Medical Subjects 1926;33:1–26.

20.

Everitt BS, Rabe-Hesketh S. Handbook of Statistical Analyses Using Stata, Fourth Edition. Taylor & Francis, 2006.

21.

R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria, R Foundation for Statistical Computing, 2013.

22.

SAS Institute: SAS/STAT 12.1 User’s Guide: Survival Analysis. SAS Institute, 2012.

23.

TIBCO Software I: SPLUS. TIBCO Softward Inc., 2008.

24.

Nelson W. Hazard plotting for incomplete failure data. Journal of Quality Technology 1969;1:27–52.

25.

Brookmeyer R, Crowley J. A Confidence Interval for the Median Survival Time. Biometrics 1982;38:29–41.CrossRefMATH

26.

Gehan EA. A Generalized Wilcoxon Test for Comparing Arbitrarily Singly-Censored Samples. Biometrika 1965;52:203–223.MathSciNetCrossRefMATH

27.

Mantel N. Evaluation of survival data and two new rank order statistics arising in its consideration. Cancer chemotherapy reports Part 1 1966;50:163–170.

28.

Cochran WG. Some Methods for Strengthening the Common-ç2 Tests. Biometrics 1954;10:417–451.MathSciNetCrossRefMATH

29.

Mantel N, Haenszel W. Statistical Aspects of the Analysis of Data From Retrospective Studies of Disease. J Natl Cancer Inst 1959;22:719–748.

30.

Crowley J, Breslow N. Remarks on the Conservatism of Sigma(0-E)2/E in Survival Data. Biometrics 1975;31:957–961.CrossRefMATH

31.

Peto R, Pike MC. Conservatism of the Approximation Sigma (O-E)2-E in the Logrank Test for Survival Data or Tumor Incidence Data. Biometrics 1973;29:579–584.MathSciNetCrossRef

32.

Mantel N. Ranking Procedures for Arbitrarily Restricted Observation. Biometrics 1967;23:65-78.CrossRef

33.

Breslow NE. A Generalized Kruskal-Wallis Test for Comparing K Samples Subject to Unequal Patterns of Censorship. Biometrika 1970;57:579–594.CrossRefMATH

34.

Peto R, Peto J. Asymptotically Efficient Rank Invariant Test Procedures. J R Stat Soc Ser A 1972;135:185–207.CrossRefMATH

35.

Harrington DP, Fleming TR. A Class of Rank Test Procedures for Censored Survival Data. Biometrika 982;69:553–566.

36.

Leurgans S. Three Classes of Censored Data Rank Tests: Strengths and Weaknesses under Censoring. Biometrika 1983;70:651–658.MathSciNetCrossRefMATH

37.

Oakes D. The Asymptotic Information in Censored Survival Data. Biometrika 1977;64:441–448.MathSciNetCrossRefMATH

38.

Prentice RL. Linear Rank Tests with Right Censored Data. Biometrika 1978;65:167–179.MathSciNetCrossRefMATH

39.

Schoenfeld DA. The Asymptotic Properties of Nonparametric Tests for Comparing Survival Distributions. Biometrika 1981;68:316–319.MathSciNetCrossRef

40.

Tarone RE, Ware J. On Distribution-Free Tests for Equality of Survival Distributions. Biometrika 1977;64:156–160.MathSciNetCrossRefMATH

41.

Simon R. Confidence Intervals for Reporting Results of Clinical Trials. Ann Intern Med 1986;105:429–435.CrossRef

42.

Cox DR. Regression Models and Life-Tables. J R Stat Soc Series B Stat Methodol 1972;34:187–220.MathSciNetMATH

43.

Feigl P, Zelen M. Estimation of Exponential Survival Probabilities with Concomitant Information. Biometrics 1965;21:826–838.CrossRef

44.

Prentice RL, Kalbfleisch JD. Hazard Rate Models with Covariates. Biometrics 1979;35:25–39.MathSciNetCrossRefMATH

45.

Zelen M. Application of Exponential Models to Problems in Cancer Research. J R Stat Soc Ser A 1966;129:368–398.CrossRef

46.

Fisher L, Van Belle G. Biostatistics—A Methodology for the Health Sciences. New York, John Wiley and Sons, 1993.MATH

47.

Breslow NE. Covariance Analysis of Censored Survival Data. Biometrics 1974;30:89–99.CrossRef

48.

Breslow NE. Analysis of Survival Data under the Proportional Hazards Model. International Statistical Review/Revue Internationale de Statistique 1975;43:45–57.MATH

49.

Efron B. The Efficiency of Cox’s Likelihood Function for Censored Data. J Am Stat Assoc 1977;72:557–565.MathSciNetCrossRefMATH

50.

Kalbfleisch JD, Prentice RL. Marginal Likelihoods Based on Cox’s Regression and Life Model. Biometrika 1973;60:267–278.MathSciNetCrossRefMATH

51.

Kay R. Proportional Hazard Regression Models and the Analysis of Censored Survival Data. J R Stat Soc Ser C Appl Stat 1977;26:227–237.

52.

Pocock SJ. Interim analyses for randomized clinical trials: the group sequential approach. Biometrics 1982;38:153–162.CrossRef

53.

Prentice RL, Gloeckler LA. Regression Analysis of Grouped Survival Data with Application to Breast Cancer Data. Biometrics 1978;34:57–67.CrossRefMATH

54.

Schoenfeld DA. Chi-Squared Goodness-of-Fit Tests for the Proportional Hazards Regression Model. Biometrika 1980;67:145–153.MathSciNetCrossRefMATH

55.

Tsiatis AA. A Large Sample Study of Cox’s Regression Model. Ann Stat 1981;9:93–108.MathSciNetCrossRefMATH

56.

Lagakos SW, Schoenfeld DA. Properties of Proportional-Hazards Score Tests under Misspecified Regression Models. Biometrics 1984;40:1037–1048.MathSciNetCrossRefMATH

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_16

16. Monitoring Committee Structure and Function

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

The investigator’s ethical responsibility to the study participants demands that safety and clinical benefit be monitored during trials. If data partway through the trial indicate that the intervention is harmful to the participants, early termination of the trial should be considered. If these data demonstrate a clear definitive benefit from the intervention, the trial may also be stopped early because continuing would be unethical to the participants in the control group. In addition, if differences in primary and possibly secondary response variables are so unimpressive that the prospect of a clear result is extremely unlikely, it may not be justifiable in terms of time, money, and effort to continue the trial. Also, monitoring of response variables can identify the need to collect additional data to clarify questions of benefit or toxicity that may arise during the trial. Finally, monitoring may reveal logistical problems or issues involving data quality that need to be promptly addressed. Thus, there are ethical, scientific, and economic reasons for interim evaluation of a trial [1–3]. In order to fulfill the monitoring function, the data must be collected and processed in a timely fashion as the trial progresses. Data monitoring would be of limited value if conducted only at a time when all or most of the data had been collected. The specific issues related to monitoring of recruitment, adherence, and quality control are covered in other chapters and will not be discussed here. The monitoring committee process has been described in detail [4] as have case studies representing trials, which were terminated for benefit, harm, or futility [5]. One of the earliest discussions of the basic rationale for data monitoring was included in a report of a committee initiated at the request of the council advisory to the then National Heart Institute and chaired by Bernard Greenberg [3]. This report outlined a clinical trial model depicted in Fig. 16.1, variations of which have been implemented widely by institutes at the National Institutes of Health (NIH). The key components are the Steering Committee, the Statistical and Data Coordinating Center, the Clinics, and the Data Monitoring Committee. Later the pharmaceutical and device industries [6] adopted a modified version of this NIH model, depicted in Fig. 16.2. The main modification was to separate the Statistical Data Coordinating Center into a Statistical Data Analysis Center and a Data Coordinating Center. Many of the early experiences have been described and formed the basis of current practice [7–34], particularly in trials of cardiovascular disease [35–37]. Over the past decade especially, the number of DMCs has increased dramatically [38]. In 2013, of over 120,000 trials registered in Clintrials.gov, more than 13,000 were interventional trials and 40% of those included a DMC. This suggests over 5,000 DMCs are or have existed in this period of time. The highest DMC utilization was in cardiovascular and oncology trials. Of the 630 trials that were NIH sponsored, 74% had a DMC. For the 55% that were industry sponsored, about a third had a DMC. Some of this difference is reflected in the written policies or guidelines by the NIH and the FDA. The Office of Inspector General (OIG) issued a report in 1998 that reviewed the adequacy of IRB oversight in clinical trials and recommended that the NIH and the FDA provide guidance on when a more focused monitoring committee might be needed. In response, NIH issued a policy statement that was consistent with their longstanding practice in many of their institutes of having an independent DMC for all Phase III randomized trials that they funded [39]. Soon after, the FDA began to develop a guidance document that was issued as a draft in 2001 and finalized in 2006 [40]. The FDA guidance recommended DMCs for trials with high risk patients or high risk or novel interventions, not all Phase III or IV trials conducted by industry.

Fig. 16.1

The NIH Clinical Trial Model

Fig. 16.2

The Industry Modified Clinical Trial Model [6]

Prior to the year 2000, the general public was generally not aware of the longstanding practice of data monitoring committees providing oversight to Phase III trials especially. However, the death of a gene transfer patient at a leading research institution changed that [41]. While this patient was not in a Phase III trial, the issues surrounding the case drew attention as to who was responsible for monitoring trials and to whom or what body should such monitoring be reported to. The proximity of this event in time with the NIH and FDA guidance certainly enhanced public awareness and DMC activity came under increased scrutiny by a variety of interested parties. The US Secretary of Health and Human Resources became aware of these events and also reaffirmed the policies and practices of the NIH and the FDA [39, 40, 42]. It has also become clear that for large multicenter trials, an individual IRB, often deluged by sporadic SAEs from the sponsor, is not able to ascertain if there is compelling evidence of risk or benefit based on accumulating data according to (often blinded) treatment in the trial. Thus, that critical role in assuring patient safety can only be played by the DMC.

While all trials need some level of monitoring, many trials such as early phase trials, single center trials, a very simple intervention trial or a trial not involving vulnerable populations, may not need an external monitoring committee. External monitoring, using an independent committee, is used mostly in later phase trials that could lead to change in clinical practice or where special expertise is needed. A survey of monitoring practices conducted by the DAMOCLES group found that the roles of monitoring committees varied widely across trials, sponsors, and regions. While there was a general agreement about the types of trials that needed formal monitoring committees, there was not a uniform practice or policy as to their function [43]. External monitoring committees go by a variety of names such as data and safety monitoring board (DSMB), data and safety monitoring committee (DSMC) or simply Data Monitoring Committee (DMC). In this text, we prefer using DMC since it does not focus on safety when in fact the challenge is to review the risks and the benefits of a new intervention.

The principles and fundamentals expressed in this book reflect the experience of the authors in monitoring numerous trials since the early 1970s.

Fundamental Point

During the trial, response variables need to be monitored for early dramatic benefits or potential harmful effects or futility. Monitoring should be done by a person or group independent of the investigator.

Monitoring Committee

Keeping in mind the scientific, ethical, and economic rationales, data and safety monitoring is not simply a matter of looking at tables or results of statistical analysis of the primary outcome. Rather, it is an active process in which additional tabulations and analysis are suggested and evolve as a result of ongoing review. Monitoring also involves an interaction between the individuals responsible for collating, tabulating, and analyzing the data. For single center studies, the monitoring responsibility could, in principle, be assumed by the investigator. However, he may find himself in a difficult situation. While monitoring the data, he may discover that the results trend in one direction or the other while participants are still being enrolled and/or treated. Presumably, he recruits participants to enter a trial on the basis that he favors neither intervention nor control, a state of clinical equipoise [44]. Knowing that a trend exists may make it difficult for him to continue enrolling participants. It is also difficult for the investigator to follow, evaluate, and care for the participants in an unbiased manner knowing that a trend exists. Furthermore, the credibility of the trial is enhanced if, instead of the investigator, an independent person monitors the response variable data. Because of these considerations, we recommend that for later phase trials the individuals who monitor a clinical trial have no formal involvement with the participants or the investigators, although some disagree [11, 19, 20].

Except for small, short-term studies which could be early or late phase, when one or two knowledgeable individuals may suffice, the responsibility for monitoring response variable data is usually placed with an independent group with expertise in various disciplines [4–6]. The independence protects the members of the monitoring committee from being influenced in the decision-making process by investigators, participants as well as federal or industry sponsors. The committee would usually include experts in the relevant clinical fields or specialties, individuals with experience in the conduct of clinical trials, epidemiologists, biostatisticians knowledgeable in design and analysis, and often for NIH funded trials a bioethicist or participant advocate. While we will describe statistical procedures that are often helpful in evaluating interim results in Chap. 17, the decision process to continue, terminate a trial early, or modify the design is invariably complex and no single statistical procedure is adequate to address all these complexities. Furthermore, no single individual is likely to have all the experiences and expertise to deal with these issues. Thus, as was recommended in the Greenberg Report [3], we suggest that the independent monitoring committee have a multidisciplinary membership.

The first priority of the monitoring committee must be to ensure the safety of the participants in the trial. The second priority is to the investigators and the Institutional Review Boards or ethics committees, who place an enormous trust in the monitoring committee both to protect their participants from harm and to ensure the integrity of the trials. Third, the monitoring committee has a responsibility to the sponsor of the trial, whether it be federal or private. Finally, the monitoring committee provides a service to the drug or device regulatory agency, especially for trials which are utilizing drugs, biologics or devices which still have investigational status.

Although many formats for monitoring committee meetings have been used, one that we recommend allows for exchange of information by all relevant parties and for appropriate confidential and independent review [4, 13]. The format utilizes an open session, a closed session, and an executive session. The open session enables interaction between investigator representatives such as the study principal investigator or chair, the sponsor, the statistical center, the relevant industrial participants, and the monitoring committee. It is uncommon but permissible for a regulatory agency to participate in an open session of the meeting. In this session, issues of participant recruitment, data quality, general adherence, toxicity issues, and any other logistical matter that may affect either the conduct or outcome of the trial are considered in a blinded fashion. After a thorough discussion, the monitoring committee would go into a closed session with DMC members and the statistical reporting statistician or team where analyses of the confidential unblinded outcome data are reviewed. This review would include comparison by intervention groups of baseline variables, primary or secondary variables, safety or adverse outcome variables, adherence measures for the entire group, and examinations of any relevant subgroups. Following this review, the monitoring committee may decide to move into an executive session with DMC members only where decisions about continuation, termination or protocol modification are made. After the DMC review has been completed in closed sessions, they may meet with a representative of the sponsor or investigator leadership to share their recommendations which are usually followed up in a letter. Regardless of how formal, most monitoring committee meetings have such components. One variation is that the DMC meeting begins with a closed session which allows the members to discuss any issues that they want to raise with the investigators and sponsors in the subsequent open session. This discussion may also serve to identify what issues will be central in the second closed session. Thus, the sequence is closed executive, open, closed and ending with an open debriefing session. This particular model, for example, has been used extensively in NIH-sponsored AIDS trials [13].

Before a trial begins and the first monitoring committee meeting is scheduled, it must be decided specifically who attends the various sessions, as outlined above. In general, attendance should be limited to those who are essential for proper monitoring. As noted, it is common for the study principal investigator and sponsor representatives to attend the first open session. If he or she does not provide care for participants in the trial, the principal investigator will sometimes attend the closed session; however, that practice is not recommended. If the study is sponsored by industry, independence and credibility of the study is best served by no industry attendance at the closed session. Industry sponsored trials that are also managed and analyzed by industry will require a biostatistician from the sponsor who prepares the monitoring report to attend. In such situations the company statistician must have a “firewall” separating her from other colleagues at the company, something that may be difficult to achieve in a way that is convincing to outsiders. However, a common practice for industry-sponsored pivotal Phase III trials is for a separate statistical analysis center to provide the interim analyses and report to the independent monitoring committee [6]. This practice reduces the possibility or perception that interim results are known within the industry sponsor, or the investigator group. Regulatory agency representatives usually do not attend the closed session because being involved in the monitoring decision may affect their regulatory role, should the product be submitted for subsequent approval.

An executive session should involve only the voting members of the monitoring committee, although the independent statistician who provided the data report may also attend. There are many variations of this general outline, including a merger of the closed and executive session since attendance may involve the same individuals.

Most monitoring committees evaluate one, or perhaps two, clinical trials. When a trial is completed, that monitoring committee is dissolved. However, as exemplified by cancer and AIDS, ongoing networks of clinical centers conduct many trials concurrently [11, 13, 18–20, 23]. Cancer trial cooperative groups may conduct trials across several cancer sites, such as breast, colon, lung or head, and neck at any given time, and even multiple trials for a given site depending upon the stage of the cancer or other risk factors. The AIDS trial networks in the United States have likewise conducted trials simultaneously in AIDS patients at different stages of the disease. In these areas, monitoring committees may follow the progress of several trials. In such instances, a very disciplined agenda and a standardized format of the data report enhance the efficiency of the review. If there is a research program of several trials evaluating a new drug, a common DMC may have the advantage of being able to monitor a larger combined experience that will provide for more precise estimates of safety and efficacy. Regardless of the model, the goals and procedures are similar.

Another factor that needs to be resolved before the start of a trial is how the intervention or treatment comparisons will be presented to the monitoring committee. In some trials, the monitoring committee knows the identity of the interventions in each table or figure of the report. In other trials, for two interventions the tables may be labelled as A and B with the identity of A and B remaining blinded until the DMC requests the unblinding on a “need to know” basis. Thus, if there are no trends in either benefit or harm, which is likely to be the case early in a trial, there is no overwhelming reason to know the identity of groups A and B. When trends begin to emerge in either direction, the monitoring committee should have full knowledge of the group identities [45].

In some trials, the monitoring committee is blinded throughout the interim monitoring. In order to achieve this, data reports have complex labeling schemes, such as A versus B for baseline tables, C versus D for primary outcomes, E versus F for toxicity, and G versus H for various laboratory results. While this degree of blinding may enhance objectivity, it may conflict with the monitoring committee’s primary purpose of protecting the participants in the trial from harm or unnecessary continuation. As pointed out by Whitehead [46], the intention of this approach is to deny the DMC a complete picture of the interim data. To assess the progress of the trial, the harm and benefit profile of the intervention must be well understood and the possible tradeoffs weighed. If each group of tables is labeled by a different code, the committee cannot easily assess the overall harm/benefit profile of the intervention, and thus may put participants at unnecessary risk or continue a trial beyond the point at which benefit outweighs risks. Such complex coding schemes also increase the chance for errors in labeling. This practice is not common and not recommended.

No simple formula can be given for how often a monitoring committee should meet. The frequency may vary depending on the phase of the trial [2, 4, 5, 47]. Participant recruitment, follow-up, and closeout phases require different levels of activity. Meetings should not be so frequent that little new data are accumulated in the interim, given the time and expense of convening a committee. If potential toxicity of one of the interventions becomes an issue during the trial, special meetings may be needed. In many long-term clinical trials, the monitoring committees have met regularly at 4- to 6-month intervals, with additional meetings or telephone conferences as needed. In some circumstances, an annual review may be sufficient. However, less frequent review is not recommended since too much time may elapse before a serious adverse effect is uncovered. As described later, another strategy is to schedule monitoring committee meetings when approximately 10, 25, 50, 75, and 100% of the primary outcomes have been observed, or some similar pattern. Thus, there might be an early analysis to check for serious immediate adverse effects with later analyses to evaluate evidence of intervention benefit or harm. Other approaches provide for additional in-between analyses if strong, but as yet non-significant trends emerge. Between committee meetings, the person or persons responsible for collating, tabulating, and analyzing the data assume the responsibility for monitoring unusual situations which may need to be brought to the attention of the monitoring committee.

A monitoring committee often reviews the data for the last time before the data file is closed, and may never see the complete data analysis except as it appears in the publication. There is currently no consistent practice as to whether a monitoring committee meets to review the final complete data set. From one perspective, the trial is over and there is no need for the committee to meet since early termination or protocol modification is no longer an option. From another perspective, the committee has become very familiar with the data, including issues of potential concern, and thus may have insight to share with the investigators and study sponsors. Some trials have scheduled this final meeting to allow the monitoring committee to see the final results before they are presented at a scientific meeting or published.

Based on our experience, we strongly recommend this latter approach. There is a great deal to be gained for the trial and the investigators at a very modest cost. Other remaining issues still need to be resolved. For example, if a worrisome safety trend or a significant finding is not reported clearly or at all in the primary publication, what are the scientific, ethical, and legal obligations for the monitoring committee to comment on what is not reported? Suppose the committee differs substantially in the interpretation of the primary or safety outcomes? What is the process for resolving differences between it and the investigators or sponsor? These are important questions and the answers are not simple or straightforward, yet are relevant for science and ethics.

Repeated Testing for Significance

In the discussion on sample size (Chap. 8) the issue of testing several hypotheses was raised and referred to as the “multiple testing” problem. Similarly, repeated significance testing of accumulating data is essential to the monitoring function has statistical implications [48–54]. These issues are discussed in more detail in Chap. 17 but the concept of repeated testing is described here. If the null hypothesis, H ₀, of no difference between two groups is, in fact, true, and repeated tests of that hypothesis are made at the same level of significance using accumulating data, the probability that, at some time, the test will be called significant by chance alone is larger than the significance level selected in the sample size computation. That is, the rate of incorrectly rejecting the null hypothesis, or making a false positive error, will be larger than what is normally considered acceptable. Trends may emerge and disappear, especially early in the trial, and caution must be used.

In a clinical trial in which the participant response is known relatively soon after entry, the difference in rates between two groups may be compared repeatedly as more participants are added and the trial continues. The usual test statistic for comparing two proportions used is the chi-square test or the equivalent normal test statistic. The null hypothesis is that the true response rates or proportions are equal. If a significance level of 5% is selected and the null hypothesis, H ₀, is tested only once, the probability of rejecting H ₀ if it is true is 5% by definition. However, if H ₀ is tested twice, first when one-half of the data are known and then when all the data are available, the probability of incorrectly rejecting H ₀ is increased from 5 to 8% [50]. If the hypothesis is tested five times, with one-fifth of the participants added between tests, the probability of finding a significant result if the usual statistic for the 5% significance level is used becomes 14%. For ten tests, this probability is almost 20%.

In a clinical trial in which long-term survival experience is the primary outcome, repeated tests might be done as more information becomes known about the enrolled participants. Canner [10] performed computer simulations of such a clinical trial in which both the control group and intervention group event rates were assumed to be 30% at the end of the study. He performed 2,000 replications of this simulated experiment. He found that if 20 tests of significance are done within a trial, the chance of crossing the 5% significance level boundaries (i.e., Z = ±1.96) is, on the average, 35%. Thus, whether one calculates a test statistic for comparing proportions or for comparing time to event data, repeated testing of accumulating data without taking into account the number of tests increases the overall probability of incorrectly rejecting H ₀ and claiming an intervention effect. If the repeated testing continues indefinitely, the null hypothesis is certain to be rejected eventually. Although it is unlikely that a large number of repeated tests will be done, even five or ten can lead to a misinterpretation of the results of a trial if the multiple testing issues are ignored.

A classic illustration of the repeated testing problem is provided by the Coronary Drug Project (CDP) for the clofibrate versus placebo mortality comparison, shown in Fig. 16.3 [10, 54]. This figure presents the standardized mortality comparisons over the follow-up or calendar time of the trial. The two horizontal lines indicate the conventional value of the test statistic, corresponding to a two-sided 0.05 significance level, used to judge statistical significance for studies where the comparison is made just one time. It is evident that the trends in this comparison emerge and weaken throughout, coming close or exceeding the conventional critical values on five monitoring occasions. However, as shown in Fig. 16.4, the mortality curves at the end of the trial are nearly identical, corresponding to the very small standardized statistic at the end of the Fig. 16.3. The monitoring committee for this trial took into consideration the repeated testing problem and did not terminate this trial early just because the conventional values were exceeded.

Fig. 16.3

Interim survival analyses comparing mortality in clofibrate- and placebo-treated participants in the Coronary Drug Project. A positive Z value favors placebo [9]

Fig. 16.4

Cumulative mortality curves comparing clofibrate- and placebo treated participants in the Coronary Drug Project [9]

For ethical, scientific, and economic reasons, all trials must be monitored so as not to expose participants to possible harm unnecessarily, waste precious fiscal and human resources, or miss opportunities to correct flaws in the design [2–5]. However, in the process of evaluating interim results to meet these responsibilities, incorrect conclusions can be drawn by overreacting to emerging or non-emerging trends in primary, secondary or adverse effect outcomes. In general, the solution to multiple testing is to adjust the critical value used in each analysis so that the overall significance level for the trial remains at the desired level. It has been suggested that a trial should not be terminated early unless the difference between groups is very significant [2, 4, 5, 55]. More formal monitoring techniques are reviewed in the next chapter, Chap. 17. They include the group sequential methods and stochastic curtailed sampling or conditional power procedures.

Decision for Early Termination

There are five major valid reasons for terminating a trial earlier than scheduled [2, 4, 5, 9, 10]. First, the trial may show serious adverse effects in the entire intervention group or in a dominating subgroup. Second, the trial may indicate greater than expected beneficial effects. Third, it may become clear that a statistically significant difference by the end of the study is improbable, sometimes referred to as being futile. Fourth, logistical or data quality problem may be so severe that correction is not feasible or participant recruitment is far behind and not likely to achieve the target. Fifth, the question posed may have already been answered elsewhere or may no longer be sufficiently important. A few trials have been terminated because the sponsor decided the trial was no longer a priority but this causes serious ethical dilemmas for investigators and leaves participants having contributed without getting an answer to the posed question.

For a variety of reasons, a decision to terminate a study early must be made with a great deal of caution and in the context of all pertinent data. A number of issues or factors must be considered thoroughly as part of the decision process:

1.

Possible differences in prognostic factors between the two groups at baseline.

2.

Any chance of bias in the assessment of response variables, especially if the trial is not double-blind.

3.

The possible impact of missing data. For example, could the conclusions be reversed if the experience of participants with missing data from one group were different from the experience of participants with missing data from the other group?

4.

Differential concomitant intervention and levels of participant adherence.

5.

Potential adverse events and outcomes of secondary response variables in addition to the outcome of the primary response variable.

6.

Internal consistency. Are the results consistent across subgroups and the various primary and secondary outcome measures? In a multicenter trial, the monitoring committee should assess whether the results are consistent across centers. Before stopping, the committee should make certain that the outcome is not due to unusual experience in only one or two centers.

7.

In long-term trials, the experience of the study groups over time. Survival analysis techniques (Chap. 15) partly address this issue.

8.

The outcomes of similar trials.

9.

The impact of early termination on the credibility of the results and acceptability by the clinical community.

Some trials request the chair of the monitoring committee to review frequently serious adverse events, by intervention, to protect the safety of the participants. While such frequent informal, or even formal, review of the data is also subject to the problems of repeated testing or analyses, the adjustment methods presented are typically not applied. Also, safety may be measured by many response variables. Rather than relying on a single outcome showing a worrisome trend, a profile of safety measures might be required. Thus, the decision to stop a trial for safety reasons can be quite complex.

The early termination of a clinical trial can be difficult [2, 9, 10, 12, 55–60], not only because the issues involved may be complex and the study complicated but also because the final decision often lies with the consensus of a committee. The statistical methods discussed in the next chapter are useful guides in this process but should not be viewed as absolute rules. A compilation of diverse monitoring experiences is available [5]. A few examples are described here to illustrate key points. One of the earlier clinical trials conducted in the United States illustrates how controversial the decision for early termination may be. The University Group Diabetes Program (UGDP) was a placebo-control, randomized, double-blind trial designed to test the effectiveness of four interventions used in the treatment of diabetes [61–64]. The primary measure of efficacy was the degree of retinal damage. The four interventions were: a fixed dose of insulin, a variable dose of insulin, tolbutamide and phenformin. After the trial was underway, study leaders formed a committee to review accumulating safety data. This committee membership consisted of individuals involved in the UGDP and external consultants. The tolbutamide group was stopped early because the monitoring committee thought the drug could be harmful and did not appear to have any benefit [64]. An excess in cardiovascular mortality was observed in the tolbutamide group as compared to the placebo group (12.7% vs. 4.9%) and the total mortality was in the same direction (14.7% vs. 10.2%). Analysis of the distribution of the baseline factors known to be associated with cardiovascular mortality revealed an imbalance, with participants in the tolbutamide group being at higher risk. This, plus questions about the classification of cause of death, drew considerable criticism. Later, the phenformin group was also stopped because of excess mortality in the control group (15.2% vs. 9.4%) [61]. The controversy led to a further review of the data by an independent group of statisticians. Although they basically concurred with the decisions made by the UGDP monitoring committee [61], the debate over the study and its conclusion continued [63]. This trial certainly highlighted the need for an independent review of the interim data to assess safety.

The decision-making process during the course of the CDP [65] a long-term randomized, double-blind, multicenter study that compared the effect on total mortality of several lipid-lowering drugs (high- and low-dose estrogen, dextrothyroxine, clofibrate, nicotinic acid) against placebo has been reviewed [5, 9, 54, 65, 66]. Three of the interventions were terminated early because of potential adverse effects and no apparent benefit. One of the issues in the discontinuation of the high dose estrogen and dextrothyroxine interventions [65, 67] concerned subgroups of participants. In some subgroups, the interventions appeared to cause increased mortality, in addition to having a number of other adverse effects. In others, the adverse effects were present, but mortality was only slightly reduced or unchanged. The adverse effects were thought to more than outweigh the minimal benefit in selected subgroups. Also, positive subgroup trends in the dextrothyroxine arm were not maintained over time. After considerable debate, both interventions were discontinued. The low dose estrogen intervention [66] was discontinued because concerns over major toxicity. Furthermore, it was extremely improbable that a significant difference in a favorable direction for the primary outcome (mortality) could have been obtained had the study continued to its scheduled termination. Using the data available at the time, the number of future deaths in the control group was projected. This indicated that there had to be almost no further deaths in the intervention group for a significance level of 5% to be reached.

The CDP experience also warns against the dangers of stopping too soon [9, 54]. In the early months of the study, clofibrate appeared to be beneficial, with the significance level reaching or exceeding 5% on five monitoring occasions (Fig. 16.3). However, because of the repeated testing issue described earlier in this chapter, the decision was made to continue the study and closely monitor the results. The early difference was not maintained, and at the end of the trial the drug showed no benefit over placebo. It is notable that the mortality curves shown in Fig. 16.4 do not suggest the wide swings observed in the interim analyses shown in Fig. 16.3. The fact that participants were entered over a period of time and thus had various lengths of follow-up at any given interim analysis, explains the difference between the two types of analyses. (See Chap. 15 for a discussion of survival analysis.).

Pocock [55] also warns about the dangers of terminating trials too early for benefit, reflecting on a systematic review of trials stopped early [59]. At an early interim analysis, the Candesartan in Heart failure Assessment of Reduction in Mortality and Morbidity (CHARM) trial [68] had a 25% mortality benefit (p < 0.001) from candesartan compared to a placebo control, but for a variety of reasons the trial continued and found after a median of 3 years of follow-up only a 9% nonsignificant difference in mortality. Continuing the trial revealed that the early mortality benefit was probably exaggerated and allowed other long-term intervention effects to be discovered. In general, trials stopped early for benefit often do not report in sufficient detail the rationale for early termination and often show implausibly large intervention effects based on only a small number of events [57]. This phenomenon is well recognized [58]. Thus, while there are sound ethical reasons to terminate trials early because of benefit, these decisions must be cautioned by our experience with early trends not being reliable or sustainable. Nevertheless, there is a natural tension between getting the estimate of treatment benefit precise and allowing too many participants to be exposed to the inferior intervention [57]. Statistical methods to be described in the next chapter are useful as guidelines but not adequate as rules and the best approach based on experience is to utilize a properly constituted monitoring committee, charged with weighing the benefits and risks of early termination.

The Nocturnal Oxygen Therapy Trial was a randomized, multicenter clinical trial comparing two levels of oxygen therapy in people with advanced chronic obstructive pulmonary disease [69, 70]. While mortality was not considered as the primary outcome in the design, a strong mortality difference emerged during the trial, notably in one particular subgroup. Before any decision was made, the participating clinical centers were surveyed to ensure that the mortality data were as current as possible. A delay in reporting mortality was discovered and when all the deaths were considered, the trend disappeared. The earlier results were an artifact caused by incomplete mortality data. Although a significant mortality difference ultimately emerged, the results were similar across subgroups in contrast to the results in the earlier review.

Early termination of a subgroup can be especially error prone if not done carefully. Peto and colleagues [71] have illustrated the danger of subgroup analysis by reporting that treatment benefit in ISIS-2 did not apply to individuals born during a certain astrologic sign. Nevertheless, treatment benefits may be observed in subgroups which may be compelling. An AIDS trial conducted by the AIDS Clinical Trial Research Group (ACTG), ACTG-019 [5, 6, 13] indicated that zidovudine (AZT) led to improved outcome in participants who had a low laboratory value (CD4 cell counts under 500, which is a measure of poor immune response). The results were not significant for participants with a higher CD4 value. Given previous experience with this drug, and given the unfavorable prognosis for untreated AIDS patients, the trial was stopped early for benefit in those with the low CD4 cell count but continued in the rest of the participants.

A scientific and ethical issue was raised in the Diabetic Retinopathy Study, a randomized trial of 1,758 participants with proliferative retinopathy [72, 73]. Each participant had one eye randomized to photocoagulation and the other to standard care. After 2 years of a planned 5 year follow-up, a highly significant difference in the incidence of blindness was observed (16.3% vs. 6.4%) in favor of photocoagulation [74]. Since the long-term efficacy of this new therapy was not known, the early benefit could possibly have been negated by subsequent adverse effects. After much debate, the monitoring committee decided to continue the trial, publish the early results, and allow any untreated eye at high risk of blindness to receive photocoagulation therapy [75]. In the end, the early treatment benefit was sustained over a longer follow-up, despite the fact that some of the eyes randomized to control received photocoagulation. Furthermore, no significant long-term adverse effect was observed.

The Beta-Blocker Heart Attack Trial provided another example of early termination [76, 77]. This randomized placebo control trial enrolled over 3,800 participants with a recent myocardial infarction to evaluate the effectiveness of propranolol in reducing mortality. After an average of a little over 2 years of a planned 3 year follow-up, a mortality difference was observed, as shown in Fig. 16.5. The results were statistically significant, allowing for repeated testing, and would, with high probability, not be reversed during the next year [77]. The data monitoring committee debated whether the additional year of follow-up would add valuable information. It was argued that there would be too few events in the last year of the trial to provide a good estimate of the effect of propranolol treatment in the third and fourth year of therapy. Thus, the committee decided that prompt publication of the observed benefit was more important than waiting for the marginal information yet to be obtained. This trial was one of the early trials to implement group sequential monitoring boundaries discussed in the next chapter and will be used as an example to illustrate the method.

Fig. 16.5

Cumulative mortality curves comparing propranolol and placebo in the Beta-Blocker Heart Attack Trial [75]

Another example of using sequential monitoring boundaries is found in chronic heart failure trials that evaluated different beta blockers. Common belief had been that administering a beta-blocker drug to a heart failure patient would cause harm, not benefit. Fortunately, early research suggested this belief may have been in error and ultimately four well designed trials were conducted to evaluate the risks and benefits. Three trials were terminated early because of beneficial intervention effect on mortality of 30–35% [78–80]. The fourth trial [81] did not go to completion in part due to the fact that the other three trials had already reported substantial benefits. Details of monitoring in one of the trials, the Metoprolol CR/XL Randomized Trial In Chronic Heart Failure (MERIT-HF) are discussed more fully in Chap. 17.

Some trials of widely used interventions have also been stopped early due to adverse events. One classic example comes from the treatment of arrhythmias following a heart attack. Epidemiological data showed an association between the presence of irregular ventricular heartbeats or arrhythmias and the incidence of sudden death, presumably due to serious arrhythmias. Drugs were developed that suppressed such arrhythmias and they became widely used after approval by the drug regulatory agency for that indication. The Cardiac Arrhythmia Suppression Trial (CAST) was a multicenter randomized double blind placebo-controlled trial evaluating the effects of three such drugs (encainide, flecainide, moricizine) on total mortality and sudden death [82]. Statistical procedures used in CAST to address the repeated testing problem [83, 84] are described in the next chapter. However, the encainide and flecainide arms of the trial were terminated after only 15% of the expected mortality events observed because of an adverse effect (63 deaths in the two active arms vs. 26 deaths in the corresponding placebo arms).

At the first monitoring committee review, the mortality trend in CAST began to appear but the number of events was relatively small [83]. Because the monitoring committee decided no definitive conclusion could be reached on the basis of so few events, it elected to remain blinded to the treatment assignment. However, before the next scheduled meeting, the statistical center alerted the committee that the trends continued and were now nearing the CAST monitoring criteria for stopping. In a conference call meeting, the monitoring committee became unblinded and learned that the trends were in the unexpected direction, that is, toward harm from the active treatment. A number of confirmatory and exploratory analyses were requested by the monitoring committee and a meeting was held a few weeks later to discuss fully these unexpected results. After a thorough review, the monitoring committee recommended immediate termination of the encainide and flecainide portions of the trial [83]. Results were consistent across outcome variables and participant subgroups, and no biases could be identified which would explain these results. The third arm (moricizine) continued since there were no convincing trends at that time, but it too was eventually stopped due to adverse experiences [85]. The CAST experience points out that monitoring committees must be prepared for the unexpected and that large trends may emerge quickly. Even in this dramatic result, the decision was not simple or straightforward. Many of the issues discussed earlier were covered thoroughly before a decision was reached [83].

Not all negative trends emerge as dramatically as in the CAST. Two other examples are provided by trials in congestive heart failure. Yearly mortality from severe congestive heart failure is approximately 40%. The Prospective RandOmized MIlrinone Survival Evaluation (PROMISE) [36] and the Prospective RandOmized FlosequInan Longevity Evaluation (PROFILE) [35] trials evaluated inotropic agents (milrinone and flosequinone). Both of these drugs had been approved by regulatory agencies for use on the basis of improved exercise tolerance, which might be considered a surrogate response for survival. PROMISE and PROFILE were randomized placebo controlled trials comparing mortality outcomes. Both trials were unexpectedly terminated early due to statistically significant harmful mortality results, even after adjusting for repeated testing of these data. Because severe heart failure has a high mortality rate and the drugs were already in use, it was a difficult decision how long and how much evidence was needed to decide that the intervention was not helpful but was in fact harmful. In both trials, the monitoring committees allowed results to achieve statistical significance since a negative, but nonsignificant trend might have been viewed as evidence consistent with no effect on mortality.

Another trial in acute coronary syndromes, the Thrombin Receptor Antagonist for Clinical Event Reduction in Acute Coronary Syndrome (TRACER) trial, evaluated a thrombin antagonist with a composite outcome of cardiovascular death, myocardial infraction, stroke, recurrent ischemia with rehospitalization or urgent coronary revascularization [86]. The trial of 12,944 patients randomized 1:1 between the thrombin antagonist and placebo was terminated for safety reasons with a positive but non-significant emerging trend in the primary outcome. There were 1,031 primary events in the treated patients and 1,102 in the placebo controls. The secondary composite of cardiovascular death, MI and stroke had 822 vs 910 events (P = 0.02). However, the rates of intracranial bleeding was 1.2% vs 0.2% yielding a hazard ratio of 3.39 (P < 0.001). The data monitoring group decided that the serious bleeding risks overwhelmed any emerging benefits.

The PROMISE and PROFILE experiences illustrate the most difficult of the monitoring scenarios, the emerging negative trend, but they are not unique [87–91]. Trials with persistent nonsignificant negative trends may have no real chance of reversing and indicating a benefit from intervention. In some circumstances, that observation may be sufficient to end the trial since if a result falls short of establishing benefit, the intervention would not be used. For example a new expensive or invasive intervention would likely need to be more effective than a standard intervention to be used. In other circumstances, a neutral result may be important, so a small negative trend, still consistent with a neutral result, would argue for continuation. If a treatment is already in clinical use on the basis of other indications, as in the case of the drugs used in PROMISE and PROFILE, an emerging negative trend may not be sufficient evidence to alter clinical practice. If a trial terminates early without resolving convincingly the harmful effects of an intervention, that intervention may still continue to be used. This practice would put future patients at risk, and perhaps even participants in the trial as they return to their usual healthcare system. In that case, the investment of participants, investigators, and sponsors would not have resolved an important question. There is a serious and delicate balance between the responsibility to safeguard the participants in the trial and the responsibility for all concurrent and future patients [87].

Trials may continue to their scheduled termination even though interim results are very positive and persuasive [92] or the intervention and control data are so similar that almost surely no significant results will emerge [93–96]. In one study of antihypertensive therapy, early significant results did not override the need for getting long-term experience with an intensive intervention strategy [92]. Another trial [95] implemented approaches to reduce cigarette smoking, change diet to lower cholesterol, and used antihypertensive medications to lower blood pressure in order to reduce the risk of heart disease. Although early results showed no trends, it was also not clear how long intervention needed to be continued before the applied risk factor modifications would take full effect. It was argued that late favorable results could still emerge. In fact, they did, though not until some years after the trial had ended [96]. In a trial that compared medical and surgical treatment of coronary artery atherosclerosis, the medical care group had such a favorable survival experience that there was little room for improvement by immediate coronary artery bypass graft intervention [94].

The Women’s Health Initiative (WHI) was one of the largest and most complex trials ever conducted, certainly in women [97, 98]. This partial factorial trial evaluated three interventions in postmenopausal women: (1) hormone replacement therapy (HRT), (2) a low fat diet, and (3) calcium and vitamin D supplementation. Each intervention, in principle, could affect multiple organ systems, each with multiple outcomes. For example, HRT was being evaluated for its effect on cardiovascular events such as mortality and fatal and non-fatal myocardial infarction. HRT can also affect bone density, the risk of fracture, and breast cancer. The HRT component was also stratified into those with an intact uterus, who received both estrogen and progestin, and those without a uterus who received estrogen alone. The estrogen–progestin arm was terminated early due to increases in deep vein thrombosis, pulmonary embolism, stroke, and breast cancer and a trend toward increased heart disease as shown in Fig. 16.6 although there was a benefit in bone fracture as expected [98]. There was no observed difference in total mortality or the overall global index, the composite outcome defined in the protocol, as shown in Fig. 16.7. The WHI is an excellent example of the challenges of monitoring trials with composite outcomes where component trends are not consistent. In such cases, the most important or most clinically relevant component may have to dominate in the decision process, even if not completely specified in the protocol or the monitoring committee charter. Later, the WHI estrogen-alone arm was also terminated, primarily due to increased pulmonary embolus and stroke, though there was no difference in myocardial infarction or total mortality [97]. The formal monitoring process had to account for multiple interventions, multiple outcomes and repeated testing.

Fig. 16.6

WHI Kaplan-Meier estimates of cumulative hazards for selected clinical outcomes [94]. HR hazard ratio, nCI nominal confidence interval, aCI adjusted confidence interval

Fig. 16.7

WHI Kaplan-Meier estimates of cumulative hazards for global index and death [94]. HR hazard ratio, nCI nominal confidence interval, aCI adjusted confidence interval

A heart failure trial evaluating the drug tezosentan used a stopping criterion that included futility [99]. That is, when there was less than a 10% chance of having a positive beneficial result, the monitoring committee was to alert the investigators and sponsors and recommend termination. In fact, at about two-thirds of the way into the trial, a slightly negative trend was sufficient to make any chance of a beneficial result unlikely and the trial was terminated.

In some instances, a trial may be terminated because the hypothesis being tested has been convincingly answered by other ongoing trials. This was the case with trials evaluating warfarin in the treatment of atrial fibrillation [100]. Between 1985 and 1987, five trials were launched to evaluate warfarin to prevent strokes in participants with atrial fibrillation. Three of the trials were terminated early by 1990, reporting significant reductions in embolic complications. One of the remaining trials was also terminated early, largely due to the ethical aspects of continuing trials when the clinical question being tested has already been answered. The window of opportunity to further evaluate the intervention had closed.

The Justification for the Use of Statin in Prevention: An Intervention Trial Evaluating Rosuvastatin (JUPITER) trial compared a statin agent, which lowers both LDL cholesterol and C-reactive protein, in 17,802 patients with elevated high-sensitivity C-reactive protein levels but without hyperlipidemia [101]. The primary outcome was the occurrence of the combination of myocardial infarction, stroke, arterial revascularization, hospitalization for unstable angina, or death from cardiovascular causes. The trial demonstrated a clear statin effect of lowering LDL even further as well as lowering C-reactive protein levels and demonstrated a corresponding lowering of the primary outcome (hazard ratio (HR) of .56, p < 0.00001). Similar reductions were observed for myocardial infarction (HR, 0.46), for stroke (HR, 0.52), for revascularization or unstable angina (HR, 0.53), for the combined end point of myocardial infarction, stroke, or death from cardiovascular causes (HR, 0.53), and for death from any cause (HR, 0.80), all being statistically significant. In addition, all of the major predefined subgroups were consistent. Still, there was criticism that the cardiovascular mortality was not significant even though overall mortality was [102, 103]. This raises the difficult question when using combined outcomes as the primary if each component or at least some components should also be statistically significant before terminating a trial. In general, trials are not designed to demonstrate statistically significant results for any of the components usually due to low events for each of them. To do so would require trials much larger than the one designed. If a component of the combined outcome is of paramount importance, then that outcome should be established as the primary and the trial designed accordingly as described in Chaps. 3 and 8. In the case of the JUPITER trial, the results for the primary outcome and nearly all of its components as well as overall mortality appear to be compelling for a trial to be terminated. This is especially the case when total mortality is significantly reduced in addition to the primary. Another approach to a focus on a component of the primary outcome was in the CHARM program, in which three trials that comprised the overall program each had cardiovascular death and heart failure hospitalization as its primary outcome, and the overall program was powered to assess all-cause mortality. The DMC focused on the effect on mortality in the overall program as the criterion for early termination [68].

As we have already discussed, the decision to terminate a trial is complex. It is never based on a single outcome and may require more than one DMC meeting before a recommendation to terminate is reached. Timing of the recommendation can also be questioned by those external to the trial. In the Investigation of Lipid Level Management to Understand its Impact in Atherosclerotic Events (ILLUMINATE) trial [104], a new agent torcetrapib, a cholesterylester transfer protein inhibitor that increases HDL cholesterol, was tested to reduce major cardiovascular events. ILLUMINATE was a randomized, double-blind study involving 15,067 patients at high cardiovascular risk, receiving either torcetrapib plus atorvastatin (a statin which lowers LDL cholesterol) or atorvastatin alone. The primary outcome was defined as time to death from coronary heart disease, nonfatal myocardial infarction, stroke, or hospitalization for unstable angina, whichever occurred first. ILLUMINATE clearly demonstrated an increase in HDL, which would be expected to cause a reduction in cardiovascular risk. However, the trial was terminated early by the DMC and the investigators because of an increased risk of death and cardiac events in patients receiving torcetrapib [104]. To conclude that torcetrapib improved HDL but caused harmful clinical effects was of course disappointing since this was the first testing of an exciting new class of drugs. However, the timing of the recommendation to terminate was challenged by a regulatory agency, which recognized the complexity of such decisions but argued that the trial could and perhaps should have been terminated earlier [105]. Determining at what point there is sufficient and compelling evidence to make a recommendation for termination is often challenging. DMCs do not have the benefit of hindsight while in process of monitoring a trial.

On occasion, trials may have achieved a significant benefit, or show strong trends for benefit, but the DMC recommend early termination for safety reasons. Two trials, the Thrombin Receptor Antagonist in Secondary Prevention of Atherothrombotic Ischemic Events (TRA 2P) trial [106] and TRACER [86] provide examples of such instances. Both trials evaluated a new platelet inhibition agent vorapaxar compared with placebo. TRA 2P had the primary outcome as a composite of death from cardiovascular causes, myocardial infarction, or stroke. TRACER had a composite outcome of death from cardiovascular causes, myocardial infarction, stroke, recurrent ischemia with rehospitalization, or urgent coronary revascularization. Both trials, TRA 2P with 26,449 patients and TRACER with 12,944 patients, had statistically significant beneficial effects in their respective primary outcomes (HR of 0.87 and 0.89). However, the DMCs for both trials recommended early termination and/or modification of the protocol for unacceptable bleeding complications including intracranial hemorrhage.

In all of these studies, the decisions were difficult and involved many analyses, thorough review of the literature, and an understanding of the biological processes. As described above, a number of questions must be answered before serious consideration should be given to early termination. As noted elsewhere, the relationship between clinical trials and practice is very complex and this complexity is evident in the monitoring process [107, 108].

Decision to Extend a Trial

The question of whether to extend a trial beyond its original sample size or planned period of follow-up may arise. Suppose the mortality rate over a 2-year period in the control group is assumed to be 40%. (This estimate may be based on data from another trial involving a similar population.) Also specified is that the sample size should be large enough to detect a 25% reduction due to the intervention, with a two-sided significance level of 5% and a power of 90%. The total sample size is, therefore, approximately 960. However, say that early in the study, the mortality rate in the control group appears somewhat lower than anticipated, closer to 30%. This difference may result from a change in the study population, selection factors in the trial, or new concomitant therapies. If no design changes are made, the intervention would have to be more effective (30% reduction rather than 25%) for the difference between groups to be detected with the same power. Alternatively, the investigators would have to be satisfied with approximately 75% power of detecting the originally anticipated 25% reduction in mortality. If it is unreasonable to expect a 30% benefit and if a 75% power is unacceptable, the design needs modification. Modifying the design to increase sample size or extend follow-up can inflate type 1 error if it is done with knowledge of the observed intervention effect and not pre-specified. Even when the process is pre-specified, because they may be aware of other data suggesting reasons not to extend the trial, the DMC is usually not involved in such decisions. Changes may be made by a third party either according to a pre-specified plan, or with access to certain summaries of follow-up data, but not to estimates of the intervention effect.

In the above example, given the lower control group mortality rate, approximately 1,450 participants would be required to detect a 25% reduction in mortality, while maintaining a power of 90%. Another option is to extend the length of follow-up, which would increase the total number of events. A combination of these two approaches can also be tried (e.g. [109]). Another approach that has been used [35, 36] is to fix the target of the trial to be a specified number of primary events in the control group or the overall number. This is often referred to as “event driven” trials. If event rates are low, it may take longer follow-up per participant or more randomized participants, or both, to reach the required number of events. In any case, the target is the number of events. In the above situations, only data from the control group or the combined groups are used. No knowledge of what is happening in the intervention group is needed. In our example, if the event rate is closer to 30% than the assumed 40%, then the expected number of events under the null hypothesis, 390, would not be achieved. The trial could achieve the prespecified target number of events by increasing recruitment, increasing the length of follow-up or a combination.

The concept of adaptive designs has already been discussed in Chap. 5. Adaptive designs can be used in trials with overall lower event rates or increased variability, or when emerging trends are smaller than planned for but yet of clinical interest. Modifying the design once the trial is underway due to lower event rates or increased variability is rather straightforward. In a trial of antenatal steroid administration [110], the incidence of infant respiratory distress in the control group was much less than anticipated. Early in the study, the investigators decided to increase the sample size by extending the recruitment phase. In another trial, the protocol specifically called for increasing the sample size if the control group event rate was less than assumed [111]. As described in the sample size chapter, power is the probability of detecting a treatment effect if there truly is an effect. This probability is computed at the beginning of the trial during the design phase. The design goal is to set this probability at a range from 0.80 to 0.95 with an appropriate sample size. Sometimes this probability, or power, is referred to as “unconditional power” to distinguish it from “conditional power” to be described in more detail in the next chapter. Adjustments to sample size based on overall event rates or variability estimates can preserve the power (or unconditional power). No account of emerging trends is used in this recalculation.

The issue of whether the control group event rate or the overall event rate should be used in this sample size reassessment must be considered. It might seem intuitive that the emerging control group event rate should be used since it was the estimated control group rate that was initially used in the sample size calculation, as described in Chap. 8. However, to reveal the control group rate to the investigators may unblind the emerging trend if they are also aware of the overall number of events. The use of the overall event rate would avoid this potential problem. Additionally, there are statistical arguments that under the null hypothesis, the overall rate is the more appropriate one to use because it is likely to be more stable, particularly if the sample size re-estimation is done early in the trial. Many prefer to use the overall event rate, but in either case, this must be decided while the protocol and data monitoring procedures are being developed.

However, modifying the design based on emerging trends is more complicated (see Chap. 5) and will be discussed in more technical detail in the next chapter. Despite the statistical literature for different approaches [112–115] and some criticism [116, 117], only a few applications of this type of adaptive design have been utilized. One such trial is the African-American Heart Failure Trial (A-HeFT) [118], a trial in African Americans with advanced heart failure using a combination of two established drugs. The primary outcome consisted of a weighted score of death, hospitalization, and quality of life. Mortality was among the secondary outcomes. The trial utilized an adaptive design [113] that required the monitoring committee to assess variability of this novel primary outcome and the emerging trend to make sample size adjustment recommendations to the trial leaders. The reason for the adaptive design was that little previous data were available for this combined outcome so estimates of variability were not adequate to compute a reliable sample size. Little experience with the outcome also limited the assessment of potential drug effect on this outcome. A group sequential boundary was established using a Lan–DeMets alpha spending function of the O’Brien–Fleming type (see Chapter 17) for monitoring benefit or harm for the composite outcome, as described in the next chapter. This adaptive procedure was followed as planned and the sample size was increased from 800 to 1,100. Meanwhile, the monitoring committee was observing a mortality trend favoring the combination drug but there was no sequential monitoring plan prespecified for this outcome. The monitoring committee elected to utilize the same sequential boundary specified for the primary composite outcome to monitor mortality. Although not ideal while the trial was ongoing, it was done before the mortality difference became nominally significant. At the last scheduled meeting of the monitoring committee, the difference was nominally significant at the 0.05 level but had not crossed the sequential boundary. The committee decided to conduct an additional review of the data. At that additional review, the mortality difference was nominally significant (p = 0.01) and had, in fact, crossed the sequential O’Brien–Fleming boundary. The committee recommended early termination both because of a significant mortality benefit and a primary outcome that was nominally significant, along with a consistency across the components of the composite outcome and relevant subgroups.

While the statistical methods for adaptive designs based on emerging trends to reset the sample size exist, the use of these methods is still evolving. A more technical discussion of specific trend adaptive designs is provided in the next chapter. One concern is whether the application of the pre-specified algorithm, according to the statistical plan, may reveal information about the size and direction of the emerging trend to those blind to the data. These algorithms can be “reverse engineered” to obtain a reasonable estimate of the emerging trend. We know of no example to date where this revelation has caused a problem but in principle this could create bias in participant selection or recruitment efforts or even participant assessment. Thus, mechanisms for implementation of trend adaptive trials are needed that protect the integrity of the trial.

If only the control group event rate and not the intervention effect is used in the recalculation of sample size, then an increase could be recommended when the observed difference between the intervention and control groups is actually larger than originally expected. Thus, in the hypothetical example described above, if early data really did show a 30% benefit from intervention, an increased sample size might not be needed to maintain the desired power of 90%. For this reason, despite the shortcomings of existing adaptive designs, one would not like to make a recommendation about extension without also considering the observed effect of intervention. Computing conditional power is one way of incorporating these results, and some methods in the adaptive design literature have formalized such as approach [112]. Conditional power is the probability that the test statistic will be larger than the critical value, given that a portion of the statistic is already known from the observed data and described in the next chapter. As in other power calculations, the assumed true difference in response variables between groups must be specified. When the early intervention experience is better than expected, the conditional power will be large. When the intervention is doing worse than anticipated, the conditional power will be small. The conditional power concept utilizes knowledge of outcome in both the intervention and control groups and is, therefore, controversial. Nevertheless, the concept attempts to quantify the decision to extend.

Whatever adjustments are made to either sample size or the length of follow-up, they should be made as early in the trial as possible or as part of a planned adaptive design strategy. Early adjustments should diminish the criticism that the trial leadership waited until the last minute to see whether the results would achieve some prespecified significance level before changing the study design. The technical details for the statistical methods to adjust sample size based on interim results are covered in the next chapter.

As mentioned earlier, one challenge in adaptive designs using interim effects that remains unresolved is who should make the calculations and perhaps recommend a sample size increase. The monitoring committee is of course aware of the interim results for the primary outcome but also for the other secondary outcomes and adverse event outcomes as well as overall conduct. The sample size calculations based on the primary events and emerging trends may recommend an increase in sample size but the overall profile of the intervention effects may not support such an increase. Knowing all of the interim results may place the DMC in an awkward and even ethical dilemma.

Traditionally, DMCs have not been directly involved in the trial design except to possible terminate a trial early. Based on our experience to date, we do not recommend that the monitoring committee engage in the adaptive design if based on emerging trends.

Accelerated Approval Paradigm

One special case of extending a trial is presented in the accelerated approval paradigm as illustrated by the recent FDA guidelines for new interventions for diabetes [119]. Diabetes is a serious disease with fatal and nonfatal consequences. Thus, getting new interventions into clinical practice is a priority. A typical regulatory approval historically may have been based on a drug’s ability to lower glucose or HbA1c, which is believed to reduce the risk of diabetes. However, trials in diabetes have raised the issue of whether a new drug might in fact raise cardiovascular (CV) risk. Thus, these new FDA guidelines propose a two-step process. The first step is to rule out a substantial increase in relative risk of 1.8. If and when that criteria is met, the pharmaceutical sponsor can submit their data to get a conditional FDA regulatory approval and be able to market their drug on the condition that they gather additional data to rule out an increase in CV risk of 1.3 or greater. This could be accomplished through two consecutive trials, one smaller trial with approximately 200 CV events to rule out a CV risk of 1.8 followed by a second larger trial with approximately 600 CV events to rule out a CV risk of 1.3. Alternatively, one large trial could be designed such that when the 95% confidence interval excludes a CV risk of 1.8, the monitoring committee could alert the sponsor to that fact and the regulatory submission for drug approval would be initiated. However, the trial would continue to gather for CV data to rule out a risk of 1.3. A problem for trial continuation arises however if the new drug is fact given regulatory conditional approval. At that point, the clinical and private community is now aware of the “interim” results submitted to rule out the CV risk of 1.8, with pressure to share some degree of detail about the results. Whether diabetes patients will continue adhere to their assigned, presumably blinded, trial medication or even agree to be entered if not already randomized is a serious question. Failure to adhere to the assigned therapy will bias the risk assessment towards the null and thus a result favoring a rule out of the 1.3 CV risk.

Employing the two sequential trial approach also has problems. First, it adds to the length of the period where a new drug is under development. Second, knowing the results of the first trial which ruled out a CV risk of 1.8 may influence the profile of the diabetes patients in the second trial, either through physician or patient decision not to participate. For example, more serious risk patients may choose not to be randomized and thus lowering the overall CV risk.

References

1.

Baum M, Houghton J, Abrams K. Early stopping rules—clinical perspectives and ethical considerations. Statist Med 1994;13:1459–1469.CrossRef

2.

Fleming TR, DeMets DL. Monitoring of clinical trials: issues and recommendations. Control Clin Trials 1993;14:183–197.CrossRef

3.

Heart Special Project Committee. Organization, review, and administration of cooperative studies (Greenberg Report): a report from the Heart Special Project Committee to the National Advisory Heart Council, May 1967. Control Clin Trials 1988;9:137–148.CrossRef

4.

Ellenberg SS, Fleming TR, DeMets DL. Data Monitoring Committees in Clinical Trials: A Practical Perspective. Wiley, 2003.

5.

DeMets DL, Furberg C, Friedman LM. Data monitoring in clinical trials: a case studies approach. New York, NY, Springer, 2006.CrossRefMATH

6.

Fisher MR, Roecker EB, DeMets DL. The role of an independent statistical analysis center in the industry-modified National Institutes of Health model. Drug Inf J 2001;35:115–129.

7.

Burke G. Discussion of ‘early stopping rules—clinical perspectives and ethical considerations’. Statist Med 1994;13:1471–1472.CrossRef

8.

Buyse M. Interim analyses, stopping rules and data monitoring in clinical trials in Europe. Statist Med 1993;12:509–520.CrossRef

9.

Canner PL. Practical Aspects of Decision-Making In Clinical Trials—The Coronary Drug Project as a Case-Study. Control Clin Trials 1981;1:363–376.CrossRef

10.

Canner PL. Monitoring of the data for evidence of adverse or beneficial treatment effects. Control Clin Trials 1983;4:467–483.CrossRef

11.

Crowley J, Green S, Liu PY, Wolf M. Data monitoring committees and early stopping guidelines: the Southwest Oncology Group experience. Statist Med 1994;13:1391–1399.CrossRef

12.

DeMets DL. Data monitoring and sequential analysis—An academic perspective. J Acquir Immune Defic Syndr 1990;3:S124–S133.

13.

DeMets DL, Fleming TR, Whitley RJ, et al. The data and safety monitoring board and acquired immune deficiency syndrome (AIDS) clinical trials. Control Clin Trials 1995;16:408–421.CrossRef

14.

Ellenberg SS, Myers MW, Blackwelder WC, Hoth DF. The use of external monitoring committees in clinical trials of the National Institute of Allergy and Infectious Diseases. Statist Med 1993;12:461–467.CrossRef

15.

Fleming TR, Green SJ, Harrington DP. Considerations for monitoring and evaluating treatment effects in clinical trials. Control Clin Trials 1984;5:55–66.CrossRef

16.

Friedman L. The NHLBI model: a 25 year history. Statist Med 1993;12:425–431.CrossRef

17.

Geller NL, Stylianou M. Practical issues in data monitoring of clinical trials: summary of responses to a questionnaire at NIH. Statist Med 1993;12:543–551.CrossRef

18.

George SL. A survey of monitoring practices in cancer clinical trials. Statist Med 1993;12:435–450.CrossRef

19.

Green S, Crowley J. Data monitoring committees for Southwest Oncology Group clinical trials. Statist Med 1993;12:451–455.CrossRef

20.

Harrington D, Crowley J, George SL, et al. The case against independent monitoring committees. Statist Med 1994;13:1411–1414.CrossRef

21.

Herson J. Data monitoring boards in the pharmaceutical industry. Statist Med 1993;12:555–561.CrossRef

22.

O’Neill RT. Some FDA perspectives on data monitoring in clinical trials in drug development. Statist Med 1993;12:601–608

23.

Parmar MK, Machin D. Monitoring clinical trials: experience of, and proposals under consideration by, the Cancer Therapy Committee of the British Medical Research Council. Statist Med 1993;12:497–504.CrossRef

24.

Pater JL. The use of data monitoring committees in Canadian trial groups. Statist Med 1993;12:505–508.CrossRef

25.

Peto R, Pike MC, Armitage P, et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Br J Cancer 1976;34:585.

26.

Pocock SJ. Statistical and ethical issues in monitoring clinical trials. Statist Med 1993;12:1459–1469.CrossRef

27.

Robinson J. A lay person’s perspective on starting and stopping clinical trials. Statist Med 1994;13:1473–1477.CrossRef

28.

Rockhold FW, Enas GG. Data monitoring and interim analyses in the pharmaceutical industry: Ethical and logistical considerations. Statist Med 1993;12:471–479.CrossRef

29.

Simon R. Some practical aspects of the interim monitoring of clinical trials. Statist Med 1994;13:1401–1409.CrossRef

30.

Souhami RL. The clinical importance of early stopping of randomized trials in cancer treatments. Statist Med 1994;13:1293–1295.CrossRef

31.

Task Force of the Working Group on Arrythmias of the European Society of Cardiology. The early termination of clinical trials: causes, consequences, and control. With special reference to trials in the field of arrhythmias and sudden death. Circulation 1994;89:2892–2907.

32.

Walters L. Data monitoring committees: the moral case for maximum feasible independence. Statist Med 1993;12:575–580.CrossRef

33.

Williams GW, Davis RL, Getson AJ, et al. Monitoring of clinical trials and interim analyses from a drug sponsor’s point of view. Statist Med 1993;12:481–492.CrossRef

34.

Wittes J. Behind closed doors: the data monitoring board in randomized clinical trials. Statist Med 1993;12:419–424.CrossRef

35.

Packer M, Rouleau J, Swedberg K, et al. Effect of flosequinan on survival in chronic heart failure: preliminary results of the PROFILE study. Circulation 1993;88 (Supp I):301.

36.

Packer M, Carver JR, Rodeheffer RJ, et al. Effect of Oral Milrinone on Mortality in Severe Chronic Heart Failure. N Engl J Med 11-21–1991;325:1468–1475.

37.

Packer M, O’Connor CM, Ghali JK, et al. Effect of Amlodipine on Morbidity and Mortality in Severe Chronic Heart Failure. N Engl J Med 1996;335:1107–1114.CrossRef

38.

Seltzer J. Clinical Trial Safety-The Goldilocks Dilemma-Balancing Effective and Efficient Safety Monitoring. Drug Development 2010;5:8.

39.

NIH: NIH policy for data and safety monitoring. NIH Guide.

40.

FDA: Guidance for clinical trial sponsors: Establishment and operation of clinical trial data monitoring committees. FDA.

41.

Raper SE, Chirmule N, Lee FS, et al. Fatal systemic inflammatory response syndrome in a ornithine transcarbamylase deficient patient following adenoviral gene transfer. Mol Genet Metab 2003;80:148–158.CrossRef

42.

Shalala D: Protecting research subjects-what must be done. N Engl J Med 2000;343:808–810.CrossRef

43.

Clemens F, Elbourne D, Darbyshire J, Pocock S. Data monitoring in randomized controlled trials: surveys of recent practice and policies. Clin Trials 2005;2:22–33.CrossRef

44.

Freedman B. Equipoise and the ethics of clinical research. N Engl J Med 1987.

45.

Meinert CL. Masked monitoring in clinical trials--blind stupidity? N Engl J Med 1998;338:1381–1382.

46.

Whitehead J. On being the statistician on a Data and Safety Monitoring Board. Statist Med 12-30–1999;18:3425–3434.

47.

Li ZQ, Geller NL. On the Choice of Times for Data Analysis in Group Sequential Clinical Trials. Biometrics 1991;47:745–750.CrossRef

48.

Anscombe FJ. Sequential medical trials. J Am Stat Assoc 1963;58:365–383.MathSciNetCrossRef

49.

Armitage P. Restricted sequential procedures. Biometrika 1957;9–26.

50.

Armitage P, McPherson CK, Rowe BC. Repeated Significance Tests on Accumulating Data. J R Stat Soc Ser A 1969;132:235–244.MathSciNetCrossRef

51.

Bross I. Sequential medical plans. Biometrics 1952;8:188–205.CrossRef

52.

Robbins H. Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society 1952;58:527–535.MathSciNetCrossRefMATH

53.

Robbins H. Statistical methods related to the law of the iterated logarithm. The Annals of Mathematical Statistics 1970;1397–1409.

54.

The Coronary Drug Project Research Group. Clofibrate and niacin in coronary heart disease. JAMA 1975;231:360–381.CrossRef

55.

Pocock SJ. When to stop a clinical trial. BMJ 1992;305:235.CrossRef

56.

DeMets DL. Stopping Guidelines Vs Stopping Rules—A Practitioners Point of View. Commun Stat Theory Methods 1984;13:2395–2417.CrossRef

57.

Freidlin B, Korn EL. Stopping clinical trials early for benefit: impact on estimation. Clin Trials 2009;6:119–125.CrossRef

58.

Goodman SN. Stopping trials for efficacy: an almost unbiased view. Clin Trials 2009;6:133–135.CrossRef

59.

Montori VM, Devereaux PJ, Adhikari NK, et al. Randomized trials stopped early for benefit: a systematic review. JAMA 2005;294:2203.CrossRef

60.

Pocock SJ. When (not) to stop a clinical trial for benefit. JAMA 2005;294:2228–2230.CrossRef

61.

Report of the committee for the assessment of biometric aspects of controlled trials of hypoglycemic agents. JAMA 1975;231:583–608.

62.

Knatterud GL, Meinert CL, Klimt CR, et al. Effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes: IV. A preliminary report on phenformin results. JAMA 1971;217:777–784.CrossRef

63.

Kolata GB. Controversy over study of diabetes drugs continues for nearly a decade. Science (New York, NY) 1979;203:986.

64.

Meinert CL, Knatterud GL, Prout TE, Klimt CR. A study of the effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes. II. Mortality results Diabetes 1970;19:789–830.

65.

The Coronary Drug Project Research Group. The coronary drug project: Initial findings leading to modifications of its research protocol. JAMA 1970;214:1303–1313.CrossRef

66.

The Coronary Drug Project Research Group. The coronary drug project: Findings leading to discontinuation of the 2.5-mg/day estrogen group. JAMA 1973;226:652–657.

67.

The Coronary Drug Project Research Group. The coronary drug project: Findings leading to further modifications of its protocol with respect to dextrothyroxine. JAMA 1972;220:996–1008.CrossRef

68.

Pocock SJ, Wang D, Wilhelmsen L, Hennekens CH. The data monitoring experience in the Candesartan in Heart Failure Assessment of Reduction in Mortality and morbidity (CHARM) program. Am Heart J 2005;149:939–943.CrossRef

69.

DeMets DL, Williams GW, Brown Jr BW. A case report of data monitoring experience: The Nocturnal Oxygen Therapy Trial. Control Clin Trials 1982;3:113–124.CrossRef

70.

Nocturnal Oxygen Therapy Trial Group. Continuous or Nocturnal Oxygen Therapy in Hypoxemic Chronic Obstructive Lung DiseaseA Clinical Trial. Ann Intern Med 1980;93:391–398.CrossRef

71.

ISIS-2 Collaborative Group. Randomised Trial Of Intravenous Streptokinase, Oral Aspirin, Both, Or Neither Among 17 187 Cases Of Suspected Acute Myocardial Infarction: ISIS-2. Lancet 1988;332:349–360.

72.

Diabetic Retinopathy Study Research Group: Preliminary report on effects of photocoagulation therapy. Am J Ophthalmol 1976;81:383–396.CrossRef

73.

Diabetic Retinopathy Study Research Group. Diabetic retinopathy study. Report Number 6. Design, methods, and baseline results. Invest Ophthalmol Vis Sci 1981;21:1–226.

74.

Diabetic Retinopathy Study Research Group. Photocoagulation treatment of proliferative diabetic retinopathy: the second report of diabetic retinopathy study findings. Ophthalmology 1978;85:82–106.CrossRef

75.

Ederer F, Podgor MJ. Assessing possible late treatment effectsin stopping a clinical trial early: A case study. Diabetic retinopathy study report no. 9. Control Clin Trials 1984;5:373–381.CrossRef

76.

Beta-Blocker Heart Attack Trial Research Group. A randomized trial of propranolol in patients with acute myocardial infarction: I. mortality results. JAMA 1982;247:1707–1714.

77.

DeMets DL, Hardy R, Friedman LM, Gordon Lan KK. Statistical aspects of early termination in the Beta-Blocker Heart Attack Trial. Control Clin Trials 1984;5:362–372.CrossRef

78.

CIBIS-II Investigators and Committees. The Cardiac Insufficiency Bisoprolol Study II (CIBIS-II): a randomised trial. Lancet 1999;353:9–13.

79.

MERIT HF Study Group. Effect of Metoprolol CR/XL in chronic heart failure: Metoprolol CR/XL Randomized Interventional Trial in congestive heart failure.(MERIT-HF). Lancet 1999;353:2001–2007.

80.

Packer M, Coats AJS, Fowler MB, et al. Effect of Carvedilol on Survival in Severe Chronic Heart Failure. N Engl J Med 2001;344:1651–1658.CrossRef

81.

Beta-Blocker Evaluation of Survival Trial Investigators. A trial of the beta-blocker bucindolol in patients with advanced chronic heart failure. N Engl J Med 2001;344:1659.

82.

The Cardiac Arrhythmia Suppression Trial (CAST) Investigators: Preliminary report: effect of encainide and flecainide on mortality in a randomized trial of arrhythmia suppression after myocardial infarction. N Engl J Med 1989;321:406–412.CrossRef

83.

Friedman LM, Bristow JD, Hallstrom A, et al. Data monitoring in the cardiac arrhythmia suppression trial. Online J Curr Clin Trials 1993;79.

84.

Pawitan Y, Hallstrom A. Statistical interim monitoring of the cardiac arrhythmia suppression trial. Statist Med 1990;9:1081–1090.CrossRef

85.

Cardiac Arrhythmia Suppression Trial Investigators. Effect of the antiarrhythmic agent moricizine on survival after myocardial infarction. N Engl J Med 1992;327:227–233.CrossRef

86.

Tricoci P, Huang Z, Held C, et al. Thrombin-Receptor Antagonist Vorapaxar in Acute Coronary Syndromes. N Engl J Med 2011;366:20–33.CrossRef

87.

DeMets DL, Pocock SJ, Julian DG. The agonising negative trend in monitoring of clinical trials. Lancet 1999;354:1983–1988.CrossRef

88.

Furberg CD, Campbell R, Pitt B. ACE Inhibitors after Myocardial Infarction. N Engl J Med 1993;328:966–969.CrossRef

89.

Pater JL. Timing the collaborative analysis of three trials comparing 5-FU plus folinic acid (FUFA) to surgery alone in the management of resected colorectal cancer: A National Cancer Institute of Canada Clinical trials group (NCIC-CTG) perspective. Statist Med 1994;13:1337–1340.

90.

Swedberg K, Held P, Kjekshus J, et al. Effects of the early administration of enalapril on mortality in patients with acute myocardial infarction: results of the Cooperative New Scandinavian Enalapril Survival Study II (CONSENSUS II). N Engl J Med 1992;327:678–684.CrossRef

91.

Sylvester R, Bartelink H, Rubens R. A reversal of fortune: practical problems in the monitoring and interpretation of an EORTC breast cancer trial. Statist Med 1994;13:1329–1335.CrossRef

92.

Hypertension Detection and Follow-up Program Cooperative Group. Five-year findings of the hypertension detection and follow-up program: I. Reduction in mortality of persons with high blood pressure, including mild hypertension. JAMA 1979;242:2562–2571.

93.

Aspirin Myocardial Infarction Study Research Group. A randomized, controlled trial of aspirin in persons recovered from myocardial infarction. JAMA 1980;243:661–669.CrossRef

94.

CASS Principle Investigators and Their Associates. Coronary artery surgery study (CASS): a randomized trial of coronary artery bypass surgery. Survival data. Circulation 1983;68:939–950.CrossRef

95.

Multiple Risk Factor Intervention Trial Research Group. Multiple risk factor intervention trial: Risk factor changes and mortality results. JAMA 1982;248:1465–1477.CrossRef

96.

Multiple Risk Factor Intervention Trial Research Group. Mortality after 16 years for participants randomized to the Multiple Risk Factor Intervention Trial. Circulation 1996;94:946–951.CrossRef

97.

The Women’s Health Initiative Steering Committee. Effects of conjugated equine estrogen in postmenopausal women with hysterectomy: the Women’s Health Initiative randomized controlled trial. JAMA 2004;291:1701–1712.CrossRef

98.

Writing Group for the Women’s Health Initiative Investigators. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: Principal results from the women’s health initiative randomized controlled trial. JAMA 2002;288:321–333.CrossRef

99.

McMurray JJ, Teerlink JR, Cotter G, et al. Effects of tezosentan on symptoms and clinical outcomes in patients with acute heart failure: the VERITAS randomized controlled trials. JAMA 2007;298:2009–2019.CrossRef

100.

Tegeler CH, Furberg CD. Lessons from warfarin trials in atrial fibrillation: Missing the window of opportunity; in Data Monitoring in Clinical Trials: Springer, 2006, pp 312–319.

101.

Ridker PM, Danielson E, Fonseca FA, et al. Rosuvastatin to prevent vascular events in men and women with elevated C-reactive protein. N Engl J Med 2008;359:2195.CrossRef

102.

Ridker PM. The JUPITER trial results, controversies, and implications for prevention. Circ Cardiovasc Qual Outcomes 2009;2:279–285.CrossRef

103.

Voss E, Rose CP, Biron P. JUPITER, a statin trial without cardiovascular mortality benefit. Circ Cardiovasc Qual Outcomes 2009;2:279–285.CrossRef

104.

Barter PJ, Caulfield M, Eriksson M, et al. Effects of Torcetrapib in Patients at High Risk for Coronary Events. N Engl J Med 2007;357:2109–2122.CrossRef

105.

Hedenmalm K, Melander H, Alvan G. The conscientious judgement of a DSMB—statistical stopping rules re-examined. Eur J Clin Pharmacol 2008;64:69–72.CrossRef

106.

Morrow DA, Braunwald E, Bonaca MP, et al. Vorapaxar in the Secondary Prevention of Atherothrombotic Events. N Engl J Med 2012;366:1404–1413.CrossRef

107.

Liberati A. Conclusions. 1: The relationship between clinical trials and clinical practice: The risks of underestimating its complexity. Statist Med 1994;13:1485–1491.CrossRef

108.

O’Neill RT. Conclusions. 2: The relationship between clinical trials and clinical practice: The risks of underestimating its complexity. Statist Med 1994;13:1493–1499.CrossRef

109.

Brancati FL, Evans M, Furberg CD, et al. Midcourse correction to a clinical trial when the event rate is underestimated: the Look AHEAD (Action for Health in Diabetes) Study. Clin Trials 2012;9:113–124.CrossRef

110.

Collaborative Group on Antenatal Steroid Therapy. Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome. Am J Obstet Gynecol 1981;141:276–287.

111.

The MIAMI Trial Research Group. Metoprolol in acute myocardial infarction (MIAMI). A randomised placebo-controlled international trial. Eur Heart J 1985;6:199–226.

112.

Chen YH, DeMets DL, Lan KK. Increasing the sample size when the unblinded interim result is promising. Stat Med 2004;23:1023–1038.CrossRef

113.

Cui L, Hung HMJ, Wang SJ. Modification of Sample Size in Group Sequential Clinical Trials. Biometrics 1999;55:853–857.CrossRefMATH

114.

Lan KKG, Trost DC. Estimation of parameters and sample size re-estimation. Proceedings—Biopharmaceutical Section American Statistical Association, 48-51. 1997. American Statistical Association.

115.

Proschan MA, Liu Q, Hunsberger S. Practical midcourse sample size modification in clinical trials. Control Clin Trials 2003;24:4–15.CrossRef

116.

Fleming TR. Standard versus adaptive monitoring procedures: a commentary. Statist Med 2006;25:3305–3312.MathSciNetCrossRef

117.

Tsiatis AA, Mehta C. On the Inefficiency of the Adaptive Design for Monitoring Clinical Trials. Biometrika 2003;90:367–378.MathSciNetCrossRefMATH

118.

Taylor AL, Ziesche S, Yancy C, et al. Combination of isosorbide dinitrate and hydralazine in blacks with heart failure. N Engl J Med 2004;351:2049–2057.CrossRef

119.

FDA: Guidance for Industry Diabetes Mellitus—Evaluating Cardiovascular Risk in New Antidiabetic Therapies to Treat Type 2 Diabetes. FDA.

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_17

17. Statistical Methods Used in Interim Monitoring

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

The original version of this chapter was revised. An erratum can be found at DOI 10.​1007/​978-3-319-18539-2_​23

In Chap. 16, the administrative structure was discussed for conducting interim analysis of data quality and outcome data for benefit and potential harm to trial participants. Although statistical approaches for interim analyses may have design implications, we have delayed discussing any details until this chapter because they really focus on monitoring accumulating data. Even if, during the design of the trial, consideration was not given to sequential methods, they could still be used to assist in the data monitoring or the decision-making process. In this chapter, some statistical methods for sequential analysis will be reviewed that are currently available and used for monitoring accumulating data in a clinical trial. These methods help support the evaluation of interim data and whether they are so convincing that the trial should be terminated early for benefit, harm, or futility or whether it should be continued to its planned termination. No single statistical test or monitoring procedure ought to be used as a strict rule for decision-making, but rather as one piece of evidence to be integrated with the totality of evidence [1–6]. Therefore, it is difficult to make a single recommendation about which should be used. However, the following methods, when applied appropriately, can be useful guides in the decision-making process.

Classical sequential methods, a modification generally referred to as group sequential methods, and curtailed testing procedures are discussed below in some detail; other approaches are also briefly considered. Classical sequential methods are given more mathematical attention in several articles and texts which can be referred to for further detail [7–20].

Fundamental Point

Although many statistical techniques are available to assist in monitoring, none of them should be used as the sole basis in the decision to stop or continue the trial.

Classical Sequential Methods

The aim of the classical sequential design is to minimize the number of participants that must be entered into a study. The decision to continue to enroll participants depends on results from those already entered. Most of these sequential methods assume that the response variable outcome is known in a short time relative to the duration of the trial. Therefore, for many trials involving acute illness, these methods are applicable. For studies involving chronic diseases, classical sequential methods have not been as useful. Detailed discussions of classical sequential methods are given, for example, by Armitage [20], Whitehead [18], and Wald [16].

The classical sequential analysis method as originally developed by Wald [16] and applied to the clinical trial by others such as Armitage [8, 9, 20] involves repeated testing of data in a single experiment. The method assumes that the only decision to be made is whether the trial should continue or be terminated because one of the groups is responding significantly better than the other. This classical sequential decision rule is called an “open plan” by Armitage [20] because there is no guarantee of when a decision to terminate will be reached. Strict adherence to the “open plan” would mean that the study could not have a fixed sample size. Very few clinical trials use the “open” or classical sequential design. The method also requires data to be paired, one observation from each group. In many instances, the pairing of participants is not appealing because the paired participants may be very different and may not be “well matched” in important prognostic variables. If stratification is attempted in order to obtain better matched pairs, each stratum with an odd number of participants would have one unpaired participant. Furthermore, the requirement to monitor the data after every pair may not be possible for many clinical trials. Silverman and colleagues [21] used an “open plan” in a trial of the effects of humidity on survival in infants with low birth weight. At the end of 36 months, 181 pairs of infants had been enrolled; 52 of the pairs had a discrepant outcome. Nine infants were excluded because they were un-matched and 16 pairs were excluded because of a mismatch. The study had to be terminated without a clear decision because it was no longer feasible to continue the trial. This study illustrates the difficulties inherent in the applying the classical sequential design for clinical trials.

Armitage [8] introduced the restricted or “closed” sequential design to assure that a maximum limit is imposed on the number of participants (2 N) to be enrolled. As with the “open plan,” the data must be paired using one observation from each study group. Criteria for early termination and rejection of no treatment effect are determined so that the design has specified levels of significance and power (α and 1 − β). This design was used in a comparison of two interventions in patients with ulcerative colitis [22]. In that trial, the criterion for no treatment effect was exceeded, demonstrating short-term clinical benefit of corticosteroids over sulphasalazine therapy. This closed design was also used in an acute leukemia trial, comparing 6-mercaptopurine with placebo (CALGB) [23]. This trial was terminated early, with the statistic comparing remission rates crossing the sequential boundary for benefit after 21 pairs of patients.

Another solution to the repeated testing problem, called “repeated significance tests,” was proposed by McPherson and Armitage [24] and also described by Armitage [20]. Although different theoretical assumptions are used, this approach has features similar to the restricted sequential model. That is, the observed data must be paired, and the maximum number of pairs to be considered can be fixed. Other modifications to the Armitage restricted plan [25–27] have also been proposed. This methodology plays an important role in a method to be described below, referred to as group sequential design.

The methods described above can in some circumstances be applied to interim analyses of censored survival data [25, 28–36]. If participants simultaneously enter a clinical trial and there is no loss to follow-up, information from interim analyses is said to be “progressively censored.” Sequential methods for this situation have been developed using, for example, modified rank statistics. In fact, most participants are not entered into a trial simultaneously, but in a staggered fashion. That is, participants enter over a period of time after which events of interest occur, subject to an independent censoring process. The log-rank statistic, described in Chap. 15, may also be used to monitor in this situation.

The classical sequential approach has not been widely used, even in clinical trials where the time to the event is known almost immediately. One major reason is that for many clinical trials, if the data are monitored by a committee which has regularly scheduled meeting, it is neither feasible nor necessary for ethical reasons to perform an analysis after every pair of outcomes. In addition, classical sequential boundaries require an alternative hypothesis to be specified, a feature not demanded by conventional statistical tests for the rejection of the null hypothesis.

Group Sequential Methods

Because of limitations with classical sequential methods, other approaches to the repeated testing problem have been proposed. Ad hoc rules have been suggested that attempt to ensure a conservative interpretation of interim results. One such method is to use a critical value of 2.6 at each interim look as well as in the final analyses [1]. Another approach [37, 38] referred to as the Haybittle–Peto procedure, favors using a large critical value, such as Z i  = +3.0, for all interim tests (i < K). Then any adjustment needed for repeated testing at the final test (i = K) is negligible and the conventional critical value can be used. These methods are ad hoc in the sense that no precise Type I error level is guaranteed. They might, however, be viewed as precursors of the more formal procedures to be described below.

Pocock [39–41] modified the repeated testing methods of McPherson and Armitage [24] and developed a group sequential method for clinical trials which avoids many of the limitations of classical methods. He discusses two cases of special interest; one for comparing two proportions and another for comparing mean levels of response. Pocock’s method divides the participants into a series of K equal-sized groups with 2n participants in each, n assigned to intervention and n to control. K is the number of times the data will be monitored during the course of the trial. The total expected sample size is 2nK. The test statistic used to compare control and intervention is computed as soon as data for the first group of 2n participants are available, and then recomputed when data from each successive group of 2n participants become known. Under the null hypothesis, the distribution of the test statistic, Z i , is assumed to be approximately normal with zero mean and unit variance, where i indicates the number of groups (i ≤ K) which have complete data. This statistic Z i is compared to the stopping boundaries, ±ZN K where ZN K has been determined so that for up to K repeated tests, the overall (two sided) significance level for the trial will be α. For example, if K = 5 and α = 0.05 (two-sided), ZN K  = 2.413. This critical value is larger than the critical value of 1.96 used in a single test of hypothesis with α = 0.05. If the statistic Z i falls outside the boundaries on the “i”-th repeated test, the trial should be terminated, rejecting the null hypothesis. If the statistic never falls outside the boundaries, the trial should be continued until i = K (the maximum number of tests). When i = K, the trial would stop and the investigator would “accept” H ₀.

O’Brien and Fleming [42] also discuss a group sequential procedure. Using the above notation, their stopping rule compares the statistic Z i with Z* √(K / i) where Z* is determined so as to achieve the desired significance level. For example, if K = 5 and a = 0.05, Z* = 2.04. If K ≤ 5, Z* may be approximated by the usual critical values for the normal distribution. One attractive feature is that the critical value used at the last test (i = K) is approximately the same as that used if a single test were done.

In Fig. 17.1, boundaries for the three methods described are given for K = 5 and α = 0.05 (two-sided). If for i < 5 the test statistic falls outside the boundaries, the trial is terminated and the null hypothesis rejected. Otherwise, the trial is continued until i = 5, at which time the null hypothesis is either rejected or “accepted”. The three boundaries have different early stopping properties. The O’Brien–Fleming model is unlikely to lead to stopping in the early stages. Later on, however, this procedure leads to a greater chance of stopping prior to the end of the study than the other two. Both the Haybittle–Peto and the O’Brien–Fleming boundaries avoid the awkward situation of accepting the null hypothesis when the observed statistic at the end of the trial is much larger than the conventional critical value (i.e., 1.96 for a two-sided 5% significance level). If the observed statistic in Fig. 17.1 is 2.3 when i = 5, the result would not be significant using the Pocock boundary. The large critical values used at the first few analyses for the O’Brien–Fleming boundary can be adjusted to some less extreme values (e.g., 3.5) without noticeably changing the critical values used later on, including the final one.

Fig. 17.1

Three group sequential stopping boundaries for the standardized normal statistic (Zi) for up to five sequential groups with two-sided significance level of 0.05 [64]

Many monitoring committees wish to be somewhat conservative in their interpretation of early results because of the uncertainties discussed earlier and because a few additional events can alter the results substantially. Yet, most investigators would like to use conventional critical values in the final analyses, not requiring any penalty for interim analyses. This means that the critical value used in a conventional fixed sample methods would be the same for that used in a sequential plan, resulting in no increase in sample size. With that in mind, the O’Brien–Fleming model has considerable appeal, perhaps with the adjusted or modified boundary as described. That is, the final critical value at the scheduled end of the trial is very close to the conventional critical value (e.g. 2.05 instead of 1.96) if the number of interim analyses is not excessive (e.g. larger than 10). The group sequential methods have an advantage over the classical methods in that the data do not have to be continuously tested and individual participants do not have to be “paired.” This concept suits the data review activity of most large clinical trials where monitoring committees meet periodically. Furthermore, in many trials constant consideration of early stopping is unnecessary. Pocock [39–41] discusses the benefits of the group sequential approach in more detail and other authors describe variations [43–47].

In many trials, participants are entered over a period of time and followed for a relatively long period. Frequently, the primary outcome is time to some event. Instead of adding participants between interim analyses, new events are added. As discussed in Chap. 15, survival analysis methods could be used to compare the experience of the intervention and the control arms. Given their general appeal, it would be desirable to use the group sequential methods in combination with survival analyses. It has been established for large studies that the log-rank or Mantel–Haenszel statistic [48–53] can be used. Furthermore, even for small studies, the log-rank procedure is still quite robust. The Gehan, or modified Wilcoxon test [54, 55], as defined in Chap. 15 does not always produce interim values with independent increments and so cannot be easily incorporated using the usual group sequential procedures. A generalization of the Wilcoxon procedure for survival data, though, is appropriate [56] and the survival methods of analyses can in general terms be applied in group sequential monitoring. Instead of looking at equal-sized participant groups, the group sequential methods described strictly require that interim analyses should be done after an additional equal number of events have been observed. Since monitoring committees usually meet at fixed calendar times, the condition of equal number of events might not be met exactly. However, the methods applied under these circumstances are approximately correct [57] if the increments are not too disparate. Other authors have also described the application of group sequential methods to survival data [58–61].

Interim log-rank tests in the Beta-Blocker Heart Attack Trial [62, 63] were evaluated using the O’Brien–Fleming group sequential procedure [42]. Seven meetings had been scheduled to review interim data. The trial was designed for a two-sided 5% significance level. These specifications produce the group sequential boundary shown in Fig. 17.2. In addition, the interim results of the log-rank statistic are also shown for the first six meetings. From the second analysis on, the conventional significance value of 1.96 was exceeded. Nevertheless, the trial was continued. At the sixth meeting, when the O’Brien–Fleming boundary was crossed, a decision was made to terminate the trial with the final mortality curves as seen earlier in Fig. 16.​5. However, it should be emphasized that crossing the boundary was not the only factor in this decision.

Fig. 17.2

Six interim log rank statistics plotted for the time of data monitoring committee meetings with a two-sided O’Brien-Fleming significance level boundary in the Beta-Blocker Heart Attack Trial. Dashed line represents Z = 1.96 [63]

Flexible Group Sequential Procedures: Alpha Spending Functions

While the group sequential methods described are an important advance in data monitoring, the Beta-blocker Heart Attack Trial (BHAT) [62, 63] experience suggested two limitations. One was the need to specify the number K of planned interim analyses in advance. The second was the requirement for equal numbers of either participants or events between each analysis. This also means that the exact time of the interim analysis must be pre-specified. As indicated in the BHAT example, the numbers of deaths between analyses were not equal and exactly seven analyses of the data had been specified. If the monitoring committee had requested an additional analysis between the fifth and sixth scheduled meetings, the O’Brien–Fleming group sequential procedure would not have directly accommodated such a modification. Yet such a request could easily have happened. In order to accommodate the unequal numbers of participants or events between analyses and the possibility of larger or fewer numbers of interim analyses than pre-specified, flexible procedures that eliminated those restrictions were developed [64–71]. The authors proposed a so-called alpha spending function which allows investigators to determine how they want to allocate or “spend” the Type I error or alpha during the course of the trial. This function guarantees that at the end of the trial, the overall Type I error will equal the prespecified value of α. As will be described, this approach is a generalization of the previous group sequential methods so that the Pocock [39] and O’Brien–Fleming [42] monitoring procedures become special cases.

We must first distinguish between calendar time and information fraction [70, 71]. The information expected from all participants at the planned end of the trial is the total information. At any particular calendar time t during the study, a certain fraction t* of the total information is observed. That may be approximated by the fraction of participants randomized at that point, n, divided by the total number expected, N, or in survival studies, by the number of events observed already, d, divided by the total number expected D. Thus, the value for t* must be between 0 and 1. The information fraction is more generally defined in terms of ratio of the inverse of the variance of the test statistic at the particular interim analysis and the final analysis. The alpha spending function, α(t*), determines how the prespecified α is allocated at each interim analyses as a function of the information fraction. At the beginning of a trial, t* = 0 and α(t*) = 0, while at the end of the trial, t* = 1 and α(t*) = α. Alpha-spending functions that correspond to the Pocock and O’Brien–Fleming boundaries shown in Fig. 17.1 are indicated in Fig. 17.3 for a two-sided 0.05 α level and five interim analyses. These spending functions correspond to interim analyses at information fractions at 0.2, 0.4, 0.6, 0.8, and 1.0. However, in practice the information fractions need not be equally spaced. We chose those information fractions to indicate the connection between the earlier discussion of group sequential boundaries and the α spending function. The Pocock-type spending function allocates the alpha more rapidly than the O’Brien–Fleming type spending function. For the O’Brien–Fleming-type spending function at t* = 0.2, the α(0.2) is less than 0.0001 which corresponds approximately to the very large critical value or boundary value of 4.56 in Fig. 17.1. At t* = 0.4, the amount of α which can be spent is α(0.4) − α(0.2) which is approximately 0.0006, corresponding to the boundary value 3.23 in Fig. 17.1. That is, the difference in α(t*) at two consecutive information fractions, t* and t** where t* is less than t**, α(t**) − α(t*), determines the boundary or critical value at t**. Obtaining these critical values consecutively requires numerically integrating a distribution function similar to that for the Pocock boundary and is described elsewhere in detail [68]. Because these spending functions are only approximately equivalent to the Pocock or O’Brien–Fleming boundaries, the actual boundary values will be similar but not exactly the same. However, the practical operational differences are important in allowing greater flexibility in the monitoring process. Programs are available for these calculations [72, 73].

Fig. 17.3

Alpha-spending functions for K = 5, two-sided α = 0.05 at information fractions 0.2, 0.4, 0.6, 0.8, and 1.0. α1(t*) ~ O’Brien-Fleming; α2(t*) ~ Pocock; α3(t*) ~ uniform [74]

Many different spending functions can be specified. The O’Brien–Fleming α ₁(t*) and Pocock α ₂ (t*) type spending functions are specified as follows:

The spending function α ₃(t*) spends alpha uniformly during the trial for θ = 1, at a rate somewhat between α ₁(t*) and α ₂(t*). Other spending functions have also been defined [75, 76].

The advantage of the alpha-spending function is that neither the number nor the time of the interim analyses needs to be specified in advance. Once the particular spending function is selected, the information fractions t ₁*, t ₂*, …. determine the critical or boundary values exactly. In addition, the frequency of the interim analyses can be changed during the trial and still preserve the prespecified α level. Even if the rationale for changing the frequency is dependent on the emerging trends, the impact on the overall Type I error rate is almost negligible [77, 78]. These advantages give the spending function approach to group sequential monitoring the flexibility in analysis times that is often required in actual clinical trial settings [79]. It must be emphasized that no change of the spending function itself is permitted during the trial. Other authors have discussed additional aspects of this approach [80–82].

Applications of Group Sequential Boundaries

As indicated in the BHAT example [62, 63], the standardized logrank test can be compared to the standardized boundaries provided by the O’Brien–Fleming, Pocock, or α spending function approach. However, these group sequential methods are quite widely applicable for statistical tests. Under very general conditions, any statistic testing a single parameter from a parametric or semiparametric model has the normal or asymptotically normal distribution with independent increments of information between interim analyses which is sufficient for this approach [83, 84]. Many of the commonly used test statistics used in clinical trials have this feature. Besides logrank and other survival tests, comparisons of means, comparison of proportions [39, 85] and comparison of linear regression slopes [86–91] can be monitored using this approach. For means and proportions, the information fraction can be approximated by the ratio of the number of participants observed to the total expected. For regression slopes, the information fraction is best determined from the ratio of the inverse of the variance of the regression slope differences computed for the current and expected final estimate [86, 90, 91]. Considerable work has extended the group sequential methodology to more general linear and nonlinear random effects models for continuous data and to repeated measure methods for categorical data [83, 84, 92]. Thus, for most of the statistical tests that would be applied to common primary outcome measures in a clinical trial setting, the flexible group sequential methods can be used directly.

If the trial continues to the scheduled termination point, a p value is often computed to indicate the extremeness of the result. If the standardized statistical test exceeds the critical value, the p value would be less than the corresponding significance level. If a trial is terminated early or continues to the end with the standardized test exceeding or crossing the boundary value, a p value can also be computed [93]. These p values cannot be the nominal p value corresponding to the standardized test statistic. They must be adjusted to account for the repeated statistical testing of the outcome measure and for the particular monitoring boundary employed. Calculation of the p value is relatively straight forward with existing software packages [72, 73].

Statistical tests of hypotheses are but one of the methods used to evaluate the results of a clinical trial. Once trials are terminated, either on schedule or earlier, confidence intervals (CIs) are often used to give some sense of the uncertainty in the estimated treatment or intervention effect. For a fixed sample study, CIs are typically constructed as

where SE is the standard error of the estimate.

In the group sequential monitoring setting, this CI will be referred to as the naïve estimate since it does not take into account the sequential testing aspects. In general, construction of CIs following the termination of a clinical trial is not as straightforward [94–107], but software exists to aid in the computations [72]. The major problem with naive CIs is that they may not give proper coverage of the unknown but estimated treatment effect. That is, the CIs constructed in this way may not include the true effect with the specified frequency (e.g., 95%). For example, the width of the CI may be too narrow. Several methods have been proposed for constructing a more proper CI [94–107] by typically ordering the possible outcomes in different ways. That is, a method is needed to determine if a treatment effect at one time is either more or less extreme than a difference at another time. None of the methods proposed appear to be universally superior but the ordering originally suggested by Siegmund [104] and adopted by Tsiatis et al. [105] appears to be quite adequate in most circumstances. In this ordering, any treatment comparison statistic which exceeds the group sequential boundary at one time is considered to be more extreme than any result which exceeds the sequential boundary at a later time. While construction of CIs using this ordering of possible outcomes can break down, the cases or circumstances are almost always quite unusual and not likely to occur in practice [107]. It is also interesting that for conservative monitoring boundaries such as the O’Brien– Fleming method, the naive CI does not perform that poorly, due primarily to the extreme early conservatism of the boundary [103]. While more exact CIs can be computed for this case, the naive estimate may still prove useful as a quick estimate to be recalculated later using the method described [105]. Pocock and Hughes [102] have suggested that the point estimate of the effect of the intervention should also be adjusted, since trials that are terminated early tend to exaggerate the size of the true treatment difference. Others have also pointed out the bias in the point estimate [96, 101]. Kim [101] suggested that an estimate of the median is less biased.

CIs can also be used in another manner in the sequential monitoring of interim data. At each interim analysis, a CI could be constructed for the parameter summarizing the intervention effect, such as differences in means, proportions, or hazard ratios. This is referred to as repeated confidence intervals (RCIs) [95, 98, 99]. If the RCI excludes a null difference, or no intervention effect, then the trial might be stopped claiming a significant effect, either beneficial or harmful. It is also possible to continue the trial unless the CI excluded not only no difference but also minimal or clinically unimportant differences. On the other hand, if all values of clinically meaningful treatment differences are ruled out or fall outside the CI, then that trial might be stopped claiming that no useful clinical effect is likely. This method is useful for non-inferiority designs as described earlier in Chap. 5. Here, as for CIs following termination, the naive CI is not appropriate. Jennison and Turnbull [98, 99] have suggested one method for RCIs that basically inverts the group sequential test. That is, the CI has the same form as the naive estimate, but the coefficient is the standardized boundary value as determined by the spending function, for example. The RCI then has the following form:

where Z(k) is the sequential boundary value at the kth interim analysis. For example, using the O’Brien–Fleming boundaries shown in Fig. 17.1, we would have a coefficient of 4.56 at k = 1, $$ {t}_1^{*}=0.2 $$ and 3.23 at k = 2, $$ {t}_2^{*}=0.4 $$. Used in this manner, the RCI and the sequential test of the null hypothesis will yield the same conclusions.

One particular application of the RCI is for trials whose goal is to demonstrate that two interventions or treatments are essentially equivalent, that is, have an effect that is considered to be within a specified acceptable range and might be used interchangeably. As indicated in Chap. 5, clinicians might select the cheaper, less toxic or less invasive intervention if the effects were close enough. One suggestion for “close enough” or “equivalence” would be treatments whose effects are within 20% [108, 109]. Thus, RCIs that are contained within a 20% range would suggest that the results are consistent with this working definition of equivalence. For example, if the relative risks were estimated along with a RCI, the working range of equivalence would be from 0.8 to 1.2, where large values indicate inferiority of the intervention being tested. The trial would continue as long as the upper limit of the RCI exceeded 1.2 since we would not have ruled out a treatment worsening by 20% or more. Depending on the trial and the interventions, the trial might also continue until the lower limit of the RCI was larger than 0.8, indicating no improvement by 20% or greater.

As described in Chap. 5, there is a fundamental difference between an “equivalence” design and a noninferiority design. The former is a two-sided test, with the aim of establishing a narrow range of possible differences between the new intervention and the standard, or that any difference is within a narrow range. The noninferiority design aims to establish that the new intervention is no worse than the standard by some prespecified margin. It may be that the margins in the two designs are set to the same value. From a data monitoring point of view, both of these designs are best handled by sequential CIs [99]. As data emerge, the RCI takes into consideration the event rate or variability, the repeated testing aspects, and the level of the CI. The upper and lower boundaries can address either the “equivalence” point of view or the noninferiority margin of indifference.

Asymmetric Boundaries

In most trials, the main purpose is to test whether the intervention is superior to the control. It is rarely ethical to continue a study in order to prove, at the usual levels of significance, that the intervention is harmful relative to a placebo or standard control. This point has been mentioned by authors [110, 111] who discuss methods for group sequential designs in which the hypothesis to be tested is one-sided; that is, to test whether the intervention is superior to the control. They proposed retaining the group sequential upper boundaries of methods such as Pocock, Haybittle–Peto, or O’Brien–Fleming for rejection of H ₀ while suggesting various forms of a lower boundary which would imply “acceptance” of H ₀. One simple approach is to set the lower boundary at an arbitrary value of Z i , such as −1.5 or −2.0. If the test statistic goes below that value, the data may be sufficiently suggestive of a harmful effect to justify terminating the trial. This asymmetric boundary attempts to reflect the behavior or attitude of members of many monitoring committees, who recommend stopping a study once the intervention shows a strong, but non-significant, trend in an adverse direction for major events. Emerson and Fleming [112] recommend a lower boundary for acceptance of the null hypothesis which allows the upper boundary to be changed in order to preserve the Type I error exactly. Work by Gould and Pecore [113] suggests ways for early acceptance of the null hypothesis while incorporating costs as well. For new interventions, trials might well be terminated when the chances of a positive or beneficial result seem remote (discussed in the next section). However, if the intervention arm is being compared to a standard but the intervention is already in widespread use, it may be important to distinguish between lack of benefit and harm [114]. For example, if the intervention is not useful for the primary outcome, and also not harmful, it may still have benefits such as on other secondary clinical outcomes, quality of life, or fewer adverse events that would still make it a therapeutic option. In such cases, a symmetric boundary for the primary outcome might be appropriate.

An example of asymmetric group sequential boundaries is provided by the Cardiac Arrhythmia Suppression Trial (CAST). Two arms of the trial (encainide and flecainide, each vs. placebo) were terminated early using a symmetric two-sided boundary, although the lower boundary for harm was described as advisory by the authors [115–117]. The third comparison (moricizine vs. placebo) continued. However, due to the experience with the encainide and flecainide arms, the lower boundary for harm was revised to be less stringent than originally, i.e., an asymmetric boundary was used [115].

MERIT-HF used a modified version of the Haybittle–Peto boundary for benefit, requiring a critical value near +3.0 and a similar but asymmetric boundary, close to a critical Z value of −2.5 for harm as shown in Fig. 17.4. In addition, at least 50% of the designed person years of exposure were to be observed before early termination could be recommended. The planned interim analyses to consider benefit were at 25, 50, and 75% of the expected target number of events. Because there was a concern that treating heart failure with a beta blocker might be harmful, the monitoring committee was required to evaluate safety on a monthly basis using the lower sequential boundary as a guide. At the 25% interim analyses, the statistic for the logrank test was +2.8, just short of the boundary for benefit. At the 50% interim analyses, the observed logrank statistic was +3.8, clearly exceeding the sequential boundary for benefit. It also met the desired person years of exposure as plotted in Fig. 17.4. Details of this experience are described elsewhere [118]. A more detailed presentation of group sequential methods for interim analysis of clinical trials may be found in books by Jennison and Turnbull [119] and Proschan, Lan, and Wittes [120].

Fig. 17.4

MERIT-HF group sequential monitoring bounds for mortality [118]

Curtailed Sampling and Conditional Power Procedures

During the course of monitoring accumulating data, one question often posed is whether the current trend in the data is so impressive that “acceptance” or rejection of H ₀ is already determined, or at least close to being determined. If the results of the trial are such that the conclusions are known for certain, no matter what the future outcomes might be, then consideration of early termination is in order. A helpful sports analogy is a baseball team “clinching the pennant” after winning a specific game. At that time, it is known for certain who has won and who has not won the pennant or league championship, regardless of the outcome of the remaining games. Playing the remaining games is done for reasons (e.g., fiscal) other than deciding the winner. This idea has been developed for clinical trials and is often referred to as deterministic curtailed sampling. It should be noted that group sequential methods focus on existing data while curtailed sampling in addition considers the data which have not yet been observed.

Alling [121, 122] developed a closely related approach when he considered the early stopping question and compared the survival experience in two groups. He used the Wilcoxon test for two samples, a frequently used non-parametric test which ranks survival times and which is the basis for one of the primary survival analysis techniques. Alling’s method allows stopping decisions to be based on data available during the trial. The trial would be terminated if future data could not change the final conclusion about the null hypothesis. The method is applicable whether all participants are entered at the same time or recruitment occurs over a longer period of time. However, when the average time to the event is short relative to the time needed to enroll participants, the method is of limited value. The repeated testing problem is irrelevant, because any decision to reject the null hypothesis is based on what the significance test will be at the end of the study. Therefore, frequent use of this procedure during the trial causes no problem with regard to significance level and power.

Many clinical trials with survival time as a response variable have observations that are censored; that is, participants are followed for some length of time and then at some point, no further information about the participant is known or collected. Halperin and Ware [123] extended the method of Alling to the case of censored data, using the Wilcoxon rank statistic. With this method, early termination is particularly likely when the null hypothesis is true or when the expected difference between groups is large. The method is shown to be more effective for small sample sizes than for large studies. The Alling approach to early stopping has also been applied to another commonly used test, the Mantel–Haenszel statistic. However, the Wilcoxon statistic appears to have better early stopping properties than the Mantel–Haenszel statistic.

A deterministic curtailed procedure has been developed [124] for comparing the means of two bounded random variables using the two sample t-test. It assumes that the response must be between two values, A and B (A < B). An approximate solution is an extreme case approach. First, all the estimated remaining responses in one group are given the maximum favorable outcome and all the remaining responses in the other take on the worst response. The statistic is then computed. Next, the responses are assigned in the opposite way and a second statistic is computed. If neither of these two extreme results alters the conclusion, no additional data are necessary for testing the hypothesis. While this deterministic curtailed approach provides an answer to an interesting question, the requirement for absolute certainty results in a very conservative test and allows little opportunity for early termination.

In some clinical trials, the final outcome may not be absolutely certain, but almost so. To use the baseball analogy again, a first place team may not have clinched the pennant but is so many games in front of the second place team that it is highly unlikely that it will not, in fact, end up the winner. Another team may be so far behind that it cannot “realistically” catch up. In clinical trials, this idea is often referred to as stochastic curtailed sampling or conditional power. It is identical to the concept of conditional power discussed in the section on extending a trial.

One of the earliest applications of the concept of conditional power was in the CDP [1, 125]. In this trial, several treatment arms for evaluating cholesterol lowering drugs produced negative trends in the interim results. Through simulation, the probability of achieving a positive or beneficial result was calculated given the observed data at the time of the interim analysis. Unconditional power is the probability at the beginning of the trial of achieving a statistically significant result at a prespecified alpha level and with a prespecified alternative treatment effect. Ideally, trials should be designed with a power of 0.80–0.90 or higher. However, once data begin to accumulate, the probability of attaining a significant result increases or decreases with emerging positive or negative trends. Calculating the probability of rejecting the null hypothesis of no effect once some data are available is conditional power.

Lan et al. [126] considered the effect of stochastic curtailed or conditional power procedures on Type I and Type II error rates. If the null hypothesis, H ₀, is tested at time t using a statistic, S(t), then at the scheduled end of a trial at time T, the statistic would be S(T). Two cases are considered. First, suppose a trend in favor of rejecting H ₀ is observed at time t < T, with intervention doing better than control. One then computes the conditional probability, γ₀ of rejecting H ₀ at time T; that is, S(T) > Z α , assuming H ₀ to be true and given the current data, S(t). If this probability is sufficiently large, one might argue that the favorable trend is not going to disappear. Second, suppose a negative trend or data consistent with the null hypothesis of no difference, at some point t. Then, one computes the conditional probability, γ₁, of rejecting H ₀ at the end of the trial, time T, given that some alternative H ₁ is true, for a sample of reasonable alternatives. This essentially asks how large the true effect must be before the current “negative” trend is likely to be reversed. If the probability of a trend reversal is highly unlikely for a realistic range of alternative hypotheses, trial termination might be considered.

Because there is a small probability that the results will change, a slightly greater risk of a Type I or Type II error rate will exist than would be if the trial continued to the scheduled end [127]. However, it has been shown that the Type I error is bounded very conservatively by α/γ₀ and the Type II error by β/γ₁. For example, if the probability of rejecting the null hypothesis, given the existing data were 0.85, then the actual Type I error would be no more than 0.05/0.85 or 0.059, instead of 0.05. The actual upper limit is considerably closer to 0.05, but that calculation requires computer simulation. Calculation of these probabilities is relatively straightforward and the details have been described by Lan and Wittes [128]. A summary of these methods, using the approach of DeMets [74], follows.

Let Z(t) represent the standardized statistic at information fraction t. The information fraction may be defined, for example, as the proportion of expected participants or events observed so far. The conditional power, CP, for some alternative intervention effect θ, using a critical value of Z α for a Type I error of alpha, can be calculated as

where θ = E(Z(t = 1)), the expected value of the test statistic at the full completion of the trial.

The alternative θ is defined for various outcomes as follows for:

1.

Survival outcome (D = total events)

λ C  and λ T are the hazard rates in the control and intervention arms, respectively.

2.

Binomial outcome (2n = N, n/arm or N = total sample size)

where P C and P T are the event rates in the control arm and intervention arm respectively and $$ \overline{p} $$ is the common event rate.

3.

Continuous outcome (means) (N = total sample size)

where μ C and μ T are the mean response levels for the control and the intervention arms, respectively, and σ is the common standard deviation.

If we specify a particular value of the conditional power as γ, then a boundary can also be produced which would indicate that if the test statistic fell below that, the chance of finding a significant result at the end of the trial is less than γ [127]. For example, in Fig. 17.5 the lower futility boundary is based on a specified conditional power γ, ranging from 10 to 30% that might be used to claim futility of finding a positive beneficial claim at the end of the trial. For example, if the standardized statistic crosses that 20% lower boundary, the conditional power for a beneficial result at the end of the trial is less than 0.20 for the specified alternative.

Fig. 17.5

Conditional power boundaries: outer boundaries represent symmetric O’Brien-Fleming type sequential boundaries (α = 0.05). Three lower boundaries represent boundaries for 10%, 20% and 30% conditional power to achieve a significant (P < 0.05) result of the trial conclusion [74]

Conditional power calculations are done for a specific alternative but in practice, a monitoring committee would likely consider a range of possibilities. These specified alternatives may range between the null hypothesis of no effect and the prespecified design based alternative treatment effect. In some cases, a monitoring committee may consider even more extreme beneficial effects to determine just how much more effective the treatment would have to be to raise the conditional power to desired levels. These conditional power results can be summarized in a table or a graph, and then monitoring committee members can assess whether they believe recovery from a substantial negative trend is likely.

Conditional power calculations were utilized in the Vesnarinone in Heart Failure Trial (VEST) [129]. In Table 17.1, the test statistics for the logrank test are provided for the information fractions at a series of monitoring committee meetings. Table 17.2 provides conditional power for VEST at three of the interim analyses. A range of intervention effects was used including the beneficial effect (hazard rate less than 1) seen in a previous vesnarinone trial to the observed negative trend (hazard rates of 1.3 and 1.5). It is clear that the conditional power for a beneficial effect was very low by the midpoint of this trial for a null effect or worse. In fact, the conditional power was not encouraging even for the original assumed effect. As described by DeMets et al. [114] the trial continued beyond this point due to the existence of a previous trial that indicated a large reduction in mortality, rather than the harmful effect observed in VEST.

Table 17.1

Accumulating results for the Vesnarinone in Heart Failure Trial (VEST) [129]

Information fraction

Log-rank Z-value (high dose)

0.43

+0.99

0.19

−0.25

0.34

−0.23

0.50

−2.04

0.60

−2.32

0.67

−2.50

0.84

−2.22

0.20

−2.43

0.95

−2.71

1.0

−2.41

Table 17.2

Conditional power for the Vesnarinone in Heart Failure Trial (VEST) [129]

RR

Information fraction

0.50

0.67

0.84

0.50

0.46

<0.01

<0.01

0.70

0.03

<0.01

<0.01

1.0

<0.01

<0.01

<0.01

1.3

<0.01

<0.01

<0.01

1.5

<0.01

<0.01

<0.01

RR = relative risk

The Beta-Blocker Heart Attack Trial [62, 63] made considerable use of this approach. As discussed, the interim results were impressive with 1 year of follow-up still remaining. One question posed was whether the strong favorable trend (Z = 2.82) could be lost during that year. The probability of rejecting H ₀ at the scheduled end of the trial, given the existing trend (γ ₀), was approximately 0.90. This meant that the false positive or Type I error was no more than α/γ ₀ = 0.05/0.90 or 0.056.

Other Approaches

Other techniques for interim analysis of accumulating data have also received attention. These include binomial sampling strategies [15], decision theoretic models [130], and likelihood or Bayesian methods [131–140]. Bayesian methods require specifying a prior probability on the possible values of the unknown parameter. The experiment is performed and based on the data obtained, the prior probability is adjusted. If the adjustment is large enough, the investigator may change his opinion (i.e., his prior belief). Spiegelhalter et al. [139] and Freedman et al. [135] have implemented Bayesian methods that have frequentist properties very similar to boundaries of either the Pocock or O’Brien–Fleming type. It is somewhat reassuring that two methodologies, even from a different theoretical framework, can provide similar monitoring procedures. While the Bayesian view is critical of the hypothesis testing methods because of the arbitrariness involved, the Bayesian approach is perhaps hampered mostly by the requirement that the investigator formally specify a prior probability. However, if a person during the decision-making process uses all of the factors and methods discussed in this chapter, a Bayesian approach is involved, although in a very informal way.

One Bayesian method to assess futility that has been used extensively is referred to as predictive power and is related to the concept of conditional power. In this case, the series of possible alternative intervention effects, θ, are represented by a prior distribution for θ, distributing the probability across the alternatives. The prior probability distribution can be modified by the current trend to give an updated posterior for θ. The conditional power is calculated as before for a specific value of θ. Then a predictive or “average” power is calculated by integrating the conditional power over the posterior distribution for θ:

This can then be utilized by the monitoring committee to assess whether the trial is still viable, as was computed for the interim analyses conducted in VEST [129] as shown in Table 17.3. In this case, the prior was taken from an earlier trial of vesnarinone where the observed reduction in mortality was over 60% (relative risk = 0.40). For these calculations, the prior was first set at the point estimate of the hazard ratio equal to 0.40. Using this approach, it is clear that VEST would not likely have shown a benefit at the end of the trial.

Table 17.3

Predictive probability for the Vesnarinone in Heart Failure Trial (VEST) [129]

Date

Ta

Probability

Hazard rate = 0.40

2/7/96

0.50

0.28

3/7/96

0.60

0.18

4/10/96

0.67

<0.0001

5/19/96

0.84

<0.0001

6/26/96

0.90

<0.0001

aT = information fraction

We have stated that the monitoring committee should be aware of all the relevant information in the use of the intervention which existed before the trial started and which emerges during the course of a trial. Some have argued that all of this information should be pooled or incorporated and updated sequentially in a formal statistical manner [141]. This is referred to as cumulative meta-analysis (see Chap. 18). We do not generally support cumulative or sequential meta-analysis as a primary approach for monitoring a trial. We believe that the results of the ongoing trial should be first presented alone, in detail, including baseline comparisons, primary and secondary outcomes, adverse events and relevant laboratory data (see Chap. 16). As supportive evidence for continuation or termination, results or other analysis from external completed trials may be used, including a pooled analysis of all available external data.

Trend Adaptive Designs and Sample Size Adjustments

Sample size adjustments based on overall event rates or outcome variability, without knowledge of interim trends, have long been performed to regain trial power with no issues regarding impact on Type I error or other design concerns. However, while sample size adjustments based on comparing emerging trends in the intervention and control groups were initially discouraged, statistical methodology now allows trialists to adjust the sample size and maintain the Type I error while regaining power [142–164]. It is possible to have a statistically efficient or nearly efficient design if the adaptations are prespecified [154]. While multiple adjustments over the course of follow-up are possible, the biggest gain comes from a single adaptive adjustment.

These methods must be implemented by some individual or party that is aware of the emerging trend. In general, we do not recommend that the monitoring committee perform this function because it may be aware of other factors that would mitigate any sample size increase but cannot share those issues with the trial investigators or sponsors. This can present an awkward if not an ethical dilemma for the monitoring committee. Rather, someone who only knows the emerging trend should make the sample size adjustment recommendation to the investigators. Whatever trend adaptive method is used must also take into account the final analyses as discussed briefly in Chap. 18, because it can affect the final critical value. We will briefly describe a few of these methods [145, 147, 159].

As proposed by Cui et al. [146, 147] for adaptive adjustments in a group sequential setting, suppose we measure an outcome variable denoted as X where X has a N(0,1) distribution and n is current sample size, N ₀ is initial total sample size, N is new target sample size, θ is hypothesized intervention effect, and t is n/N ₀. In this case, we can have an estimate of the intervention effect and a test statistic

based on n observations.

We then compute a revised sample size N based on the current trend, assuming the same initial Type I error and desired power. A new test statistic is defined that combines the already observed data and the yet to be obtained data.

In this setting, we would reject the null hypothesis H ₀ of no treatment effect if $$ {Z}_W^{(N)}>{Z}_{\alpha } $$. This revised test statistic controls the Type I error at the desired level α. However, less weight is assigned to the new or additional observations, yielding what is referred to as a weighted Z statistic. That is, the weight given to each trial participant prior to any sample size adjustment is greater than weight given to participants after the adjustment, violating a “one participant—one vote” principle. This discounting may not be acceptable for scientific and ethical reasons [144, 164].

Other approaches have also been proposed. A modification proposed by Chen et al. [145] requires that both the weighted and un-weighted test statistics exceed the standard critical value.

In this case, the Type I error < α and there is no loss of power. Another approach, an adjusted p value method, proposed by Proschan and colleagues [159, 160] requires a “promising” p value before allowing an increase in sample size. However, this approach requires stopping if the first stage p value is not promising. It also requires a larger critical value at the second stage to control the Type I error. As an example, consider a one-sided significance level a = 0.05, which would ordinarily have a critical value of 1.645 for the final test statistic. In this case the promising p value, p′, and the final critical values, Z′, are as follows, regardless of the sample size in the second stage:

p′:

0.10

0.15

0.20

0.25

0.50

Z′:

1.77

1.82

1.85

1.875

1.95

This simple method will control the Type I error but in fact may make Type I error substantially less than 0.05. A method can be developed to obtain an exact Type I error as a function of Z(t) and the adjusted sample size N, using a conditional power type calculation [127] as described below.

Conditional power, CP, is a useful calculation to assess the likelihood of exceeding a critical value at the scheduled end of a trial, given the current data or value of the interim test statistic and making assumptions about the future intervention effect as described earlier in this chapter [67, 126, 128]. The computation of conditional power in this case is relatively simple. Let θ be a function of the intervention affect, as described earlier, and then

Applying the idea of conditional power to the trend adaptive design, we can define an algorithm to adjust the sample size and still control the Type I error [146]. For example,

Let Δ = observed effect and δ = assumed effect. If we observe that for θ(Δ) as a function of the observed effect Δ, and θ (δ) as a function of the assumed δ, then if

where N is the final targeted sample size. The properties of this procedure have not been well investigated but the idea is related to other conditional power approaches [153]. These conditional power procedures adjust the sample size if the computed conditional power for the current trend is marginal, with only a trivial impact on Type I error. For example, define a lower limit (c ℓ) and an upper limit (c u) such that for the current trend θ(Δ):

if $$ \mathrm{C}\mathrm{P}\left(Z(t),\theta \left(\varDelta \right)\right)<{c}_{\mathrm{\ell}} $$, then terminate for futility and accept the null (required),

if $$ \mathrm{C}\mathrm{P}\left(Z(t),\theta \left(\varDelta \right)\right)>{c}_{\mathrm{u}} $$, then continue with no change in sample size, or

if $$ {c}_{\mathrm{\ell}}<\mathrm{C}\mathrm{P}\left(Z(t),\theta \left(\varDelta \right)\right)<{c}_{\mathrm{u}} $$, then increase sample size from N ₀, to N to get conditional power to the desired level.

Chen et al. [145] suggested a modest alternative. If the conditional power is 50% or larger, then increase the sample size to get the desired power. An upper cap is typically placed on the size of the increase in sample size. Increase N ₀ if the interim result is “promising,” defined as conditional power >50% for the current trend but the increase in N ₀ cannot be greater than 1.75-fold. Under these conditions, Type I error is not increased and there is no practical loss in power. This approach is one that we favor since it is simple to implement, easy to understand and preserves the design characteristics.

Adaptive designs have appeal because the assumptions made during protocol development often fail to hold precisely for the implemented trial, making adjustments useful or even necessary for the study to succeed. However, adaptive designs also rely on assumptions which prove to be unmet in practice, so that theoretical gains are not necessarily realized. For example, it is often found that the observed event rate is less than expected, or the intervention effect not as great as had been assumed. Tsiatis and Mehta [163] have provided conditions under which a properly designed group sequential trial is more efficient than these adaptive designs, though Mehta has also argued that factors such as allocation of financial and participant resources may be as important as statistical efficiency [157]. In any case, a clear need exists for adaptive designs, including trend adaptive designs. We are fortunate that technical advances have been made through several new methods. Research continues on finding methods which can applied to different trial settings [143, 150–152, 154–158, 161, 164].

Perhaps the largest challenge is how to implement the trend adaptive design without introducing bias or leaving the door open for bias. If one utilizes one of the described trend adaptive designs, anyone who knows the details of the method can “reverse engineer” the implementation and obtain a reasonable estimate of what the current trend (Z(t)) must have been to generate the adjusted sample size (N). Given that these trend adaptive designs have as yet not been widely used, there is not enough experience to recommend what can be done to best minimize bias. However, as suggested earlier, a third party who knows only the emerging trend and none of the other secondary or safety data is probably best suited to make these calculations and provide them to the investigators.

References

1.

Canner PL. Practical Aspects of Decision-Making In Clinical Trials—The Coronary Drug Project as a Case-Study. Control Clin Trials 1981;1:363–376.

2.

DeMets DL. Data monitoring and sequential analysis—An academic perspective. J Acquir Immune Defic Syndr 1990;3:S124–S133.

3.

DeMets DL, Furberg C, Friedman LM. Data monitoring in clinical trials: a case studies approach. New York, NY, Springer, 2006.MATH

4.

Ellenberg SS, Fleming TR, DeMets DL. Data Monitoring Committees in Clinical Trials: A Practical Perspective. Wiley, 2003.

5.

Fisher MR, Roecker EB, DeMets DL. The role of an independent statistical analysis center in the industry-modified National Institutes of Health model. Drug Inf J 2001;35:115–129.

6.

Fleming TR, DeMets DL. Monitoring of clinical trials: issues and recommendations. Control Clin Trials 1993;14:183–197.

7.

Anscombe FJ. Sequential medical trials. J Am Stat Assoc 1963;58:365–383.MathSciNet

8.

Armitage P. Restricted sequential procedures. Biometrika 1957;9–26.

9.

Armitage P, McPherson CK, Rowe BC. Repeated Significance Tests on Accumulating Data. J R Stat Soc Ser A 1969;132:235–244.MathSciNet

10.

Bross I Sequential medical plans. Biometrics 1952;8:188–205.

11.

Cornfield J. Sequential trials, sequential analysis and the likelihood principle. Am Stat 1966;20:18–23.MATH

12.

DeMets DL, Lan KKG. An Overview of Sequential-Methods and Their Application in Clinical-Trials. Commun Stat Theory Methods 1984;13:2315–2338.MATH

13.

Robbins H. Some aspects of the sequential design of experiments. Bulletin of the American Mathematical Society 1952;58:527–535.MathSciNetMATH

14.

Robbins H. Statistical methods related to the law of the iterated logarithm. Ann Math Stat 1970;1397–1409.

15.

Simon R, Weiss GH, Hoel DG. Sequential Analysis of Binomial Clinical Trials. Biometrika 1975;62:195–200.MathSciNetMATH

16.

Wald A. Sequential Analysis. Dover Publications, 2013.

17.

Whitehead J, Stratton I. Group Sequential Clinical Trials with Triangular Continuation Regions. Biometrics 1983;39:227–236.MathSciNet

18.

Whitehead J. The Design and Analysis of Sequential Clinical Trials. Wiley, 1997.

19.

Whitehead J, Jones D. The analysis of sequential clinical trials. Biometrika 1979;66:443–452.MATH

20.

Armitage P. Sequential medical trials, ed 2. New York, Wiley, 1975.

21.

Silverman WA, Agate FJ, Fertig JW. A sequential trial of the nonthermal effect of atmospheric humidity on survival of newborn infants of low birth weight. Pediatrics 1963;31:719–724.

22.

Truelove SC, Watkinson G, Draper G. Comparison of corticosteroid and sulphasalazine therapy in ulcerative colitis. Br Med J 1962;2:1708.

23.

Freireich EJ, Gehan E, Frei E, et al. The effect of 6-mercaptopurine on the duration of steroid-induced remissions in acute leukemia: A model for evaluation of other potentially useful therapy. Blood 1963;21:699–716.

24.

McPherson CK, Armitage P. Repeated significance tests on accumulating data when the null hypothesis is not true. J R Stat Soc Ser A 1971;15–25.

25.

Chatterjee SK, Sen PK. Nonparametric testing under progressive censoring. Calcutta Statist Assoc Bull 1973;22:13–50.MathSciNetMATH

26.

Dambrosia JM, Greenhouse SW. Early stopping for sequential restricted tests of binomial distributions. Biometrics 1983;695–710.

27.

Whitehead J, Jones DR, Ellis SH. The analysis of a sequential clinical trial for the comparison of two lung cancer treatments. Statist Med 1983;2:183–190.

28.

Breslow NE, Haug C. Sequential comparison of exponential survival curves. J Am Stat Assoc 1972;67:691–697.

29.

Canner PL. Monitoring treatment differences in long-term clinical trials. Biometrics 1977;603–615.

30.

Davis CE. A two sample Wilcoxon test for progressively censored data. Commun Stat Theory Methods 1978;7:389–398.MATH

31.

Joe H, Koziol JA, Petkau AJ. Comparison of procedures for testing the equality of survival distributions. Biometrics 1981;327–340.

32.

Jones D, Whitehead J. Sequential forms of the log rank and modified Wilcoxon tests for censored data. Biometrika 1979;66:105–113.MathSciNetMATH

33.

Koziol JA, Petkau AJ. Sequential testing of the equality of two survival distributions using the modified Savage statistic. Biometrika 1978;65:615–623.MathSciNetMATH

34.

Muenz LR, Green SB, Byar DP. Applications of the Mantel-Haenszel statistic to the comparison of survival distributions. Biometrics 1977;617–626.

35.

Nagelkerke NJD, Hart AAM. The sequential comparison of survival curves. Biometrika 1980;67:247–249.MathSciNetMATH

36.

Sellke T, Siegmund D. Sequential analysis of the proportional hazards model. Biometrika 1983;70:315–326.MathSciNetMATH

37.

Haybittle JL. Repeated assessment of results in clinical trials of cancer treatment. Br J Radiol 1971;44:793–797.

38.

Peto R, Pike MC, Armitage P, et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Br J Cancer 1976;34:585.

39.

Pocock SJ. Group Sequential Methods in Design and Analysis of Clinical-Trials. Biometrika 1977;64:191–200.

40.

Pocock SJ. Size of cancer clinical trials and stopping rules. Br J Cancer 1978;38:757.

41.

Pocock SJ. Interim analyses for randomized clinical trials: the group sequential approach. Biometrics 1982;38:153–162.

42.

O’Brien PC, Fleming TR. A multiple testing procedure for clinical trials. Biometrics 1979;35:549–556.

43.

DeMets DL. Practical aspects in data monitoring: a brief review. Stat Med 1987;6:753-760.

44.

Emerson SS, Fleming TR. Interim analyses in clinical trials. Oncology (Williston Park, NY) 1990;4:126.

45.

Fleming TR, Watelet LF. Approaches to monitoring clinical trials. J Natl Cancer Inst 1989;81:188–193.

46.

Freedman LS, Lowe D, Macaskill P. Stopping rules for clinical trials. Statist Med 1983;2:167–174.

47.

Jennison C, Turnbull BW. Statistical approaches to interim monitoring of medical trials: a review and commentary. Stat Sci 1990;299–317.

48.

Gail MH, DeMets DL, Slud EV. Simulation studies on increments of the two-sample logrank score test for survival time data, with application to group sequential boundaries. Lecture Notes-Monograph Series 1982;2:287–301.

49.

Harrington DP, Fleming TR, Green SJ. Procedures for serial testing in censored survival data; in Crowley J, Johnson RA, Gupta SS (eds): Survival Analysis. Hayward, CA, Institute of Mathematical Statistics, 1982, pp 269–286.

50.

Mantel N. Evaluation of survival data and two new rank order statistics arising in its consideration. Cancer chemotherapy reports Part 1 1966;50:163–170.

51.

Tsiatis AA. The asymptotic joint distribution of the efficient scores test for the proportional hazards model calculated over time. Biometrika 1981;68:311–315.MathSciNetMATH

52.

Tsiatis AA. Group sequential methods for survival analysis with staggered entry. Lecture Notes-Monograph Series 1982;2:257–268.

53.

Tsiatis AA. Repeated Significance Testing for A General-Class of Statistics Used in Censored Survival Analysis. J Am Stat Assoc1982;77:855–861.MathSciNetMATH

54.

Gehan EA. A Generalized Wilcoxon Test for Comparing Arbitrarily Singly-Censored Samples. Biometrika 6-1-1965;52:203–223.

55.

Slud E, Wei LJ. Two-Sample Repeated Significance Tests Based on the Modified Wilcoxon Statistic. J Am Stat Assoc 1982;77:862–868.MathSciNetMATH

56.

Peto R, Peto J: Asymptotically Efficient Rank Invariant Test Procedures. J R Stat Soc Ser A 1972;135:185–207.MATH

57.

DeMets DL, Gail MH. Use of logrank tests and group sequential methods at fixed calendar times. Biometrics 1985;41:1039–1044.

58.

George SL. Sequential Methods Based on the Boundaries Approach for the Clinical Comparison Of Survival Times-Discussion. Statistics in Medicine 13[13-14], 1369–1370. 1994, John Wiley & Sons Ltd.

59.

Kim K, Tsiatis AA. Study Duration for Clinical-Trials with Survival Response and Early Stopping Rule. Biometrics 1990;46:81–92.MathSciNetMATH

60.

Kim K. Study duration for group sequential clinical trials with censored survival data adjusting for stratification. Statist Med 1992;11:1477–1488.

61.

Whitehead J: Sequential methods based on the boundaries approach for the clinical comparison of survival times. Statist Med 1994;13:1357–1368.

62.

Beta-Blocker Heart Attack Trial Research Group. A randomized trial of propranolol in patients with acute myocardial infarction: I. mortality results. JAMA 1982;247:1707–1714.

63.

DeMets DL, Hardy R, Friedman LM, Gordon Lan KK. Statistical aspects of early termination in the Beta-Blocker Heart Attack Trial. Control Clin Trials 1984;5:362–372.

64.

DeMets DL, Lan KK. Interim analysis: the alpha spending function approach. Stat Med 1994;13:1341–1352.

65.

Kim K, DeMets DL: Design and Analysis of Group Sequential Tests Based on the Type I Error Spending Rate Function. Biometrika 1987;74:149–154.MathSciNetMATH

66.

Lan KKG, Rosenberger WF, Lachin JM. Use of spending functions for occasional or continuous monitoring of data in clinical trials. Stat Med 1993;12:2219–2231.

67.

Lan KKG, Zucker DM. Sequential monitoring of clinical trials: the role of information and Brownian motion. Stat Med 1993;12:753–765.

68.

Lan KKG, DeMets DL. Discrete Sequential Boundaries for Clinical-Trials. Biometrika 1983;70:659–663.MathSciNetMATH

69.

Lan KKG, DeMets DL, Halperin M. More Flexible Sequential and Non-Sequential Designs in Long-Term Clinical-Trials. Commun Stat Theory Methods 1984;13:2339–2353.

70.

Lan KKG, Reboussin DM, DeMets DL. Information and Information Fractions for Design and Sequential Monitoring of Clinical-Trials. Commun Stat Theory Methods 1994;23:403–420.MATH

71.

Lan KKG, DeMets D. Group sequential procedures: calendar versus information time. Stat Med 1989;8:1191–1198.

72.

Reboussin DM, DeMets DL, Kim K, Lan KKG. Lan-DeMets Method—Statistical Programs for Clinical Trials. [2.1]. 11–17–2003.

73.

Reboussin DM, DeMets DL, Kim K, Lan KKG. Computations for group sequential boundaries using the Lan-DeMets spending function method. Control Clin Trials 2000;21:190–207.

74.

DeMets DL. Futility approaches to interim monitoring by data monitoring committees. Clin Trials 2006;3:522–529.

75.

Hwang IK, Shih WJ, De Cani JS. Group sequential designs using a family of type I error probability spending functions. Statist Med 1990;9:1439–1445.

76.

Wang SK, Tsiatis AA: Approximately optimal one-parameter boundaries for group sequential trials. Biometrics 1987;193–199.

77.

Lan KKG, DeMets DL. Changing frequency of interim analysis in sequential monitoring. Biometrics 1989;45:1017–1020.

78.

Proschan MA, Follmann DA, Waclawiw MA. Effects of assumption violations on type I error rate in group sequential monitoring. Biometrics 1992;1131–1143.

79.

Geller NL. Discussion of “Interim analysis: the alpha spending approach”. Statist Med 1994;13:1353–1356.

80.

Falissard B, Lellouch J. A new procedure for group sequential analysis in clinical trials. Biometrics 1992;373–388.

81.

Lan KKG, Lachin JM. Implementation of group sequential logrank tests in a maximum duration trial. Biometrics 1990;46:759–770.

82.

Li ZQ, Geller NL. On the Choice of Times for Data Analysis in Group Sequential Clinical Trials. Biometrics 1991;47:745–750.

83.

Jennison C, Turnbull BW. Group-sequential analysis incorporating covariate information. J Am Stat Assoc 1997;92:1330–1341.MathSciNetMATH

84.

Scharfstein DO, Tsiatis AA, Robins JM. Semiparametric Efficiency and Its Implication on the Design and Analysis of Group-Sequential Studies. J Am Stat Assoc 1997;92:1342–1350.MATH

85.

Kim K, DeMets DL. Sample size determination for group sequential clinical trials with immediate response. Stat Med 1992;11:1391–1399.

86.

Lee JW, DeMets DL. Sequential Comparison of Changes with Repeated Measurements Data. J Am Stat Assoc 1991;86:757–762.MathSciNet

87.

Lee JW, DeMets DL. Sequential Rank-Tests with Repeated Measurements in Clinical-Trials. J Am Stat Assoc 1992;87:136–142.MathSciNetMATH

88.

Lee JW. Group sequential testing in clinical trials with multivariate observations: a review. Statist Med 1994;13:101–111.

89.

Su JQ, Lachin JM. Group Sequential Distribution-Free Methods for the Analysis of Multivariate Observations. Biometrics 1992;48:1033–1042.MATH

90.

Wei LJ, Su JQ, Lachin JM. Interim Analyses with Repeated Measurements in A Sequential Clinical-Trial. Biometrika 1990;77:359–364.MathSciNetMATH

91.

Wu MC, Lan G KK. Sequential Monitoring for Comparison of Changes in a Response Variable in Clinical Studies. Biometrics 1992;48:765–779.

92.

Gange SJ, DeMets DL. Sequential monitoring of clinical trials with correlated responses. Biometrika 1996;83:157–167.MathSciNetMATH

93.

Fairbanks K, Madsen R. P values for tests using a repeated significance test design. Biometrika 1982;69:69–74.MathSciNet

94.

Chang MN, O’Brien PC. Confidence intervals following group sequential tests. Control Clin Trials 1986;7:18–26.

95.

DeMets DL, Lan KKG. Discussion of: Interim analyses: The repeated confidence interval approach by C. Jennison and BW Turnbull. J R Stat Soc Series B Stat Methodol 1989;51:344.

96.

Emerson SS, Fleming TR: Parameter Estimation Following Group Sequential Hypothesis Testing. Biometrika 1990;77:875–892.MathSciNet

97.

Hughes MD, Pocock SJ. Stopping rules and estimation problems in clinical trials. Stat Med 1988;7:1231–1242.

98.

Jennison C, Turnbull BW. Repeated Confidence-Intervals for Group Sequential Clinical-Trials. Control Clin Trials 1984;5:33–45.

99.

Jennison C, Turnbull BW. Interim Analyses: The Repeated Confidence Interval Approach. J R Stat Soc Series B Stat Methodol 1989;51:305–361.MathSciNetMATH

100.

Kim K, DeMets DL. Confidence Intervals Following Group Sequential Tests in Clinical Trials. Biometrics 12-1-1987;43:857–864.

101.

Kim K. Point Estimation Following Group Sequential Tests. Biometrics 1989;45:613–617.MATH

102.

Pocock SJ, Hughes MD. Practical problems in interim analyses, with particular regard to estimation. Control Clin Trials 1989;10:209S–221S.

103.

Rosner GL, Tsiatis AA. Exact confidence intervals following a group sequential trial: A comparison of methods. Biometrika 1988;75:723–729.MATH

104.

Siegmund D. Estimation following sequential tests. Biometrika 1978;65:341–349.MathSciNetMATH

105.

Tsiatis AA, Rosner GL, Mehta CR. Exact confidence intervals following a group sequential test. Biometrics 1984;797–803.

106.

Whitehead J: On the bias of maximum likelihood estimation following a sequential test. Biometrika 1986;73:573–581.MathSciNetMATH

107.

Whitehead J, Facey KM. Analysis after a sequential trial: A comparison of orderings of the sample space. Joint Society for Clinical Trials/International Society for Clinical Biostatistics, Brussels 1991.

108.

Fleming TR. Treatment evaluation in active control studies. Cancer Treat Rep 1987;71:1061–1065.

109.

Fleming TR. Evaluation of active control trials in AIDS. J Acquir Immune Defic Syndr 1990;3:S82–S87.

110.

DeMets DL, Ware JH. Group Sequential Methods for Clinical Trials with A One-Sided Hypothesis. Biometrika 1980;67:651–660.MathSciNet

111.

DeMets DL, Ware JH. Asymmetric Group Sequential Boundaries for Monitoring Clinical-Trials. Biometrika 1982;69:661–663.

112.

Emerson SS, Fleming TR. Symmetric Group Sequential Test Designs. Biometrics 1989;45:905–923.MathSciNetMATH

113.

Gould AL, Pecore VJ. Group sequential methods for clinical trials allowing early acceptance of Ho and incorporating costs. Biometrika 1982;69:75–80.

114.

DeMets DL, Pocock SJ, Julian DG. The agonising negative trend in monitoring of clinical trials. Lancet 1999;354:1983–1988.

115.

Cardiac Arrhythmia Suppression Trial Investigators. Effect of the antiarrhythmic agent moricizine on survival after myocardial infarction. N Engl J Med 1992;327:227–233.

116.

Friedman LM, Bristow JD, Hallstrom A, et al. Data monitoring in the cardiac arrhythmia suppression trial. Online Journal of Current Clinical Trials 7–31–1993;79.

117.

Pawitan Y, Hallstrom A. Statistical interim monitoring of the cardiac arrhythmia suppression trial. Statist Med 1990;9:1081–1090.

118.

Feyzi J, Julian DG, Wikstrand J, Wedel H. Data monitoring experience in the Metoprolol CR/XL randomized intervention trial in chronic heart failure: Potentially high-risk treatment in high-risk patients; in Data Monitoring in Clinical Trials: Springer, 2006, pp 136–147.

119.

Jennison C, Turnbull BW. Group Sequential Methods with Applications to Clinical Trials. Taylor & Francis, 1999.

120.

Proschan MA, Lan KKG, Wittes JT. Statistical Monitoring of Clinical Trials: A Unified Approach. Springer New York, 2006.

121.

Alling DW. Early decision in the Wilcoxon two-sample test. J Am Stat Assoc 1963;58:713–720.MathSciNet

122.

Alling DW. Closed sequential tests for binomial probabilities. Biometrika 1966;73–84.

123.

Halperin M, Ware J. Early decision in a censored Wilcoxon two-sample test for accumulating survival data. J Am Stat Assoc 1974;69:414–422.MathSciNetMATH

124.

DeMets DL, Halperin M. Early stopping in the two-sample problem for bounded random variables. Control Clin Trials 1982;3:1–11.

125.

Canner PL. Monitoring of the data for evidence of adverse or beneficial treatment effects. Control Clin Trials 1983;4:467–483.

126.

Lan KKG, Simon R, Halperin M. Stochastically curtailed tests in long-term clinical trials. Seq Anal 1982;1:207–219.MathSciNetMATH

127.

Halperin M, Gordon Lan KK, Ware JH, et al. An aid to data monitoring in long-term clinical trials. Control Clin Trials 1982;3:311–323.

128.

Lan KKG, Wittes J. The B-value: a tool for monitoring data. Biometrics 1988;44:579–585.

129.

Cohn JN, Goldstein SO, Greenberg BH, et al. A dose-dependent increase in mortality with vesnarinone among patients with severe heart failure. N Engl J Med 1998;339:1810–1816.

130.

Colton T. A model for selecting one of two medical treatments. J Am Stat Assoc 1963;58:388–400.MathSciNet

131.

Carlin BP, Louis TA. Bayes and Empirical Bayes Methods for Data Analysis, Second Edition. Taylor & Francis, 2000.

132.

Choi SC, Pepple PA. Monitoring Clinical Trials Based on Predictive Probability of Significance. Biometrics 1989;45:317–323.MATH

133.

Cornfield J. A Bayesian test of some classical hypotheses—with applications to sequential clinical trials. J Am Stat Assoc 1966;61:577–594.MathSciNet

134.

Cornfield J. Recent methodological contributions to clinical trials. Am J Epidemiol 1976;104:408–421.

135.

Freedman LS, Spiegelhalter DJ, Parmar MK. The what, why and how of Bayesian clinical trials monitoring. Statist Med 1994;13:1371–1383.

136.

George SL, Li C, Berry DA, Green MR. Stopping a clinical trial early: Frequentist and bayesian approaches applied to a CALGB trial in non-small-cell lung cancer. Statist Med 1994;13:1313–1327.

137.

Grieve AP, Choi SC, Pepple PA. Predictive Probability in Clinical Trials. Biometrics 1991;47:323–330.MATH

138.

Machin D. Discussion of “The what, why and how of Bayesian clinical trials monitoring”. Statist Med 1994;13:1385–1389.

139.

Spiegelhalter DJ, Freedman LS, Blackburn PR. Monitoring clinical trials: Conditional or predictive power? Control Clin Trials 1986;7:8–17.

140.

Spiegelhalter DJ. Probabilistic prediction in patient management and clinical trials. Statist Med 1986;5:421–433.

141.

Lau J, Antman EM, Jimenez-Silva J, et al. Cumulative meta-analysis of therapeutic trials for myocardial infarction. N Engl J Med 1992;327:248–254.

142.

Bauer P, Kohne K. Evaluation of Experiments with Adaptive Interim Analyses. Biometrics 1994;50:1029–1041.MATH

143.

Berry DA. Adaptive clinical trials: the promise and the caution. J Clin Oncol 2011;29:606–609.

144.

Burman CF, Sonesson C. Are Flexible Designs Sound? Biometrics 2006;62:664–669.MathSciNetMATH

145.

Chen YH, DeMets DL, Lan KK. Increasing the sample size when the unblinded interim result is promising. Stat Med 2004;23:1023–1038.

146.

Cui L, Hun HMJ, Wang SJ. Impact of changing sample size in a group sequential clinical trial. Proceedings of the Biopharmaceutical Section, American Statistical Association, 1997, 52–57. 1997.

147.

Cui L, Hung HMJ, Wang SJ. Modification of Sample Size in Group Sequential Clinical Trials. Biometrics 1999;55:853–857.MATH

148.

Fisher LD. Self-designing clinical trials. Statist Med 1998;17:1551–1562.

149.

Fleming TR. Standard versus adaptive monitoring procedures: a commentary. Statist Med 2006;25:3305–3312.MathSciNet

150.

Hu F, Zhang LX, He X. Efficient randomized-adaptive designs. Ann Stat 2009;2543-2560.

151.

Hung HMJ, Wang SJ. Sample Size Adaptation in Fixed-Dose Combination Drug Trial. J Biopharm Stat 2012;22:679–686.MathSciNet

152.

Irle S, Schafer H: Interim design modifications in time-to-event studies. J Am Stat Assoc 2012;107:341–348.MathSciNetMATH

153.

Lan KKG, Trost DC. Estimation of parameters and sample size re-estimation. Proceedings–Biopharmaceutical Section American Statistical Association, 48–51. 1997. American Statistical Association.

154.

Levin GP, Emerson SC, Emerson SS. Adaptive clinical trial designs with pre-specified rules for modifying the sample size: understanding efficient types of adaptation. Statist Med 2013;32:1259–1275.MathSciNet

155.

Lui KJ Sample size determination under an exponential model in the presence of a confounder and type I censoring. Control Clin Trials 1992;13:446–458.

156.

Luo X, Li M, Shih WJ, Ouyang P. Estimation of Treatment Effect Following a Clinical Trial with Adaptive Design. J Biopharm Stat 2012;22:700–718.MathSciNet

157.

Mehta CR. Adaptive clinical trial designs with pre-specified rules for modifying the sample size: a different perspective. Statist Med 2013;32:1276–1279.

158.

Posch M, Proschan MA. Unplanned adaptations before breaking the blind. Statist Med 2012;31:4146–4153.MathSciNet

159.

Proschan MA, Hunsberger SA. Designed Extension of Studies Based on Conditional Power. Biometrics 1995;51:1315–1324.MATH

160.

Proschan MA, Liu Q, Hunsberger S. Practical midcourse sample size modification in clinical trials. Control Clin Trials 2003;24:4–15.

161.

Proschan MA. Sample size re-estimation in clinical trials. Biom J 2009;51:348–357.MathSciNet

162.

Shen Y, Fisher. Statistical Inference for Self-Designing Clinical Trials with a One-Sided Hypothesis. Biometrics 1999;55:190–197.

163.

Tsiatis AA, Mehta C. On the Inefficiency of the Adaptive Design for Monitoring Clinical Trials. Biometrika 2003;90:367–378.MathSciNetMATH

164.

van der Graaf R, Roes KC, van Delden JJ. Adaptive trials in clinical research: scientific and ethical issues to consider. JAMA 2012;307:2379–2380.

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_18

18. Issues in Data Analysis

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

The analysis of data obtained from a clinical trial represents the outcome of the planning and implementation already described. Primary and secondary questions addressed by the clinical trial can be tested and new hypotheses generated. Data analysis is sometimes viewed as simple and straightforward, requiring little time, effort, or expense. However, careful analysis usually requires a major investment in all three. It must be done with as much care and concern as any of the design or data-gathering aspects. Furthermore, inappropriate statistical analyses can introduce bias, result in misleading conclusions and impair the credibility of the trial.

In the context of clinical trials, the term bias has several senses. One is what we might call experimenter bias. This sense applies to a difference in the behavior, conscious or unconscious, of investigators depending on what they believe about the intervention. For example, in an unblinded trial, or a trial in which the intervention assignment can be guessed, the investigator may treat a participant or data from a participant differently depending on whether she believes that participant has received the experimental intervention. These differences in behavior can lead to a second sense of the term bias, one with a more technical definition which we may call estimation bias. If the goal of the trial is to estimate how the intervention affects an outcome measured in a specified population, bias is any difference between that estimate and the true effect which is not attributable to random variation.

Even in a randomized, double-blind clinical trial which has been conducted without any experimenter bias, the estimated effect can be biased by excluding randomized participants or observed outcomes, by inappropriate choice of analytic techniques, and by missing or poor quality data caused by mechanisms which were not the same in the intervention and control groups. This chapter will focus on how to avoid introducing estimation bias into the analysis of clinical trial data.

Several introductory textbooks of statistics [1–8] provide excellent descriptions for many basic methods of analysis. Chapter 15 presents essentials for analysis of survival data, since they are frequently of interest in clinical trials. This chapter will focus on some issues in the analysis of data which seem to cause confusion in the medical research community. Some of the proposed solutions are straightforward; others require judgment. They reflect a point of view toward bias developed by the authors of this text and many colleagues in numerous collaborative efforts over three to four decades. Whereas some [9–12] have taken similar positions, others [13–16] have opposing views on several issues, and some published trials are more consistent with these opposing views (e.g. [17]).

The analytic approaches discussed here primarily apply to late phase (III and IV) trials. Various exploratory analysis approaches may be entirely reasonable in early phase (I and II) studies where the goal is to obtain information and insight to design better subsequent trials. However, some the fundamentals presented may still be of value in these early phase trials. We have used early examples that were instrumental in establishing many of the analytic principles and added new examples which reinforce them. However, given the multitude of clinical trials, it is not possible to include even a small proportion of the many excellent examples that exist.

Fundamental Point

Removing randomized participants or observed outcomes from analysis and subgrouping on the basis of outcome or other response variables can lead to biased results. Those biases can be of unknown magnitude and direction.

Which Participants Should Be Analyzed?

In the context of clinical trials the term ‘withdrawal’ is used in different ways. For this chapter, ‘withdrawing a participant’ generally means removing from an analysis the data contributed by a participant who has been randomized and perhaps followed for some length of time. A common and related meaning of the term ‘withdrawal’ refers to a participant who is randomized but from whom follow-up data is deliberately not collected, or only partially collected, as a result of decisions made by the study investigators. Yet another meaning indicates someone who is randomized but refuses to continue participating in the trial. The term ‘excluded’ can also be ambiguous, sometimes referring to a participant who does not meet eligibility criteria during screening, sometimes to a participant’s data which was not used in an analysis. Since removing or not using data collected from a randomized participant can lead to bias, the question of which participants’ data should be analyzed is an important one. This chapter has adopted, in part, the terminology used by Peto and colleagues [12] to classify participants according to the nature and extent of their participation.

Discussion about which participants are to be included in the data analysis often arises in clinical trials. Although a laboratory study may have carefully regulated experimental conditions, even the best designed and managed clinical trial cannot be perfectly implemented. Response variable data may be missing, adherence to protocol may not be complete, and some participants, in retrospect, will not have met the entrance criteria. Some investigators may, after a trial has been completed, be inclined to remove from the analysis participants who did not meet the eligibility criteria or did not follow the protocol perfectly. In contrast, others believe that once a participant is randomized, that participant should always be followed and included in the analysis.

The intention-to-treat principle states that all participants randomized and all events, as defined in the protocol, should be accounted for in the primary analysis [12]. This requirement is stated in the International Conference on Harmonisation and FDA guidelines [18, 19]. There are often proposed “modified intention-to-treat” analyses, or “per protocol” or “on treatment” analyses, that suggest that only participants who received at least some of the intervention should be included. However, as we will discuss, any deviations from strict intention-to-treat offer the potential for bias and should be avoided, or at a minimum presented along with an intention-to-treat analysis. Many published analyses claim to have followed the intention-to-treat principle yet do not include all randomized participants and all events. Although the phrase is widely used, “per protocol” analysis suggests that the analysis is the one preferred in the trial’s protocol. For such analyses we think that “on treatment” analysis more accurately reflects what is done.

Exclusions refer to people who are screened as potential participants for a randomized trial but who do not meet all of the entry criteria and, therefore, are not randomized. Reasons for exclusion might be related to age, severity of disease, refusal to participate, or any of numerous other determinants evaluated before randomization. Since these potential participants are not randomized, their exclusion does not bias any intervention-control group comparison (sometimes called internal validity). Exclusions do, however, influence the broader interpretation and applicability of the results of the clinical trial (external validity). In some circumstances, follow-up of excluded people, as was done in the Women’s Health Initiative [20, 21], can be helpful in determining to what extent the results can be generalized. If the event rate in the control group is considerably lower than anticipated, an investigator may want to determine whether most high risk people were excluded or whether she was incorrect in her initial assumption.

Withdrawals from analysis refer to participants who have been randomized but are deliberately excluded from the analysis. As the fundamental point states, omitting participants from analyses can bias the results of the study [22]. If participants are withdrawn, the burden rests with the investigator to convince the scientific community that the analysis has not been biased. However, this is essentially impossible, because no one can be sure that participants were not differentially withdrawn from the study groups. Differential withdrawal can occur even if the number of omitted participants is the same in each group, since the reasons for withdrawal in each group may be different and consequently their risk of primary, secondary and adverse events may be different. As a result, the participants remaining in the randomized groups may not be comparable, undermining one of the reasons for randomization.

Many reasons are given for not including certain participants’ data in the analysis, among them ineligibility and nonadherence.

Ineligibility

A previously common cited reason for withdrawal is that some participants did not meet the entry criteria, a protocol violation unknown at the time of enrollment. Admitting unqualified participants may be the result of a simple clerical error, a laboratory error, a misinterpretation, or a misclassification. Clerical mistakes such as listing wrong sex or age may be obvious. Other errors can arise from differing interpretation of diagnostic studies such as electrocardiograms, x-rays, or biopsies. It is not difficult to find examples in earlier literature [23–31]. The practice of withdrawal for ineligibility used to be common, but appears to be less frequent now, at least in papers published in major journals.

Withdrawals for ineligibility can involve a relatively large number of participants. In an early trial by the Canadian Cooperative Study Group [30], 64 of the 649 enrolled participants (10%) with stroke were later found to have been ineligible. In this four-armed study, the numbers of ineligible participants in the study groups ranged from 10 to 25. The reasons for the ineligibility of these 64 participants were not reported, nor were their outcome experiences. Before cancer cooperative groups implemented phone-in or electronic eligibility checks, 10–20% of participants entered into a trial may have been ineligible after further review. By taking more careful care at the time of randomization, the number of ineligible participants was reduced to a very small percent [32]. Currently, web based systems or Interactive Voice Recording Systems are used for multicenter and multinational clinical trials [33]. These interactive systems can lead clinic staff through a review of key eligibility criteria before randomization is assigned, cutting down on the ineligibility rate. For example, several trials employed these methods [34–38].

A study design may require enrollment within a defined time period following a qualifying event. Because of this time constraint, data concerning a participant’s eligibility might not be available or confirmable at the time the decision must be made to enroll him. For example, the Beta-Blocker Heart Attack Trial looked at a 2–4 year follow-up mortality in people administered a beta-blocking drug during hospitalization for an acute myocardial infarction [23]. Because of known variability in interpretation, the protocol required that the diagnostic electrocardiograms be read by a central unit. However, this verification took several weeks to accomplish. Local investigators, therefore, interpreted the electrocardiograms and decided whether the patient met the necessary criteria for inclusion. Almost 10% of the enrolled participants did not have their myocardial infarction confirmed according to a central reading, and were “incorrectly” randomized. The question then arose: Should the participants be kept in the trial and included in the analysis of the response variable data? The Beta-Blocker Heart Attack Trial protocol required follow-up and analysis of all randomized participants. In this case, the observed benefits from the intervention were similar in those eligible as well as in those “ineligible.”

A more complicated situation occurs when the data needed for enrollment cannot be obtained until hours or days have passed, yet the study design requires initiation of intervention before then. For instance, in the Multicenter Investigation for the Limitation of Infarct Size [26], propranolol, hyaluronidase, or placebo was administered shortly after participants were admitted to the hospital with possible acute myocardial infarctions. In some, the diagnosis of myocardial infarction was not confirmed until after electrocardiographic and serum enzyme changes had been monitored for several days. Such participants were, therefore, randomized on the basis of a preliminary diagnosis of infarction. Subsequent testing may not have supported the initial diagnosis. Another example of this problem involves a study of pregnant women who were likely to deliver prematurely and, therefore, would have children who were at a higher than usual risk of being born with respiratory distress syndrome [24]. Corticosteroids administered to the mother prior to delivery were hypothesized to protect the premature child from developing this syndrome. Although, at the time of the mother’s randomization to either intervention or control groups, the investigator could not be sure that the delivery would be premature, she needed to make a decision whether to enroll the mother into the study. Other examples include trials where thrombolytic agents are being evaluated in reducing mortality and morbidity during and after an acute myocardial infarction. In these trials, agents must be given rapidly before diagnosis can be confirmed [39].

To complicate matters still further, the intervention given to a participant can affect or change the entry diagnosis. For example, in the above mentioned study to limit infarct size, some participants without a myocardial infarction were randomized because of the need to begin intervention before the diagnosis was confirmed. Moreover, if the interventions succeeded in limiting infarct size, they could have affected the electrocardiogram and serum enzyme levels. Participants in the intervention groups with a small myocardial infarction may have had the infarct size reduced or limited and therefore appeared not to have had a qualifying an infarction. Thus, they would not seem to have met the entry criteria. However, this situation could not exist in the placebo control group. If the investigators had withdrawn participants in retrospect who did not meet the study criteria for a myocardial infarction, they would have withdrawn more participants from the intervention groups (those with no documented infarction plus those with small infarction) than from the control group (those with no infarction). This would have produced a bias in later comparisons. On the other hand, it could be assumed that a similar number of truly ineligible participants were randomized to the intervention groups and to the control group. In order to maintain comparability, the investigators might have decided to withdraw the same number of participants from each group. The ineligible participants in the control group could have been readily identified. However, the participants in the intervention groups who were truly ineligible had to be distinguished from those made to appear ineligible by the effects of the interventions. This would have been difficult, if not impossible. In the Multicenter Investigation for the Limitation of Infarct Size (MILIS) for example, all randomized participants were retained in the analysis [26].

An example of possible bias because of withdrawal of ineligible participants is found in the Anturane Reinfarction Trial, which compared sulfinpyrazone with placebo in participants who had recently suffered a myocardial infarction [27–29]. As seen in Table 18.1, of 1,629 randomized participants (813 to sulfinpyrazone, 816 to placebo), 71 were subsequently found to be ineligible. Thirty-eight had been assigned to sulfinpyrazone and 33 to placebo. Despite relatively clear definitions of eligibility and comparable numbers of participants withdrawn, mortality among these ineligible participants was 26.3% in the sulfinpyrazone group (10 of 38) and 12.1% in the placebo group (4 of 33) [27]. The eligible placebo group participants had a mortality of 10.9%, similar to the 12.1% seen among the ineligible participants. In contrast, the eligible participants on sulfinpyrazone had a mortality of 8.3%, less than one-third that of the ineligible participants. Including all 1,629 participants in the analysis gave 9.1% mortality in the sulfinpyrazone group, and 10.9% mortality in the placebo group (p = .20). Withdrawing the 71 ineligible participants (and 14 deaths, 10 vs. 4) gave an almost significant p = .07.

Table 18.1

Mortality by study group and eligibility status in the Anturane Reinfarction Trial

Randomized

Percent mortality

Ineligible

Percent mortality

Eligible

Percent mortality

Sulfinpyrazone

813

9.1

38

26.3

775

8.3

Placebo

816

10.9

33

12.1

783

10.9

Stimulated by criticisms of the study, the investigators initiated a reevaluation of the Anturane Reinfarction Trial results. An independent group of reviewers examined all reports of deaths in the trial [29]. Instead of 14 deceased participants who were ineligible, it found 19; 12 in the sulfinpyrazone group and seven in the placebo group. Thus, supposedly clear criteria for ineligibility can be judged differently. This trial was an early example that affirmed the value of the intention-to-treat principle.

Three trial design policies that relate to withdrawals because of entry criteria violations have been discussed by Peto et al. [12]. The first policy is not to enroll participants until all the diagnostic tests have been confirmed and all the entry criteria have been carefully checked. Once enrollment takes place, no withdrawals from the trial are allowed. For some studies, such as the one on limiting infarct size, this policy cannot be applied because firm diagnoses cannot be ascertained prior to the time when intervention has to be initiated.

The second policy is to enroll marginal or unconfirmed cases and later withdraw from analysis those participants who are proven to have been misdiagnosed. This would be allowed, however, only if the decision to withdraw is based on data collected before enrollment. Any process of deciding upon withdrawal of a participant from a study group should be done blinded with respect to the participant’s outcome and group assignment.

A third policy is to enroll some participants with unconfirmed diagnoses and to allow no withdrawals. This procedure is always valid in that the investigator compares two randomized groups which are comparable at baseline. However, this policy is conservative because each group contains some participants who might not benefit from the intervention. Thus, the overall trial may have less power to detect differences of interest.

A modification to these three policies has been recommended [22]. Every effort should be made to establish the eligibility of participants prior to any randomization. No withdrawals should be allowed and the analyses should include all participants enrolled. Analyses based on only those truly eligible may be performed. If the analyses of data from all enrolled participants and from those eligible agree, the interpretation of the results is clear, at least with respect to participant eligibility. If the results differ, however, the investigator must be very cautious in her interpretation. In general, she should emphasize the analysis with all the enrolled participants because that analysis is always valid.

Any policy on withdrawals should be stated in the study protocol before the start of the study. Though the enrolled cohort is never a random sample, in general, the desired aim is to make the recruited cohort as similar as possible to the population in which the intervention will be used in clinical practice, so withdrawal of participants from the trial or participants’ data from an analysis after the decision to treat should be extremely rare. The actual decision to withdraw specific participants should be done without knowledge of the study group, ideally by someone not directly involved in the trial. Of special concern is withdrawal based on review of selected cases, particularly if the decision rests on a subjective interpretation. Even in double-blind trials, blinding may not be perfect, and the investigator may supply information for the eligibility review differentially depending upon study group and health status. Therefore, withdrawal should be done early in the course of follow-up, before a response variable has occurred, and with a minimum of data exchange between the investigator and the person making the decision to withdraw the participant. This withdrawal approach does not preclude a later challenge by readers of the report, on the basis of potential bias. It should, however, remove the concern that the withdrawal policy was dependent on the outcome of the trial. The withdrawal rules should not be based on knowledge of study outcomes. Even when these guidelines are followed, if the number of withdrawals is high, if the number of entry criteria violations is substantially different in the study groups, or if the event rates in the withdrawn participants are different between the groups, the question will certainly be raised whether bias played a role in the decision to withdraw participants.

Nonadherence

Nonadherence to the prescribed intervention or control regimen is another reason that participants are withdrawn from analysis [40–59]. One version of this is to define an “on treatment” analysis that eliminates any participant who does not adhere to the intervention by some specified amount, as defined in the protocol. One form of nonadherence may be drop-outs and drop-ins (Chap. 14). Drop-outs are participants in the intervention arm who do not adhere to the regimen. Drop-ins are participants in the control arm who receive the intervention. The decision not to adhere to the protocol intervention may be made by the participant, his primary care physician, or the trial investigator. Nonadherence may be due to adverse events in either the intervention or control group, loss of participant interest or perceived benefit, changes in the underlying condition of a participant, or a variety of other reasons.

Withdrawal from analysis of participants who do not adhere to the intervention regimens specified in the study design is often proposed. The motivation for withdrawal of nonadherent participants is that the trial is not a “fair test” of the ideal intervention with these participants included. For example, there may be a few participants in the intervention group who took little or no therapy. One might argue that if participants do not take their medication, they certainly cannot benefit from it. There could also be participants in the control group who frequently receive the study medication. The intervention and control groups are thus “contaminated.” Proponents of withdrawal of nonadherent participants argue that removal of these participants keeps the trial closer to what was intended; that is, a comparison of optimal intervention versus control. The impact of nonadherence on the trial findings is that any observed benefit of the intervention, as compared to the control, will be reduced, making the trial less powerful than it planned. Newcombe [11], for example, discusses the implication of adherence for the analysis as well as the design and sample size. We discuss this in Chap. 8.

A policy of withdrawal from analysis because of participant nonadherence can lead to bias. The overwhelming reason is that participant adherence to a protocol may be related to the outcome. In other words, there may an effect of adherence on the outcome which is independent of the intervention. Certainly, if nonadherence is greater in one group than another, for example if the intervention produces many adverse events, then withdrawal of nonadherent participants could lead to bias. Even if the frequency of nonadherence is the same for the intervention and control groups, the reasons for nonadherence in each group may differ and may involve different types of participants. The concern would always be whether the same types of participants were withdrawn in the same proportion from each group or whether an imbalance had been created. Of course, an investigator could probably neither confirm nor refute the possibility of bias.

For noninferiority trials (see Chaps. 3 and 5), nonadherence may make the two interventions arms to look more alike and thus create bias towards the claim of noninferiority [13, 60]. Any attempt to use only adherers in a noninferiority trial, though, could be biased in unknown directions, thus rendering the results uninterpretable. Again, the best policy is to design a trial to have minimum nonadherence, power the trial to overcome non-preventable nonadherence and then accept the results using the principle of intention-to-treat.

The Coronary Drug Project evaluated several lipid-lowering drugs in people several years after a myocardial infarction. In participants on one of the drugs, clofibrate, total 5-year mortality was 18.2%, as compared with 19.4% in control participants [31, 57]. Among the clofibrate participants, those who had at least 80% adherence to therapy had a mortality of 15%, whereas the poor adherers had a mortality of 24.6% (Table 18.2). This seeming benefit from taking clofibrate was, unfortunately, mirrored in the group taking placebo, 15.1% vs. 28.2%. A similar pattern (Table 18.3) was noted in the Aspirin Myocardial Infarction Study [58]. Overall, no difference in mortality was seen between the aspirin-treated group (10.9%) and the placebo-treated group (9.7%). Good adherers to aspirin had a mortality of 6.1%; poor adherers had a mortality of 21.9%. In the placebo group, the rates were 5.1% and 22%.

Table 18.2

Percent mortality by study group and level of adherence in the Coronary Drug Project

Drug adherence

Overall

≥80%

<80%

Clofibrate

18.2

15.0

24.6

Placebo

19.4

15.1

28.2

Table 18.3

Percent mortality by study group and degree of adherence in the Aspirin Myocardial Infarction Study

Overall

Good adherence

Poor adherence

Aspirin

10.9

6.1

21.9

Placebo

9.7

5.1

22.0

A trial of antibiotic prophylaxis in cancer patients also demonstrated a relationship between adherence and benefit in both the intervention and placebo groups [43]. Among the participants assigned to intervention, efficacy in reducing fever or infection was 82% in excellent adherers, 64% in good adherers, and 31% in poor adherers. Among the placebo participants, the corresponding figures were 68%, 56%, and 0%.

Another pattern is noted in a three-arm trial comparing two beta-blocking drugs, propranolol and atenolol, with placebo [59]. Approximately equal numbers of participants in each group stopped taking their medication. In the placebo group, adherers and nonadherers had similar mortality: 11.2% and 12.5%, respectively. Nonadherers to the interventions, however, had death rates several times greater than did the adherers: 15.9% to 3.4% in those on propranolol and 17.6% to 2.6% in those on atenolol. Thus, even though the numbers of nonadherers in each arm were equal, their risk characteristics as reflected by their morality rates were obviously different.

Pledger [51] provides an analogous example for a trial of schizophrenia. Participants were randomized to chlorpromazine or placebo and the 1-year relapse rates were measured. The overall comparison was a 27.8% relapse rate on active medication and 52.8% for those on placebo. The participants were categorized into low or high adherence subgroups. Among the active medication participants, the relapse rate was 61.2% for low adherence and 16.8% for high adherence. However, the relapse rate was 74.7% and 28.0% for the corresponding adherence groups on placebo.

Another example of placebo adherence versus nonadherence is reported by Oakes et al. [49]. A trial of 2,466 heart attack participants compared diltiazem with placebo over a period of 4 years with time to first cardiac event as the primary outcome. Cardiac death or all-cause mortality were additional outcome measures. The trial was initially analyzed according to intention-to-treat with no significant effect of treatment. Qualitative interaction effects were found with the presence or absence of pulmonary congestion which favored diltiazem for patients without pulmonary congestion and placebo in patients with pulmonary congestion. Interestingly, for participants without pulmonary congestion, the hazard ratio or relative risk for time to first cardiac event was 0.92 for those off placebo compared to those on placebo. For participants with pulmonary congestion, the hazard ratio was 2.86 for participants off placebo compared to those on placebo. For time to cardiac death and to all-cause mortality, hazard ratios exceeded 1.68 in both pulmonary congestion subgroups. This again suggests that placebo adherence is a powerful prognostic indicator and argues for the intention-to-treat analysis.

The definition of nonadherence can also have a major impact on the analysis. This is demonstrated by reanalysis of a trial in breast cancer patients by Redmond et al. [52]. This trial compared a complex chemotherapy regimen with placebo as adjuvant therapy following surgery with disease-free survival as the primary outcome. To illustrate the challenges of trying to adjust analyses for adherence, two measures of adherence were created. Adherence was defined as the fraction of chemotherapy taken while on the study to what was defined by the protocol as a full course. One analysis (Method I) divided participants into good adherers (≥85%), moderate adherers (65–84%) and poor adherers (<65%). Using this definition, placebo adherers had a superior disease-free survival than moderate adherers who did better than poor adherers (Fig. 18.1). This pattern of outcome in the placebo group is similar to the CDP clofibrate example. The authors performed a second analysis (Method II) changing the definition of adherence slightly. In this case, adherence was defined as the fraction of chemotherapy taken while on study to what should have been taken while still on study before being taken off treatment for some reason. Note that the previous definition (Method I) compared chemotherapy taken to what would have been taken had the participant survived to the end and adhered perfectly. This subtle difference in definition changed the order of outcome in the placebo group. Here, the poor placebo adherers had the best disease-free survival and the best adherers had a disease-free survival in-between the moderate and poor adherers. Of special importance is that the participants in this example were all on placebo. Thus, adherence is itself an outcome and trying to adjust one outcome (the primary response variable) for another outcome (adherence) can lead to misleading results.

Fig. 18.1

Percentage of disease free survival related to adherence levels of placebo; methods I and II definition of compliance in National Surgical Adjuvant Breast Program (NSABP). Three levels of compliance are: Good (>85%) Moderate (65–84%) Poor (<65%) [52]

Detre and Peduzzi have argued that, although as a general rule nonadherent participants should be analyzed according to the study group to which they were assigned, there can be exceptions. They presented an example from the VA coronary bypass surgery trial [40]. In that trial, a number of participants assigned to medical intervention crossed over to surgery. Contrary to expectation, these participants were at similar risk of having an event, after adjusting for a variety of baseline factors, as those who did not crossover. Therefore, the authors argued that the non-adherers should be kept in their original groups, but can be censored at the time of crossover. This may be true, but, as seen in the Coronary Drug Project [57], adjustment for known variables does not always account for the observed response. The differences in mortality between adherers and nonadherers remained even after adjustment. Thus, other unknown or unmeasured variables were of critical importance.

Some might think that if rules for withdrawing participants are specified in advance, withdrawals for nonadherence are legitimate. However, the potential for bias cannot be avoided simply because the investigator states, ahead of time, the intention to withdraw participants. This is true even if the investigator is blinded to the group assignment of a participant at the time of withdrawal. Participants were not withdrawn from the analyses in the above examples. However, had a rule allowing withdrawal of participants with poor adherence been specified in advance, the results described above would have been obtained. The type of participants withdrawn would have been different in the intervention and control groups and would have resulted in the analysis of non-comparable groups of adherers. Unfortunately, as noted, the patterns of possible bias can vary and depend on the precise definition of adherence. Neither the magnitude nor direction of that bias is easily assessed or compensated for in analysis.

Adherence is also a response to the intervention. If participant adherence to an intervention is poor compared to that of participants in the control group, widespread use of this therapy in clinical practice may not be reasonably expected. An intervention may be effective, but may be of little value if it cannot be tolerated by a large portion of the participants. For example, in the Coronary Drug Project, the niacin arm showed a favorable trend for mortality over 7 years, compared with placebo, but niacin caused “hot flashes” and was not easily tolerated [31]. The development of slow release formulations that reduce pharmacologic peaks has lessened the occurrence of side effects.

It is therefore recommended that no participants be withdrawn from analysis in superiority trials for lack of adherence. The price an investigator must pay for this policy is possibly reduced power because some participants who are included in the analysis may not be on optimal intervention. For limited or moderate nonadherence, one can compensate by increasing the sample size, as discussed in Chap. 8, although doing so is costly.

Missing or Poor Quality Data

In most trials, participants have data missing for a variety of reasons. Perhaps they were not able to keep their scheduled clinic visits or were unable to perform or undergo the particular procedures or assessments. In some cases, follow-up of the participant was not completed as outlined in the protocol. The challenge is how to deal with missing data or data of such poor quality that they are in essence missing. One approach is to withdraw participants who have poor data completely from the analysis [26, 61, 62]. However, the remaining subset may no longer be representative of the population randomized and there is no guarantee that the validity of the randomization has been maintained in this process.

There is a vast literature on approaches to dealing with missing data [63–73]. Many of these methods assume that the data are missing at random; that is, the probability of a measurement not being observed does not depend on what its value would have been. In some contexts, this may be a reasonable assumption, but for clinical trials, and clinical research in general, it would be difficult to confirm. It is, in fact, probably not a valid assumption, as the reason the data are missing is often associated with the health status of the participant. Thus, during trial design and conduct, every effort must be made to minimize missing data. If the amount of missing data is relatively small, then the available analytic methods will probably be helpful. If the amount of missing data is substantial, there may be no method capable of rescuing the trial. In this section, we discuss some of the issues that must be kept in mind when analyzing a trial with missing data.

Rubin [72] provided a definition of missing data mechanisms. If data are missing for reasons unrelated to the measurement that would have been observed and unrelated to covariates, then the data are “missing completely at random.” Statistical analyses based on likelihood inference are valid when the data are missing at random or missing completely at random. If a measure or index allows a researcher to estimate the probability of having missing data, say in a participant with poor adherence to the protocol, then using methods proposed by Rubin and others might allow some adjustment to reduce bias [66, 71, 72, 74]. However, adherence, as indicated earlier, is often associated with a participant’s outcome and attempts to adjust for adherence can lead to misleading results.

If participants do not adhere to the intervention and also do not return for follow-up visits, the primary outcome measured may not be obtained unless it is survival or some easily ascertained event. In this situation, an intention-to-treat analysis is not feasible and no analysis is fully satisfactory. Because withdrawal of participants from the analysis is known to be problematic, one approach is to “impute” or fill in the missing data such that standard analyses can be conducted. This is appealing if the imputation process can be done without introducing bias. There are many procedures for imputation. Those based on multiple imputations are more robust than single imputation [75].

A commonly used single imputation method is to carry the last observed value forward. This method, also known as an endpoint analysis, requires the very strong and unverifiable assumption that all future observations, if they were available, would remain constant [51]. Although commonly used, the last observation carried forward method is not generally recommended [71, 73]. Using the average value for all participants with available data, or using a regression model to predict the missing value are alternatives, but in either case, the requirement that the data be missing at random is necessary for proper inference.

A more complex approach is to conduct multiple imputations, typically using regression methods, and then perform a standard analysis for each imputation. The final analysis should take into consideration the variability across the imputations. As with single imputation, the inference based on multiple imputation depends on the assumption that the data are missing at random. Other technical approaches are not described here, but in the context of a clinical trial, none is likely to be satisfactory.

Various other methods for imputing missing values have been described [63–73, 75]. Examples of some of these methods are given by Espeland et al. for a trial measuring carotid artery thickness at multiple anatomical sites using ultrasound [61]. In diagnostic procedures of this type, typically not all measurements can be made. Several imputation methods, based on a mixed effects linear model where regression coefficient and a covariance structure (i.e., variances and correlations), were estimated. Once these were known, this regression equation was the basis for the imputation. Several imputation strategies were used based on different methods of estimating the parameters and whether treatment differences were assumed or not. Most of the imputation strategies gave similar results when the trial data were analyzed. The results indicated up to a 20% increase in efficiency compared to using available data in cross sectional averages.

For repeated measures, imputation techniques such as these are useful if the data are missing at random; that is, the probability of missing data is not dependent on the measurement that would have been observed or on the preceding measurements. Unfortunately, it is unlikely that data are missing at random. The best that can be offered, therefore, is a series of analyses, each exploring different approaches to the imputation problem. If all, or most, are in general agreement qualitatively, then the results are more persuasive. All analyses should be presented, not just the one with the preferred results.

In long-term trials participants may be lost to follow-up or refuse to continue their participation. In this situation, the status of the participant with regard to any response variable cannot be determined. If mortality is the primary response variable and if the participant fails to return to the clinic, his survival status may still be obtained. If a death has occurred, the date of death can be ascertained. In the Coronary Drug Project [31] where survival experience over 60 months was the primary response variable, four of 5,011 participants were lost to follow-up (one in a placebo group, three in one treatment group, and none in another treatment group). The Lipid Research Clinics Coronary Primary Prevention Trial [47] followed over 3,800 participants for an average of 7.4 years, and was able to assess vital status on all. The Physicians’ Health study of over 20,000 US male physicians had complete follow-up for survival status [76]. Many other large simple trials, such as GUSTO [39], have similar nearly complete follow-up experience. Obtaining such low loss to follow-up rates, however, required special effort. In the Women’s Health Initiative (WHI), one portion evaluated the possible benefits of hormone replacement therapy (estrogen plus progestin) compared with placebo in post-menopausal women. Of the 16,025 participants, 3.5% were lost to follow-up and did not provide 18 month data [77].

For some conditions, e.g., trials of treatment for substance abuse, many participants fail to return for follow-up visits, and missing data can be 25–30% or even more. Efforts to account for missing data must be made, recognizing that biases may very well exist.

An investigator may not be able to obtain any information on some kinds of response variables. For example, if a participant is to have blood pressure measured at the last follow-up visit 12 months after randomization and the participant does not show up for that visit, this blood pressure can never be retrieved. Even if the participant is contacted later, the later measurement does not truly represent the 12-month blood pressure. In some situations, substitutions may be permitted, but, in general, this will not be a satisfactory solution. An investigator needs to make every effort to have participants come in for their scheduled visits in order to keep losses to follow-up at a minimum. In the Intermittent Positive Pressure Breathing (IPPB) trial, repeated pulmonary function measurements were required for participants with chronic obstructive pulmonary disease [62]. However, some participants who had deteriorated could not perform the required test. A similar problem existed for the Multicenter Investigation of the Limitation of Infarct Size (MILIS) where infarct size could not be obtained in many of the sickest participants [26].

Individuals with chronic obstructive pulmonary disease typically decline in their pulmonary function and this decline may lead to death, as happened to some participants in the IPPB trial. In this case, the missing data were not missing at random and censoring was said to be informative. One simple method for cases such as the IPPB study is to define a decreased performance level considered to be a clinical event. Then the analysis can be based on time to the clinical event of deterioration or death, incorporating both pieces of information. Survival analysis, though, assumes that loss of follow-up is random and independent of risk of the event. Methods relaxing the missing at random assumption have been proposed [78, 79], but require other strong assumptions, the details of which are beyond the scope of this text.

If the number of participants lost to follow-up differs in the study groups, the analysis of the data could be biased. For example, participants who are taking a new drug that has adverse effects may, as a consequence, miss scheduled clinic visits. Events may occur but be unobserved. These losses to follow-up would probably not be the same in the control group. In this situation, there may be a bias favoring the new drug. Even if the number lost to follow-up is the same in each study group, the possibility of bias still exists because the participants who are lost in one group may have quite different prognoses and outcomes than those in the other group.

An example of differential follow-up was reported by the Comparison of Medical Therapy, Pacing, and Defibrillation in Chronic Heart Failure (COMPANION) trial [80]. COMPANION compared a cardiac pacemaker or a pacemaker plus defibrillator with best pharmacologic treatment in people with chronic heart failure. Over 1,500 participants were randomized. Two primary outcomes were assessed; death and death plus hospitalization. Individuals randomized to one of the device arms did not know to which device they had been assigned, but those on the pharmacologic treatment arm were aware that no device had been installed. During the course of the trial, the pacemaker plus defibrillator devices, made by two different manufacturers, were approved by a regulatory agency. As a result, participants in the pharmacologic treatment arm began to drop-out from the trial and some also withdrew their consent. Many requested one of the newly approved devices. Thus, when the trial was nearing completion, the withdrawal rate was 26% in the pharmacologic treatment arm and 6–7% in the device arms. Additionally, no further follow-up information could be collected on those who withdrew consent. Clearly, censoring at the time of withdrawal was not random and the possibility that it was related to disease status could not be ruled out, thus creating the possibility of serious bias. This situation could have jeopardized an otherwise well designed and conducted trial in people with a serious medical condition. However, the investigators initiated a complicated process of reconsenting the participants to allow for collection of the primary outcomes. After completing this process, assessment of the status for death plus hospitalization and vital status were 91% and 96%, respectively, in the pharmacologic treatment group. Outcome ascertainment for the two device arms was 99% or better. The final results demonstrated that both the pacemaker and the defibrillator plus pacemaker reduced death plus hospitalization and further that the defibrillator plus pacemaker reduced mortality. These results were important in the treatment of chronic heart failure. However, not correcting for the initial differential loss to follow-up would have rendered the COMPANION trial data perhaps uninterpretable. In Fig. 18.2, the Kaplan Meier curves for mortality for the two intervention arms are provided with the most complete data available.

Fig. 18.2

Kaplan-Meier estimates in COMPANION trial. (a) The time to primary end point of death from hospitalization for any cause. (b) The time to the secondary end point of death from any cause [80]

Often, protocol designs call for follow-up to terminate at some period, for example 7, 14, or 30 days, after a participant has stopped adhering to his or her intervention, even though the intended duration of intervention would not have ended. The concept is that “off intervention” means “off study”; i.e., assessment for nonadherent participants ends when intervention ends. We do not endorse this concept. Although time to event analysis may be censored at the time of last follow-up, going off intervention or control is not likely random and may be related to participant health status. Important events, including serious adverse events, may occur beyond the follow-up period and might be related to the intervention. As noted above, though, survival analysis assumes that censoring is independent of the primary event. The practice of not counting events at the time of, or shortly after, intervention discontinuation is all too common, and can lead to problems in the interpretation of the final results. An instructive example is the Adenomatous Polyp Prevention on Vioxx (APPROVe) trial [81]. This randomized double blind trial compared a cyclo-oxygenase (COX)-II inhibitor with placebo in people with a history of colorectal adenomas. Previous trials of COX-II inhibitors had raised concern regarding long term cardiovascular risk. Thus, while the APPROVe trial was a cancer prevention trial, attention also focused on cardiovascular events, in particular thrombotic events and cardiovascular death, nonfatal myocardial infarction, and nonfatal stroke. However, participants who stopped taking their medication during the trial were not followed beyond 14 days after the time of discontinuation. The Kaplan-Meier cardiovascular risk curve is shown in Fig. 18.3. Note that for the first 18 months the two curves are similar and then begin to diverge. Controversy arose as to whether there was an 18-month lag time in the occurrence of cardiovascular events for this particular COX-II inhibitor [82, 83].

Fig. 18.3

APPROVe—Kaplan-Meier estimates of time to death from the AntiPlatelet Trialists’ Collaborative (APTC) outcomes (cardiovascular causes, nonfatal myocardial infarction of nonfatal stroke) with censoring 14 days after participants stopped therapy [84]. Reproduced with the permission of Elsevier Ltd. for Lancet

Due to the controversy, the investigators and sponsor launched an effort to collect information on all trial participants for at least a year after stopping study treatment. This extended follow-up, referred to here as APPROVe + 1, was able to collect selected cardiovascular events of nonfatal myocardial infarction, nonfatal stroke and cardiovascular death [84], as shown in Fig. 18.4. The time to event curves begin to separate from the beginning and continue throughout the extended follow-up, with a hazard ratio of 1.8 (p = 0.006). There was a corresponding statistically nonsignificant increase in mortality.

Fig. 18.4

APPROVe Kaplan-Meier estimates of time to death for the AntiPlatelet Trialists’ Collaborative (APTC) outcome (cardiovascular causes, nonfatal myocardial infarction or nonfatal stroke) counting all events observed for an additional year of follow-up after trial was initially terminated [84]. Reproduced with the permission

Censoring follow-up when participants go off their intervention is a common error that leads to problems like those encountered by the APPROVe trial. Going off intervention, and thus censoring follow-up at some number of days afterwards, is not likely to be independent of the disease process or how a participant is doing. At least, it would be difficult to demonstrate such independence. Yet, survival analysis and most other analyses assume that the censoring is independent. The principle lesson here is that “off intervention should not mean off study.”

An outlier is an extreme value significantly different from the remaining values. The concern is whether extreme values in the sample should be included in the analysis. This question may apply to a laboratory result, to the data from one of several areas in a hospital or from a clinic in a multicenter trial. Removing outliers is not recommended unless the data can be clearly shown to be erroneous. Even though a value may be an outlier, it could be correct, indicating that on occasions an extreme result is possible. This fact could be very important and should not be ignored. Long ago Kruskall [85] suggested carrying out an analysis with and without the “wild observation.” If the conclusions vary depending on whether the outlier values are included or excluded, one should view any conclusions cautiously. Procedures for detecting extreme observations have been discussed [86–89], and the publications cited can be consulted for further detail.

An interesting example given by Canner et al. [86] concerns the Coronary Drug Project. The authors plotted the distributions of four response variables for each of the 53 clinics in that multicenter trial. Using total mortality as the response variable, no clinics were outlying. When nonfatal myocardial infarction was the outcome, only one clinic was an outlier. With congestive heart failure and angina pectoris, response variables which are probably less well defined, there were nine and eight outlying clinics, respectively.

In conclusion, missing data can create problems. Though methods which allow for missing data exist, they require certain assumptions which are not likely to be true. Every attempt should made to minimize missing data, and investigators should be aware of the potential for bias

Competing Events

Competing events are those that preclude the assessment of the primary response variable. They can reduce the power of the trial by decreasing the number of participants available for follow-up. If the intervention can affect the competing event, there is also the risk of bias. In some clinical trials, the primary response variable may be cause-specific mortality, such as death due to myocardial infarction or sudden death, rather than total mortality [90–93]. The reason for using cause-specific death as a response variable is that a therapy often has specific mechanisms of action that may be effective against a disease or condition. In this situation, measuring death from all causes, at least some of which are not likely to be affected by the intervention, can “dilute” the results. For example, a study drug may be anti-arrhythmic and thus sudden cardiac death might be the selected response variable. Other causes of death such as cancer and accidents would be competing events.

Even if the response variable is not cause-specific mortality, death may be a factor in the analysis. This is particularly an issue in long term trials in the elderly or high risk populations. If a participant dies, future measurements will be missing. Analysis of nonfatal events in surviving participants has the potential for bias, especially if the mortality rates are different in the two groups.

In a study in which cause-specific mortality is the primary response variable, deaths from other causes are treated statistically as though the participants were lost to follow-up from the time of death (Chap. 15) and these deaths are not counted in the analysis. In this situation, the analysis, however, must go beyond merely examining the primary response variable. An intervention may or may not be effective in treating the condition of interest but could be harmful in other respects. Therefore, total mortality should be considered as well as cause-specific fatal events. Similar considerations need to be made when death occurs in studies using nonfatal primary response variables. This can be done by considering tables that show the number of times the individual events occur, one such event per person. No completely satisfactory solution exists for handling competing events. At the very least, the investigator should report all major outcome categories; for example, total mortality, as well as cause-specific mortality and morbid events.

In many cases, there may be recurring events. Many trials simply evaluate the time to the first event and do not count the additional events in the time to event analysis. Tables may show the total number of events in each intervention arm. Some attempts to further analyze recurrent events have been made, using for example the data from the COMPANION trial [80, 90]. Software exists for these methods [94, 95]; however, the technical details of these methods are complicated and will not be covered in this text.

Composite Outcomes

In recent years, many trials have used combinations of clinical and other outcomes as a composite response variable [90–93]. One major motivation is to increase the event rate and thus reduce the sample size that might otherwise have been required had just one of the components been selected as the primary outcome. Another motivation is to combine events that have a presumed common etiology and thus get an overall estimate of effect. The sample size is usually not based on any single component.

There are challenges in using a composite outcome [96, 97]. The components may not have equal weight or clinical importance, especially as softer outcomes are added. The components may go in opposite directions or at least not be consistent in indicating intervention effect. One component may dominate the composite. Results with any single component are based on a smaller number of events and thus the power for that component is greatly reduced. Rarely do we find significance for a component, nor should we expect it in general. Regardless of the composition of the composite, analyses should be conducted for each component, or in some cascading sequence. For example, if the composite were death, myocardial infarction, stroke or heart failure hospitalization, the analysis sequence might be death, death plus myocardial infarction or stroke, and death plus heart failure hospitalization. The reason for including death is to take into account competing risk of death for the other components, in addition to its obvious clinical importance.

As pointed out in Chap. 3, it is essential that follow-up continue after the first event in the composite outcome occurs. Analysis will include looking at the contribution of each component to the overall but should also include time to event for each component separately. As indicated, if follow-up does not continue, only partial results are available for each component and analysis of those events separately could be misleading.

There are several examples where the use of a composite such as death, myocardial infarction and stroke has been used as a primary or leading secondary outcome [34, 36–38]. These outcomes are all clinically relevant. In these trials, the outcomes all trended in the same direction. However, that is not always the case.

In the Pravastatin or Atorvastatin Evaluation and Infection Therapy (PROVE IT) trial, the 80 mg atorvastatin arm was more effective than the 40 mg pravastatin arm in reducing the incidence of death, myocardial infarction, stroke, required hospitalization due to unstable angina and revascularization [91]. Stroke results, one of the key components, went in the opposite direction. These results complicate the interpretation. Should investigators think that the atorvastatin improves the composite or just those components that are in the same direction as the composite? As would be expected, the differences for the components were not, in themselves, statistically significant.

Another interesting example is provided by the Women’s Health Initiative (WHI) which was a large factorial design trial post-menopausal women [77]. As discussed earlier and in Chap. 16, one part involved hormone replacement therapy which contained two strata, women with a uterus and those without. Women with a uterus received estrogen plus progestin or matching placebo; those without a uterus received estrogen alone or a matching placebo. Due to the multiple actions of hormone replacement therapy, one response variable was a global outcome mortality, coronary heart disease, bone loss reflected by hip fracture rates, breast cancer, colorectal cancer, pulmonary embolism, and stroke. As seen in Fig. 16.​7, for the estrogen plus progestin stratum, there was essentially no effect on mortality and a small but nonsignificant effect in the global index, when compared to placebo. However, as shown in Fig. 16.​6, the various components went in different directions. Hip fracture and colorectal cancer had a favorable response to hormone replacement therapy. Pulmonary embolism, stroke and coronary heart disease went in an unfavorable direction. Thus, any interpretation of the global index, which is a composite, requires careful examination of the components. Of course, few trials would have been designed with adequate power for the individual components so the interpretation must be qualitative, looking for consistency and biological plausibility.

The Look AHEAD trial examined whether a long-term lifestyle intervention for weight loss would decrease cardiovascular morbidity and mortality in overweight or obese patients with type 2 diabetes [98]. The primary outcome was a composite of death from cardiovascular causes, nonfatal myocardial infarction, or nonfatal stroke. During follow-up, the Data and Safety Monitoring Board (DSMB) alerted the investigators that the event rate for the primary outcome was dramatically lower than expected, less than a third [99]. The protocol was changed to include hospitalization for angina as a way of increasing the event rate, and this turned out to be the most frequent component in the revised composite, which had an incidence about 50% higher than the original composite. Unfortunately, hospitalization for angina showed markedly less effect of the intervention [100]. Using the original composite would not have changed the trial’s outcome, which was negative, but this experience underscores the importance of giving full consideration of a candidate component’s likely response to the intervention, as well as to its incidence rate.

Experience suggests that composite outcome variables should be used cautiously and only include those components that have relatively equal clinical importance, frequency, and anticipated response to the presumed mechanism of action of the intervention [96]. As softer and less relevant outcomes are added, the interpretation becomes less clear, particularly if the less important component occurs more frequently than others, driving the overall result. Significance by any individual component cannot be expected but there should be a plausible consistency across the components.

Covariate Adjustment

The goal in a clinical trial is to have study groups that are comparable except for the intervention being studied. Even if randomization is used, all of the prognostic factors may not be perfectly balanced, especially in smaller studies. Even if no prognostic factors are significantly imbalanced in the statistical sense, an investigator may, nevertheless, observe that one or more factors favor one of the groups. In either case, covariate adjustment can be used in the analysis to minimize the effect of the differences. However, covariate adjustment is not likely to eliminate the effect of these differences. Covariance analysis for clinical trials has been reviewed in numerous articles [101–122].

Adjustment also reduces the variance in the test statistic. If the covariates are highly correlated with outcome, this can produce more sensitive analyses. The specific adjustment procedure depends on the type of covariate being adjusted for and the type of response variable being analyzed. If a covariate is discrete, or if a continuous variable is converted into intervals and made discrete, the analysis is sometimes referred to as “stratified.” A stratified analysis, in general terms, means that the study participants are subdivided into smaller, more homogeneous groups, or strata. A comparison of study groups is made within each stratum and then averaged over all strata to achieve a summary result for the response variable. This result is adjusted for group imbalances in the discrete covariates. If a response variable is discrete, such as the occurrence of an event, the stratified analysis might take the form of a Mantel-Haenszel statistic.

If the response variable is continuous, the stratified analysis is referred to as analysis of covariance. This uses a model which, typically, is linear in the covariates. A simple example for a response Y and covariate X would be Y = α j  + β(X − μ) + error where β is a coefficient representing the importance of the covariate X and is assumed to be the same in each group, μ is the mean value of X, and α j is a parameter for the contribution of the overall response variable jth group (e.g., j = 1 or 2). The basic idea is to adjust the response variable Y for any differences in the covariate X between the two groups. Under appropriate assumptions, the advantage of this method is that the continuous covariate X does not have to be divided into categories. Further details can be found in statistics textbooks [1–6, 8, 123]. If time to an event is the primary response variable, then survival analysis methods that allow for adjustments of discrete or continuous covariates may be used [106]. However, whenever models are employed, the investigator must be careful to evaluate the assumptions required and how closely they are met. Analysis of covariance can be attractive, but may be abused if linearity is assumed when the data are nonlinear, if the response curve is not parallel in each group, or if assumptions of normality are not met [122]. If measurement errors in covariates are substantial, the lack of precision can be increased [112]. For all of these reasons, covariate adjustment models may be useful in the interpretation of data, but should not be viewed as absolutely correct.

Regardless of the adjustment procedure, covariates should be measured at baseline. Except for certain factors such as age, sex, or race, any variables that are evaluated after initiation of intervention should be considered as response variables. Group comparisons of the primary response variable, adjusted for other response variables, are discouraged. Interpretation of such analyses is difficult because group comparability may be lost.

Surrogates as a Covariate

Adjustment for various surrogate outcomes may be proposed. In a trial of clofibrate [101], the authors reported that those participants who had the largest reduction in serum cholesterol had the greatest clinical improvement. However, reduction in cholesterol is probably highly correlated with adherence to the intervention regimen. Since, as discussed earlier, adherers in one group may be different from adherers in another group, analyses that adjust for a surrogate for adherence can be biased. This issue was addressed in the Coronary Drug Project [56]. Adjusted for baseline factors, the 5-year mortality was 18.8% in the clofibrate group (N = 997) and 20.2% in the placebo group (N = 2,535), an insignificant difference. For participants with baseline serum cholesterol greater than or equal to 250 mg/dl, the mortality was 17.5% and 20.6% in the clofibrate and placebo groups, respectively. No difference in mortality between the groups was noted for participants with baseline cholesterol of less than 250 mg/dl (20.0% vs. 19.9%). Those participants with lower baseline cholesterol in the clofibrate group who had a reduction in cholesterol during the trial had 16.0% mortality, as opposed to 25.5% mortality for those with a rise in cholesterol (Table 18.4). This would fit the hypothesis that lowering cholesterol is beneficial. However, in those participants with high baseline cholesterol, the situation was reversed. An 18.1% mortality was seen in those who had a fall in cholesterol, and a 15.5% mortality was noted in those who had a rise in cholesterol. The best outcome, therefore, appeared to be in participants on clofibrate whose low baseline cholesterol dropped or whose high baseline cholesterol increased. As seen earlier, adherence can affect outcomes in unexpected ways, and the same is true of surrogates for adherence.

Table 18.4

Percent 5-year mortality in the clofibrate group, by baseline cholesterol and change in cholesterol in the Coronary Drug Project’s

Baseline cholesterol

<250 mg/dl

≥250 mg/dl

Total

20.0

17.5

Fall in cholesterol

16.0

18.1

Rise in cholesterol

25.5

15.5

Modeling the impact of adherence on a risk factor and thus on the response has also received attention [109, 115]. Regression models have been proposed that attempt to adjust outcome for the amount of risk factor change that could have been attained with optimum adherence. One example of this was suggested by Efron and Feldman [109] for a lipid research study. However, Albert and DeMets [115] showed that these models are very sensitive to assumptions about the independence of adherence and health status or response. If these assumptions using these regression models are violated, misleading results emerge, such as that for the clofibrate and serum cholesterol example described above.

Clinical trials of cancer treatment commonly analyze results by comparing responders to nonresponders [104, 108]. That is, those who go into remission or have a reduction in tumor size are compared to those who do not. One early survey indicated that such analyses were done in at least 20% of published reports [122]. The authors of that survey argued that statistical problems, due to lack of random assignment, and methodological problems, due both to classification of response and inherent differences between responders and nonresponders, can occur. These will often yield misleading results, as shown by Anderson et al. [104]. They pointed out that participants “who eventually become responders must survive long enough to be evaluated as responders.” This factor can invalidate some statistical tests comparing responders to nonresponders. Those authors present two statistical tests that avoid bias. They note, though, that even if the tests indicate a significant difference in survival between responders and nonresponders, it cannot be concluded that increased survival is due to tumor response. Thus, aggressive intervention, which may be associated with better response, cannot be assumed to be better than less intensive intervention, which may be associated with poorer response. Anderson and colleagues state that only a truly randomized comparison can say which intervention method is preferable. What is unsaid, and illustrated by the Coronary Drug Project examples, is that even comparison of good responders in the intervention group with good responders in the control can be misleading, because the reasons for good response may be different.

Morgan [48] provided a related example of comparing duration of response in cancer patients, where duration of response is the time from a favorable response such as tumor regression (partial or total) to remission. This is another form of defining a subgroup of post-treatment outcome, that is, tumor response. In a trial comparing two complex chemotherapy regiments (A vs. B) in small cell lung cancer, the tumor response rates were 64% and 85%, with median duration of 245 days and 262 days respectively. When only responders were analyzed, the slight imbalance in prognostic factors was substantially increased. Extensive disease was evident at baseline in 48% of one and 21% of the other treatment responder groups. Thus, while it may be theoretically possible to adjust for prognostic factors, in practice, such adjustment may decrease bias, but will not eliminate it. Because not all prognostic factors are known, any model is only an approximation to the true relationship.

The Cox proportional hazards regression model for the analysis of survival data (Chap. 15) allows for covariates in the regression to vary with time [116]. This has been suggested as a way to adjust for factors such as adherence and level of response. It should be pointed out that, like simple regression models, this approach is vulnerable to the same biases described earlier in this chapter. For example, if cholesterol level and cholesterol reduction in the CDP example were used as time dependent covariates in the Cox model, the estimator of treatment effect would be biased due to the effects shown in Table 18.4.

Rosenbaum [121] provides a nice overview of adjustment for concomitant variables that have been affected by treatment in both observational and randomized studies. He states that “adjustments for post-treatment concomitant variables should be avoided when estimating treatment effects except in rather special circumstances, since adjustments themselves can introduce a bias where none existed previously.”

A number of additional methodologic attempts to adjust for adherence have also been conducted. Newcombe [11], for example, suggested adjusting estimates of intervention effect on the extent of nonadherence. Robins and Tsiatis [110] proposed a causal inference model. Lagakos et al. [46] evaluated censoring survival time, or time to an event, at the point when treatment is terminated. The rationale is that participants are no longer able to completely benefit from the therapy. However, the hazard ratio estimated by this approach is not the hazard that would have been estimated if participants had not terminated treatment. The authors stated that it is not appropriate to evaluate treatment benefit by comparing the hazard rates estimated by censoring for treatment termination [46]. Models for causal interference have also been used to explore the effects of adherence in clinical trials [124–127]. Though promising, these approaches require strong assumptions usually either known to be untrue or difficult to validate and so are not recommended as part of a primary analysis.

Baseline Variables as Covariates

The issue of stratification was first raised in the discussion of randomization (Chap. 6). For large studies, the recommendation was that stratified randomization is usually unnecessary because overall balance would nearly always be achieved and that stratification would be possible in the analysis. For smaller studies, baseline adaptive methods could be considered but the analysis should include the covariates used in the randomization. In a strict sense, analysis should always be stratified if stratification was used in the randomization. In such cases, the adjusted analysis should include not only those covariates found to be different between the groups, but also those stratified during randomization. Of course, if no stratification is done at randomization, the final analysis is less complicated since it would involve only those covariates that are imbalanced at baseline or to be of special interest associated with the outcomes.

As stated in Chap. 6, randomization tends to produce comparable groups for both measured and unmeasured baseline covariates. However, not all baseline covariates will be closely matched. Adjusting treatment effect for these baseline disparities continues to be debated. Canner [111] describes two points of view, one which argues that “if done at all, analyses should probably be limited to covariates for which there is a disparity between the treatment groups and that the unadjusted measure is to be preferred.” The other view is “to adjust on only a few factors that were known from previous experience to be predictive.” Canner [111], as well as Beach and Meier [107], indicate that even for moderate disparity in baseline comparability, or even if the covariates are moderately predictive, it is possible for covariate adjustment to have a nontrivial impact on the measure of treatment effect. However, Canner [111] also points out that it is “often possible to select specific covariates out of a large set in order to achieve a desired result.” In addition, he shows that this issue is true for both small and large studies. For this reason, it is critical that the process for selecting covariates be specified in the protocol and adhered to in the primary analyses. Other adjustments may be used in exploratory analyses.

Another issue is testing for covariate interaction in a clinical trial [105, 113, 114, 118, 119]. Treatment-covariate interaction is defined when the response to treatment varies according to the value of the covariate [105]. Peto [118] defines treatment covariate interactions as quantitative or qualitative. Quantitative interactions indicate that the magnitude of treatment effect varies with the covariate but still favors the same intervention (Fig. 18.5a). Qualitative interaction involves a favorable intervention effects for some values of the covariate and unfavorable effects for others (Fig. 18.5b). A quantitative interaction, for example, would be if the benefit of treatment for blood pressure on mortality varied in degree by the level of baseline blood pressure but still favoring the same intervention (See Fig. 18.5a). A qualitative interaction would exist if lowering blood pressure was beneficial for severe hypertension, but less beneficial or even harmful for mild hypertension. Intervention effects can vary by chance across levels of the covariate, even changing direction, so a great deal of caution must be taken in the interpretation. One can test formally for interaction, but requiring a significant interaction test is much more cautious than reviewing the magnitude of intervention effect within each subgroup. Byar [105] presents a nice illustration example shown in Table 18.5. Two treatments, A and B, are being compared by the difference in mean response, $$ Y={\overline{X}}_A-{\overline{X}}_B $$, and S is the standard error of Y. In the upper panel, the interaction test is not significant, but examination of the subgroups is highly suggestive of interaction. The lower panel is more convincing for interaction, but we still need to examine each subgroup to understand what is going on.

Fig. 18.5

Two types of intervention–covariate interactions [118]

Table 18.5

Examples of apparent treatment-covariate interactions [105]

Let $$ Y={\overline{X}}_A-{\overline{X}}_B $$

Statistic

SE of Y

P value (2 tail)

Unconvincing

Overall test

Y = 2S

S

0.045

Subsets

Y ₁ = 3S

0.034

Y ₂ = 1S

0.480

Interaction

Y ₁ − Y ₂ = 2S

2S

0.317

More convincing

Overall test

Y = 2S

S

0.045

Subsets

Y ₁ = 4S

0.005

Y ₂ = 0

1.000

Interaction

Y ₁ − Y ₂ = 4S

2S

0.045

Methods have been proposed for testing for overall interactions [114, 119]. However, Byar’s concluding remarks [105] are noteworthy when he says,

one should look for treatment-covariate interactions, but, because of the play of chance in multiple comparisons, one should look very cautiously in the spirit of exploratory data analysis rather than that of formal hypothesis testing. Although the newer statistical methods may help decide whether the data suffice to support a claim of qualitative interactions and permit a more precise determination of reasonable p values, it seems to me unlikely that these methods will ever be as reliable a guide to sensible interpretation of data as will medical plausibility and replication of the findings in other studies. We are often warned to specify the interactions we want to test in advance in order to minimize the multiple comparisons problem, but this is often impossible in practice and in any case would be of no help in evaluating unexpected new findings. The best advice remains to look for treatment-covariate interactions but to report them skeptically as hypotheses to be investigated in other studies.

As indicated in Chap. 6, the randomization in multicenter trials should be stratified by clinic. The analysis of such a study should, strictly speaking, incorporate the clinic as a stratification variable. Furthermore, the randomization should be blocked in order to achieve balance over time in the number of participants randomized to each group. These “blocks” are also strata and, ideally, should be included in the analysis as a covariate. However, there could be a large number of strata, since there may be many clinics and the blocking factor within any clinic is usually anywhere from four to eight participants. Use of these blocking covariates is probably not necessary in the analysis. Some efficiency will be lost for the sake of simplicity, but the sacrifice should be small.

As Fleiss [10] describes, clinics differ in their demography of participants and medical practice as well as adherence to all aspects of the protocol. These factors are likely to lead to variation in treatment response from clinic to clinic. In the Beta-blocker Heart Attack Trial (BHAT) [23], most, but not all, of the 30 clinics showed a trend for mortality benefit from propranolol. A few indicated a negative trend. In the Aspirin Myocardial Infarction Study (AMIS) [102], data from a few clinics suggested a mortality benefit from aspirin, although most clinics indicated little or no benefit. Most reported analyses probably do not stratify by clinic, but simply combine the results of all clinics. However, at least one of the primary analyses should average within-clinic differences, an analysis that is always valid, even in the presence of clinic-treatment interaction [114].

Subgroup Analyses

While covariance or stratified analysis adjusts the overall comparison of main outcomes for baseline variables, another common analytic technique is to subdivide or subgroup the enrolled participants defined by baseline characteristics [128–156]. Here the investigator looks specifically at particular subgroups rather than the overall comparison to assess whether different groups of patients respond differently to the intervention. One of the most frequently asked questions during the design of a trial and when the results are analyzed is, “Are the intervention effects the same across levels of important baseline factors?” It is important that subgroups be examined. Clinical trials require considerable time and effort to conduct and the resulting data deserve maximum evaluation. Subgroup analyses can support or elaborate a trial’s overall primary result, or provide exploratory results for the primary outcome that may have special interest for a particular subgroup. For example, analysis of data from the V-HeFT I trial suggested that the combination of isosorbide dinitrate and hydralazine might reduce mortality in blacks but not whites [157, 158]. This lead to the initiation of a follow-up trial of the combination which enrolled only blacks with advanced heart failure [159]. The A-HeFT trial concluded that this therapy increased survival [160]. However, such success stories are not common, and care must be exercised in the interpretation of subgroup findings.

How to perform subgroup analyses when reporting clinical trial data has long been a controversial topic [140, 156]. Manuscripts reporting the results of clinical trials commonly include statements about and estimates of effects in subgroups, but the results of subgroup analyses are often misleading, having been over-interpreted or presented in a way that makes their interpretation ambiguous [129, 149]. Most published advice since the early 1980’s has included a common set of specific recommendations for subgroup analyses: they should be adjusted for multiple comparisons, they should be prespecified, and they should be assessed using interaction tests (rather than by within group estimates of the treatment effect) [142, 143, 153, 155, 156]. Making public a well-written protocol which specifies the proposed subgroups together with biologically plausible hypotheses for each and including plans for performing and presenting the subgroup analyses is often recommended as well.

As the number of subgroups increases, the potential for chance findings increases due to multiple comparisons [132, 143, 144, 155]. Therefore, if one were to perform tests of significance on a large number of subgroup analyses, there will be an increased probability of false positive results unless adjustments are made. Adjustment for multiple interaction tests on a set of variables defining subgroups is necessary to control the number of false positive results. This can be done by such familiar methods as the Bonferroni correction or variants of it. An alternative suggested by guidelines for the New England Journal of Medicine is to report the expected number of false positives associated with a set of tests reported with nominal p-values [143, 153]; for example, this approach was taken for the ACCORD BP results [152]. Even with adjustments for multiplicity, however, over-interpretation of the results of treatment effects within subgroups can lead to irreproducible conclusions.

Ideally, the subgroups to be analyzed should be pre-specified. Since it is almost always possible to find at least one suggestive subgroup effect by persistent exploration of the data after a trial is over, even when the intervention is completely inert, defining the groups to be analyzed in advance, preferably with argument for their biological plausibility, confers the greatest credibility. There is likely to be, however, low power for detecting differences in subgroups [132, 155], and they are more likely to be affected by imbalances in baseline characteristics [161, 162]. Therefore, investigators should not pay as much attention to statistical significance for subgroup questions as they do for the primary question. Recognizing the low chance of seeing significant differences, descriptions of subgroup effects are often qualitative. On the other hand, as mentioned previously, testing multiple questions can increase the chance of a Type I error. Even when prespecified, there are reasons to be cautious.

The Clopidogrel for High Atherothrombotic Risk and Ischemic Stabilization, Management, and Avoidance (CHARISMA) trial [131] tested the effectiveness of long term dual antiplatelet therapy with clopidogrel plus low-dose aspirin to aspirin alone for the prevention of cardiovascular events among patients with either clinically evident CVD or multiple risk factors. Enrolled patients had either clinically evident cardiovascular disease (symptomatic) or multiple risk factors for atherothrombotic disease (asymptomatic). There was no difference between the two randomized arms, but 20 subgroup analyses were pre-specified in the protocol. For symptomatic vs. asymptomatic patients, the p-value for the interaction test was 0.045 and the p-value for benefit among the symptomatic patients was 0.046. This was reported as a suggestion of benefit for clopidogrel. Two accompanying editorials [143, 148] took issue with this conclusion for several reasons. The authors made no adjustment for multiple comparisons: had any correction been done, none of the subgroup analyses would have been even close to significant. The subsequent interpretation of the p-value for the symptomatic patients overstated its significance, which was marginal in any case. Furthermore the significance of the interaction test seemed to be driven more by a harmful effect in the asymptomatic patients than by any beneficial effect in the symptomatic patients. Finally, from the clinical point of view, the distinction between symptomatic and asymptomatic was not clear, since some of the patients in the asymptomatic group had a history of major cardiovascular events at baseline. A subsequent re-analysis of subgroups with patients identified as primary prevention and secondary prevention found no within-subgroup benefit for the primary endpoint [153].

Even if not explicitly pre-specified, subgroup analyses may be identified in several ways with different implications for the reliability of their results. For example, it might be reasonable to infer that subgroup hypotheses related to factors used to stratification of the randomization, such as age, sex or stage of disease, were in fact considered in advance. Factors that are integrated into the study design may be implied as subgroups even if they are not explicitly stated in the protocol.

Of course, the same problems in interpretation apply here as with formally prespecified subgroups. The Prospective Randomized Amlodipine Survival Evaluation Study (PRAISE), a large multicenter trial [146], pre-specified several subgroups, but in addition analyzed a baseline characteristic used to stratify the randomization, ischemic vs. non-ischemic etiology of chronic heart failure, as an additional subgroup. The randomization of participants with chronic heart failure was stratified by ischemic and non-ischemic etiology. While the primary outcome of death or cardiovascular hospitalization was nonsignificant and the secondary outcome of overall survival outcome was nearly significant (p = 0.07), almost all of the risk reduction was in the non-ischemic subgroup. The risk reduction was 31% for the primary outcome (p = 0.04) and 46% for mortality (p < 0.001). However, the more favorable result was expected to be in the ischemic subgroup, not the non-ischemic subgroup. Thus, the investigators recommended that a second trial be conducted in the patient population with non-ischemic chronic heart failure using a nearly identical protocol to confirm this impressive subgroup result [146]. The results of the PRAISE-II trial proved disappointing with no reduction in either the primary or secondary outcome [147]. Thus, the previous predefined subgroup result could not be confirmed.

On occasion, during the monitoring of a trial, particular subgroup findings may emerge and be of special interest. If additional participants remain to be enrolled into the trial, one approach is to test the new subgroup hypothesis in the later participants. With small numbers of participants, it is unlikely that significant differences will be noted. If, however, the same pattern emerges in the newly created subgroup, the hypothesis is considerably strengthened. Subgroups may also emerge during a trial by being identified by other, similar trials. If one study reports that the observed difference between intervention and control appears to be concentrated in a particular subgroup of participants, it is appropriate to see if the same findings occur in another trial of the same intervention, even though that subgroup was not pre-specified. Problems here include comparability of definition. It is unusual for different trials to have baseline information sufficiently similar to allow for characterization of identical subgroups. In the Raloxifene Use for The Heart (RUTH) [133], age groups were among a number of pre-specified subgroups, but the definition of the groups was modified to match what was used for the Women’s Health Initiative [77]. Though the subgroup effects from RUTH and WHI were consistent, their interpretation as real clinical effects was vigorously challenged [155].

The weakest type of subgroup analysis involves post hoc analysis, sometimes referred to as “data-dredging” or “fishing.” With this approach subgroups are suggested by the data themselves. Because many comparisons are theoretically possible, tests of significance become difficult to interpret and should be challenged. Such analyses should serve primarily to generate hypotheses for evaluation in other trials. An example of subgrouping that was challenged comes from a study of diabetes in Iceland. Male children under the age of 14 and born in October were claimed to be at highest risk of ketosis-prone diabetes. Goudie [138] challenged whether the month of October emerged from post-study analyses biased by knowledge of the results. The ISIS-2 trial [141] illustrated a spurious subgroup finding that suggested treatment benefit of aspirin after myocardial infarction was not present in individuals born under Gemini or Libra astrological signs. A similar example [135] suggests twice as many participants with bronchial carcinoma were born in the month of March (p < 0.01) although this observation could not be reproduced [130, 134]. Subgroups unsupported by a biologically plausible hypothesis should be regarded with heightened caution.

Even subgroups supported by a biologically plausible rationale and suggesting beneficial effects can turn out to be irreproducible. Post-hoc subgroup analyses were performed for a number of trials of beta-blocking drugs were conducted in people who had had a myocardial infarction. One found that the observed benefit was restricted to participants with anterior infarctions [145]. Another claimed improvement only in participants 65 years or younger [128]. In the Beta-Blocker Heart Attack Trial, it was observed that the greatest relative benefit of the intervention was in participants with complications during the infarction [137]. These subgroup findings however, were not consistently confirmed in other trials [136].

Post-hoc subgroups may be specified by comparing participants from two groups who experience the same event, or have similar outcomes; an early example is the discriminant analysis done for the Multicentre International Study [145]. Investigators frequently want to do this in an attempt to understand the mechanisms of action of an intervention. Sometimes this retrospective look can suggest factors or variables by which the participants could be subgrouped. As discussed earlier in this chapter, categorization of participants by any outcome variable, e.g., adherence, can lead to biased conclusions. If some subgroup is suggested in this way, the investigator should create that subgroup in each randomized arm and make the appropriate comparison. For example, she may find that participants in the intervention arm who died were older than those in the control arm who died. This retrospective observation might suggest that age is a factor in the usefulness of the intervention. The appropriate way to test this hypothesis would be to subgroup all participants by age and compare intervention versus control for each age subgroup.

An interesting post hoc subgroup analyses was reported by the Metoprolol CR/XL Randomized Intervention Trial (MERIT) [154]. This trial, which evaluated the effect of a beta-blocker in participants with chronic heart failure, had two primary outcomes. One was all-cause mortality and the other was death plus hospitalization. MERIT was terminated early by the monitoring committee due to a highly significant reduction in mortality, as shown in Fig. 18.6, and similar reductions in death plus hospitalization. The results are remarkably consistent across all of the predefined subgroups for mortality, mortality plus hospitalization and mortality plus heart failure hospitalization as shown in Fig. 18.7. Moreover, the results were very consistent with those from two other beta-blocker trials [37, 38], as shown in Figs. 18.8 and 18.9. However, post hoc analyses during review by regulatory agencies compared results among countries. These results are shown in Fig. 18.10. Of note is that for mortality, the relative risk in the United States slightly favors placebo, in contrast to the mortality results for the trial as a whole. With respect to outcomes of mortality plus hospitalization, and mortality and hospitalization for heart failure, the U.S. data are consistent with the overall trial results. As noted by Wedel et al. [154] the analysis for interaction depends on how the regional subgroups are formed. Whether the observed regional difference is due to chance or real has been debated, but Wedel and colleagues argued that is not consistent with other external data, not internally consistent within MERIT and not biologically plausible, and thus is most likely due to chance. This result does however point out the risks of post hoc subgroup analyses.

Fig. 18.6

MERIT Kaplan-Meier estimates of cumulative percentage of total mortality after randomization—p value nominal and adjusted for two interim analyses (MERIT) [37]. Reproduced with permission of Elsevier Ltd. for Lancet

Fig. 18.7

Relative risk and 95% confidence intervals for selected subgroups in the MERIT trial, for total mortality, total mortality and all hospitalization, and total mortality and heart failure hospitalization [154]. Reproduced with the permission of Elsevier Ltd. for the Amer Heart Journal

Fig. 18.8

Kaplan-Meier survival curves for the CIBIS-II trial, comparing bisoprolol and placebo [34]. Reproduced with the permission of Elsevier Ltd. for Lancet

Fig. 18.9

Kaplan-Meier Analysis of Time to Death for COPERNICUS trial, comparing Placebo and Carvedilol Group. The 35% lower risk in the carvedilol group was significant: p = 0.00013 (unadjusted) and p = 0.0014 (adjusted) [38]

Fig. 18.10

Relative risk and 95% confidence intervals for the MERIT trial, for outcomes of total mortality, total mortality and hospitalization for any cause, and total mortality and heart failure hospitalization [154]. Reproduced with the permission of Elsevier Ltd. for the Amer Heart Journal

Regardless of how subgroups are selected, several factors can provide supporting evidence for the validity of the findings. As mentioned, similar results obtained in several studies strengthen interpretation. Internal consistency within a study is also a factor. If similar subgroup results are observed at most of the sites of a multicenter trial, they are more likely to be true. And of course, not all follow-up analyses and replication studies refute the initial subgroup finding. In contrast, however, plausible post hoc biological explanations for the findings, while necessary, are not sufficient. Given almost any outcome, reasonable sounding explanations can be put forward.

The two most common approaches to analysis of subgroup effects are (1) multiple hypothesis tests for effects within subgroups and (2) interaction tests for homogeneity of effects across subgroups defined by each variable of interest. Of these two the consensus in the literature strongly favors the interaction test. The interaction test provides a single, global assessment of whether a categorical variable partitioning the study cohort is associated with different magnitudes of treatment effect. Estimates of those effects, with confidence intervals, provide exploratory indications of the consistency of the treatment effect across the population. Testing for treatment effects within subgroups, in contrast, requires a greater number of hypothesis tests and inflates the probability of a false positive result over the nominal significance level [132]. Statistical power and other considerations make the overall trial result a better guide to the effect in subgroups than the subgroup specific treatment effects [155, 156].

Often, attention is focused on subgroups with the largest intervention-control differences. However, even with only a few subgroups, the likelihood of large but spurious differences in effects of intervention between the most extreme subgroup outcomes can be considerable [136, 140, 150]. Because large, random differences can occur, subgroup findings may easily be over-interpreted. Peto has argued that observed quantitative differences in outcome between various subgroups are to be expected, and they do not imply that the effect of intervention is truly inhomogeneous [118].

It has also been suggested that, unless the main overall comparison for the trial is significant, investigators should be particularly conservative in reporting significant subgroup findings [150, 163]. Lee and colleagues conducted a simulated randomized trial, in which participants were randomly allocated to two groups, although no intervention was initiated [144]. Despite the expected lack of overall difference, a subgroup was found which showed a significant difference. Further simulations [132] have emphasized the potential for spurious results even when the main comparison is significant, and the importance of basing statements about significance on interaction tests rather than subgroup-specific tests.

In summary, subgroup analyses are important. However, they must be defined using baseline data and interpreted cautiously.

Not Counting Some Events

In prevention trials, the temptation is not to count events that are observed in the immediate post-randomization follow-up period. The rational for this practice is that events occurring that rapidly must have existed at screening, but were not detected. For example, if a cancer prevention trial randomized participants into a vitamin versus placebo trial, any immediate post randomization cancer events could not have been prevented since the cancer had to have already been present subclinically at entry. Because the intervention could not have prevented these cases, their inclusion in the design only dilutes the results and decreases power. While such an argument has some appeal, it must be viewed with caution. Rarely are mechanisms of action of therapies or interventions fully understood. More importantly, negative impact of interventions having a more immediate effect might not be seen as easily or as quickly with this approach. If used at all, and this should be rarely, the data must be presented both ways; i.e., with and without the excluded events.

An extreme case of dropping early events might be in a surgical or procedure trial. Participants assigned to the procedure might be put at higher risk of a fatal or irreversible event. These early risks to the participant are part of the overall intervention effect and should not be eliminated from the analysis.

Some trials have defined various counting rules for events once participants have dropped out of the study or reached some level of nonadherence. For example, the Anturane Reinfarction Trial [28] suggested that no events after 7 days going off study medication should be counted. It is not clear what length of time is appropriate to eliminate events to avoid bias. For example, if a participant with an acute disease continues to decline and is removed from therapy, bias could be introduced if the therapy itself is contributing to the decline due to adverse effects and toxicity. In the APPROVe trial [81–84] described earlier in this chapter, the decision not to count events after 14 days and not to follow participants after that period of time led to controversy. In fact, the results and the interpretation were different once the almost complete follow-up was obtained [84].

Comparison of Multiple Variables

If enough hypothesis tests are done, some may be significant by chance even if all the hypotheses being tested are false. This issue of multiple comparisons includes repeated looks at the same response variable (Chap. 15) and comparisons of multiple variables. Many clinical trials have more than one response variable, and prespecify several subgroups of interest. Thus, a number of statistical comparisons are likely to be made. For example, when performing 20 independent comparisons, one of them, on the average, will be significantly different by chance alone using 0.05 as the level of significance. The implication of this is that the investigator should be cautious in the interpretation of results if she is making multiple comparisons. The alternative is to require a more conservative significance level. As noted earlier, lowering the significance level will reduce the power of a trial. The issue of multiple comparisons has been discussed by Miller [164], who reviewed many proposed approaches.

One way to counter the problem is to increase the sample size so that a smaller significance level can be used while maintaining the power of the trial. However, in practice, most investigators could probably not afford to enroll the number of participants required to compensate for all the possible comparisons that might be made. As an approximation, if investigators are making k comparisons, each comparison should be made at the significance level α/k, a procedure known as the Bonferonni correction [164]. Thus, for k = 10 and α = 0.05, each test would need to be significant at the 0.005 level. Sample size calculations involving a significance level of 0.005 will dramatically increase the required number of participants. The Bonferonni correction is quite conservative in controlling the overall α level or false positive error rate if the test statistics are correlated, which is often the case. Therefore, it may be more reasonable to calculate sample size based on one primary response variable, limit the number of comparisons and be cautious in claiming significant results for other comparisons.

However, there are other procedures to control the overall α level and we summarize briefly two of them [165, 166]. Assume that we prespecify m hypotheses to be tested, involving multiple outcomes, multiple subgroups, or a combination. The goal is to control the overall α level. One implementation of the Holm procedure [166] is to order the p values from smallest to largest as p(1), p(2),…..,p(m), corresponding to the m hypotheses H(1), H(2),…,H(m). Then the Holm procedure would reject H(1), if p(1) ≤ α/m. If and only if H(1) is rejected can we consider the next hypothesis. In that case, H(2) can be rejected if p(2) ≤ α /(m − 1). This process continues until we fail to reject and then the testing must stop. The Holm procedure can also be applied if the m hypotheses can be ordered according to their importance. Here, the most important hypothesis H(1) can be rejected only if the corresponding p value is less than α/m. If rejected, the next most important hypothesis H(2) can be rejected if the p value is less than α/(m − 1).

Hochberg’s procedure [165] also requires that the m hypotheses be specified in advance and orders the p-values from smallest to largest as does Holm’s. The Hochberg procedure allows all m hypotheses to be rejected if p(m) ≤ α/m. If this is not the case, then the remaining m − 1 hypotheses can be rejected if p(m − 1) ≤ α/(m − 1). This process is carried out for all of the m hypotheses until a rejection is obtained and then stops. These two procedures will not give exactly the same rejection pattern so it is important to prespecify which one will be used.

In considering multiple outcomes or subgroups, it is important to evaluate the consistency of the results qualitatively, and not stretch formal statistical analysis too far. Most formal comparisons should be stated in advance. Beyond that, one engages in observational data analysis to generate ideas for subsequent testing.

Use of Cutpoints

Splitting continuous variables into two categories, for example by using an arbitrary “cutpoint,” is often done in data analysis. This can be misleading, especially if the cutpoint is suggested by the data. As an example, consider the constructed data set in Table 18.6. Heart rate, in beats per minute, was measured prior to intervention in two groups of 25 participants each. After therapies A and B were administered, the heart rate was again measured. The average changes between groups A and B are not sufficiently different from each other (p = 0.75) using a standard t-test. However, if these same data are analyzed by splitting the participants into “responders” and “non-responders,” according to the magnitude of heart rate reduction, the results can be made to vary. Table 18.7 shows three such possibilities, using reductions of 7, 5, and 3 beats per minute as definitions of response. As indicated, the significance levels, using a chi-square test or Fisher’s exact test, change from not significant to significant and back to not significant. This created example suggests that by manipulating the cutpoint one can observe a significance level less than 0.05 when there does not really seem to be a difference.

Table 18.6

Differences in pre- and post-therapy heart rate, in beats per minute (HR), for Groups A and B, with 25 participants each

Observation number

A

B

Pre HR

Post HR

Change in HR

Pre HR

Post HR

Change in HR

1

72

72

0

72

70

2

2

74

73

1

71

68

−3

3

77

71

6

75

74

1

4

73

78

−5

74

71

3

5

70

66

4

71

73

−2

6

72

76

−4

73

78

−5

7

72

72

0

71

69

2

8

78

76

2

70

74

−4

9

72

80

−8

79

78

1

10

78

71

7

71

72

−1

11

76

70

6

78

79

−1

12

73

77

−4

72

75

−3

13

77

75

2

73

72

1

14

73

79

−6

72

69

3

15

76

76

0

77

74

3

16

74

76

−2

79

75

4

17

71

69

2

77

75

2

18

72

71

1

75

75

0

19

68

72

−4

71

70

1

20

78

75

3

78

74

4

21

76

76

0

75

80

−5

22

70

63

7

71

72

−1

23

76

70

6

77

77

0

24

78

73

5

79

76

3

25

73

73

0

79

79

0

Mean

73.96

73.20

.76

74.40

73.96

0.44

Standard deviation

2.88

3.96

4.24

3.18

3.38

2.66

Table 18.7

Comparison of change in heart rate in Group A versus B by three choices of cutpoints

Beats/min

<7

≥7

<5

≥5

<3

≥3

Group A

25

2

19

6

17

8

Group B

25

0

25

0

18

7

Chi-square

p = 0.15

p = 0.009

p = 0.76

Fisher’s exact

p = 0.49

p = 0.022

p = 0.99

Noninferiority Trial Analysis

As discussed in Chap. 5, noninferiority trials are challenging to design, conduct and analyze. We pointed out the special challenges in setting the margin of noninferiority. However, once that margin of noninferiority is established prior to the start of the trial, there remain several issues that must be included in a rigorous analysis and reported because of the clinical and regulatory implications [13, 167–183]. If we define I to be the new intervention, C to be the control or standard, and P to be the placebo or no treatment, then we obtain from the noninferiority trial an estimate of the relative risk (RR) of I to C, RR(I/C) or an absolute difference. In the design, the metric must be established since the sample size and the interim monitoring depend on it. The first analytic challenge is to establish whether the new intervention met the criteria for noninferiority, a part of which is demonstrating that the upper limit of the 95% confidence interval of the estimate was less than the noninferiority margin.

As shown in Fig. 18.11, from Pocock and Ware [181], if the upper limit of the 95% confidence interval for the relative risk is less than unity, various degrees of evidence exist for superiority (See case A). For noninferiority trials, if the upper limit of the 95% confidence interval is less than the margin of non-inferiority, δ, then there is evidence for noninferiority (see cases B and C). Failure to be less than this margin does not provide evidence for noninferiority (see case D). The design must have sufficient sample size and power to rule out a margin of noninferiority as discussed in Chap. 8. Although not expected when the study was designed, a noninferiority trial might also indicate harm (See E).

Fig. 18.11

Relative risks and 95% confidence intervals for a series of superiority and non-inferiority trials [181]

The second desired goal of a noninferiority analysis is to demonstrate that the new intervention would have beaten a placebo or no treatment if it had been included; that is, the estimate of RR(I/P). Analytically, this can be accomplished by recognizing that RR(I/P) = RR(I/C) RR(C/P). However, for this imputation step to work requires at least two critical assumptions: (1) there is constancy of the control effect over time, and (2) the population where the control was tested against placebo is relevant to the current use where the intervention (I) is being tested. These assumptions are difficult, perhaps impossible, to establish (see Chap. 5). In this chapter, we will focus our attention on the first challenge of establishing whether or not the intervention versus control comparison was less than the noninferiority margin.

Assuming that an appropriate active control was selected, the trial must implement it according to best practice and as good or better than that what was done in the initial trial establishing its benefit [172]. Otherwise, the new intervention is being compared to a control that is handicapped, making it easier for the new intervention to appear similar or even better than the control. Poor adherence and conduct will favor the new intervention in a noninferiority trial, instead of handicapping the new intervention as in a superiority trial [179]. Thus, as discussed in Chap. 14, adequate measures of adherence must be collected during the trial in order to make this critical assessment. Adherence in this case does not only mean whether the participant took all or almost all of the intervention and control drugs. What else participants were taking as concomitant medication is also a consideration. If there is a substantial imbalance, interpretation of the results would be difficult.

Another key factor is whether the outcomes chosen are true measures of the effect of both the new intervention and the control. This is sometimes referred to as assay sensitivity [177]. Thus, whether consciously or not, an investigator might select an outcome that would show no change no matter what intervention was being studied, and thus guarantee that the noninferiority margin would be achieved. Outcomes should be similar to those used in the positive control versus placebo trials.

There is a debate whether the intention-to-treat analysis or the “on treatment” analysis is most appropriate for a noninferiority designed trial. If intention-to-treat is used, nonadherence dilutes whatever differences may exist and thus is biased towards noninferiority. An “on treatment” analysis compares only those who are good adherers, or at least took some predefined portion of the intervention and thus is closer to testing the true effect. However, as we demonstrated earlier in this chapter, analyzing trials by adherence to an intervention can be substantially biased, the direction of which cannot be predicted. Thus, we do not recommend such an analysis because of the uncertainty of bias and its direction, and instead recommend that a trial be designed to minimize nonadherence. The true comparison of the new intervention may be somewhere in between the intention-to-treat and the “on treatment” but there is no dependable way to tease that estimate out. If both analytic approaches confirm noninferiority, then the conclusion is more robust, assuming that the noninferiority margin is reasonable [178].

Any trial relies on an adequate sample size to have power to test hypotheses of interest, whether for superiority or noninferiority. For a superiority trial, inadequate sample size works against finding differences but for noninferiority, inadequate sample size favors finding noninferiority. There is a difficult balance between having a noninferiority margin that is too small and thus requiring an unachievable sample size and having a margin that is so large that the sample size is appealing but the results would not be convincing.

There are many examples of noninferiority trials but we will use one to illustrate the challenges. The Stroke Prevention using an ORal Thrombin Inhibitor in atrial Fibrillation (SPORTIF)-V trial in participants with atrial fibrillation comparing a new intervention, ximelegatran, against a standard warfarin intervention [183], with a primary outcome of stroke incidence. A number of issues were involved. First, there were no very good warfarin versus placebo trials to set the noninferiority margin. Second, the trial used absolute difference as the metric, assuming the event rate would be around 3%, but instead observed an event rate less than half that. Thus, the noninferiority margin of 2% that was prespecified was too large given the small event rate. If the observed event rate of 1.5% had been assumed, the prespecified margin would have been much less, perhaps closer to 1%. The observed stroke rates were 1.2% in the warfarin group and 1.6% in the ximelegatran group with a 95% CI of −.13% to 1.03% which would meet the initial margin of noninferiority. However, this was not adequate for a margin of 1%. Therefore, even though margins may be set in advance, results may invalidate the assumptions and thus the margin itself.

As discussed in Chap. 21, presentation of the results of noninferiority trials are more complex than for superiority trials because all of the assumptions must be so carefully and clearly laid out [181].

Analysis Following Trend Adaptive Designs

As discussed in Chaps. 5 and 17, the design of a trial may have an adaptive element. This might be a group sequential design for early termination due to overwhelming benefit or a strong signal for harm, or perhaps futility. Among the adaptive designs discussed some involved changing the sample size. Some of these sample size changes are due to overall lower event rates or higher variability in the primary outcome than was assumed in the original sample size estimate. In these instances, the final analysis proceeds as normal. However, another method for sample size change relies on trend adaptive designs. In these designs, which depend on the emerging trend in the data, the final critical value or significance level will be affected and thus must be kept in mind for the final analysis.

For example, some trials may monitor accumulating interim data and may terminate the trial early for evidence of benefit or harm. If a group sequential design using a 0.05 two-sided significance level O’Brien-Fleming boundary were used five times during the trial, approximately equally spaced, the final critical value would not be +1.96 and −1.96 for the upper and lower bounds but a value closer to 2.04.

For trend adaptive sample size changes, the final critical value depends on which methodology was used but all will require typically a more conservative value, for example, than a two-sided nominal alpha level of 0.05 (a critical value of 1.96).

Other than adjusting the final critical value, the analyses for these trend adaptive designs may also utilize a modified test statistic. For example, if the method of Cui et al. [184] is used in increasing the sample size, a weighted test statistic as described in Chap. 17 is required. Future observations are given less weight than the early existing observations. The usual test statistic is not appropriate in this situation. For the other trend adaptive methods described in Chaps. 5 and 17, the final analysis can proceed with the standard statistics in a usual straightforward fashion, adjusting for the final critical value from sequential testing as appropriate.

Meta-analysis of Multiple Studies

Often in an area of clinical research several independent trials using similar participants and similar intervention strategies are conducted over a period of a few years. Some may be larger multicenter trials, but there may be a substantial number of small trials none of which were conclusive individually, though they may have served as a pilot for a larger subsequent study. Investigators from a variety of medical disciplines often review the cumulative data on similar trials and try to develop a consensus conclusion of the overall results [185–193]. If this overview is performed by a formal process and with statistical methods for combining all the data with a single analysis, the analysis is usually referred to as a meta-analysis or systematic review. Methods suitable for this purpose were described in 1954 by Cochran [194] and later by Mantel and Haenszel [195]. Other authors have summarized the methodologic approaches [196–207]. The Cochrane Collaboration has been a major contributor to systematic reviews of controlled trials [208], often organized around a specific health care area or issue, including systematic reviews of adverse effects and advice on how to conduct such systematic reviews. Guidelines intended to improve the conduct and reporting of meta-analyses have been published [209, 210]. There are numerous examples of meta-analysis in a variety of medical disciplines and a few are referenced here [211–221]. A great deal has been written and discussed about the usefulness and challenges of meta-analyses [222–233].

Rationale and Issues

Researchers conduct systematic reviews and meta-analyses to address a number of important questions [190]. Probably the most common reason is to obtain more precise estimates of an intervention effect and to increase the power to observe small but clinically important effects. Very often the potential for increased power to detect small but clinically important effects motivates the meta-analysis. However, meta-analyses can also evaluate the generalizability of results across trials, populations, and specific interventions. Subgroup analyses based on small numbers of participants may not lead to firm conclusions and miss qualitative differences in effect. Post hoc subgroup analyses are unreliable due to multiplicity of testing. Prespecified meta-analysis offers the opportunity to examine a limited number of hypotheses identified in individual trials. Meta-analysis of subgroups can guide clinicians in their practice by selecting participants most suitable for the intervention. In addition, meta-analysis can support submissions to the U.S. Food and Drug Administration. If a major clinical trial is being initiated, a sensible approach is to base many aspects of the design on the summary of all existing data. Meta-analysis is a systematic process that can provide critical information on definitions of population and intervention, control group response rates, expected size of the intervention effect, and length of follow-up. Finally, if a new treatment or intervention gains widespread popularity early in its use, a meta-analysis may provide a balanced perspective and may suggest the need for a single, large, properly designed clinical trial to provide a definitive test. Furthermore, meta-analyses are mandated if the opportunity to conduct a new large study no longer exists due to a loss of equipoise, even if this loss is not well justified. In this case, a meta-analysis may be the only solution for salvaging a reliable consensus.

As indicated, a meta-analysis is the combination of results from similar participants evaluated by similar protocols and interventions. The standard analysis of a multicenter trial, stratified by clinical center is in some ways an ideal meta-analysis. Each center plays the role of a small study. Protocols and treatment strategies are identical, and participants are more similar than those in a typical collection of trials.

This contrast between a meta-analysis and a multicenter trial points out some limitations of the former. While the implementation of a clinical protocol can vary across centers, such differences are negligible compared to those in a collection of independently conducted large or small trials. Even when the analysis is done by pooling participant-level data from each trial [212, 217], meta-analysis cannot be expected to produce the same level of evidence as a single, large clinical trial. In a typical meta-analysis, important differences exist in actual treatment, study population, length of follow-up, measures of outcome, level of background medical care in international trials and quality of data [222, 225–228, 233]. Because of these differences, the potential for meta-analysis should never be a justification for conducting a series of small, loosely connected studies with the expectation that a definitive result can be produced by combining after the fact. Perhaps the most fundamental problem is the potential to create bias when deciding on which studies to include in a meta-analysis. Two examples of such bias are selection bias and investigator bias.

Many support the concept that the most valid overview and meta-analysis requires all relevant studies conducted be available for inclusion or at least for consideration [190, 226]. Failing to do so can produce selection bias; that is, a mis-estimation caused by analysis of a non-representative sample. For example, Furberg [228] provides a review of seven meta-analyses of lipid lowering trials. Each article presents different inclusion criteria, such as the number of participants or the degree of cholesterol reduction. The results vary depending on the criteria used. Another example of selection bias in meta-analysis involves the investigation of whether adding manual thrombus aspiration to primary percutaneous coronary intervention (PPCI) reduces total mortality. Between 1996 and 2009, about 20 small clinical trials and one larger trial, the Thrombus Aspiration during Percutaneous Coronary Intervention in Acute Myocardial Infarction Study (TAPAS) trial [234], were conducted to address whether PPCI with thrombus aspiration might have benefits over PPCI alone. These trials were not powered for total mortality and the smaller trials were not consistently positive; however, the largest suggested a possible 50% mortality benefit for manual thrombus aspiration. A series of meta-analyses sought to clarify the situation [212, 235–240]. Despite having identical aims, nearly identical inclusion criteria, and access to the same small set of trial results, no two meta-analyses included the same set of studies, and results varied. Because there were conflicting conclusions, no consensus was produced. The Thrombus Aspiration in ST-Elevation Myocardial Infarction in Scandinavia (TASTE) trial, designed with mortality as its primary outcome, concluded that there was no effect [20, 241], but a subsequent meta-analysis including the TASTE trial, while finding a non-significant effect on mortality, concluded that a modest reduction in clinical outcomes exists [242].

While it is clearly difficult enough to decide which well-known and published trial results to include, a further serious complication is that some relevant trial results may not be readily accessible in the literature due to publication bias [223, 231]. Published trials are more likely to be statistically significant (p < 0.05) or to favor a novel intervention. Trials that yield neutral or indifferent results are less likely to be published. One example described by Furberg and Morgan [227] illustrates this problem. An overview [223] of the use of propranolol in patients following a heart attack reported 7 of 45 patients died in the hospital compared to a non-randomized, placebo-control where 17 of 46 died, indicating a clear benefit of propranolol. Controversy over design limitations motivated the investigator to conduct two additional randomized trials. One showed no difference and the other a negative (harmful) trend. Neither was ever published. Identifying yet another obstacle to inclusion of all relevant studies, Chalmers et al. [224] pointed out that a MEDLINE literature search may only find 30–60% of published trials. This is due in part to the way results are presented and searches of typical key words may not uncover relevant papers. Although search engines may be better now, there are undoubtedly still limitations. Work by Gordon and colleagues found that only 57% of 244 NHLBI-supported trials completed between January 2000 and December 2011 published their main results within 30 months after completion [243]. These difficulties in determining and accessing the entire population of relevant studies may lead to analysis of a subset of trial results which are not representative, producing conclusions which do not reflect the totality of evidence because of selection bias.

Another type of bias, referred to as investigator bias, occurs when an investigator ignores or goes beyond any pre-specified plan and makes subjective decisions about which trials and outcome variables will get reported. If protocols were written well and adhered to strictly, investigator bias would not be a problem. However, post-hoc repeated testing of multiple subgroups and multiple outcomes may not be easy to detect from the published report [229]. Promising early results may draw major attention, but if later results show smaller intervention effects, they may go unnoticed or be harder to find for the systematic review. Furthermore, authors of systematic reviews are also to subject to investigator bias. That is, unless the goals of the meta-analysis are clearly stated a priori in a protocol, a positive result can be found in this analysis by sifting through numerous attempts. A great deal of time and persistence are required in order to get access to all known conducted trials and accurately extract the relevant data. Not all meta-analyses are conducted with the same degree of thoroughness.

The medical literature is filled with meta-analysis of trials covering a wide range of disciplines [211–221]. Several examples from the cardiology literature will provide an overview. Chalmers and colleagues [214] reviewed six small studies that used anticoagulants in an effort to reduce mortality in heart attack patients. While only one of the six was individually significant, the combined overall results suggested a statistically significant 4.2% absolute reduction in mortality. The authors suggested no further trials were necessary. However, due to issues raised, this analysis drew serious criticism [229]. Several years later, Yusuf and colleagues [221] reviewed 33 fibrinolytic trials, focusing largely on the use of streptokinase. This overview included trials with much dissimilarity in dose, route and time of administration, and setting. Although the meta-analysis for intravenous use of fibrinolytic drugs was impressive, and the authors concluded that results were not due to reporting biases, they nevertheless discussed the need for future large-scale trials before widespread use should be recommended. There were issues, for example, as to how quickly such an intervention needed to be started after onset of a heart attack. That is, timing needed to be resolved. Canner [213] conducted an overview of 6 randomized clinical trials testing aspirin use in participants with a previous heart attack to reduce mortality. His overall meta-analysis suggested a 10% reduction that was not significant (p = 0.11). However, there was an apparent heterogeneity of results and the largest trial had a slightly negative mortality result. The Canner overview was repeated by Hennekens et al. [215] after several more trials had been conducted. This updated analysis demonstrated favorable results. May et al. [218] conducted an early overview of several modes of therapy for secondary prevention of mortality after a heart attack. Their overview covered anti-arrhythmic drugs, lipid-lowering drugs, anticoagulant drugs, beta-blocker drugs, and physical exercise. Although statistical methods were available to combine studies within each treatment class, they chose not to combine results, but simply provided relative risks and confidence interval results graphically for each study. A visual inspection of the trends and variation in trial results suggests a summary analysis. Yusuf et al. [220] later provided a more detailed overview of beta blocker trials. While using a similar graphical presentation, they calculated a summary odds ratio and its confidence interval. Meta-analysis of cancer trials have also been conducted including the use of adjuvant therapy for breast cancer [216]. While using multiple chemotherapeutic agents indicated improved relapse-free survival after 3 and 5 years of follow-up, as well as for survival, the dissimilarity among the trials led the authors to call for more trials and better data.

Thompson [232] pointed out the need to investigate sources of heterogeneity. These differences may be in populations studied, intervention strategies, outcomes measured, or other logistical aspects. Given such differences, inconsistent results among individual studies might be expected. Statistical tests for heterogeneity often have low statistical power even in the presence of a moderate heterogeneity. Thompson [232] argued that we should investigate the influence of apparent clinical differences between studies and not rely on formal statistical tests to give us assurance of no heterogeneity. In the presence of apparent heterogeneity, overall summary results should be interpreted cautiously. Thompson described an example of a meta-analysis of 28 studies evaluating cholesterol lowering and the impact on risk of coronary heart disease. A great deal of heterogeneity was present, so a simple overall estimate of risk reduction may be misleading. He showed that factors such as age of the cohort, length of treatment, and size of study were contributing factors. Taking these factors into account made the heterogeneity less extreme and results more interpretable. One analysis showed that the percent reduction in risk decreased with the age of the participant at the time of the event, a point not seen in the overall meta-analysis. However, he also cautioned that such analyses of heterogeneity must be interpreted cautiously, just as for subgroup analyses in any single trial.

Meta-analysis, as opposed to typical literature reviews, usually puts a p-value on the conclusion. The statistical procedure may allow for calculation of a p-value, but it implies a precision which may be inappropriate. The possibility that not all relevant studies have been included may make the interpretation of the p-value tenuous. Quality of data may vary from study to study. Data from some trials may be incomplete without being recognized as such. Thus, only very simple and unambiguous outcome variables, such as all-cause mortality and major morbid events ought to be used for meta-analysis.

Statistical Methods

Since meta-analysis became a popular approach to summarizing a collection of studies, numerous statistical publications have been produced addressing technical aspects [186, 194–196, 198–201, 203, 205, 207]. Most of this is beyond the technical scope of this text, but a number of texts on the subject of meta-analysis are available [197, 202, 204, 206]. Two common technical approaches were first suggested by Cochran [194] in 1954. If all trials included in the meta-analysis are estimating the same true (but unknown) fixed effect of an intervention, the Mantel-Haenszel method [195] can be used with a slight variation. This is similar to the logrank or Mantel-Haenszel method in the chapter on survival analysis. If the trials are assumed to have dissimilar or heterogeneous true intervention effects, the effects are described by a random effects model, as suggested by DerSimonian and Laird [200]. Another valid but less common approach relies on a Bayesian analysis [204] which was used to assess the literature on adjunctive thrombotomy for acute myocardial infarction [239].

The method of DerSimonian and Laird [200] compares rate differences within each study, and obtains a pooled estimate of the rate difference as well as the standard error. The pooled estimate of the rate difference is a weighted average of the individual study rate differences. The weights are the inverse of the sum of the between and within study variance components of intervention effect. If the studies are relatively similar or homogeneous in intervention effect, this approach and the fixed effects method produce very similar results [196]. Heterogeneity tests generally are not as powerful as the test for main effects. However, if studies vary in intervention effect, these two methods can produce different results as illustrated by Berlin et al. [196] as well as Pocock and Hughes [203].

Typically, when presenting the results of a meta-analysis, the OR estimate and 95% confidence interval are plotted in a single graph for each trial to provide a visual summary. Figure 18.12, from Yusuf et al. [221], summarizes the effects of 24 trials of fibrinolytic treatment on mortality in people with an acute heart attack. The hash mark represents the estimated OR and the line represents the 95% confidence interval. They [221] include a single estimate of the OR, combining all studies. The size of the symbol in this plots, sometimes referred to as “forest plots,” is an indication of the size of each individual studies. In the presence of serious heterogeneity of treatment effect, however, the appropriateness of obtaining a single point estimate must be questioned. If the heterogeneity is qualitative; that is, some estimates of the OR are larger than unity and others less than unity, then a combined single estimate is perhaps not wise. This would be especially true if these estimates indicated a time trend, which could occur if dose and participant selection changed as more experience with the new intervention was obtained.

Fig. 18.12

Apparent effects of fibrinolytic treatment on mortality in the randomized trials of IV treatment of acute myocardial infarction (Reproduced with permission of the Editor, European Heart Journal and Dr. S. Yusuf)

Which model to use for meta-analysis is a matter of debate, but none are exactly correct. The random effects model has an undesirable aspect, in that small trials may dominate the final estimate. With the fixed effect model, larger trials get greater weight. Since the meta-analysis is conducted on available trials, however, the sample of participants included is not likely to be very representative of the general population to which the intervention may be applied. That is, the trials that are available do not contain a random sample of people from the targeted population but rather are participants who volunteered and who in other respects may not be representative. Thus, the estimate of the intervention effect is not as relevant as whether or not the intervention has an effect. We prefer a fixed effects model but suggest that both models should be conducted to examine what, if any, differences exist.

Chalmers, a strong advocate of clinical trials, argued that participants should be randomized early in the evolution and evaluation of a new intervention [244]. Both as a result of that kind of advocacy and the fact that small trials are always done before large ones in the development of new interventions, an early meta-analysis is likely to consist of many small studies. Sometimes, meta-analyses of just small trials might yield significant results.

Thus, meta-analyses are seen by many as alternatives to the extraordinary effort and cost often required to conduct adequately powered individual trials. Rather than providing a solution, they perhaps ought to be viewed as a way of summarizing existing data; a way that has strengths and weaknesses, and must be critically evaluated. It would clearly be preferable to combine resources prospectively and collaborate in a single large study. Pooled results from distinct studies cannot replace individual, well-conducted multicenter trials.

Analysis for Harmful Effects

While the analyzing the primary and secondary outcome variables for benefit is challenging, the analysis of adverse event data for safety is even more complex and challenging. Of course, if any of the primary or secondary outcome variables trend in the wrong direction, then there is evidence of harm, not benefit. However, harmful effects may manifest themselves in other variables than these primary or secondary outcomes. Some adverse event measures can be prespecified such as changes in the QT interval in an ECG or an elevated liver function test (LFT). But there are many other possibilities.

The typical way that adverse event data are collected in current Phase III trials is a passive system where patient complaints or physician observations are summarized in text fields which are later coded by various adverse event coding systems (See Chap. 12). Such events are usually not solicited actively so that if the patient does not complain or the physician does not record the event or problems, they do not get coded. In fact, if a patient complains about the adverse event in a different manner from one visit to the next, the event may be coded differently. If the physician records the event using different language, the event may get coded differently. It can be challenging to even track an adverse event from one visit to the next within a patient. Another one of the problems of these types of coding systems is that a very large number of categories can be generated for the same essential problem, depending on how the patient complained or the physician recorded his observations in the patient chart.

Thus, tables of adverse events using these systems can have very many rows with only a few events in each row, even for the same basic adverse problem. Such data are not likely to produce statistically significant comparisons or flag potential problems. The data are so granular that an adverse event signal cannot be seen easily. These coding systems can collapse these detailed categories into higher order terms but in doing so add adverse events that are a real signal with typically a lot more events that are not very serious or clinical important. That is, the noise drowns out the signal.

Thus, analysis of this type of data requires a careful scrutiny of the numerous detailed categories to find ones that seem to indicate a meaningful clinical issue, and these items may come from different higher level categories. This process is or can be very subjective and may be hard for another investigative team to reproduce this same categorization.

One alternative to this passive adverse event reporting is to specify in the protocol the special adverse events of interest, and actively solicit the participants for information on their occurrence or conduct whatever laboratory measures are necessary to assess whether that event did occur. Examples of a deal breaker might be QT interval increase or an increase in LFT measures. Any substantial, statistically significant or clinically important imbalance in these type of events would be sufficient to perhaps terminate a trial early or kill the further development of the intervention, whether drug, device or biologic. There are probably more than 10 such “deal breakers” but less than 100, depending on the disease and intervention. Of course other adverse event data may be collected in a patient chart as text and later retrieved as necessary using more recent developed natural language processing (NLP) algorithms. If imbalances are found in such review, confirmation should be sought whenever possible using warehouse data from large electronic health record (EHR) systems.

References

1.

Geller NL. Advances in Clinical Trial Biostatistics. Taylor & Francis, 2003.

2.

Van Belle G, Fisher LD, Heagerty PJ, Lumley T. Biostatistics: A Methodology For the Health Sciences. Wiley, 2004.

3.

Piantadosi S. Clinical Trials: A Methodologic Perspective. Wiley, 2013.

4.

Cook TD, DeMets DL. Introduction to Statistical Methods for Clinical Trials. Taylor & Francis, 2007.

5.

Pagano M, Gauvreau K. Principles of Biostatistics. Duxbury, 2000.

6.

Hill AB. Principles of medical statistics. Oxford University Press, 1971.

7.

Armitage P, Berry G, Matthews JNS. Statistical Methods in Medical Research. Wiley, 2008.

8.

Woolson RF, Clarke WR. Statistical Methods for the Analysis of Biomedical Data. Wiley, 2011.

9.

Armitage P. The analysis of data from clinical trials. Statistician 1979;171–183.

10.

Fleiss JL. Analysis of data from multiclinic trials. Control Clin Trials 1986;7:267–275.

11.

Newcombe RG. Explanatory and pragmatic estimates of the treatment effect when deviations from allocated treatment occur. Statist Med 1988;7:1179–1186.

12.

Peto R, Pike MC, Armitage P, et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design. Br J Cancer 1976;34:585.

13.

Temple R, Ellenberg SS. Placebo-Controlled Trials and Active-Control Trials in the Evaluation of New Treatments. Part 1: Ethical and Scientific Issues. Ann Intern Med 2000;133:455–463.

14.

Jones B, Jarvis P, Lewis JA, Ebbutt AF. Trials to assess equivalence: the importance of rigorous methods. BMJ 1996;313:36–39.

15.

Sackett DL, Gent M. Controversy in Counting and Attributing Events in Clinical Trials. N Engl J Med 1979;301:1410–1412.

16.

Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Chronic Dis1967;20:637–648.

17.

Gillespie SH, Crook AM, McHugh TD, et al. Four-Month Moxifloxacin-Based Regimens for Drug-Sensitive Tuberculosis. N Engl J Med 2014.

18.

FDA: International Conference on Harmonization - Efficacy: Statistical principles for clinical trials. U S Food and Drug Administration.

19.

Lewis JA. Statistical principles for clinical trials (ICH E9): an introductory note on an international guideline. Statist Med 1999;18:1903–1942.

20.

Frobert O, Lagerqvist B, Olivecrona GrK, et al. Thrombus Aspiration during ST-Segment Elevation Myocardial Infarction. N Engl J Med 2013;369:1587–1597.

21.

The Women’s Health Initiative Study Group. Design of the Women’s Health Initiative Clinical Trial and Observational Study. Control Clin Trials 1998;19:61–109.

22.

May GS, DeMets DL, Friedman LM, et al. The randomized clinical trial: bias in analysis. Circulation 1981;64:669–673.

23.

Beta Blocker Heart Attack Trial Research Group. A randomized trial of propranolol in patients with acute myocardial infarction: I. mortality results. JAMA 1982;247:1707–1714.

24.

Collaborative Group on Antenatal Steroid Therapy. Effect of antenatal dexamethasone administration on the prevention of respiratory distress syndrome. Am J Obstet Gynecol 1981;141:276–287.

25.

Ingle JN, Ahmann DL, Green SJ, et al. Randomized Clinical Trial of Diethylstilbestrol versus Tamoxifen in Postmenopausal Women with Advanced Breast Cancer. N Engl J Med 1981;304:16–21.

26.

Roberts R, Croft C, Gold HK, et al. Effect of Propranolol on Myocardial-Infarct Size in a Randomized Blinded Multicenter Trial. N Engl J Med 1984;311:218–225.

27.

Temple R, Pledger GW. The FDA’s Critique of the Anturane Reinfarction Trial. N Engl J Med 1980;303:1488–1492.

28.

The Anturane Reinfarction Trial Research Group. Sulfinpyrazone in the Prevention of Sudden Death after Myocardial Infarction. N Engl J Med 1980;302:250–256.

29.

The Anturane Reinfarction Trial Research Group. The Anturane Reinfarction Trial: Reevaluation of Outcome. N Engl J Med 1982;306:1005–1008.

30.

The Canadian Cooperative Study Group. A Randomized Trial of Aspirin and Sulfinpyrazone in Threatened Stroke. N Engl J Med 1978;299:53–59.

31.

The Coronary Drug Project Research Group. Clofibrate and niacin in coronary heart disease. JAMA 1975;231:360–381.

32.

Soran A, Nesbitt L, Mamounas EP, et al. Centralized medical monitoring in phase III clinical trials: the National Surgical Adjuvant Breast and Bowel Project (NSABP) experience. Clin Trials 2006;3:478–485.

33.

Reboussin D, Espeland MA. The science of web-based clinical trial management. Clin Trials 2005;2:1–2.

34.

CIBIS-II Investigators and Committees. The Cardiac Insufficiency Bisoprolol Study II (CIBIS-II): a randomised trial. Lancet 1999;353:9–13.

35.

Ambrosius WT, Sink KM, Foy CG, et al. The SPRINT Study Research Group. The design and rationale of a multicenter clinical trial comparing two strategies for control of systolic blood pressure: The Systolic Blood Pressure Intervention Trial (SPRINT). Clin Trials 2014;11:532–546.

36.

Kjekshus J, Apetrei E, Barrios V, et al. Rosuvastatin in Older Patients with Systolic Heart Failure. N Engl J Med 2007;357:2248–2261.

37.

MERIT HF Study Group. Effect of Metoprolol CR/XL in chronic heart failure. Metoprolol CR/XL Randomized Interventional Trial in congestive heart failure (MERIT-HF). Lancet 1999;353:2001–2007.

38.

Packer M, Coats AJS, Fowler MB, et al. Effect of Carvedilol on Survival in Severe Chronic Heart Failure. N Engl J Med 2001;344:1651–1658.

39.

The GUSTO Investigators. An International Randomized Trial Comparing Four Thrombolytic Strategies for Acute Myocardial Infarction. N Engl J Med 1993;329:673–682.

40.

Detre K, Peduzzi P. The problem of attributing deaths of nonadherers: The VA coronary bypass experience. Control Clin Trials 1982;3:355–364.

41.

Diggle PJ. Testing for Random Dropouts in Repeated Measurement Data. Biometrics 1989;45:1255–1258.

42.

Dillman RO, Seagren SL, Propert KJ, et al. A Randomized Trial of Induction Chemotherapy plus High-Dose Radiation versus Radiation Alone in Stage III Non-Small-Cell Lung Cancer. N Engl J Med 1990;323:940–945.

43.

Dolin R, Reichman RC, Madore HP, et al. A Controlled Trial of Amantadine and Rimantadine in the Prophylaxis of Influenza a Infection. N Engl J Med 1982;307:580–584.

44.

Heyting A, Tolboom JTBM, Essers JGA. Statistical handling of drop-outs in longitudinal clinical trials. Statist Med 1992;11:2043–2061.

45.

Hoover DR, Munoz A, Carey V, et al. Using Events from Dropouts in Nonparametric Survival Function Estimation with Application to Incubation of AIDS. J Am Stat Assoc 1993;88:37–43.

46.

Lagakos SW, Lim LLY, Robins JM. Adjusting for early treatment termination in comparative clinical trials. Statist Med 1990;9:1417–1424.

47.

Lipid Research Clinics Program. The lipid research clinics coronary primary prevention trial results: I. reduction in incidence of coronary heart disease. JAMA 1984;251:351–364.

48.

Morgan TM. Analysis of duration of response: A problem of oncology trials. Control Clin Trials 1988;9:11–18.

49.

Oakes D, Moss AJ, Fleiss JL, et al. Use of Compliance Measures in an Analysis of the Effect of Diltiazem on Mortality and Reinfarction After Myocardial Infarction. J Am Stat Assoc 1993;88:44–49.

50.

Pizzo PA, Robichaud KJ, Edwards BK, et al. Oral antibiotic prophylaxis in patients with cancer: a double-blind randomized placebo-controlled trial. J Pediatr 1983;102:125–133.

51.

Pledger GW. Basic statistics: importance of compliance. Journal of clinical research and pharmacoepidemiology 1992;6:77–81.

52.

Redmond C, Fisher B, Wieand HS. The methodologic dilemma in retrospectively correlating the amount of chemotherapy received in adjuvant therapy protocols with disease-free survival. Cancer Treat Rep 1983;67:519–526.

53.

Ridout MS, Diggle PJ: Testing for Random Dropouts in Repeated Measurement Data. Biometrics 1991;47:1617–1621.

54.

Simon R, Makuch RW. A non-parametric graphical representation of the relationship between survival and the occurrence of an event: Application to responder versus non-responder bias. Statist Med 1984;3:35–44.

55.

Sommer A, Zeger SL. On estimating efficacy from clinical trials. Statist Med 1991;10:45–52.

56.

The Coronary Drug Project Research Group. The coronary drug project: Initial findings leading to modifications of its research protocol. JAMA 1970;214:1303–1313.

57.

The Coronary Drug Project Research Group. Influence of Adherence to Treatment and Response of Cholesterol on Mortality in the Coronary Drug Project. N Engl J Med 1980;303:1038–1041.

58.

Verter J, Friedman L. Adherence measures in the aspirin myocardial infarction study (AMIS). Control Clin Trials 1984;5:306.

59.

Wilcox RGR, Roland J, Banks D, et al. Randomised trial comparing propranolol with atenolol in immediate treatment of suspected myocardial infarction. BMJ 1980;280:885-888.

60.

FDA: Guidance for Industry: Non-Inferiority Trials. FDA.

61.

Espeland MA, Byington RP, Hire D, et al. Analysis strategies for serial multivariate ultrasonographic data that are incomplete. Statist Med 1992;11:1041–1056.

62.

The Intermittent Positive Pressure Breathing Trial Group. Intermittent Positive Pressure Breathing Therapy of Chronic Obstructive Pulmonary Disease. A Clinical Trial. Ann Intern Med 1983;99:612–620.

63.

Conaway MR, Rejeski WJ, Miller ME. Statistical issues in measuring adherence: Methods for incomplete longitudinal data; in Riekert KA, Judith KO, Shumaker SA (eds): The Handbook of Health Behavior Change. New York, Springer Publishing Company, 2008, pp 375–391.

64.

Dempster AP, Laird NM, Rubin DB. Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc Series B Stat Methodol 1977;1–38.

65.

Efron B. Missing Data, Imputation, and the Bootstrap. J Am Stat Assoc 1994;89:463–475.MathSciNetMATH

66.

Fitzmaurice GM, Laird NM, Ware JH. Applied longitudinal analysis, ed 2nd. John Wiley & Sons, 2011.

67.

Greenlees JS, Reece WS, Zieschang KD. Imputation of Missing Values when the Probability of Response Depends on the Variable Being Imputed. J Am Stat Assoc 1982;77:251–261.

68.

Laird NM. Missing data in longitudinal studies. Statist Med 1988;7:305–315.

69.

Little RJA, Rubin DB. Statistical Analysis with Missing Data. Wiley, 2002.

70.

Little RJA. Modeling the Drop-Out Mechanism in Repeated-Measures Studies. J Am Stat Assoc 1995;90:1112–1121.MathSciNetMATH

71.

Molenberghs G, Kenward M. Missing Data in Clinical Studies. Wiley, 2007.

72.

Rubin DB. Inference and missing data. Biometrika 1976;63:581–592.MathSciNetMATH

73.

Shao J, Zhong B. Last observation carry-forward and last observation analysis. Statist Med 2003;22:2429–2441.

74.

O’Kelly M, Ratitch B. Clinical Trials with Missing Data: A Guide for Practitioners. Wiley, 2014.

75.

Rubin DB. Multiple Imputation for Nonresponse in Surveys. Wiley, 2009.

76.

Steering Committee of the Physicians’ Health Study. Final Report on the Aspirin Component of the Ongoing Physicians’ Health Study. N Engl J Med 1989;321:129–135.

77.

Writing Group for the Women’s Health Initiative Investigators. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: Principal results from the women’s health initiative randomized controlled trial. JAMA 2002;288:321–333.

78.

Wu MC, Carroll RJ. Estimation and Comparison of Changes in the Presence of Informative Right Censoring by Modeling the Censoring Process. Biometrics 1988;44:175–188.MathSciNetMATH

79.

Wu MC, Bailey KR. Estimation and comparison of Changes in the Presence of Informative Right Censoring: Conditional Linear Model. Biometrics 1989;45:939–955.MathSciNetMATH

80.

Bristow MR, Saxon LA, Boehmer J, et al. Cardiac-Resynchronization Therapy with or without an Implantable Defibrillator in Advanced Chronic Heart Failure. N Engl J Med 2004;350:2140–2150.

81.

Bresalier RS, Sandler RS, Quan H, et al. Cardiovascular Events Associated with Rofecoxib in a Colorectal Adenoma Chemoprevention Trial. N Engl J Med 2005;352:1092–1102.

82.

Lagakos SW. Time-to-Event Analyses for Long-Term Treatments—The APPROVe Trial. N Engl J Med 2006;355:113–117.

83.

Nissen SE. Adverse Cardiovascular Effects of Rofecoxib. N Engl J Med 2006;355:203-205.

84.

Baron JA, Sandler RS, Bresalier RS, et al. Cardiovascular events associated with rofecoxib: final analysis of the APPROVe trial. Lancet 2008;372:1756–1764.

85.

Kruskal WH. Some remarks on wild observations. Technometrics 1960;2:1–3.MathSciNet

86.

Canner PL, Huang YB, Meinert CL. On the detection of outlier clinics in medical and surgical trials: I. Practical considerations. Control Clin Trials 1981;2:231–240.

87.

Canner PL, Huang YB, Meinert CL. On the detection of outlier clinics in medical and surgical trials: II. Theoretical considerations. Control Clin Trials 1981;2:241–252.

88.

Dixon WJ. Processing Data for Outliers. Biometrics 1953;9:74–89.

89.

Grubbs FE. Procedures for detecting outlying observations in samples. Technometrics 1969;11:1–21.

90.

Anand IS, Carson P, Galle E, et al. Cardiac Resynchronization Therapy Reduces the Risk of Hospitalizations in Patients With Advanced Heart Failure: Results From the Comparison of Medical Therapy, Pacing and Defibrillation in Heart Failure (COMPANION) Trial. Circulation 2009;119:969–977.

91.

Cannon CP, Braunwald E, McCabe CH, et al. Intensive versus Moderate Lipid Lowering with Statins after Acute Coronary Syndromes. N Engl J Med 2004;350:1495–1504.

92.

Shepherd J, Cobbe SM, Ford I, et al. Prevention of Coronary Heart Disease with Pravastatin in Men with Hypercholesterolemia. N Engl J Med 1995;333:1301–1308.

93.

Scandinavian Simvastatin Survival Study Group. Randomised trial of cholesterol lowering in 4444 patients with coronary heart disease: the Scandinavian Simvastatin Survival Study (4S). Lancet 1994;344:1383–1389.

94.

R Core Team. R: A Language and Environment for Statistical Computing. Vienna, Austria, R Foundation for Statistical Computing, 2013.

95.

SAS Institute: SAS/STAT 12.1 User’s Guide: Survival Analysis. SAS Institute, 2012.

96.

Ferreira-Gonzalez I, Busse JW, Heels-Ansdell D, et al. Problems with use of composite end points in cardiovascular trials: systematic review of randomised controlled trials. BMJ 2007;334:786.

97.

Tomlinson G, Detsky AS. Composite end points in randomized trials: There is no free lunch. JAMA 2010;303:267–268.

98.

The Look AHEAD Research Group. Look AHEAD (Action for Health in Diabetes): design and methods for a clinical trial of weight loss for the prevention of cardiovascular disease in type 2 diabetes. Control Clin Trials 2003;24:610–628.

99.

Brancati FL, Evans M, Furberg CD, et al. Midcourse correction to a clinical trial when the event rate is underestimated: the Look AHEAD (Action for Health in Diabetes) Study. Clin Trials 2012;9:113–124.

100.

Look AHEAD Research Group. Cardiovascular Effects of Intensive Lifestyle Intervention in Type 2 Diabetes. N Engl J Med 2013;369:145–154.

101.

Committee of Principal Investigators, World Health Organization. A co-operative trial in the primary prevention of ischaemic heart disease using clofibrate. Br Heart J 1978;40:1069–1118.

102.

Aspirin Myocardial Infarction Study Research Group. A randomized, controlled trial of aspirin in persons recovered from myocardial infarction. JAMA 1980;243:661–669.

103.

Crager MR. Analysis of Covariance in Parallel-Group Clinical Trials with Pretreatment Baselines. Biometrics 1987;43:895–901.MathSciNetMATH

104.

Anderson JR, Cain KC, Gelber RD. Analysis of survival by tumor response. J Clin Oncol 1983;1:710–719.

105.

Byar DP. Assessing apparent treatment—covariate interactions in randomized clinical trials. Statist Med 1985;4:255–263.

106.

Thall PF, Lachin JM. Assessment of stratum-covariate interactions in Cox’s proportional hazards regression model. Statist Med 1986;5:73–83.

107.

Beach ML, Meier P. Choosing covariates in the analysis of clinical trials. Control Clin Trials 1989;10:161–175.

108.

Weiss GB, Bunce III H, James A. Comparing survival of responders and nonresponders after treatment: A potential source of confusion in interpreting cancer clinical trials. Control Clin Trials 1983;4:43–52.

109.

Efron B, Feldman D. Compliance as an Explanatory Variable in Clinical Trials. J Am Stat Assoc 1991;86:9–17.

110.

Robins JM, Tsiatis AA. Correcting for non-compliance in randomized trials using rank preserving structural failure time models. Commun Stat Theory Methods 1991;20:2609–2631.MATH

111.

Canner PL. Covariate adjustment of treatment effects in clinical trials. Control Clin Trials 1991;12:359–366.

112.

Morgan TM, Elashoff RM. Effect of covariate measurement error in randomized clinical trials. Statist Med 1987;6:31–41.

113.

Canner PL. Further aspects of data analysis. Control Clin Trials 1983;4:485-503.

114.

Shuster J, van Eys J. Interaction between prognostic factors and treatment. Control Clin Trials 1983;4:209–214.

115.

Albert JM, DeMets DL. On a model-based approach to estimating efficacy in clinical trials. Statist Med 1994;13:2323–2335.

116.

Cox DR. Regression Models and Life-Tables. J R Stat Soc Series B Stat Methodol 1972;34:187–220.MathSciNetMATH

117.

Oye RK, Shapiro MF. Reporting results from chemotherapy trials: Does response make a difference in patient survival? JAMA 1984;252:2722–2725.

118.

Peto R. Statistical aspects of cancer trials; in Halnan KE (ed): Treatment of Cancer. London, Chapman and Hall, 1982.

119.

Gail MH, Simon R. Testing for Qualitative Interactions between Treatment Effects and Patient Subsets. Biometrics 1985;41:361–372.MATH

120.

Yates F. The Analysis of Multiple Classifications with Unequal Numbers in the Different Classes. J Am Stat Assoc 1934;29:51–66.MATH

121.

Rosenbaum PR. The Consequences of Adjustment for a Concomitant Variable That Has Been Affected by the Treatment. J R Stat Soc Ser A 1984;147:656–666.

122.

Egger MJ, Coleman ML, Ward JR, et al. Uses and abuses of analysis of covariance in clinical trials. Control Clin Trials 1985;6:12–24.

123.

Armitage P, Berry G, Mathews J. Statistical Methods in Medical Research, ed 4th. Malden MA, Blackwell Publishing, 2002.

124.

Balke A, Pearl J. Bounds on Treatment Effects From Studies With Imperfect Compliance. J Am Stat Assoc 1997;92:1171–1176.MATH

125.

Loeys T, Goetghebeur E. A Causal Proportional Hazards Estimator for the Effect of Treatment Actually Received in a Randomized Trial with All-or-Nothing Compliance. Biometrics 2003;59:100–105.MathSciNetMATH

126.

Sagarin BJ, West SG, Ratnikov A, et al. Treatment noncompliance in randomized experiments: Statistical approaches and design issues. Psychol Methods 2014;19:317–333.

127.

Ten Have TR, Normand SL, Marcus SM, et al. Intent-to-treat vs. non-intent-to-treat analyses under treatment non-adherence in mental health randomized trials. Psychiatr Ann 2008;38:772.

128.

Andersen MP, Frederiksen J, Jnrgensen H, et al. Effect of Alprenolol on Mortality Among Patients With Definite or Suspected Acute Myocardial Infarction. Lancet 1979;314:865–868.

129.

Assmann SF, Pocock SJ, Enos LE, Kasten LE. Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet 2000;355:1064–1069.

130.

Baas C, Strackee J, Jones I. Lung Cancer and Month of Birth. Lancet 1964;283:47.

131.

Bhatt DL, Fox KAA, Hacke W, et al. Clopidogrel and Aspirin versus Aspirin Alone for the Prevention of Atherothrombotic Events. N Engl J Med 2006;354:1706–1717.

132.

Brookes ST, Whitely E, Egger M, et al. Subgroup analyses in randomized trials: risks of subgroup-specific analyses; power and sample size for the interaction test. J Clin Epidemiol 2004;57:229–236.

133.

Collins P, Mosca L, Geiger MJ, et al. Effects of the Selective Estrogen Receptor Modulator Raloxifene on Coronary Outcomes in The Raloxifene Use for the Heart Trial: Results of Subgroup Analyses by Age and Other Factors. Circulation 2009;119:922–930.

134.

Davies JM. Cancer and Date of Birth. BMJ 1963;2:1535.

135.

Dijkstra BKS. Origin of carcinoma of the bronchus. J Natl Cancer Inst 1963;31:511–519.

136.

Furberg CD, Byington RP. What do subgroup analyses reveal about differential response to beta-blocker therapy? The Beta-Blocker Heart Attack Trial experience. Circulation 1983;67:I98–101.

137.

Furberg CD, Hawkins CM, Lichstein E. Effect of propranolol in postinfarction patients with mechanical or electrical complications. Circulation 1984;69:761–765.

138.

Goudie RB. The Birthday Fallacy and Statistics of Icelandic Diabetes. Lancet 1981;318:1173.

139.

Helgason T, Jonasson MR. Evidence For A Food Additive As A Cause Of Ketosis-Prone Diabetes. Lancet 1981;318:716–720.

140.

Ingelfinger JA, Mosteller F, Thibodeau LA, Ware JH. Biostatistics in clinical medicine. New York, MacMillan, 1983.

141.

ISIS-2 Collaborative Group. Randomised Trial Of Intravenous Streptokinase, Oral Aspirin, Both, Or Neither Among 17 187 Cases Of Suspected Acute Myocardial Infarction: ISIS-2. Lancet 1988;332:349–360.

142.

Kaul S, Diamond GA. Trial and Error: How to Avoid Commonly Encountered Limitations of Published Clinical Trials. J Am Coll Cardiol 2-2-2010;55:415–427.

143.

Lagakos SW. The Challenge of Subgroup Analyses—Reporting without Distorting. N Engl J Med 2006;354:1667–1669.

144.

Lee KL, McNeer JF, Starmer CF, et al. Clinical judgment and statistics. Lessons from a simulated randomized trial in coronary artery disease. Circulation 1980;61:508–515.

145.

Multicentre International Study. Improvement in prognosis of myocardial infarction by long-term beta-adrenoreceptor blockade using practolol. A multicentre international study. BMJ 1975;3:735–740.

146.

Packer M, O’Connor CM, Ghali JK, et al. Effect of Amlodipine on Morbidity and Mortality in Severe Chronic Heart Failure. N Engl J Med 1996;335:1107–1114.

147.

Packer M, Carson P, Elkayam U, et al. Effect of Amlodipine on the Survival of Patients With Severe Chronic Heart Failure Due to a Nonischemic Cardiomyopathy: Results of the PRAISE-2 Study (Prospective Randomized Amlodipine Survival Evaluation 2). JACC: Heart Fail 2013;1:308–314.

148.

Pfeffer MA, Jarcho JA. The Charisma of Subgroups and the Subgroups of CHARISMA. N Engl J Med 2006;354:1744–1746.

149.

Pocock SJ, Assmann SE, Enos LE, Kasten LE. Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Statist Med 2002;21:2917–2930.

150.

Simon R. Patient subsets and variation in therapeutic efficacy. Br J Clin Pharmacol 1982;14:473–482.

151.

Thackray S, Witte K, Clark AL, Cleland JG. Clinical trials update: OPTIME-CHF, PRAISE-2, ALLHAT. Euro J Heart Fail 2000;2:209–212.

152.

The ACCORD Study Group. Effects of Intensive Blood-Pressure Control in Type 2 Diabetes Mellitus. N Engl J Med 2010;362:1575–1585.

153.

Wang R, Lagakos SW, Ware JH, et al. Statistics in medicine—Reporting of subgroup analyses in clinical trials. N Engl J Med 2007;357:2189–2194.

154.

Wedel H, DeMets D, Deedwania P, et al. Challenges of subgroup analyses in multinational clinical trials: experiences from the MERIT-HF trial. Am Heart J 2001;142:502–511.

155.

Wittes J: On Looking at Subgroups. Circulation 2009;119:912–915.

156.

Yusuf S, Wittes J, Probstfield J, Tyroler HA. Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. JAMA 1991;266:93–98.

157.

Carson P, Ziesche S, Johnson G, Cohn JN. Racial differences in response to therapy for heart failure: Analysis of the vasodilator-heart failure trials. J Card Fail 1999;5:178–187.

158.

Cohn JN, Archibald DG, Francis GS, et al. Veterans Administration Cooperative Study on Vasodilator Therapy of Heart Failure: influence of prerandomization variables on the reduction of mortality by treatment with hydralazine and isosorbide dinitrate. Circulation 1987;75:IV49-IV54.

159.

Franciosa JA, Taylor AL, Cohn JN, et al. African-American Heart Failure Trial (A-HeFT): Rationale, design, and methodology. J Card Fail 2002;8:128–135.

160.

Taylor AL, Ziesche S, Yancy C, et al. Combination of isosorbide dinitrate and hydralazine in blacks with heart failure. N Engl J Med 2004;351:2049–2057.

161.

Hennekens CH, DeMets D. The need for large-scale randomized evidence without undue emphasis on small trials, meta-analyses, or subgroup analyses. JAMA 2009;302:2361–2362.

162.

Sedgwick P. Randomised controlled trials: subgroup analyses. BMJ 2014;349.

163.

Peto R, Pike MC, Armitage P, et al. Design and analysis of randomized clinical trials requiring prolonged observation of each patient. II. Analysis and examples. Br J Cancer 1977;35:1.

164.

Miller RG. Simultaneous Statistical Inference, ed 2nd. Springer New York, 2011.

165.

Hochberg Y. A sharper Bonferroni procedure for multiple tests of significance. Biometrika 1988;75:800–802.MathSciNetMATH

166.

Holm S. A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian Journal of Statistics 1979;6:65–70.MathSciNetMATH

167.

Blackwelder WC. “Proving the null hypothesis” in clinical trials. Control Clin Trials 1982;3:345–353.

168.

Califf RM. A perspective on the regulation of the evaluation of new antithrombotic drugs. Am J Cardiol 1998;82:25P–35P.

169.

D’Agostino RB, Massaro JM, Sullivan LM. Non-inferiority trials: design concepts and issues—the encounters of academic consultants in statistics. Statist Med 2003;22:169–186.

170.

Diamond GA, Kaul S. An Orwellian discourse on the meaning and measurement of noninferiority. Am J Cardiol 2007;99:284–287.

171.

Ellenberg SS, Temple R. Placebo-controlled trials and active-control trials in the evaluation of new treatments. Part 2: practical issues and specific cases. Ann Intern Med 2000;133:464–470.

172.

Fleming TR. Current issues in non-inferiority trials. Statist Med 2008;27:317-332.MathSciNet

173.

Hasselblad V, Kong DF. Statistical methods for comparison to placebo in active-control trials. Drug Inf J 2001;35:435–449.

174.

James Hung HM, Wang SJ, Tsong Y, et al. Some fundamental issues with non-inferiority testing in active controlled trials. Stat Med 2003;22:213–225.

175.

Hung HMJ, Wang S, O’Neill R. A Regulatory Perspective on Choice of Margin and Statistical Inference Issue in Non-inferiority Trials. Biom J 2005;47:28–36.MathSciNet

176.

Kaul S, Diamond GA, Weintraub WS. Trials and tribulations of non-inferiority: the ximelagatran experience. J Am Coll Cardiol 2005;46:1986–1995.

177.

Kaul S, Diamond GA. Good enough: a primer on the analysis and interpretation of noninferiority trials. Ann Intern Med 2006;145:62–69.

178.

Kaul S, Diamond GA. Making sense of noninferiority: a clinical and statistical perspective on its application to cardiovascular clinical trials. Prog Cardiovasc Dis 2007;49:284–299.

179.

Koch A, Röhmel J: The impact of sloppy study conduct on noninferiority studies. Drug Inf J 2002;36:3–6.

180.

Piaggio G, Elbourne DR, Altman DG, et al., CONSORT Group. Reporting of noninferiority and equivalence randomized trials: An extension of the consort statement. JAMA 2006;295:1152–1160.

181.

Pocock SJ, Ware JH. Translating statistical findings into plain English. Lancet 2009;373:1926–1928.

182.

Siegel JP. Equivalence and noninferiority trials. Am Heart J 2000;139:S166-S170.

183.

SPORTIF Executive Steering Committee for the SPORTIF. Ximelagatran vs warfarin for stroke prevention in patients with nonvalvular atrial fibrillation: A randomized trial. JAMA 2005;293:690–698.

184.

Cui L, Hung HMJ, Wang SJ. Modification of Sample Size in Group Sequential Clinical Trials. Biometrics 1999;55:853–857.MATH

185.

Altman DG, Dore CJ. Randomisation and baseline comparisons in clinical trials. Lancet 1990;335:149–153.

186.

DeMets DL. Methods for combining randomized clinical trials: Strengths and limitations. Statist Med 1987;6:341–348.

187.

Goodman SN. Meta-analysis and evidence. Control Clin Trials 1989;10:188–204.

188.

Hennekens CH, Buring JE, Hebert PR. Implications of overviews of randomized trials. Statist Med 1987;6:397–402.

189.

Meinert CL. Meta-analysis: Science or religion? Control Clin Trials 1989;10:257–263.

190.

Peto R. Why do we need systematic overviews of randomized trials? (Transcript of an oral presentation, modified by the editors). Statist Med 1987;6:233–240.

191.

Sacks HS, Berrier J, Reitman D, et al. Meta-Analyses of Randomized Controlled Trials. N Engl J Med 1987;316:450–455.

192.

Simon R. The role of overviews in cancer therapeutics. Statist Med 1987;6:389-393.

193.

Yusuf S. Obtaining medically meaningful answers from an overview of randomized clinical trials. Statist Med 1987;6:281–286.

194.

Cochran WG. Some Methods for Strengthening the Common χ² Tests. Biometrics 1954;10:417–451.MathSciNetMATH

195.

Mantel N, Haenszel W. Statistical Aspects of the Analysis of Data From Retrospective Studies of Disease. J Natl Cancer Inst 1959;22:719–748.

196.

Berlin JA, Laird NM, Sacks HS, Chalmers TC. A comparison of statistical methods for combining event rates from clinical trials. Statist Med 1989;8:141–151.

197.

Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to Meta-Analysis. Wiley, 2011.

198.

Brand R, Kragt H. Importance of trends in the interpretation of an overall odds ratio in the meta-analysis of clinical trials. Statist Med 1992;11:2077–2082.

199.

Carroll RJ, Stefanski LA. Measurement error, instrumental variables and corrections for attenuation with applications to meta-analyses. Statist Med 1994;13:1265–1282.

200.

DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986;7:177-188.

201.

Galbraith RF. A note on graphical presentation of estimated odds ratios from several clinical trials. Statist Med 1988;7:889–894.

202.

Hedges LV, Olkin I. Statistical Methods for Meta-analysis. Academic Press, 1985.

203.

Pocock SJ, Hughes MD. Estimation issues in clinical trials and overviews. Statist Med 1990;9:657–671.

204.

Stangl D, Berry DA. Meta-Analysis in Medicine and Health Policy. Taylor & Francis, 2000.

205.

Thompson SG. Meta-analysis of clinical trials; in Armitage P, Colton T (eds): Encyclopedia of Biostatistics. New York, Wiley, 1998, pp 2570–2579.

206.

Whitehead A. Meta-Analysis of Controlled Clinical Trials. Wiley, 2003.

207.

Whitehead A, Whitehead J. A general parametric approach to the meta-analysis of randomized clinical trials. Statist Med 1991;10:1665–1677.

208.

Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions. Wiley, 2008.

209.

Liberati A, Altman DG, Tetzlaff J, et al. The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. Ann Intern Med 2009;151:W–65.

210.

Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. Ann Intern Med 2009;151:264–269.

211.

Baum ML, Anish DS, Chalmers TC, et al. A Survey of Clinical Trials of Antibiotic Prophylaxis in Colon Surgery: Evidence against Further Use of No-Treatment Controls. N Engl J Med 1981;305:795–799.

212.

Burzotta F, De Vita M, Gu YL, et al. Clinical impact of thrombectomy in acute ST-elevation myocardial infarction: an individual patient-data pooled analysis of 11 trials. Eur Heart J 2009;30:2193–2203.

213.

Canner PL. Aspirin in coronary heart disease. Comparison of six clinical trials. Isr J Med Sci1983;19:413–423.

214.

Chalmers TC, Matta RJ, Smith H, Kunzler AM. Evidence Favoring the Use of Anticoagulants in the Hospital Phase of Acute Myocardial Infarction. N Engl J Med 1977;297:1091–1096.

215.

Hennekens CH, Buring JE, Sandercock P, et al. Aspirin and other antiplatelet agents in the secondary and primary prevention of cardiovascular disease. Circulation 1989;80:749–756.

216.

Himel HN, Liberati A, Gelber RD, Chalmers TC. Adjuvant chemotherapy for breast cancer: A pooled estimate based on published randomized control trials. JAMA 1986;256:1148–1159.

217.

Kotecha D, Holmes J, Krum H, et al. Efficacy of ß blockers in patients with heart failure plus atrial fibrillation: an individual-patient data meta-analysis. Lancet 2014;384:2235–2243.

218.

May GS, Furberg CD, Eberlein KA, Geraci BJ. Secondary prevention after myocardial infarction: A review of short-term acute phase trials. Prog Cardiovasc Dis 1983;25:335–359.

219.

Wang PH, Lau J, Chalmers TC. Meta-analysis of effects of intensive blood-glucose control on late complications of type I diabetes. Lancet 1993;341:1306–1309.

220.

Yusuf S, Peto R, Lewis J, et al. Beta blockade during and after myocardial infarction: An overview of the randomized trials. Prog Cardiovasc Dis 1985;27:335–371.

221.

Yusuf S, Collins R, Peto R, et al. Intravenous and intracoronary fibrinolytic therapy in acute myocardial infarction: overview of results on mortality, reinfarction and side-effects from 33 randomized controlled trials. Eur Heart J 1985;6:556–585.

222.

Bailey KR: Inter-study differences: How should they influence the interpretation and analysis of results? Statist Med 1987;6:351–358.

223.

Berlin JA, Begg CB, Louis TA. An Assessment of Publication Bias Using a Sample of Published Clinical Trials. J Am Stat Assoc 1989;84:381–392.

224.

Chalmers TC, Frank CS, Reitman D. Minimizing the three stages of publication bias. JAMA 1990;263:1392–1395.

225.

Chalmers TC, Levin H, Sacks HS, et al. Meta-analysis of clinical trials as a scientific discipline. I: Control of bias and comparison with large co-operative trials. Statist Med 1987;6:315–325.

226.

Collins R, Gray R, Godwin J, Peto R. Avoidance of large biases and large random errors in the assessment of moderate treatment effects: The need for systematic overviews. Statist Med 1987;6:245–250.

227.

Furberg CD, Morgan TM. Lessons from overviews of cardiovascular trials. Statist Med 1987;6:295–303.

228.

Furberg CD. Lipid-lowering trials: Results and limitations. Am Heart J 1994;128:1304–1308.

229.

Goldman L, Feinstein AR. Anticoagulants and Myocardial Infarction. The Problems of Pooling, Drowning, and Floating. Ann Intern med 1979;90:92–94.

230.

Johnson RT, Dickersin K. Publication bias against negative results from clinical trials: three of the seven deadly sins. Nat Clin Pract Neurol 2007;3:590–591.

231.

Simes RJ. Confronting publication bias: A cohort design for meta-analysis. Statist Med 1987;6:11–29.

232.

Thompson SG. Systematic Review: Why sources of heterogeneity in meta-analysis should be investigated. BMJ 1994;309:1351–1355.

233.

Wittes RE. Problems in the medical interpretation of overviews. Statist Med 1987;6:269-276.

234.

Svilaas T, Vlaar PJ, van der Horst IC, et al. Thrombus Aspiration during Primary Percutaneous Coronary Intervention. N Engl J Med 2008;358:557–567.

235.

Bavry AA, Kumbhani DJ, Bhatt DL. Role of adjunctive thrombectomy and embolic protection devices in acute myocardial infarction: a comprehensive meta-analysis of randomized trials. Eur Heart J 2008;29:2989–3001.

236.

Costopoulos C, Gorog DA, Di Mario C, Kukreja N. Use of thrombectomy devices in primary percutaneous coronary intervention: A systematic review and meta-analysis. Int J Cardiol 2013;163:229–241.

237.

De Luca G, Navarese EP, Suryapranata H. A meta-analytic overview of thrombectomy during primary angioplasty. Int J Cardiol 2013;166:606–612.

238.

Kumbhani DJ, Bavry AA, Desai MY, et al. Role of Aspiration and Mechanical Thrombectomy in Patients With Acute Myocardial Infarction Undergoing Primary Angioplasty: An Updated Meta-Analysis of Randomized Trials. J Am Coll Cardiol 2013;62:1409–1418.

239.

Mongeon FP, Bélisle P, Joseph L, et al. Adjunctive Thrombectomy for Acute Myocardial Infarction: A Bayesian Meta-Analysis. Circ Cardiovasc Interv 2010;3:6–16.

240.

Tamhane U, Chetcuti S, Hameed I, et al. Safety and efficacy of thrombectomy in patients undergoing primary percutaneous coronary intervention for Acute ST elevation MI: A Meta-Analysis of Randomized Controlled Trials. BMC Cardiovasc Disord 2010;10:10.

241.

Lagerqvist B, Fröbert O, Olivecrona GrK, et al. Outcomes 1 Year after Thrombus Aspiration for Myocardial Infarction. N Engl J Med 2014;371:1111–1120.

242.

Kumbhani DJ, Bavry AA, Desai MY, et al. Aspiration thrombectomy in patients undergoing primary angioplasty: Totality of data to 2013. Cathet Cardiovasc Intervent 2014;84:973–977.

243.

Gordon D, Taddei-Peters W, Mascette A, et al. Publication of Trials Funded by the National Heart, Lung, and Blood Institute. N Engl J Med 2013;369:1926–1934.

244.

Chalmers TC. Randomization of the first patient. Med Clin North Am 1975;59:1035–1038.

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_19

19. Closeout

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

The closeout phase starts with the final follow-up visit of the first participant enrolled and lasts until all analyses have been completed. It is evident that well before the scheduled end of the trial, there needs to be a detailed plan for this phase if the study is to be completed in an orderly manner. Importantly, one must be prepared to implement or modify this plan prior to the scheduled termination since unexpected trial results, either beneficial or harmful, may require the trial to be stopped early.

This chapter addresses a number of topics on the closeout process. Although many of them relate primarily to large single-center or multicenter trials, they also apply to smaller studies. The topics discussed include technical procedures for the termination of the trial, cleanup and verification of data, dissemination of trial results, storage of study material, and post-study follow-up. Obviously, the details of the closeout plan have to be tailored to each particular trial.

Fundamental Point

The closeout of a clinical trial is usually a fairly complex process that requires careful planning if it is to be accomplished in an orderly and effective fashion.

Termination Procedures

Planning

Many details of closeout will depend on factors that only become known once the trial is underway or participant enrollment is completed. Nevertheless, general planning for the closeout ought to start early. There are arguments for initiating this process on day one of the trial. Data management processes can be optimized for rapid database finalization at the end the trial. One major issue is that the trial may not continue through its scheduled termination. Greater-than-expected benefit or unexpected harm may lead to early termination. A more subtle reason is that developing plans for closeout after the trial is well underway may be interpreted by the blinded investigators as a signal of imminent trial termination. Thus, another recommendation is to develop the general closeout plans prior to the first meeting of the data monitoring committee [1].

The closeout phase needs its own written protocol or operating procedures with respect to termination activities, dissemination of results, and data cleanup and storage. The literature on the topic of closeout is scant but there are a few good descriptions of the process [2].

Scheduling of Closeout Visits

If each participant in a clinical trial is to be followed for a fixed period of time, the closeout phase will be of the same duration as the enrollment phase. If recruitment took 2 years, the closeout phase would last 2 years. This fixed follow-up design may not be desirable, since terminating the follow-up of some participants while others are still being actively followed can create problems. In some blinded trials, the code for each participant is broken at the last scheduled follow-up visit. If the unblinding must occur over a span of many months or years, there is the possibility of the investigator learning information that could suggest the identity of the drugs taken by participants still actively followed in the trial. This may happen even if drug codes are unique for each participant. The investigator may start associating a certain symptom or constellation of symptoms and signs with particular drug codes.

An alternative and frequently used plan involves following all participants to a shortened closeout period to avoid the problems described above. Another advantage of this design is the added power of the trial and more information about the effects of longer intervention. The follow-up period is extended beyond the minimum time for all but the last participant enrolled. In a trial with 2 years of uniform recruitment, the additional follow-up period would increase by an average of up to 1 year. In addition, this approach might be more cost-efficient when clinic staff is supported solely by the sponsor of the trial. With all participants followed to a shortened closeout period, full support of personnel can be justified until all participants have been seen for the last time. In trials where the participants are phased out after a fixed time of follow-up, an increase in the staff/participant ratio may be unavoidable.

Despite the problems with following all participants for a fixed length of time, this approach may be preferable in certain trials, particularly those with a relatively short follow-up phase and when the effect of the intervention is believed to be restricted to a short period of time. In such studies, there may be no realistic alternative. In addition, it may not be logistically feasible to conduct a large number of closeout visits in a short time. Depending on the extent of data collection at the last visit and availability of staff and weekly clinic hours, seeing 100–150 participants at a clinic may require a month or 2. A decision on the type of follow-up plan should be based on the scientific question as well as logistics.

Final Response Ascertainment

At trial termination, it is important in any trial to obtain, to the extent possible, response variable data on every enrolled participant. It is particularly so in trials where the main response variables are continuous, such as laboratory data or a performance measure. By necessity, the response variable data must be obtained for each participant at the last follow-up visit because it marks the end of treatment and follow-up. If the participant fails to show up for the last visit, the investigator will have missing data. When the response variable is the occurrence of a specific event, such as a nonfatal stroke or death, the situation may be different if the information can be obtained without having the participant complete a visit.

If a participant suffers an event after his or her last follow-up visit, but before all participants have been seen for the final visit, the study must have a firm a priori rule as to whether that response variable should be included in the data analysis. For the participants who complete their participation, the simplest solution is to let the last follow-up visit denote each participant’s termination of the trial. For participants who do not show up for the last visit, the investigator has to decide when to make the final ascertainment. If death is a response variable, vital status is usually determined as of the last day that the participant was eligible to be seen. The counting rule must be clearly specified in the study protocol or the manual of procedures.

Another approach is to have a common cut-off date (for example, the date of the first planned final follow-up visit). A common cut-off date—such as was used in the Apixaban for Reduction in Stroke and Other Thromboembolic Events in Atrial Fibrillation (ARISTOTLE) trial [3]—may be an advantage for clear definitions in the statistical analysis and for dealing with participants who may never appear for their final visits. In favor of a variable cut-off date (used in most trials) is that including all events until the last participant’s final visit allows maximal capture of the exposure period and therefore optimizes the precision of the estimate of the intervention effect. It is possible that the timing of the final visit could be affected by assignment to intervention or control and therefore could create bias when a variable cut-off date is used, although this would likely be small.

A number of means have been used to track participants and to determine their vital status. These include the use of a person’s identification number (e.g., Social Security number in the United States) or contact with relatives, employers, or health care providers. It has been discovered that participants have died through searching obituaries. Electronic medical records and various other electronic databases can be searched, with appropriate permissions. In countries with national death registries, including the United States, mortality surveillance is simpler and probably more complete than in countries without such registries. Agencies that specialize in locating people have been used in several trials. As has been used in many trials, the Digitalis Investigation Group trial [4] used a search agency, but the searches were limited to records only. It used directory assistance, credit header reports, property records, obituary searches, database mailing lists for magazine subscriptions, and other similar means. No personal contact was allowed. These constraints probably limited the success of finding participants lost to follow-up. This process can be sensitive, since a search may be looked upon as an intrusion into the privacy of the participant. The integrity of a trial and the importance of its results plus the participant’s initial agreement to participate in the trial have to be weighed against a person’s right to protect his or her privacy. Investigators should consider including in the informed consent form a sentence stating that the participant agrees to have her vital status determined at the end of the trial even if he or she has by then stopped participating actively or withdrawn general consent. It pays to initiate the process of obtaining information on vital status on inactive participants well in advance of the closeout phase.

The uncertainty of the overall results rises as the number of participants for whom response variable data are missing at trial termination increases. For example, assume that death from any cause is the primary response variable in a trial and the observed mortality is 15% in one group and 10% in the other group. Depending on study size, this group difference might be statistically significant. However, if 10% of the participants in each group were lost to follow-up, the observed outcome of the trial may be in question.

The Anti-Xa Therapy to Lower Cardiovascular Events in Addition to Standard Therapy in Subjects with Acute Coronary Syndrome–Thrombolysis in Myocardial Infarction 51 (ATLAS ACS 2–TIMI 51) trial of rivaroxaban following acute coronary syndromes [5] highlights the importance of completeness of follow-up data. The trial had a primary outcome of cardiovascular death, myocardial infarction, and stroke, using an on-treatment plus 30-day analysis with an intention-to-treat sensitivity analysis. In the original data from which the primary manuscript was published and the first United States Food and Drug Administration (FDA) filing was made [5], 1509 (or about 10%) of 15,526 participants had incomplete follow-up, and 799 participants had incomplete follow-up limiting the observation period of up to 30 days after early discontinuation. For the primary analysis, there was a 1.8% absolute and a 16% relative risk reduction with rivaroxaban (p = 0.008) that was counterbalanced by more major bleeding. There was a 0.8% lower mortality with rivaroxaban than placebo (p = 0.04) in the primary analysis. The FDA review raised important issues about missing data [6]. First, when the FDA declared that the 10% missing data made it impossible to interpret the mortality data, the sponsor was able to go back to the sites and establish vital status for 843 of 1338 patients with a missing status at the end of the trial, showing that it was possible to have more complete follow-up with more intense effort. When this was done, 22 additional rivaroxaban and 9 additional placebo participants were found to have died, and the p-value increased from 0.045 to 0.076 (for the main “stratum 2” with background thienopyridine therapy). Secondly, it was more common to have missing data in the rivaroxaban arm than in the placebo arm, raising further questions about interpretability. Third, there were three sites in India with questionable data that could neither be verified nor proven to be fraud, which raised additional questions about whether to exclude these data. To address the potential impact of the missing data with regard to mortality, the FDA presented a variety of imputation scenarios to address the missing data in stratum 2 [7] (see Table 19.1). Using the conservative approach of assuming the same mortality for all patients with missing data, the p-values increased from 0.076 to 0.100.

Table 19.1

ATLAS ACS 2–TIMI 51 trial mortality data with various imputations (adapted from an FDA slide presentation at a January 16, 2014 Cardiovascular and Renal Drugs Advisory Committee meeting [7])

Imputed mortality rate used for missing data

Mortality rates when applied (placebo vs. rivaroxaban) (%)

Additional deaths imputed

Hazard ratio (95 % confidence interval)

Nominal p-value

No imputation

3.80 vs. 3.20

0

0.85 (0.71–1.02)

0.076

Observed rate for each treatment group

3.80 vs. 3.20

5 vs. 11

0.85 (0.71–1.02)

0.087

Pooled rate for all participants

3.40 vs. 3.40

5 vs. 12

0.86 (0.72–1.03)

0.093

Placebo rate

3.80 vs. 3.80

5 vs. 13

0.86 (0.72–1.03)

0.100

Another example of the problem of participants being lost to follow-up, and specifically of withdrawal of consent, comes from the Comparison of Medical Therapy, Pacing, and Defibrillation in Heart Failure (COMPANION) trial [8] of defibrillator versus pacemaker versus best medical care. Withdrawal of consent was four times higher in the medical care group than in the other two groups when the trial was terminated and follow-up ended. At a recommendation by the data safety and monitoring board, the investigators approached the participants who had withdrawn their consent and obtained their permission to collect data on vital status and hospitalizations retrospectively for the duration of the trial. This was done at a substantial extra cost and loss of time and stresses the importance of prevention of withdrawal of consent.

It is a mistaken concept that when a participant goes off study medication or intervention that he or she is out of the study and thus no longer followed, or at least not followed beyond some short period of time such as 7 days and 30 days. In the Adenomatous Polyp Prevention on Vioxx (APPROVe) study, participants who stopped their study medication (rofecoxib) due to adverse effects and other reasons were not followed beyond 14 days of going off drug [9]. In the re-analysis, the problem with this “informative censoring” was revealed and a full extra year of follow-up of all randomized participants after stopping study treatment was added. This analysis suggested that the excess of drug-induced major cardiovascular events observed during rofecoxib treatment continued to increase during the first year after treatment was stopped. The adjusted hazard ratio for the extra year was 1.41 (95% confidence interval [CI] 0.77–2.59), in addition to the hazard ratio of 2.12 (95% CI 1.20–3.74) on treatment and during the following 14 days.

The National Academy of Sciences, prompted by FDA officials, has published a comprehensive statement concerning missing data in clinical trials with a focus on phase III confirmatory trials [10]. The report states that “there is no ‘foolproof’ way to analyze data subject to substantial amounts of missing data; that is, no method recovers the robustness and unbiasedness of estimates derived from randomized allocation of treatments. Hence, the panel’s first set of recommendations emphasizes the role of design and trial conduct to limit the amount and impact of missing data” [10]. We stress the need to have systems in place from the beginning of the trial to minimize missing response variables, which for phase III trials begins with carefully structuring the informed consent to allow follow-up, at least for vital status, even if a participant otherwise withdraws from study procedures.

Transfer of Post-trial Care

Termination of a long-term study can be difficult due to the bonding that often develops between participants and clinic staff. The final visit needs to be carefully planned to deal not only with this issue, but also with the need in many trials to inform the participants of which medication they were on (in a blinded study), their individual study data, and the overall study findings (often at a later time). Referral of the participant to a regular source of medical care is another important issue (see Chap. 2).

If the closeout is extended over a long period, as it would be if each participant were followed for the same duration, any early recommendation to an individual participant would have to be based on incomplete follow-up data, which may not reflect the final conclusions of the trial. Moreover, any information given could “leak” to participants still actively treated, thus affecting the integrity of the trial. Although it is highly desirable to provide each participant with a recommendation regarding continued treatment, doing so may not be possible until the study is completely over and the trial results have been published. When unblinding occurs over a span of months or years, the investigator is in the uncomfortable position of ending a participant’s involvement in the trial and asking him or her to wait months before being told the study results and being advised about what to do. On the other hand, if the incomplete results are clear cut, it would be easy to arrive at such recommendations. However, in such an instance, the investigator would be confronted with an ethical dilemma. How can the investigator recommend that a participant start, continue, or discontinue a new intervention while keeping other participants active in the trial? For this reason, we generally prefer a shortened period of trial closeout.

Data and Other Study Material

Cleanup and Verification

Verification of data may be time-consuming, and it can conflict with the desire of the investigator to publish his findings as early as possible. While publication of important information should not be delayed unnecessarily, results should not be put into print before key data have been verified. Despite attempts to collect complete, consistent, and error-free data, perfection is unlikely to be achieved. Traditional monitoring systems are likely to reveal missing forms, unanswered items on forms, and conflicting data. In isolated cases, they may also uncover falsification of individual data [11, 12] and, in the worst cases, fabrication of all data on fictitious participants [13–15]. Data cleanup and verification typically continue for months after completion of the closeout visits, although the use of electronic records has substantially reduced the burden of this cleanup and verification. It is necessary to be realistic in the cleanup process. This means “freezing” and “locking” the files at a reasonable time after the termination of participant follow-up and accepting some incomplete data. Obviously, the efforts during cleanup should be directed toward the most critical areas—those crucial to answering the primary questions and serious adverse effects.

We strongly recommend that study forms and data be continuously monitored throughout a trial as pointed out in Chap. 11. Data editing should be initiated as soon as possible, because it is difficult to get full staff cooperation after a trial and its funding are over. Early monitoring may reveal systematic problems that can be corrected. Staff feedback is also important. Approaches for statistical process control audits are now available, and they have been shown to reduce the overall database error rates significantly [16].

Any clinical trial may be faced with having its results reviewed, questioned, and even audited. Traditionally, this review has been a scientific one. However, since regulatory and special interest groups may want to look at the data, the key results should be properly verified, documented, and filed in an easily retrievable manner. The extent of this additional documentation of important data will depend on the design of each trial. Electronic data provide verification opportunities that are more efficient than paper records, but storage remains important. Various models have been used. In one multicenter study, the investigators were asked at the end of follow-up to send a list of all deceased participants along with date of death to an office independent of the data coordinating center. In other trials, key data were independently audited before the results were published. Common to all models is an attempt to maintain credibility.

Procedures for data cleanup and verification in trials conducted for regulatory approval add substantially to the trial cost and complexity. Many such trials collect a large quantity of data. Final verification of these data is both time-consuming and costly [17, 18]. As noted in Chap. 11, investigators should, when designing such trials, both limit the amount of data and decide which data are essential and require full final verification.

Storage

Investigators should consider storing various kinds of material after a trial has ended. One set of documents—such as the trial protocol, manual of procedures, study forms, and the analytic material, including electronic records—should be kept by the investigator and sponsor. In addition, a list containing identifying information for all participants who enrolled in a trial ought to be stored at the institution where the investigation took place. Local regulations sometimes require that individual participant data such as copies of study forms, laboratory reports, electrocardiograms, and X-rays be filed for a defined period of time with the participants’ medical records. Storage of these data electronically clearly eases the problem of inadequate space. The actual trial results and their interpretation should be published and then can be retrieved through a library search, although it is all-to-common for trial results to remain unpublished [19]. As of 2012, less than two-thirds of National Heart, Lung, and Blood Institute (NHLBI)-sponsored clinical trials were published within 30 months [20]. Recognition of this major problem, pressure from sponsors and the clinical trial community to publish all trial results, and transparency and data sharing are all important steps to dealing with lack of publication (see Chap. 20).

In planning for a new trial, an investigator may want to obtain unpublished data from other investigators who have conducted trials in a similar population or tested the same intervention. Tables and figures in actual manuscripts seldom include everything that may be of interest. The situation is changing with online material available on journal websites. Many journals now publish full protocols, forms, manuals, and even raw data [21]. However, no uniform mechanism exists today for getting access to such study material from terminated trials. If information is available, it may not be in a reasonable and easily retrievable form. Substantial cooperation is usually required from the investigators originally involved in the data collection and analysis [22], and standards for data sharing and open access to trial data are evolving [23] (see Chap. 20).

The storage of biological material has raised new issues as it relates to genetic analyses. Biospecimens from well-characterized populations followed for long durations in clinical trials are in demand. These can be used to determine whether participant subgroups with a specific genotype are more likely to benefit or to experience serious adverse effects. The availability of these specimens for specific analysis depends on the wording of the informed consent (see Chap. 2). Patient privacy has to be considered, as always.

Storage of biomaterials may be costly. Freezers must be maintained, and a system for labeling and retrieving specimens or aliquots without damaging the remaining material must be implemented. Unlike with retrieval and distribution of data, many specimens may only be used once. Therefore, investigators need to develop a system for deciding when and how to use or distribute biospecimens. The cost and benefit, as well as the duration of storage must be considered. Central specimen repositories have been created to which investigators may be able to send their materials.

In summary, most trials collect an excess of study material, and it may not make sense to store everything. The investigator has to consider logistics, the length of the storage period, and cost. He also has to keep in mind that biological material, for example, deteriorates with time and laboratory methods change.

Dissemination of Results

The reporting of findings from a small single-center trial is usually straightforward. The individual participants are often told about the results shortly after the last follow-up visit, and the medical community is informed through scientific publications. However, there are situations that make the dissemination of findings difficult, especially the order in which the various interested parties are informed. Particularly in multicenter studies where the participants are referred by physicians not involved in the trial, the investigators have an obligation to tell these physicians about the conclusions, preferably before they read about them in the newspaper or are informed by their patients. In trials with clinics geographically scattered, investigators may have to be brought together to learn the results. In certain instances, the sponsoring party has a desire to make the findings known publicly at a press conference or through a press release. However, although an early press conference followed by an article in a newspaper may be politically important to the sponsor of the trial, it may offend the participants, the referring physicians, and the medical community. They may all feel that they have a right to be informed before the results are reported in the lay press. Companies may perceive a fiduciary responsibility to let the public know the “top line” results of a trial once they know them in order to control the risk of leaks.

We have had good experiences from the following sequence. First, the study leadership informs the other investigators who, in turn, inform the participants. Second, the private physicians of the participants are also told of the findings. Third, the results are then published in the scientific literature, after which they may be more widely disseminated in other forums. With journals now being available electronically, publication can be timed to coincide with presentation of the results at major scientific meetings.

However, there are sometimes unavoidable long delays between the presentation of trial findings at a scientific meeting and the publication of the full trial reports in peer-reviewed journals. The medical community may be placed in difficult positions by having to make treatment decisions if the lay press reports on elements of findings many months prior to the publication of the trial data in full. The messages released by the lay press are typically very simple. To minimize this problem, three recommendations have been made [24]: (1) “congress organizers should insist that published abstracts contain sufficient data to justify the conclusions of the presentation,” (2) “investigators should not present results of any study that is likely to influence clinical management until they are in a position to write a full paper,” and (3) “journal editors must be willing…to expedite the publication of such papers.” These recommendations are reasonable, but there may be exceptions.

In order to facilitate expedited translation of research results, the National Institutes of Health (NIH) introduced a data sharing policy in October 2003 [25] that has since been updated [26]. The agency’s position is that “Data should be made as widely and freely available as possible while safeguarding the privacy of participants, and protecting confidential and proprietary data.” The risk of wide dissemination of databases is that other investigators might analyze the data and arrive at different interpretations of the results. However, after a certain period of time has passed to allow for the trial investigators to analyze and publish, further analysis and discussion of various interpretations of trial data are usually scientifically sound and ought to be encouraged.

In special situations, when a therapy of public health importance is found to be particularly effective or harmful in a trial sponsored by the NIH, physicians and the public need to be alerted in a timely manner. The NIH would promptly post a release on its news website [27]. When the Adenoma Prevention with Celecoxib trial sponsored by the National Cancer Institute was terminated due to a 2.5-fold increased risk of major fatal and nonfatal cardiovascular events for participants taking celecoxib compared with those on a placebo, the release was issued the day after the decision was made to stop treatment [28]. Three months later, the results were published in The New England Journal of Medicine [29].

At the NIH, individual institutes may also issue their own press releases. These are generally released to coincide with the publication of an article in a medical journal. However, institutes, with journal permission, have issued brief press announcements prior to journal publication. To avoid criticism from physician groups, an institute may also notify the leadership of relevant medical societies before the release. The United States National Library of Medicine also releases timely scientific news on its MedlinePlus website [30]. These releases are not limited to NIH-sponsored research.

The FDA also informs physicians and the public about regulatory actions and news. FDA MedWatch Safety Alerts for Human Medical Products are posted on the website [31]. Included are brief summaries of products in question and FDA alerts. This and the general FDA drug website [32] provide recommendations and information for health care providers as well as information for patients to consider. If a serious adverse event has been uncovered by investigators in a trial, the FDA and other regulatory agencies or the trial sponsor may communicate this information to medical professionals, and thereby indirectly to the lay public, through a “Dear Healthcare Provider” letter.

Wide dissemination of trial findings to the public by investigators and study sponsors is increasingly common, even if the results are of modest scientific or public health importance. Press releases have become part of highly orchestrated marketing campaigns in both industry- and government-funded trials. We strongly support making trial results, and indeed data, widely available, with the expectation that broad discussion (and reanalysis as appropriate) will assist clinicians and the public in arriving at appropriate decisions as to the value of a trial’s intervention.

As emphasized in Chap. 1, clinical trials must be registered. Worldwide, there are a large number of registries [33–35]. Until the enactment of the FDA Amendments Act (FDAAA) in September 2007, the registration was limited to design information from the trial protocols [35]. The FDAAA expanded the scope to include a trial results database with information on participant demographics and baseline characteristics, primary and secondary outcomes, and statistical analyses. These data should be posted within 12 months of trial completion. The database should also be linked to publically available information from the FDA website. This would include summary safety and effectiveness data, public health advisories, and action packages for drug approval. Serious and frequent adverse effect data observed during a trial are to be added within 2 years.

Post Study Follow-up

There are three main reasons for short-term follow-up after completion of the intervention period. One is to find out how soon treatment-induced changes in laboratory values or symptoms return to pretrial level or status. The effect of the intervention may last long after a drug has been stopped, and abnormalities revealed by laboratory measurements or adverse drug effects may not disappear until weeks after the intervention has ended. Second, for certain drugs, such as beta-blockers and steroids, the intervention should not be stopped abruptly. A tapering of the dosage may require additional clinic visits. Third, clinical events may occur differentially in the study groups after the intervention is stopped due to lingering drug effects or to a hazard in switching patients back to standard of care [36]. Drug effects may be seen for weeks or months after treatment is stopped or there may be unfavorable withdrawal reactions [9]. These activities are separate from the moral obligation of the investigator to facilitate, when necessary, a participant’s return to the usual medical care system, to ensure that study recommendations are communicated to his or her private physician, and at times to continue the participant on a beneficial new intervention.

Long-term post-study follow-up of participants is a rather complex process in most, but not all, countries. The investigators and the sponsor have to decide what should be monitored. Mortality surveillance can be cumbersome globally but can easily be performed in selected regions, for example in Scandinavia. Usually, the justification for long-term post-study surveillance is based on a trend or unexpected finding in the trial or from a finding from another source. Since most clinical outcome trials of chronic therapies are relatively short in duration, extended follow-up can provide important additional information.

Obtaining information on nonfatal events is even more complicated and, in general, its value is questionable. However, a classical illustration that post-study follow-up for harm can prove valuable is the finding of severe adverse effects attributed to diethylstilbestrol. The purported carcinogenic effect occurred 15–20 years after the drug was administered and occurred in female offspring who were exposed in utero [37]. Similarly, use of unopposed estrogen has been reported to be associated with an increased risk of endometrial cancer 15 or more years after therapy was stopped [38]. One article reported an association between in utero exposure to valproate, an antiepileptic drug, and impaired cognitive function in offspring at 3 years of age [39].

In 1978, the results of a trial of clofibrate in people with elevated lipids indicated an excess of cases of cancer in the clofibrate group compared with the control group [40]. The question was raised whether the participants assigned to clofibrate in the Coronary Drug Project also showed an increase in cancer incidence. This was not the case [41]. Only 3% of deaths during the trial were cancer-related. Subsequently, a World Health Organization study of clofibrate reported that all-cause mortality was increased in the intervention group [42]. At the same time, Coronary Drug Project investigators decided that post-study follow-up was scientifically and ethically important, and such a study was undertaken. No increase in cancer incidence was noted in the clofibrate group [43]. A more recent example is the Women’s Health Initiative, which extended follow-up for 5 years after it reached its scheduled termination in 2005. This example brings up a question: should investigators of large-scale clinical trials make arrangements for surveillance in case, at some future time, the need for such a study were to arise? The implementation of any post-study surveillance plan has challenges. A key one is finding a way to keep participants’ names and addresses, or their Social Security or other national identification numbers, in a central registry without infringement upon the privacy of the individuals. The investigator must also decide, with little evidence, on the optimal duration of surveillance after the termination of a trial (e.g., 2, 5, or 20 years).

Another issue of post-study surveillance relates to a possible beneficial effect of intervention. In any trial, assumptions must be made with respect to time between initiation of intervention and the occurrence of full beneficial effect. For many drugs, this so-called “lag time” is assumed to be zero. However, if the intervention is smoking cessation, a lipid-lowering drug, or a dietary change, and if the response variable is coronary mortality, the lag time might be a year or longer. The problem with such an intervention is that the maximum practical follow-up may not be long enough for a beneficial effect to appear. Extended surveillance after completion of active treatment may be considered in such studies. At the scheduled termination of the Multiple Risk Factor Intervention Trial, the results favored the special intervention group over usual care but did not reach statistical significance [44]. Almost 4 years later, a statistically significant effect emerged [45]. A late benefit was also evident in a passive follow-up phase after stopping enalapril versus placebo in the Studies of Left Ventricular Dysfunction (SOLVD) [46].

The post-study surveillance in the Coronary Drug Project [43] showed unexpected benefit in one of the intervention groups. At the conclusion of the trial, the participants assigned to nicotinic acid had significantly fewer nonfatal re-infarctions, but no difference in survival was detected. Total mortality after an average of six-and-a-half years in the trial on drug, plus an additional 9 years after the trial, however, was significantly lower in the group assigned to nicotinic acid than in the placebo group. There are several possible interpretations of the Coronary Drug Project finding. It may be that this observation is real, and that the benefit of nicotinic acid simply took longer than expected to appear. Of course, the results may also be due to chance, a possibility that seems more likely with the lack of benefit and evidence of harm with niacin in the much larger Heart Protection Study 2–Treatment of HDL to Reduce the Incidence of Vascular Events (HPS2-THRIVE) trial [47]. A major difficulty in interpreting the data relates to the lack of knowledge about what the participants in the intervention and control groups did with respect to lipid lowering and other regimens in the intervening 9 years.

Knowledge of the response variable of interest for a substantial portion of participants is required if long-term surveillance after completion of regular follow-up is to be worthwhile. The degree of completeness attainable depends on several factors, such as the response variable itself, the length of surveillance time, the community where the trial was conducted, and the aggressiveness of the investigator. Many of the very large trials have successfully monitored participants (or subsets thereof) after closeout to determine whether behavioral effects of the study intervention have been sustained or participants have adhered to recommendations regarding continued treatment.

References

1.

Shepherd R, Macer JL, Grady D. Planning for closeout--from day one. Contemp Clin Trials 2008;29:136–139.

2.

Pressel SL, Davis BR, Wright JT, et al; ALLHAT Collaborative Research Group. Operational aspects of terminating the doxazosin arm of The Antihypertensive and Lipid Lowering Treatment to Prevent Heart Attack Trial (ALLHAT). Control Clin Trials 2001;22:29–41.

3.

Granger CB, Alexander JH, McMurray JJ, et. al. Apixaban versus warfarin in patients with atrial fibrillation. N Engl J Med 2011;365:981–992.

4.

Collins JF, Howell CL, Horney RA; Digitalis Investigation Group Investigators. Determination of vital status at the end of the DIG trial. Control Clin Trials 2003;24:726–730.CrossRef

5.

Mega JL, Braunwald E, Wiviott SD, et al; ATLAS ACS 2–TIMI 51 Investigators. Rivaroxaban in patients with a recent acute coronary syndrome. N Engl J Med 2012;366:9–19.

6.

U.S. Food and Drug Administration. U.S. Department of Health & Human Services. Briefing Information for the May 23, 2012 Meeting of the Cardiovascular and Renal Drugs Advisory Committee. Available at: http://​www.​fda.​gov/​AdvisoryCommitte​es/​CommitteesMeetin​gMaterials/​Drugs/​Cardiovascularan​dRenalDrugsAdvis​oryCommittee/​ucm304754.​htm. Accessed January 13, 2015.

7.

Bai S, Grant S, Marciniak T; U.S. Food and Drug Administration. Rivaroxaban for ACS sNDA 202439/S-002. Cardiovascular and Renal Drugs Advisory Committee Meeting, January 16, 2014 slide presentation. http://​www.​fda.​gov/​downloads/​AdvisoryCommitte​es/​CommitteesMeetin​gMaterials/​Drugs/​Cardiovascularan​dRenalDrugsAdvis​oryCommittee/​UCM386282.​pdf. Accessed January 13, 2015.

8.

Bristow MR, Saxon LA, Boehmer J, et al; Comparison of Medical Therapy, Pacing, and Defibrillation in Heart Failure (COMPANION) Investigators. Cardiac-resynchronization therapy with or without an implantable defibrillator in advanced chronic heart failure. N Engl J Med 2004;350:2140–2150.

9.

Baron JA, Sandler RS, Bresalier RS, et al. Cardiovascular events associated with rofecoxib: final analysis of the APPROVe trial. Lancet 2008;372:1756–1764.CrossRef

10.

Panel on Handling Missing Data in Clinical Trials, Committee on National Statistics, Division of Behavioral and Social Sciences and Education, National Research Council of the National Academies. The Prevention and Treatment of Missing Data in Clinical Trials. National Academies Press, Washington, DC. January 4, 2011.

11.

Fisher B, Redmond CK. Fraud in breast-cancer trials. N Engl J Med 1994;330:1458–1462.CrossRef

12.

Buyse M, George SL, Evans S, et al. The role of biostatistics in the prevention, detection and treatment of fraud in clinical trials. Stat Med 1999;18:3435–3451.CrossRef

13.

Sheldon T. Dutch neurologist found guilty of fraud after falsifying 438 case records. BMJ 2002;325:734.CrossRef

14.

Ross DB. The FDA and the case of Ketek. N Engl J Med 2007;356:1601–1604.

15.

POISE Study Group, Devereaux PJ, Yang H, et al. Effects of extended-release metoprolol succinate in patients undergoing non-cardiac surgery (POISE trial): a randomised controlled trial. Lancet 2008;371:1839–1847 (Webappendix 1).

16.

Rostami R, Nahm M, Pieper CF. What can we learn from a decade of database audits? The Duke Clinical Research Institute experience, 1997--2006. Clin Trials 2009;6:141–150.

17.

Eisenstein EL, Lemons PW 2nd, Tardiff BE, et al. Reducing the cost of phase III cardiovascular clinical trials. Am Heart J 2005;149:482–488.CrossRef

18.

Eisenstein EL, Collins R, Cracknell BS, et al. Sensible approaches for reducing clinical trial costs. Clin Trials 2008;5:75–84.CrossRef

19.

Zarin DA, Tse T, Williams RJ, et al. The ClinicalTrials.gov results database--update and key issues. N Engl J Med 2011;364:852–860.

20.

Gordon D, Taddei-Peters W, Mascette A, et al. Publication of trials funded by the National Heart, Lung, and Blood Institute. N Engl J Med 2013;369:1926–1934.CrossRef

21.

Trials. http://​www.​trialsjournal.​com.

22.

Hrynaszkiewicz I, Altman DG. Towards agreement on best practice for publishing raw clinical trial data. Trials 2009;10:17.CrossRef

23.

Krumholz HM, Peterson ED. Open access to clinical trials data. JAMA 2014;312:1002–1003.CrossRef

24.

Reporting clinical trials: message and medium. Lancet 1994;344:347–348.

25.

National Institutes of Health. Final NIH statement on sharing research data. February 26, 2003. Available at: http://​grants.​nih.​gov/​grants/​guide/​notice-files/​NOT-OD-03–032.​html. Accessed January 13, 2015.

26.

National Institutes of Health. NIH Data Sharing Policy. Available at: http://​grants.​nih.​gov/​grants/​policy/​data_​sharing. Accessed January 13, 2015.

27.

National Institutes of Health. News & Events website. Available at: www.​NIH.​gov/​news. Accessed January 13, 2015.

28.

US Department of Health and Human Services. NIH News website. National Institutes of Health. NIH halts use of COX-2 inhibitor in large cancer prevention trial. December 17, 2004. Available at: http://​www.​nih.​gov/​news/​pr/​dec2004/​od-17.​htm. Accessed January 13, 2015.

29.

Solomon SD, McMurray JJ, Pfeffer MA, et al. Cardiovascular risk associated with celecoxib in a clinical trial for colorectal adenoma prevention. N Engl J Med 2005;352:1071–1080.CrossRef

30.

National Institutes of Health, U.S. National Library of Medicine. MedlinePlus website. Available at: http://​www.​nlm.​nih.​gov/​medlineplus/​news. Accessed January 13, 2015.

31.

U.S. Food and Drug Administration, U.S. Department of Health and Human Services. MedWatch: The FDA Safety Information and Adverse Event Reporting Program. Available at: http://​www.​fda.​gov/​medwatch. Accessed January 13, 2015.

32.

U.S. Food and Drug Administration, U.S. Department of Health and Human Services. FDA Drugs website. Available at: http://​www.​fda.​gov/​Drugs. Access January 13, 2015.

33.

Zarin DA, Ide NC, Tse T, et al. Issues in the registration of clinical trials. JAMA 2007;297:2112–2120.CrossRef

34.

World Health Organization. International Clinical Trials Registry Platform (ICTRP) website. Available at: http://​www.​who.​int/​ictrp/​en/​. Accessed January 13, 2015.

35.

Zarin DA, Tse T. Medicine. Moving toward transparency of clinical trials. Science 2008;319;1340–1342.CrossRef

36.

Granger CB, Lopes RD, Hanna M, et al. Clinical events after transitioning from apixaban versus warfarin to warfarin at the end of the Apixaban for Reduction in Stroke and Other Thromboembolic Events in Atrial Fibrillation (ARISTOTLE) trial. Am Heart J 2015;169:25-30.CrossRef

37.

Herbst AL, Ulfelder H, Poskanzer DC. Adenocarcinoma of the vagina. Association of maternal stilbestrol therapy with tumor appearance in young women. N Engl J Med 1971;284:878–881.CrossRef

38.

Paganini-Hill A, Ross RK, Henderson BE. Endometrial cancer and patterns of use of estrogen replacement therapy: a cohort study. Br J Cancer 1989;59:445–447.CrossRef

39.

Meador KJ, Baker GA, Browning N, et al; NEAD Study Group. Cognitive function at 3 years of age after fetal exposure to antiepileptic drugs. N Engl J Med 2009;360:1597–1605.CrossRef

40.

A co-operative trial in the primary prevention of ischaemic heart disease using clofibrate. Report from the Committee of Principal Investigators. Br Heart J 1978;40:1069–1118.

41.

Clofibrate and niacin in coronary heart disease. JAMA 1975;231:360–381.

42.

W.H.O. cooperative trial on primary prevention of ischaemic heart disease using clofibrate to lower serum cholesterol: mortality follow-up. Report of the Committee of Principal Investigators. Lancet 1980;2:379–385.

43.

Canner PL, Berge KG, Wenger NK, et al. Fifteen year mortality in Coronary Drug Project patients: long-term benefit with niacin. J Am Coll Cardiol 1986;8:1245–1255.CrossRef

44.

Multiple risk factor intervention trial. Risk factor changes and mortality results. Multiple Risk Factor Intervention Trial Research Group. JAMA 1982;248:1465–1477.

45.

Mortality rates after 10.5 years for participants in the Multiple Risk Factor Intervention Trial. Findings related to a priori hypotheses of the trial. The Multiple Risk Factor Intervention Trial Research Group. JAMA 1990;263:1795–1801.

46.

Jong P, Yusuf S, Rousseau MF, et al. Effect of enalapril on 12-year survival and life expectancy in patients with left ventricular systolic dysfunction: a follow-up study. Lancet 2003;361:1843–1848.CrossRef

47.

HPS2-THRIVE Collaborative Group, Landray MJ, Haynes R, et al. Effects of extended-release niacin with laropiprant in high-risk patients. N Engl J Med 2014;371:203–212.

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_20

20. Reporting and Interpreting of Results

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

The final phase in any experiment is to interpret and report the results. Finding the answer to a challenging question is the goal of any research endeavor. Proper communication of the results to clinicians also provides the basis for advances in medicine [1]. To communicate appropriately, investigators have to review their results critically and avoid the temptation to overinterpret benefit or underreport harm. They are in the privileged position of knowing the quality and limitations of the data better than anyone else. Therefore, they have the responsibility for presenting the results clearly and concisely, together with any issues that might bear on their interpretation. Investigators should devote adequate care, time and attention to this critical part of the conduct of clinical trials. We believe that a policy of “conservative” interpretation and reporting best serves science, public health, clinical medicine, and the interests of readers.

A study may be reported in a scientific journal, but publication is in no way an endorsement of its results or conclusions. Even if the journal uses referees to assess each prospective publication, there is no assurance that they have sufficient experience and knowledge of the issues of design, conduct and analysis to judge fully the reported study [2]. Only the investigators are likely to recognize subtle, or even not so subtle, weaknesses and problems. As pointed out over 35 years ago by a former Editor of The New England Journal of Medicine [3],

In choosing manuscripts for publication we make every effort to winnow out those that are clearly unsound, but we cannot promise that those we do publish are absolutely true … Good journals try to facilitate this process [of medical progress] by identifying noteworthy contributions from among the great mass of material that now overloads our scientific communication system. Everyone should understand, however, that this evaluative function is not quite the same thing as endorsement.

This point has been illustrated by Ellenberg et al. [4]. The favorable results of a multicenter trial accompanied by a very positive editorial were published in The New England Journal of Medicine only 2 weeks before an Advisory Committee of the FDA voted unanimously against recommending that the intervention, a respiratory syncytial virus immune globulin, be licensed. A trial showed superiority of cangrelor against clopidogrel in people undergoing percutaneous coronary intervention for the primary outcome of a composite of death, myocardial infarction, revascularization, or stent thrombosis [5]. Despite the apparent benefit, the FDA had concerns about benefit from cangrelor and did not approve the drug [6]. In the end, it is up to the authors to be as objective as possible and the readers of a scientific article to assess it critically and to decide how to make best use of the reported findings.

In this chapter, we discuss guidelines for reporting, interpretation of findings, and publication bias, as well as the answers to three specific questions that should be considered in preparing a report: (1) Did the trial work as planned? (2) How do the findings compare with those from other studies? (3) What are the clinical implications of the findings? A checklist of what should be included in a report of a clinical trial is provided by the Consolidated Standards of Reporting Trials (CONSORT) group [7–13]. Similar guidelines have been prepared for publications of meta-analyses [14]. Included in the CONSORT website [10] is a checklist of essential items. Briefly, it lists study background and objectives, methods (trial design, participants, interventions, primary and secondary outcomes, sample size, randomization procedures, blinding, statistics), results (baseline data, outcomes, harms), and interpretation (limitations, generalizations). Also required are trial registration and funding source.

Fundamental Point

Investigators have an obligation to review their study and its findings critically and to present sufficient information so that readers can properly evaluate the trial and its findings.

Guidelines for Reporting

Any report of a clinical trial should include sufficient information about the study rationale, design, population and conduct, so the readers can assess the adequacy of the methods employed. The quality of a trial is typically judged based on the thoroughness and completeness of the Materials and Methods sections of the report. Unfortunately, thorough reporting does not always occur. A survey of 253 randomized trials published in five general medicine journals after revised CONSORT recommendations found that several aspects (e.g., allocation concealment and various components of blinding) were inadequately discussed [15]. Others [16] have noted that eligibility criteria are sometimes poorly described. Wang et al. [17] conducted a survey of subgroup analyses reported in The New England Journal of Medicine over a 1-year period. Subgroup analyses were common, but highly variable in completeness of information presented. As a result, The Journal implemented guidelines for reporting subgroup analyses [17].

Terms often used in clinical trial reports are misused. Many authors claim that they performed an “intention-to-treat” or “ITT” analysis, when in fact data from randomized participants have been excluded from the analysis. There may be good reasons why not all data are available, but unless the absent data are such a small percentage of the total, such that regardless of what they might show, no change in overall trial outcome could occur, this analysis should not be called intention-to-treat. Readers must look carefully despite claims of an ITT analysis. Sometimes, “modified ITT analysis” is used, which is a contradiction. If not all participants and not all follow-up events are accounted for, the report of the analysis should not say “intention-to-treat.” Some participants might be lost to follow-up. The number (ideally small) of those should be clearly indicated. Another term that is misleading is “per protocol analysis.” Authors use that phrase to apply to analyses that omit data from those who fail to adhere fully to the intervention or otherwise leave the study. We consider this to be an unfortunate use of the term, as it implies that such an analysis is the preferred one specified in the protocol. As we have argued in this book (Chap. 18), it is almost never the preferred analysis and should not be so specified in the protocol. When such an analysis is performed, we prefer the term “on treatment analysis” as it more accurately reflects what is done.

Traditional journals impose page limitations, forcing authors to exclude some important information. On-line journals that do not have such page limitations are becoming more common. In addition, many print journals allow supplemental material (e.g., details of methods, extra data) to be included in their electronic versions. Therefore, space limitations are no longer justification for withholding pertinent information.

As noted above, guidelines on how to report a clinical trial exist [7–14]. The International Committee of Medical Journal Editors has issued a set of uniform requirements that are endorsed by a large number of journals [18]. One of the guidelines is assurance that the trial has been listed in a formal registry [19, 20]. In addition, journals have their Instructions for Authors that address issues on format as well as content.

With the enormous number of scientific articles published annually, it is impossible for clinicians to keep up with the flow of information. Journals to which one subscribes may have online services to help identify articles of particular interest. Other online listings of publications in selected areas to which readers can subscribe may help, but the clinician still has the obligation to review carefully clinical trial publications. More informative abstracts help clinicians who browse through journals on a regular basis. Valid and informative abstracts are important, since clinical decisions are often influenced by abstracts alone [21]. For reporting clinical investigations, many journals have adopted the recommendation [22] for structured abstracts, which include information on objective, design, setting, participants, intervention(s), measurements and main results, and conclusion(s). The early experience of structured abstracts was reviewed by Haynes et al. and comments were “supportive and appreciative.” Those authors recommended some modifications of the guidelines [23]. We strongly endorse the now common use of the structured abstract.

Authorship

Decisions of authorship are both sensitive and important [24, 25]. It is critical that decisions are made at an early stage. Cases of scientific fraud have reminded us that being an author carries certain responsibilities and should not be used as a means to show gratitude. Guidelines regarding qualifications for authorship are included in general instructions for manuscripts [18]. In the past, a number of journals attempted to prohibit group authorship, on the grounds that those taking responsibility for the actual conduct of the trial and the writing of the manuscript ought to be clearly identified. Meinert [26] came to the defense of group authorship and expressed concern over the possible effect of this policy on multicenter work. We believe that group authorship is an important part of clinical trials research. Fairness and equity require proper crediting to those who have made major contributions to the design, conduct, and analysis, not just the few that served on the writing group. A compromise accepted by many journals and recommended by the International Committee of Medical Journal Editors is to allow group authorship but list those who served on the writing committee. A distinction may also be made between “authors” and “collaborators.” Some journals ask about the contributions of each person listed as an author or member of a writing group. If authorship is by a research group name, journal policies may ask that a corresponding author be listed, as well as those who accept responsibility for the paper. The International Committee of Medical Journal Editors [18] states the policy clearly:

Some large multi-author groups designate authorship by a group name, with or without the names of individuals. When submitting a manuscript authored by a group, the corresponding author should specify the group name if one exists, and clearly identify the group members who can take credit and responsibility for the work as authors. The byline of the article identifies who is directly responsible for the manuscript, and MEDLINE lists as authors whichever names appear on the byline. If the byline includes a group name, MEDLINE will list the names of individual group members who are authors or who are collaborators, sometimes called non-author contributors, if there is a note associated with the byline clearly stating that the individual names are elsewhere in the paper and whether those names are authors or collaborators.

“Ghost authorship,” or the failure to properly credit as an author those who wrote or coauthored a manuscript or who otherwise played a major role in the trial such that they deserve notice, has received considerable attention. Gøtzsche and colleagues [27] conducted a survey of 44 industry-initiated trials and found evidence of ghost authorship in three quarters of the publications. Ross et al. [28] describe publications concerning rofecoxib that were written by the industry sponsor’s employees, who were not acknowledged as authors.

The flip side of ghost authorship is “guest authorship,” where usually highly respected investigators who had little or no role in the study or in writing of the manuscript are given visible authorship. We deplore both of these practices.

Duplicate Publication

Journals typically prohibit duplicate publications. They routinely ask manuscript submitters whether the paper has been published or even submitted elsewhere. Nevertheless, a survey in 2003 looked at publications from 1983 to 1999 of trials that were relevant to submissions for approval of serotonin reuptake inhibitors by the Swedish drug regulatory agency [29]. Only one of the five drugs submitted did not involve multiple publications of the same or overlapping data. Depending on the journal and the nature and extent and importance of new information, updates of trials that were previously published may be accepted. The practice of many journals requiring a trial registration number serves to minimize, if not completely avoid, the concern that updates could lead to double counting of trials and participants in meta-analyses. One proposal that also might help, at least for electronic publications, is to better enable linkage of publications by means of trial registration numbers [30].

Disclosure of Conflict of Interest

Many journals have policies requiring clear statements of possible conflicts of interest [31]. The Uniform Requirements for Manuscripts Submitted to Biomedical Journals [18] contains guidelines regarding disclosure of potential conflicts related to individual authors and to the identification and role of the sponsor of the trial. Authors must be forthcoming in disclosing any potential conflicts, as they can affect how readers interpret study findings. Unfortunately, there have been instances where important conflicts were not disclosed and were subsequently discovered [32, 33]. These cases serve both to embarrass the investigators and perhaps unfairly tarnish good research, a situation that could have been avoided had openness been followed in the beginning. We recommend that all authors disclose freely all real, potential, or apparent conflicts of interest. Others may perceive conflicts that the authors did not consider to be such, so it is helpful to be as forthcoming as possible. Independent assessment is always preferable to lack of disclosure.

Presentation of Data

Presentation of the data analysis is important [34–42]. There is a common misunderstanding of the meaning of p-values. Only about one-fifth of the physician respondents to a multiple choice question understood the proper meaning of a p-value [43]. The p-value tells us how likely an observed difference may have occurred by chance. It conveys information about the level of doubt, not the magnitude of clinical importance of this difference. A p-value of 0.05 in a very large trial may be weak evidence of an effect while in a small sample it can be quite strong evidence [35]. The point estimate (the observed result) with its 95% confidence interval (CI) provides us with the best estimates of the size of a difference. The width of the CI is another measure of uncertainty. The p-value and the CI are inherently related; thus, if the 95% CI of the difference excludes 0, the difference is statistically significant with p < 0.05. The CI permits the readers to use their own judgment for the smallest clinically important difference in making treatment decisions [34]. Some journals have taken the lead and now require more extensive use of CIs. We advocate reporting of p-values, point estimates, and CIs for the major results. They all convey important information and help in evaluating a trial’s result.

Interpretation

Articles have been written to help clinicians in their appraisal of a clinical study [44–49]. Readers should be aware that many publications have deficiencies and can even be misleading. Pocock [50] has given three reasons why readers need to be cautious: (1) some authors produce inadequate trial reports, (2) journal editors and referees allow them to be published, (3) journals favor positive findings. For example, a review of trials of antibiotic prophylaxis found that 20% of the abstracts omitted important information or implied unjustified conclusions [51]. In another review, Pocock and colleagues [52] examined 45 trials and concluded that the reporting “appears to be biased toward an exaggeration of treatment differences” and that there was an overuse of significance levels. In a 1982 report, statistical errors were uncovered in a large proportion of 86 controlled trials in obstetrics and pediatrics journals and only 10% of the conclusions were considered justified [53]. Gøtzsche found that reports of 76% of 196 trials of nonsteroidal anti-inflammatory drugs in rheumatoid arthritis contained “doubtful or invalid statements” [54]. As mentioned in Chap. 9, inadequate reporting of the methods of randomization and baseline comparability was found in 30–40% of 80 randomized clinical trials in leading medical journals [55]. In the oncology field, the criteria for tumor response from articles published in three major journals were incompletely reported, variable and contributed to the wide variations in reported response rates [56].

Baar and Tannock [57] constructed a hypothetical trial and reported its results in two separate articles; one with errors of reporting and omissions similar to those “extracted from” leading cancer journals and the other with appropriate methods. This exercise illustrates how the same results can be interpreted and reported differently.

The way in which results are presented can affect treatment decisions [58–60]. Almost half of a group of surveyed physicians were more impressed and indicated a higher likelihood of treating their patients when the results of a trial were presented as a relative change in outcome rate compared to an absolute change (difference in the incidence of the outcome event) [59]. A relative treatment effect is difficult to interpret without knowledge of the event rate in the comparison group. The use of a “summary measure,” such as the number of persons who need to be treated to prevent one event, had the weakest impact on clinicians’ views of therapeutic effectiveness [60]. We recommend that authors report both absolute and relative changes in outcome rates.

Publication Bias

Timely preparation and submission of the trial results—whether positive, neutral, or negative—ought to be every investigator’s obligation. The written report is the public forum that all the work of a clinical trial finally faces. Regrettably, negative trials are more likely to remain unpublished than positive trials. Early evidence of this publication bias came from a survey of the psychological literature. Sterling [61] noted in 1959 that 97% of 294 articles involving hypothesis testing reported a statistically significant result. The situation was similar for medical journals decades later; about 85% of articles—clinical trials and observational studies—reported statistically significant results [62]. Simes [63] compared the results of published trials with those from trials from an international cancer registry. A pooled analysis of published therapeutic trials in advanced ovarian cancer demonstrated a significant advantage for a combination therapy. However, the survival ratio was lower and statistically nonsignificant when the pooled analysis was based on the findings of all registered trials. Several surveys have identified selective reporting and/or multiple publications of the same trial [29, 64–67]. A review of reporting bias found that it was widespread in many medical conditions [68]. Heres et al. [69] found that in head-to-head comparisons between anti-psychotic agents, 90% of the 33 trials sponsored by a drug company showed benefit from the sponsor’s drug. That is, the better drug was whichever was the one produced by the sponsor of the trial. These apparently contradictory result suggested bias in study design, analysis, and/or reporting.

Even multicenter trials conducted at a major academic center remained unpublished over 40% of the time. Those trials sponsored by government were published only modestly more often than those sponsored by industry [20, 66]. These findings were confirmed by Gordon and colleagues [70], who reviewed the publication records of 244 clinical trials funded by the National Heart, Lung, and Blood Institute of the NIH from 2000 through the end of 2011. Fifteen months later, only 156 of the trials had published main results, with the median time to publication being 25 months. Those trials with clinical outcomes were published more rapidly than those with other outcomes (e.g., biomarkers). The authors also found that after adjustment for other factors, those trials with “positive” results (defined as “a significant between-group difference in the primary end point favoring the investigators’ stated hypothesis”) were published quicker than those with “negative” results.

Turner et al. [67] looked at 74 studies of antidepressant agents that had been registered with the U.S. Food and Drug Administration. Twenty-three of the trials had not been published. In addition, those that were published claimed to show results more positive toward the intervention than did a subsequent FDA analysis of the data. Perlis et al. found that financial conflict of interest was common in clinical trials in psychiatry and was associated with clinical trial results that were highly favorable to the intervention [71]. According to Chan and colleagues [65], there were frequent discrepancies (62%) between the primary response variable stated in the trial protocol and that reported in the publication of results. Analyses used in publications have also differed from those used in internal company documents [72]. It has been shown that many abstracts are never followed by full publications [73]. In a survey of 156 investigators who acknowledged participating in trials whose results were not published, Dickersin et al. found that among 178 unpublished trials with a trend specified, 14% favored the new therapy compared to 55% among 767 published reports (p < 0.001) [74]. Analysis of factors associated with this bias are, in addition to neutral and negative findings, small sample size and possibly pharmaceutical source of funding [74]. Rejection of a manuscript by a journal is an infrequent reason [75, 76]. However, authors are no doubt aware that it is difficult to publish neutral results. A survey of the reference lists of trials of nonsteroidal anti-inflammatory drugs revealed a bias toward references with positive outcomes [77].

Selective reporting is viewed as a serious issue. In a survey of clinical trialists, selective reporting was considered among the two most important forms of scientific misconduct [78]. Investigators have the primary responsibility for ensuring that they do not engage in this practice. Journals too have a responsibility to encourage full and honest reporting. They ought to select trials for publication according to the quality of their conduct rather than according to whether the p-value is significant. We expect that the common use of clinical trial registries will encourage more complete reporting of trial results, as those trials begun but not reported are more easily identified, though as of yet, the record is mixed [20, 79].

A specific source of potentially biased reporting involves early phase studies and pilot trials. Particularly, unless these studies show positive trends or lead to full-scale late phase trials, they are likely to be unreported. Prayle and colleagues [80] reviewed trials listed in ClinicalTrials.gov that were subject to mandatory reporting of results by the FDA. They found that only 22% (163 of 738) studies reported results within 1 year of the end of the trial. Later phase trials were more likely to report results within a year (38%) than phase II studies (10%). Lack of reporting or publication is justified on the basis that the studies are small, of short duration, and may not use optimal doses of the intervention. Nevertheless, such studies may contain important data that should be made available to other researchers and clinicians. For example, if there are design flaws, disclosing those could save other researchers from repeating them. If there are problems with a drug, device, or procedure, it would be important for others to learn about them. We recognize that many journals will reject publication of these kinds of studies, but hope that in the era of on-line publishing, enough journals will accept them. We strongly encourage publication of early phase and pilot studies [81], in addition to publication of all late phase trials.

Did the Trial Work as Planned?

Baseline Comparability

The foundation of any clinical trial is the effort to make sure that the study groups are initially comparable so that differences between the groups over time can be reasonably attributed to the effect of the intervention. Randomization is the preferred method used to obtain baseline comparability. The use of randomization does not necessarily guarantee balance at baseline in the distribution of known or unknown prognostic factors. Baseline imbalance is fairly common in small trials but may also exist in large trials (see Chap. 9). Therefore, both a detailed description of the randomization process, including efforts made to prevent prior knowledge on the part of the investigator of the intervention assignment, and a presentation of baseline comparability are essential. Should the study be nonrandomized, the credibility of the findings hinges even more upon an adequate documentation of this comparability. For each group, baseline data should include means and standard deviations of known and possible prognostic factors. Small trends for individual factors can have an impact if they are in the same direction. A multivariate analysis to evaluate balance may be advantageous. Of course, the fact that major prognostic factors may be unknown will produce some uncertainty with regard to baseline balance. Adjustment of the findings on the basis of observed baseline imbalance should be performed and any difference between unadjusted and adjusted analyses should be carefully explained (see Chap. 18).

Blinding

Double-blinding is a desirable feature of a clinical trial design because, as already discussed, it diminishes bias in the assessment of response variables that require some element of judgment. However, many studies are not truly double-blinded to all parties from start to finish. While an individual side-effect may be insufficient to unblind the investigator, a constellation of effects often reveals the group assignment. A specific drug effect such as a marked fall in blood pressure in an antihypertensive drug trial—or the absence of such an effect—might also indicate which is the active intervention group and which is not. Although the success of blinding may be difficult for the investigator to assess, and some disagree with assessment of blinding [82–84], we think that an evaluation could have value. The reasons for not assessing and reporting the success of blinding are that such efforts might stimulate investigators and participants to make extra efforts to unblind and that responses from participants are often unreliable. We believe, however, that readers of a publication ought to be informed about the degree of unblinding. An evaluation such as the one provided by Karlowski and colleagues for a trial of vitamin C [85] is commendable.

It is important to emphasize that assessment of blinding should not be done while the trial is ongoing, but only at the end. If assessment is conducted at the end of the trial, effects on trial conduct are minimal or nonexistent and there is less incentive for participants to attempt to mislead the investigator.

Adherence and Concomitant Treatment

In estimating sample size, investigators often make assumptions regarding the rate of nonadherence. Throughout follow-up, efforts are made to maintain optimal adherence to the intervention under study and to monitor adherence. When interpreting the findings, one can then gauge whether the initial assumptions were borne out by what actually happened. When adherence assumptions have been too optimistic, the ability of the trial to test adequately the primary question may be less than planned. The study results must be reported and discussed with the power of the trial in mind. In trials showing a beneficial effect of a specific intervention, nonadherence is usually a minor concern. Two interpretations of the effect of nonadherence are possible. It may be argued that the intervention would have been even more beneficial had adherence been higher. On the other hand, if all participants (including those who for various reasons did not adhere entirely to the dosage schedule or duration of intervention of a trial) had been on full dose, there could have been further adverse events or harmful effects in the intervention group.

Also of interest is the comparability of groups during the follow-up period with respect to concomitant interventions. Use of drugs other than the study intervention, changes in lifestyle and general medical care—if they affect the response variable—need to be measured and reported. Of course, as mentioned in Chap. 18, adjustment on postrandomization variables is inappropriate. As a consequence, when imbalances exist, the study results must be interpreted cautiously.

What Are the Limitations?

When the results of a “superiority” trial (i.e., one in which an intervention is evaluated to see if it differs from a control) indicate no statistically significant difference between the study groups there are several possible explanations. In addition to the conclusion that the studied intervention may be of little or no value, the dose of the intervention may have been too low or too high; the technical skills of those providing the intervention (e.g., surgical procedure) may have been inadequate; the sample size may have been too small, giving the trial insufficient power to test the hypothesis (Chap. 8); there may have been major adherence problems; concomitant interventions may have reduced the effect that would otherwise have been seen; or the outcome measurements may not have been sensitive enough or the analyses may have been inadequate. Finally, chance is another obvious explanation. The authors should provide the readers with enough information in the Methods and Results sections for them to judge for themselves why an intervention may not have worked. In the Discussion section, the authors should also offer their best understanding of why no difference was found.

For equivalence or noninferiority trials, inadequate design or conduct, or poor adherence on the part of participants, can lead to what the investigators and sponsors consider as the “desired” outcome, that is, no discernable difference between intervention groups. As discussed in Chap. 5, attention to these factors is extraordinarily important in noninferiority trials. Perhaps even more than in superiority trials, the authors must recognize and clearly acknowledge any study limitations and problems that could have contributed to the lack of difference. In some cases, an “on treatment” analysis might be warranted, in addition to the intention-to-treat analysis.

What are the limitations of the trial findings? One needs to know the degree of completeness of data in order to evaluate a trial. A typical shortcoming, particularly in long-term trials, is that the investigator may lose contact with some participants or for other reasons have missing data. These participants are usually different from those who remain in the trial, and their event rate or outcome measurements may not be the same. Vigorous attempts should be made to keep the number of persons lost to follow-up to a minimum. The credibility of the findings may be questioned in trials in which the number of participants lost to follow-up is large in relation to the number of events. A conservative approach in this context is to assume the “worst case.” This approach assumes the occurrence of an event in each participant lost to follow-up in the group with lower incidence of the response variable, and it assumes no events in the comparison group. After application of the “worst case” approach, if the overall conclusions of the trial remain unchanged, they are strengthened. However, if the worst-case analysis changes the conclusions, the trial may have less credibility. Other approaches to handling missing outcome data are discussed in Chap. 18. The degree of confidence in the conclusion will depend upon the extent to which the outcome could be altered by the missing information.

What Kinds of Analyses?

As addressed in Chap. 18, results may be questionable if participants randomized into a trial are withdrawn from the analysis. Withdrawal after randomization undermines the goal of conducting a valid, unbiased trial. It should be avoided. Investigators who support the concept of allowing withdrawals from the analysis should be required to report analyses both with, and without, withdrawals. If both analyses give approximately the same result, the findings are confirmed. However, if the results of the two analyses differ, believe the intention-to-treat analysis while exploring the reasons for the differences.

In evaluating possible benefit of an intervention, more than one response variable is often assessed which raises the issue of multiple comparisons (Chap. 18). In essence, the chance of finding a nominally statistically significant result increases with the number of comparisons. This is true whether there are multiple response variables, repeated comparisons for the same response variable, subgroup analyses or whether various combinations of response variables are tested. In the survey of 45 trials in three leading medical journals, the median number of significance tests per trial was eight; more than 20 tests were reported in six trials [51]. The potential impact of this multiple testing on the findings and conclusion of a trial ought to be considered. A conservative approach in the interpretation of statistical tests is again recommended. When several comparisons have been made, a more extreme statistic might be required before a statistically significant difference could be claimed. One approach is to require a p-value <0.01 for a limited number of secondary outcomes or a Bonferroni correction in order to declare a treatment difference statistically significant. An alternative approach is to consider all of the subsidiary analyses exploratory and hypothesis generating [52]. Authors of a report should indicate the total number of comparisons made during a trial and in the analysis phase (not just those selected for reporting). Readers should focus attention on p-values for protocol-specified comparisons.

The main objective of any trial is to answer the primary question. Findings related to one of the secondary questions may be interesting, but they should be put in the proper perspective. Are the findings for the related primary and secondary response variables consistent? If not, attempts ought to be made to explain discrepancies. Explaining inconsistencies was particularly important in the Cooperative Trial in the Primary Prevention of Ischaemic Heart Disease [86]. In that trial, the intervention group showed a statistically significant reduction in the incidence of major ischemic heart disease (primary response variable), but a significant increase in all-cause mortality (secondary response variable).

An area of some controversy concerns the analysis and reporting of composite outcomes (Chap. 18). Cordoba and colleagues [87] reviewed trials published in 2008 that employed composite outcomes. Of 40 such trials, 28 used components of the composite outcome that were of different importance. Thirteen trials used inconsistent definitions of the components in different parts of the publication (in five of them, the components were not the same). Nine of the trials did not present clear data for the individual components. Particularly when components are of different clinical importance, clear presentation of individual component data, as well as the composite data, is essential. Obviously, there will be limited power to detect group differences among the separate components, but authors should provide complete and consistent reporting. Are the trends in the individual components in the same direction, even though statistical significance in not observed?

In all studies, evidence for possible serious adverse events from the intervention needs to be presented. Comparison of adverse events among those participants who adhered to the intervention may provide a more conservative assessment, in the sense that it leans toward safety. Authors might consider analyzing adverse event data both using intention-to-treat and on-treatment approaches. In the final conclusion, the overall benefit should be weighed against the risk of harm. This assessment of the balance, however, is too infrequently done (Chap. 12).

How Do the Findings Compare with Those from Other Studies?

The findings from a clinical trial should be placed in the context of current knowledge. Are they consistent with knowledge of basic science, including presumed mechanism of action of the intervention? Although the precise mechanism may be unclear, when the outcome can be explained in terms of known biological actions, the conclusions are strengthened. Do the findings confirm the results of studies with similar interventions or different interventions in similar populations?

It is important here to keep in mind that a substantial proportion of initiated and even completed trials are never published. Additionally, a review of the completeness of articles cited in reference lists of clinical trial publications suggests that studies with neutral or negative results tend not to be cited [73]. Among published trials the response to a given drug or drug combination can vary markedly [88, 89]. Much of the variation may be explained by differences in participant selection, including genetic variation, treatment regimen and concomitant intervention, but major differences may also reflect the way the data were analyzed and reported. In a review of 51 randomized clinical trials in congestive heart failure, the authors attributed conflicting results to lack of uniform diagnostic criteria [88]. In a thoughtful editorial, Packer [89] pointed out that several other factors could explain discordant results. He suggested that the characteristics of the enrolled participants may be more important than the definition of congestive heart failure. Differences in design—sample size, dose and duration of intervention—may affect the trial findings. Other factors might be differences in criteria of efficacy and publication policy. Results of positive trials tend to be published several times, for example, both in a regular journal report and in a journal supplement funded by the pharmaceutical industry.

Generally, credibility of a particular finding increases with the proportion of good independent studies that come to the same conclusion. Inconsistent results are not uncommon in clinical research and medicine. In such cases, the problem for both the investigators and the readers is to try to determine the true effect of an intervention. How and why results differ need to be explored. The use of confidence limits has the advantage of allowing the readers to compare findings and assess whether the results of different trials could, in fact, be consistent.

What Are the Clinical Implications of the Findings?

It is appropriate, of course, to generalize the results to the study population, that is, those people who would have been eligible for and could have participated in the trial. The next step, suggesting that the trial results be applied to a more general population (the majority of which would not even meet the eligibility criteria of the trial) is more tenuous. Readers must judge for themselves whether or not such an extrapolation is appropriate. As seen in Fig. 4.​1 in Chap. 4, there is often a considerable winnowing from the initial study population to the final sample. A similar argument applies to the intervention itself. How general are the findings? If the intervention involved a special procedure, such as surgery or counseling, is its application outside the trial setting likely to produce the same response? In a drug trial the question of dose-effect relationship is often raised. Would a higher or lower dose of the drug have given different results, perhaps by altering the balance between benefit and harm? Can the same claims be made for different drugs of the same class or that have a similar structure or pharmacological action? Can the results of an intervention be generalized even more broadly? For example, there have been many trials comparing different statins in the prevention of coronary disease sequelae. If the goal LDL-cholesterol is the same in the groups being compared, should one expect similar outcomes? Based on the experience with cerivastatin [90], statins are unlikely to be the same, at least with respect to adverse events. One problem in trials of devices is that the devices are constantly being modified or improved, with respect to the technology or the software algorithm. Does the trial using the old model have any implications for the latest model or the model to come in the future? For a further discussion of generalization, see Chap. 4.

In 1987, a review found that the majority of therapeutic interventions had not been properly tested in randomized clinical trials [91]; approval may have been granted on the basis of surrogate endpoints or drugs may have multiple indications, only some of which are proven. As discussed in this book, there continue to be examples of drugs that had been approved but when assessed in an adequately designed clinical trial turn out not to be as wonderful as hoped. Skillful marketing has a major impact on practice patterns. The marked regional differences in drug sales can not be explained on the basis of science, since regions, in principle, have access to the same scientific information. It is difficult to tease out the impact of clinical trials on medical practice from other factors such as marketing and treatment guidelines. There are several examples of trials that have changed practice patterns [92, 93]. Similarly, there are examples where practice was predominantly influenced by the other factors [94].

As noted by Rothwell [95], clinicians must decide if clinical trial results are relevant to their patients. Rothwell points out that issues such as trial setting, kinds of patients, details regarding the intervention and control, nature of the primary and secondary outcomes, and adverse events are important in arriving at clinical decisions. Therefore, authors should include the necessary information in their publications. Obviously, no trial is large enough or has a broad enough population to enable readers to evaluate every kind of patient that might be treated. But the information can be helpful.

As with all research, a clinical trial will often raise as many questions as it answers. Suggestions for further research should be discussed. Finally, the investigator might allude to the social, economic and medical impact of the study findings. How many lives can be saved? How many working days will be gained? Can symptoms be alleviated? Economic implications or cost-effectiveness are important. Any benefit has to be weighed against the cost and feasibility of use in routine medical practice rather than in the special setting of a clinical trial.

Data Sharing

An issue that has received considerable attention is data sharing. Even an exemplary scientific report can contain only limited data that might conceivably be important to other researchers and clinicians. Therefore, data sharing among investigators and public access to data and publications have been proposed, and even required by some clinical trials sponsors [96–102]. A study jointly sponsored by the National Institute of Allergy and Infectious Diseases of the NIH, the Juvenile Diabetes Research Foundation, Genetech, and Biogen Idec, at the time of publication, indicated that data sets were accessible on a public website [103]. Also encouraging was a report from GlaxoSmithKline that it would “provide access to deidentified patient-level data” [104] and a supportive letter from a representative from Hoffmann-La Roche [105]. It should be noted that the National Heart, Lung, and Blood Institute of the NIH has for decades provided data sets of many of its major clinical trials and observational studies to investigators [106]. In 2014, the National Institutes of Health stated its intent to increase sharing of clinical trial results for studies that it funds. It proposed to require submission of “summary results information to ClinicalTrials.gov for any applicable clinical trial that is required to be registered, regardless of whether the drugs, biological products, or devices under study have been approved, licensed, or cleared for marketing…” [107]. The proposed requirements deal with summary data only. It was acknowledged that sharing of individual participant data would be important, and that future efforts to accomplish that were under consideration [108]. The European Medicines Agency has clarified its position on the need for publication of clinical data and clinical study reports on which regulatory decisions are made [109].

The Institute of Medicine has released a report on data sharing that addresses many of the issues [110]. While the rationale for data sharing may be compelling, the process is very challenging because there are so many stakeholders and levels of data in a typical clinical trial. Stakeholders involved with data sharing, from sponsors, whether federal or private, include clinical investigators in the trial, other interested clinical investigators not part of the trial, patient advocacy groups, trial participants, regulatory agencies, journals and graduate students in training. There are also different levels of data, ranging from the raw data which may include medical images, electrocardiogram tracings, and quality of life assessment tests to the more standard baseline data and clinical and laboratory outcome data. Not all of these data are used, some perhaps only rarely, in scientific presentations or publications, and many are unlikely to be used even in regulatory review. The Institute of Medicine Report calls for data and their metadata (documentation about the data file and the study) to be made available within 6 months of the publication date or when the study has been presented for regulatory review. For publication, these shared data would be the analyzable data set used in the publication. The 6 month moratorium is to provide trial investigators time to prepare and submit their additional secondary analysis papers for publication. For regulatory review, the shared data would be what was in the complete study report. In general, the IOM report calls for data to be made available no longer than 18 months after trial completion described as last participant’s last visit or the predefined follow up cutoff date. The logistics of how this process should be carried out must evolve, including what group or groups are the curators for the trial data, what review process if any is required before data are released, and who funds this process, among many other challenging issues.

As discussed in the IOM report, the benefits and limitations of the data sharing policies are contentious, but all investigators whose trial was funded by an agency or company requiring data sharing must keep abreast of the requirements and policies of funding agencies such as NIH, drug regulatory agencies, and pharmaceutical companies.

References

1.

Comroe JH Jr. The road from research to new diagnosis and therapy. Science 1978;200:931–937.CrossRef

2.

Glantz SA. Biostatistics: how to detect, correct and prevent errors in the medical literature. Circulation 1980;61:1–7.CrossRef

3.

Relman AS. What a good medical journal does. The New York Times March 19, 1978; Section IV; p.22.

4.

Ellenberg SS, Epstein JS, Fratantoni JC, et al. A trial of RSV immune globulin in infants and young children: the FDA view. N Engl J Med 1994;331:203–204.CrossRef

5.

Bhatt DL, Stone GW, Mahaffey KW, et al. Effect of platelet inhibition with cangrelor during PCI on ischemic events. N Engl J Med 2013;368:1303–1313.CrossRef

6.

FDA Briefing Document Addendum for the Cardiovascular and Renal Drugs Advisory Committee (CRDAC). http://​www.​fda.​gov/​downloads/​advisorycommitte​es/​committeesmeetin​gmaterials/​drugs/​cardiovascularan​drenaldrugsadvis​orycommittee/​ucm385235.​pdf

7.

Moher D, Schulz KF, Altman DG, for the CONSORT Group. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet 2001;357:1191–1194.

8.

Altman DG, Schulz KD, Moher D, et al, for the CONSORT Group. The revised CONSORT statement for reporting randomized trials: explanation and elaboration. Ann Intern Med 2001;134:663–694.CrossRef

9.

Schulz KF, Altman DG, Moher D, for the CONSORT Group. CONSORT 2010 statement: updated guidelines for reporting parallel group randomized trials. PLoS Med 2010; 7 (3): e1000251.CrossRef

10.

CONSORT Statement. http://​www.​consort-statement.​org/​consort-statement/​overview0/​#checklist

11.

Ioannidis JPA, Evans SJW, Gøtzsche PC, et al, for the CONSORT Group. Better reporting of harms in randomized trials: an extension of the CONSORT statement. Ann Intern Med 2004;141:781–788.CrossRef

12.

Piaggio G, Elbourne DR, Altman DG, et al, for the CONSORT Group. Reporting of noninferiority and equivalence randomized trials: an extension of the CONSORT statement. JAMA 2006;295:1152–1160.

13.

Piaggio G, Elbourne DR, Pocock SJ, Evans SJW, Altman DG, for the CONSORT Group. Reporting of noninferiority and equivalence randomized trials: extension of the CONSORT 2010 statement. JAMA 2012;308:2594–2604.CrossRef

14.

Moher D, Liberati A, Tetziaff J, Altman DG, The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med 2009; 6:e1000097.

15.

Mills EJ, Wu P, Gagnier J, Devereaux PJ. The quality of randomized trial reporting in leading medical journals since the revised CONSORT statement. Contemp Clin Trials 2005;26:480–487.CrossRef

16.

Van Spall HGC, Toren A, Kiss A, Fowler RA. Eligibility criteria of randomized controlled trials published in high-impact general medical journals: a systematic sampling review. JAMA 2007;297:1233–1240.CrossRef

17.

Wang R, Lagakos SW, Ware JH, et al. Special Report: Statistics in medicine – reporting of subgroup analyses in clinical trials. N Engl J Med 2007;357:2189–2194.CrossRef

18.

International Committee of Medical Journal Editors. Uniform requirements for manuscripts submitted to biomedical journals: writing and editing for biomedical publication (updated December, 2013) http://​www.​icmje.​org/​icmje-recommendations.​pdf

19.

International Committee of Medical Journal Editors. Is this clinical trial fully registered?: a statement from the International Committee of Medical Journal Editors http://​www.​icmje.​org/​about-icmje/​faqs/​clinical-trials-registration/​.

20.

Chalmers I, Glasziou P, Godlee F. All trials must be registered and the results published. BMJ 2013; 346:f105 doi;10.​1136/​bmj.​f105.

21.

Haynes RB, McKibbon KA, Walker CJ, et al. A study on the use and usefulness of online access to MEDLINE in clinical settings. Ann Intern Med 1990;112:78–84.CrossRef

22.

Ad hoc Working Group for Critical Appraisal of the Medical Literature. A proposal for more informative abstracts of clinical articles. Ann Intern Med 1987;106:598–604.CrossRef

23.

Haynes RB, Mulrow CD, Huth EJ, et al. More informative abstracts revisited. Ann Intern Med 1990;113:69–76.CrossRef

24.

Huth EJ. Preparing to write. In: How to write and publish papers in medical sciences. Philadelphia: ISI Press, 1982, p.37–40.

25.

Huth EJ. Guidelines for authorship of medical papers. Ann Intern Med 1986;104:269–274.CrossRef

26.

Meinert CL. In defense of the corporate author for multicenter trials. Control Clin Trials 1993;14:255–260.CrossRef

27.

Gøtzsche PC, Hrobjartsson A, Johansen HK, et al. Ghost authorship in industry-initiated randomised trials. PLoS Med 2007; 4:e19.CrossRef

28.

Ross JS, Hill KP, Egilman DS, Krumholz HM. Guest authorship and ghostwriting in publications related to rofecoxib. JAMA 2008;299:1800–1812.CrossRef

29.

Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B. Evidence b(i)ased medicine—selective reporting from studies sponsored by pharmaceutical industry: review of studies in new drug applications. BMJ 2003;326:1171–1173.CrossRef

30.

Altman DG, Furberg CD, Grimshaw JM, Shanahan DR. Editorial: Linked publications from a single trial: a thread of evidence. Trials 2014:15:369.CrossRef

31.

Drazen JM, Van Der Weyden MB, Sahni P, et al. Editorial: Uniform format for disclosure of competing interests in ICMJE journals. N Engl J Med 2009; http://​content.​nejm.​org/​cgi/​reprint/​NEJMe0909052.​pdf?​resourcetype=​HWCIT

32.

DeAngelis CD, Fontanarosa PB. Editorial: Resolving unreported conflicts of interest. JAMA 2009;302:198–199.CrossRef

33.

Weinfurt KP, Seils DM, Tzeng JP, et al. Consistency of financial interest disclosures in the biomedical literature: the case of coronary stents. PLoS One 2008; 3:e2128.CrossRef

34.

Berry G. Statistical significance and confidence intervals. Med J Aust 1986;144:618–619.

35.

Gardner MJ, Altman DG. Confidence interval rather than p-values: estimation rather than hypothesis testing. Br Med J 1986;292:746–750.CrossRef

36.

Simon R. Confidence intervals for reporting results of clinical trials. Ann Intern Med 1986;105:429–435.CrossRef

37.

Bulpitt CJ. Confidence intervals. Lancet 1987;i:494–497.

38.

Braitman LE. Confidence intervals extract clinically useful information from data. Ann Intern Med 1988;108:296–298.CrossRef

39.

Freeman PR. The role of p-values in analyzing trial results. Stat Med 1993;12:1443–1452.CrossRef

40.

Braitman LE. Statistical estimates and clinical trials. J Biopharm Stat 1993;3:249–256.CrossRef

41.

Goodman SN, Berlin JA. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med 1994;121:200–206.CrossRef

42.

Pocock SJ, Ware JH. Translating statistical findings into plain English. Lancet 2009;373:1926–1928.CrossRef

43.

Wulff HR, Anderson B, Brandenholf P, Guttler F. What do doctors know about statistics? Stat Med 1987;6:3–10.CrossRef

44.

Haynes RB, McKibbon KA, Fitzgerald D, et al. How to keep up with the medical literature. Ann Intern Med 1986;105:149-153, 309-312, 474-478, 636-640, 810-816, 978-984.

45.

Moon TE. Interpretation of cancer prevention trials. Prev Med 1989;18:721–731.CrossRef

46.

Fowkes FGR, Fulton PM. Critical appraisal of published research: introductory guidelines. Br Med J 1991;302:1136–1140.CrossRef

47.

Cuddy PG, Elenbaas RM, Elenbaas JK. Evaluating the medical literature. Ann Emerg Med 1993;12:549-555, 610-620, 679-686.

48.

Oxman AD, Sackett DL, Guyatt GH, for the Evidence-Based Medicine Working Group. Users’ guides to the medical literature: I. How to get started. JAMA 1993;270:2093–2095.

49.

Guyatt GH, Sackett DL, Cook DJ., for the Evidence-Based Medicine Working Group. Users’ guide to the medical literature: II. How to use an article about therapy or prevention: A. Are the results of the study valid? JAMA 1993;270:2598–2601.CrossRef

50.

Pocock SJ. Clinical Trials. A Practical Approach. Chichester: John Wiley & Sons, 1983.

51.

Evans M, Pollock AV. Trials on trial. A review of trials of antibiotic prophylaxis. Arch Surg 1984;119:109–113.CrossRef

52.

Pocock SJ, Hughes MD, Lee RJ. Statistical problems in the reporting of clinical trials. A survey of three medical journals. N Engl J Med 1987;317:426–432.CrossRef

53.

Altman DG. Statistics in medical journals. Stat Med 1982;1:59–71.CrossRef

54.

Gøtzsche PC. Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal anti-inflammatory drugs in rheumatoid arthritis. Control Clin Trials 1989;10:31–56.CrossRef

55.

Altman DG, Doré CJ. Randomization and baseline comparisons in clinical trials. Lancet 1990;335:149–153.CrossRef

56.

Tonkin K, Tritchler D, Tannock I. Criteria of tumor response used in clinical trials of chemotherapy. J Clin Oncol 1985;3:870–875.

57.

Baar J, Tannock I. Analyzing the same data in two ways: a demonstration model to illustrate the reporting and misreporting of clinical trials. J Clin Oncol 1989;7:969–978.

58.

Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med 1988;318:1728–1733.CrossRef

59.

Forrow L, Taylor WC, Arnold RM. Absolutely relative: how research results are summarized can affect treatment decisions. Am J Med 1992;92:121–124.CrossRef

60.

Naylor CD, Chen E, Strauss B. Measured enthusiasm: Does the method of reporting trial results alter perceptions of therapeutic effectiveness? Ann Intern Med 1992;117:916–921.CrossRef

61.

Sterling TD. Publication decisions and their possible effects on inferences drawn from test of significance -- or vice versa. J Am Stat Assoc 1959;54:30–34.

62.

Dickersin K, Min YI. Publication bias: The problem that won’t go away. Ann N.Y. Acad Sci 1993;703:135–146.CrossRef

63.

Simes RJ. Publication bias: the case for an international registry of clinical trials. J Clin Oncol 1987;4:1529–1541.

64.

Chan A-W, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ 2005;330:753. Epub 2005 Jan 28.CrossRef

65.

Chan A-W, Hrόbjartsson A, Haahr MT, et al. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004;291:2457–2465.CrossRef

66.

Turer AT, Mahaffey KW, Compton KL, et al. Publication or presentation of results from multicenter clinical trials: evidence from an academic medical center. Am Heart J 2007;153:674–689.CrossRef

67.

Turner EH, Matthews AM, Linardatos E, et al. Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 2008;358:252–260.CrossRef

68.

McGauran N, Wieseler B, Kreis J, Schüler Y-B, Kӧlsch H, Kaiser T. Reporting bias in medical research-a narrative review. Trials 2010;11:37.CrossRef

69.

Heres S, Davis J, Maino K, et al. Why olanzapine beats risperidone, risperidone beats quetiapine, and quetiapine beats olanzapine: an exploratory analysis of head-to-head comparison studies of second-generation antipsychotics. Am J Psychiatry 2006;163:185–194.CrossRef

70.

Gordon D, Taddei-Peters W, Mascette A, Antman M, Kaufmann PG, Lauer MS. Publication of trials funded by the National Heart, Lung, and Blood Institute. N Engl J Med 2013;369:1926–1934.CrossRef

71.

Perlis RH, Perlis CS, Wu Y, et al. Industry sponsorship and financial conflict of interest in the reporting of clinical trials in psychiatry. Am J Psychiatry 2005;162:1957–1960.CrossRef

72.

Vedula SS, Li T, Dickersin K. Differences in reporting of analyses in internal company documents versus published trial reports: comparisons in industry-sponsored trials in off-label uses of gabapentin. PLoS Med 2013;10(1):e1001378, doi:10.​1371/​journal.​pmed.​1001378.CrossRef

73.

Goldman L, Loscalzo A. Fate of cardiology research originally published in abstract form. N Engl J Med 1980;303:255–259.CrossRef

74.

Dickerson K, Chan S, Chalmers TC, et al. Publication bias and clinical trials. Control Clin Trials 1987;8:343–353.CrossRef

75.

Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA 1990;263:1385–1389.CrossRef

76.

Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet 1991;337:867–872.CrossRef

77.

Gøtzsche PC. Reference bias in reports of drug trials. Br Med J 1987;295:652–659.CrossRef

78.

Al-Marzouki S, Roberts I, Marshall T, Evans S. The effect of scientific misconduct on the results of clinical trials: a Delphi survey. Contemp Clin Trials 2005;26:331–337.CrossRef

79.

Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial publication after registration in ClinicalTrials.gov: a cross-sectional analysis. PLoS Med 2009;e1000144. doi:10.​1371/​journal.​pmed.​1000144.

80.

Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ 2011;344:d7373 doi: 10.​1136/​bmj.​d7373.CrossRef

81.

Friedman L. Commentary: Why we should report results from clinical trial pilot studies. Trials 2013;14:14:doi:10.​1186/​1745-6215-14-14.

82.

Sackett DL. Turning a blind eye: why we don’t test for blindness at the end of our trials. BMJ 2004;328:1136.CrossRef

83.

Sackett DL. Commentary: Measuring the success of blinding in RCTs: don’t, must, can’t or needn’t? Int J Epidemiol 2007;36:664–665.CrossRef

84.

Schultz KF, Altman DG, Moher D, Fergusson D. Comment: CONSORT 2010 changes and testing blindness in RCTs. Lancet 2010;375:1144–1146.CrossRef

85.

Karlowski TR, Chalmers TC, Frenkel LD, et al. Ascorbic acid for the common cold: a prophylactic and therapeutic trial. JAMA 1975;231:1038–1042.CrossRef

86.

Report from the Committee of Principle Investigators. A cooperative trials in the primary prevention of ischaemic heart disease using clofibrate. Br Heart J 1978;40:1069–1118.

87.

Cordoba G, Schwartz L, Woloshin S, Bae H, Gøtzsche PC. Definition, reporting, and interpretation of composite outcomes in clinical trials: systematic review. BMJ 2010;341:c3920 doi:10.​1136/​bmj.​c3920.

88.

Marantz PR, Alderman MH, Tobin JN. Diagnostic heterogeneity in clinical trials for congestive heart failure. Ann Intern Med 1988;109:55–61.CrossRef

89.

Packer M. Clinical trials in congestive heart failure: why do studies report conflicting results. Ann Intern Med 1988;109:3–5.CrossRef

90.

Psaty BM, Furberg CD, Ray WA, Weiss NS. Potential for conflict of interest in the evaluation of suspected adverse drug reactions: use of cerivastatin and risk of rhabdomyolysis. JAMA 2004;292:2622–2631.CrossRef

91.

Fineberg HV. Clinical evaluation: how does it influence medical practice? Bull Cancer 1987;74:333–346.

92.

Collins R, Julian D. British Heart Foundation surveys (1987 and 1989) of United Kingdom treatment policies for acute myocardial infarction. Br Heart J 1991;66:250–255.CrossRef

93.

Lamas GA, Pfeffer MA, Hamm P, et al. Do the results of randomized clinical trials of cardiovascular drugs influence medical practice? N Engl J Med 1992;27:241–247.CrossRef

94.

Manolio TM, Cutler JA, Furberg CD, et al. Trends in pharmacologic management of hypertension in the United States. Arch Intern Med 1995;155:829–837.CrossRef

95.

Rothwell PM. External validity of randomised controlled trials: “To whom do the results of this trial apply?” Lancet 2005;365:82–93.CrossRef

96.

National Institutes of Health Public Access. http://​publicaccess.​nih.​gov/​

97.

Zarin DA, Tse T. Moving toward transparency of clinical trials. Science 2008;319:1340–1342.CrossRef

98.

Gøtzsche PC. Why we need easy access to all data from all clinical trials and how to accomplish it. Trials 2011;12:249.CrossRef

99.

Mello MM, Francer JK, Wilenzick M, Teden P, Bierer BE, Barnes M. Preparing for responsible sharing of clinical trial data. N Engl J Med 2013;369:1651–1658.CrossRef

100.

Eichler H-G, Pétavy F, Pignatti F, Rasi G. Access to patient-level trial data—a boon to drug developers. N Engl J Med 2013;369:1577–1579.

101.

Krumholz HM. The Opinion Pages: Give the data to the people. The New York Times February 2, 2014.

102.

Zarin DA. Participant-level data and the new frontier in trial transparency. N Engl J Med 2013; 369:468–469.CrossRef

103.

Specks U, Merkel PA, Seo P, et al. Efficacy of remission-induction regimens for ANCA-associated vasculitis. N Engl J Med 2013;369:417–427.CrossRef

104.

Nisen P, Rockhold F. Access to patient-level data from GlaxoSmithKline clinical trials. N Engl J Med 2013; 369:475–478.CrossRef

105.

Barron H. Letter to the editor: Access to patient-level trial data. N Engl J Med 2014;370:485–486.CrossRef

106.

Biologic Specimen and Data Repository Information Coordinating Center (BioLINCC). https://​biolincc.​nhlbi.​nih.​gov/​studies/​

107.

Summary of HHS/NIH proposals to enhance transparency of Clinical trial results. http://​www.​nih.​gov/​news/​health/​nov2014/​od-19_​summary.​htm.

108.

Hudson KL, Collins FS. Viewpoint: Sharing and reporting the results of clinical trials. JAMA 2015;313:355–356.CrossRef

109.

European Medicines Agency policy on publication of clinical data for medicinal products for human use. http://​www.​ema.​europa.​eu/​docs/​en_​GB/​document_​library/​Other/​2014/​10/​WC500174796.​pdf

110.

Institute of Medicine Committee on Strategies for Responsible Sharing of Clinical Trial Data. Sharing clinical trial data: maximizing benefits, minimizing risk. Washington, D.C.: The National Academies Press, 2015.

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_21

21. Multicenter Trials

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

A multicenter trial is a collaborative effort that involves more than one independent center in enrolling and following study participants. Multicenter randomized clinical trials have a long and rich history, with Hill [1] and Greenberg [2] providing general discussions of methods in the middle of the twentieth century.

In the last four decades, there has been a dramatic increase in the number of multicenter and multinational trials. Multicenter studies are more difficult and more expensive to perform than single-center studies, and they may bring less individual professional reward due to the need to share credit among many investigators. However, multicenter trials are necessary primarily because single sites cannot enroll enough participants to assess clinically important outcomes [3]. Over 40 years ago, Levin and colleagues provided many examples of “the importance and the need for well-designed cooperative efforts to achieve clinical investigations of the highest quality” [4].

The reasons for conducting multicenter trials apply even more today, with much of medicine being global in scope. It is common for large late-phase trials sponsored by industry to include a wide geographical representation. Several hundred sites might be involved, each site entering anywhere from several to a few dozen participants. While such dispersion of sites presents logistical challenges for training of personnel and data quality control, the benefits of rapid participant recruitment have generally outweighed these challenges. Another potential advantage of multicenter trials is that investigators at multiple sites, with standardized protocols, may be less prone to bias that could affect trial conduct and event ascertainment, especially in open-label trials. Participants enrolled at a single center, all under the oversight of an investigator who is academically invested in the hypothesis, may be subject to a greater likelihood of bias.

Much of the ground work for the development, organization, and conduct of a multicenter trial was laid many years ago in trials like the Coronary Drug Project [5] and the International Studies of Infarct Survival (ISIS) [6, 7]. This chapter will discuss the reasons why such studies are conducted and briefly review some key steps in their planning, design and conduct.

Fundamental Point

Multicenter trials are needed to enroll adequate numbers of participants in care settings that are likely to reflect diverse practice. Investigators responsible for organizing and conducting a multicenter study should have a full understanding of the complexity of the undertaking and the need for systems to assure that a common protocol is followed at each site.

Reasons for Multicenter Trials

1.

The main rationale for multicenter trials is to recruit the adequate numbers of participants within a reasonable time. Many clinical trials have been—and still are—performed without a good estimate of the number of participants likely to be required to adequately test the main hypothesis. Yet, if the primary response variable is an event that occurs relatively infrequently, or small group differences are to be detected, sample size requirements will be large (Chap. 8).

Studies requiring hundreds of participants usually cannot be done at one center, although there are some exceptions like the Deutsches Herzzentrum, in Munich, Germany, that has enrolled over 50,000 participants in a series of single site trials [8]. This site has also successfully participated in multicenter trials [9].

Some multicenter trials have been very large. The Global Utilization of Streptokinase and Tissue Plasminogen Activator for Occluded Coronary Arteries (GUSTO) trial enrolled 41,021 patients with acute myocardial infarction at 1081 hospitals in 15 countries, with enrollment ranging from 1 to over 200 patients per center [10]. This trial had four treatment groups, and treatment with accelerated t-PA (versus the streptokinase arms) resulted in a 14% relative risk reduction (and 1% absolute reduction) in 30-day mortality, a result that changed practice. The large sample size was required to be convincingly significant (p = 0.001). The Women’s Health Initiative (WHI) [11] was an ambitious 15-year project mandated by Congress in 1991 and sponsored by the National Institutes of Health (NIH). WHI included 161,000 postmenopausal women enrolled in 40 centers across the United States. A set of clinical trials, using a partial factorial design, included 68,132 women participants, addressed dietary modification, calcium and vitamin D supplementation, or hormone replacement therapy. The WHI provided important results that changed practice. And the program was a good investment, as shown by the fact that the $260 million cost of the WHI postmenopausal therapy trial was estimated to have a total net economic return of $37.1 billion [12, 13]. This was mainly the result of a change in practice such that women were no longer being exposed to the harmful effects of hormone replacement therapy. The Heart Protection Study 2–Treatment of HDL to Reduce the Incidence of Vascular Events (HPS2-THRIVE) investigators took a different approach to site selection [14]. They selected 245 high-performing centers who demonstrated they had adequate patient volume to enroll large numbers of participants. These investigators randomized 25,673 patients (of the 42,424 entering the run-in phase) with prior vascular disease over 3 years at sites in the United Kingdom (89 sites), Scandinavia (84 sites), and China (72 sites) to niacin plus laropiprant versus placebo. While these trials enroll very large numbers of participants to be able to detect modest treatment effects (15% relative risk reductions), they illustrate the importance of having many sites, and selected sites, involved. The National Cancer Institute Cooperative Group (now the National Clinical Trials Network) [15] and HIV/AIDS Clinical Trials Networks [16] provide other examples of the rich history of multicenter trials.

2.

A multicenter study may enable a more generalizable sample of the study population. Although no trial is completely representative, geography, race, socioeconomic status, and lifestyle of participants may be more similar to the general population if participants are enrolled by many centers. These factors may be important in the ability to generalize the findings of the trial. Concern has been raised that site selection for practical purposes like improving enrollment could negatively impact on generalizability of results [17].

In the GUSTO trial, 23,000 participants were enrolled in the United States, and most of these were enrolled over a 1-year period [10]. During that year of 1992, it has been estimated that nearly 10% of all patients in the country with acute myocardial infarction treated with fibrinolytic therapy were enrolled in the trial. The participants in this “pragmatic” trial with few exclusion criteria were well represented by high-risk groups such as the elderly (12% were at least 75 years of age, and the oldest was 110 years old) [18].

Another example of the need to anticipate how participant make-up may affect generalizability is in racial distribution. For instance, it is known that hypertension and its treatment response may vary according to race. A study of participants with hypertension from either a totally black or totally white community is likely to yield findings that may not necessarily be applicable to a more diverse population. Anticipating this, there was a special effort to enroll black participants in the Antihypertensive and Lipid Lowering Treatment to Prevent Heart Attack Trial (ALLHAT). Ultimately, 35% of participants in ALLHAT were black, [19] which allowed for exploration of racial heterogeneity of intervention effects.

3.

A multicenter study enables investigators with similar interests and skills to work together on a common problem. Science and medicine, like many other disciplines, are competitive. Nevertheless, most major research accomplishments in clinical medicine now require a collaborative team approach. A multicenter trial also gives capable, clinically-oriented persons, who might otherwise not become involved in research activities, an opportunity to contribute to science. In the early years, multicenter clinical trials typically involved only major academic centers. Now, many clinical practices based in the community successfully participate in trials, and in many trials, organized community hospitals or clinics are the best enrollers.

Conduct of Multicenter Trials

One of the earlier multicenter clinical trials was the Coronary Drug Project [5]. This study provided an initial model for many of the techniques currently employed. As in all active disciplines, concepts are frequently changing and some techniques have been refined in subsequent trials. The following series of steps, a distillation of experience from a number of studies, is one reasonable way to approach the planning and conduct of a multicenter trial.

First, a planning committee should be established to be responsible for organizing and overseeing the various phases of the study (planning, participant recruitment, participant follow-up, phase out, data analysis, paper writing) and its various centers and committees. This group often consists of representatives from the sponsoring organization (e.g., government agencies, private research organizations, educational institutions, private industry), with input from appropriate consultants. Use of consultants who are expert in the field of study, in biostatistics, and in the management of multicenter clinical trials is encouraged. The planning committee needs to have authority in order to operate effectively and for the study to function efficiently.

Second, to determine the feasibility of a study, the planning committee should make a thorough search of the literature and review of other information. Sample size requirements should be calculated. Reasonable estimates must be made regarding control group event rate, anticipated effect of intervention, and participant adherence to therapy. The planning committee also has to evaluate key issues such as participant availability, availability of competent cooperating investigators, timeliness of the study, possible competing trials, regulatory requirements, and total cost. After such an assessment, is the trial worth pursuing? Are there sufficient preliminary indications that the intervention under investigation indeed might work? On the other hand, is there so much suggestive (though inconclusive) evidence in favor of the new intervention that it might be difficult ethically to allocate participants to a control group? Might such suggestive evidence seriously impede participant recruitment? Since planning for the study may take a year or more, feasibility needs to be constantly re-evaluated, even up to the time of the actual start of participant recruitment. New or impending evidence may at any time cause cancellation, postponement, or redesign of the trial. In some instances, a pilot, or feasibility study is useful in answering specific questions important for the design and conduct of a full-scale trial.

Third, multicenter studies require not only clinical centers to recruit participants, but also one or two coordinating centers to help design and manage the trial and to collect and analyze data from all other centers. There may be regional sites, academic centers that serve as academic research organizations, or contract research organizations that conduct site visits and receive data from the clinical centers. Additional centers are often needed to perform specialized activities such as key laboratory tests, imaging, and distributing study drugs. While the specialized centers may perform multiple services, it may not be advisable to permit a clinical center to perform these services. If a specialized center and a clinical center are in the same institution, it may be important for each to have a separate staff in order to protect against unblinding and, therefore, bias. Even if unblinding or bias is avoided, there might be criticism that such a bias might have occurred and thus raise unnecessary questions about the entire clinical trial.

As reported by Croke [20], a major consideration when selecting clinical center investigators is availability of appropriate participants. Although this report is now old, the message remains relevant. The trial has to go where the participants are. Clearly, experience in clinical trials and scientific expertise are desirable features for investigators, but they are not crucial to overall success. Well-known scientists who add stature to a study are not always successful in collaborative ventures. The chief reason for this lack of success is often their inability to devote sufficient time to the trial. In a comprehensive study of factors associated with enrollment of eligible participants with documented myocardial infarction, Shea et al. [21] found positive correlations with institutions in which patients were cared for by staff other than private attending physicians and with the presence of a committed nurse-coordinator. While many factors have been associated with successful enrollment, none is more revealing than prior performance in conducting collaborative trials.

The selection of the coordinating center is of utmost importance. This is often a single entity, but sometimes the coordinating center functions are split between two or more units; a clinical coordinating center, a data coordinating center, and, often, a separate data analysis center. The responsibilities described here apply to any of the models, but clearly communication becomes more of an issue when there are multiple units.

In addition to helping design the trial, the coordinating center, or combination of centers, is responsible for implementing the randomization scheme; carrying out day-to-day trial activities; and collecting, monitoring, editing, and analyzing data. The coordinating center, or, when there are two or more units, the clinical coordinating center/data management center needs to be in constant communication with all other centers. Its staff has to have expertise in areas such as biostatistics, computer technology, epidemiology, regulatory policy, medicine, and management in order to respond expeditiously to daily problems that arise in a trial. These might range from simple questions, such as how to code a particular item on a questionnaire, to monitoring clinical site conduct. The single coordinating center, or the separate data analysis center, has responsibilities such as preparing data monitoring guidelines, conducting data analyses, and developing or modifying statistical methods. The staffs at these centers must be experienced, capable, responsive, and dedicated in order to handle their workloads in a timely fashion. A trial can succeed despite inadequate performance of one or two clinical centers, but a poorly performing coordinating center or data management center can materially affect the success of a multicenter trial. In extreme cases, a coordinating center may have to be changed midway through the trial. This causes serious delay and logistical problems. Thus, proper selection of the coordinating center is extraordinarily important.

A key element in any coordinating or analysis center is not only the presence of integrity, but the appearance of integrity. Any suspicion of conflict of interest can damage the trial. This is one of the reasons that pharmaceutical firms who support trials sometimes use outside institutions or organizations as coordinating centers. Because the personnel in the centers control the data and the analyses, they should be seen to have no overriding interest in the outcome of a trial. Meinert [22] has described the functions of the coordinating center in detail. See also Fisher et al. for a description of the operations of an independent data analysis center [23].

As noted, certain functions in a multicenter trial are best carried out by properly selected special centers. The advantages of centrally performing laboratory tests, reading x-rays, evaluating pathology specimens, or coding electrocardiograms include unbiased assessment, standardization and reduced variability, ease of quality control, and high-quality performance. The disadvantages of centralized determinations include the cost and time required for shipping, as well as the risk of losing study material. Even with electronic transfer of data, glitches may occur. It is also obvious that the centers selected to perform specialized activities need expertise in their particular fields. Equally important is the capacity to handle the large workloads of a multicenter trial with research-level quality. Despite careful selection of these centers, backlogs of work are a frequent source of frustration during the course of a trial.

Fourth, it is preferable for the planning committee to provide prospective investigators with a fairly detailed outline of the key elements of the study design as early as possible. This results in more efficient initiation of the trial and allows each investigator to better plan staffing and cost requirements. Rather than presenting a final protocol to the investigators, we recommended that all or selected representatives be given time to discuss and, if necessary, modify the trial design. This process allows them to contribute their own ideas, to have an opportunity to participate in the design of the trial, strengthening their commitment to it, and to become familiar with all aspects of the study. It may also improve the design. The investigators need a protocol that is acceptable to them and their colleagues at their local institution. This “buy-in” will improve participant recruitment, data collection, and final acceptance of the trial results. Depending on the complexity of the trial, several planning sessions prior to the start of participant recruitment may be needed for this process.

If there are many investigators and a number of difficult protocol decisions, it is useful to have specific groups or subsets of investigators address these issues during the planning stage. Working groups can focus on individual problems and prepare reports for the total body of investigators. Of course, if the initial outline has been well thought out and developed, few major design modifications will be necessary. Any design change needs to be carefully examined to ensure that the basic objectives and feasibility of the study are not threatened. This caveat applies particularly to modifications of participant eligibility criteria. Investigators are understandably concerned about their ability to enroll a sufficient number of participants. In an effort to make recruitment easier, they may favor less stringent eligibility criteria. Any such decisions need to be examined to ensure that they do not have an adverse impact on the objectives of the trial and on sample size requirements. The benefit of easier recruitment may be outweighed by the need for a larger sample size. Planning meetings also serve to make all investigators aware of the wide diversity of opinions. Inevitably, compromises consistent with good science must be reached on difficult issues, and some investigators may not be completely satisfied with all aspects of a trial. However, all are usually able to support the final design. All investigators in a cooperative trial must agree to follow the common study protocol.

Although a good protocol will provide guidance for all major issues that are anticipated, investigators will always have questions as they begin a trial that need to be addressed in a systematic way. This information should be shared with all investigators, in newsletters, in a question and answer format that could exist on a website, or (when necessary) with protocol amendments. This is part of a broader theme in multicenter trials: the importance of effective communication. It is the responsibility of coordinating centers to keep in frequent contact (by telephone, e-mail, texting, visits) with all the enrolling centers. An informative and interactive website can be helpful. The study leaders also need to maintain contact with the various centers and committees, closely monitoring the conduct of the trial.

Fifth, an organizational structure for the trial should be established with clear areas of responsibility and lines of authority and communication. Many have been developed [24–26]; the one outlined below has stood the test of time.

Steering Committee—This committee provides scientific direction for the study at the operational level. Its membership may be made up of some or all of those who were on the planning committee (including sponsor representation), plus a subset of investigators participating in the trial. In international trials, it has become conventional to have at least one “national coordinator” investigator from each major country to represent those investigators and to address country-specific issues. Depending on the length of the study, some investigators may be chosen or elected for part of the trial. Subcommittees are often established to consider on a study-wide level specific issues such as adherence, quality control, classification of response variables, and publication policies and review and then report to the Steering Committee.

It may also be important to authorize a small subgroup to make executive decisions between Steering Committee meetings. These committees are sometimes referred to as executive committees or as operations committees. Most “housekeeping” tasks and day-to-day decisions can be more easily accomplished in this manner. A large committee, for example, is unable to monitor a trial on a daily basis, write memoranda, or prepare agendas. Since committee meetings can rarely be called at short notice, issues requiring rapid decisions must be addressed by an executive group. It is important, however, that major questions be discussed with the investigators.

Subcommittees—Often, subcommittees of the Steering Committee are established. For example, there is often the need for a system for central evaluation of events, and this could be done by an Events Classification Subcommittee. Adjudication of events, with the participants’ identities and intervention groups blinded, helps to assure unbiased classification of reported events and to ensure consistent application of criteria for particular events. Other subcommittees might look for ways to improve participant accrual or adherence. In some trials, the subcommittee structure has become too complex and can lead to inefficiencies. Trials with few centers function best with a simple structure. If committees, subcommittees, and task forces multiply, the process of handling routine problems becomes difficult. Studies that involve multiple disciplines especially need a carefully thought out organizational structure. Investigators from different fields tend to look at issues from various perspectives. Although this variety can be beneficial, under some circumstances it can obstruct the orderly conduct of a trial. Investigators may seek to increase their own areas of responsibility and, in the process, change the scope of the study. What starts out as a moderately complex trial can end up being an almost unmanageable undertaking.

Data Monitoring Committee—This scientific body, which goes by various names (see Chap. 16), should be independent of the investigators and any sponsor of the trial. Its primary role, to the extent possible, is to ensure participant safety and study integrity. To accomplish this, it is charged with reviewing and approving the protocol; periodically monitoring baseline, harmful effects, and response variable data; and evaluating center performance [27]. In light of concerns about clinical-trial integrity, [28–30] the independence of this group is especially important. It usually reports to either the study sponsor or the chairperson of the planning or steering committee. The coordinating or data analysis center should present tabulated and graphic data and appropriate analyses to the data monitoring committee for review. The committee has the responsibility to recommend early termination in case of unanticipated harm, greater-than-expected benefit, or high likelihood of indifferent results (see Chap. 16). Members of this committee should be knowledgeable in the field under study, in clinical trials methodology, and in biostatistics. An ethicist and/or a participant advocate may also be part of this group. The responsibilities of the monitoring committee to the participants, as well as to the integrity of the study, should be clearly established and communicated to the participants. These responsibilities for participant safety are particularly important in double-blind studies, since the individual investigators are unaware of the group assignments and which group is associated with various adverse events.

Unfortunately, the organizational structure of many trials conducted by industry excludes meaningful involvement of independent experts in trial design, conduct, and analysis. There is a need for academic trialists, including those at agencies such as the National Institutes of Health, to work with the public and health care providers to advance the conduct, quality, and relevance of clinical trials that address health care priorities [31].

Sixth, despite special problems, multicenter trials should try to maintain standards of quality that do not differ from those in carefully conducted single-center trials (see Chap. 11). Strong emphasis should be placed on training and standardization so that the protocol is carried out in the intended fashion across centers and regions. It is obviously extremely important that staff at all centers understand the protocol definitions, and how to complete forms and perform tests. Differences in performance among centers, as well as between individuals in a single center, are unavoidable. They can, however, be minimized by proper training, certification procedures, retesting, and when necessary, retraining of staff. An attractive, functional, interactive website with updated training materials and other resources is an important tool in large trials. The National Heart, Lung, and Blood Institute (NHLBI)-sponsored International Study of Comparative Health Effectiveness with Medical and Invasive Approaches (ISCHEMIA) trial website provides such an example [32]. These efforts need to be implemented before a trial gets underway (See Chap. 11 for a discussion of quality control). In trials that require specific training and expertise, a clinical center should not be allowed to begin enrolling participants until it has demonstrated the capability of performing necessary procedures. Investigator meetings are generally important to the successful conduct of the trial because they provide opportunities to discuss common problems and review proper ways to collect data and complete study forms.

Seventh, there needs to be close monitoring of the performance of all centers. Participant recruitment, quality of data collection and processing, quality of laboratory procedures, adherence of participants to protocol, and loss to follow-up should be evaluated on an ongoing basis. Regulatory requirements for investigators are outlined in Chap. 22. Table 21.1 lists some of the major responsibilities of the principal investigator at enrolling centers.

Table 21.1

Principal investigator’s major responsibilities at research sites

1.  Be familiar with ethical principles (see Chap. 2), including as outlined in the Belmont Report: respect for persons, beneficence, justice

2.  Be familiar with US federal regulations as defined in the Code of Federal Regulations (CFR) (see Chap. 22)

  (a) Health and Human Services, for federally funded research

    i. 45 CRF 46 (Health and Human Services, Protection of Human Research Subjects)

    1. Subpart B (pregnant women)

    2. Subpart C (prisoners)

    3. Subpart D (children)

  (b) FDA, for FDA regulated products

    i. 21 CFR 50 (informed consent)

    ii. 21 CFR 54 (financial disclosure)

    iii. 21 CRF 56 (IRB)

  (c) Health and Human Services, for privacy including for all human subjects research

   i. 45 CFR 46, 160, 164 (HIPAA)

3.  Understand the requirements of the responsible Institutional Review Board (IRB) and the need to follow them

4.  Be responsible for oversight of the trial and delegation of research responsibilities, with appropriate training and experience of staff

5.  Recruit participants in a fair and equitable way (see Chap. 10)

6.  Develop process of informed consent (see Chap. 2), with IRB approval of that process, with consent obtained by the PI or a delegated research staff member who is identified as “key personnel” in the IRB approval; and maintain documentation of informed consent (generally for at least 3 years)

7.  Do not enroll patients without prior IRB approval, and not make changes to the protocol without prior IRB approval

8.  Comply with reporting requirements of adverse events, protocol deviations, unanticipated problems involving risks to participants or others, or irregularities (like loss of consent documentation) (see Chap. 12)

9.  Be available, or have a designated research staff member available, to participants to answer questions

10. Notify the IRB and seek approval for change to a new principal investigator

(The focus is on the essential need to protect the rights and welfare of research participants (adapted from Duke University and from United States Health and Human Services clinical research training materials))

Electronic tracking tools allow this to be done in an efficient and systematic way, as long as standardized reports effectively capture and display the information. It is important to track overall performance as well as performance by center.

Many industry-sponsored multicenter trials that employ contract research organizations conduct extensive auditing and quality assurance. This is quite costly and how much benefit it provides has been questioned [33]. See Chap. 11 for further discussion of this topic.

In most clinical trials, recruitment of participants is difficult. In a cooperative clinical trial, however, there is an opportunity for some clinical centers to compensate for the inadequate performance of other centers by exceeding their predetermined recruitment goals. The clinical centers should understand that, while friendly competition keeps everybody working, the real goal is overall success, and what some centers cannot do, another perhaps can. Therefore, it is important to encourage the good centers to recruit as many participants as possible. There may be a limit, however, if one center, region, or country (in the case of international trials) starts to dominate enrollment. At some point, recruitment might need to be capped if the study is to be seen as truly multicenter.

Eighth, publication, presentation, and authorship policies should be agreed upon in advance. Authorship becomes a critical issue when there are multiple investigators, many of whose academic careers depend on publications. There is no completely satisfactory way to recognize the contribution of each investigator. A common compromise is to put the study name immediately under the paper title and to acknowledge the writers of the paper, either in a footnote or under the title, next to the study name. All key investigators are then listed at the end of the paper. The policy may also vary according to the type of paper (main or subsidiary). The group authorship of manuscripts from multicenter trials was challenged by some medical journals and defended by others [34–36]. It remains common, but typically with an identified writing committee to take responsibility (see Chap. 20).

Involvement of representatives of the sponsor as authors of the main manuscripts from a major trial can be contentious, especially if it is a commercial firm that stands to benefit from a favorable presentation of the trial results. Most sponsors accept a hands-off policy and leave it to the investigators to write the scientific papers, although including sponsor members of the research team who provided important intellectual contributions can be appropriate. Typically, an industry sponsor is given 1 month to preview the main results manuscript, to allow time to deal with patent or regulatory issues. This review should not unnecessarily delay the publication of the main trial results. Regrettably, there are examples of interference that are in conflict with academic freedom. These policies should be clearly defined in the contract between the sponsor and the investigators.

In one four-center trial, the investigators at one of the centers reported their own findings before the total group had an opportunity to do so [37, 38]. Such an action is not compatible with a collaborative effort. It undermines the goal of a multicenter trial of having enough participants to answer a question and, perhaps more importantly, the trust among investigators. Academic institutions have taken a strong stand against this principle of collaboration and in defense of academic freedom for each investigator. However, we believe that those unwilling to abide by the rule for common authorship should not participate in collaborative studies.

Creating a publication charter in advance and having all parties agree to abide by it provides important protection against misunderstandings. However, fair recognition of junior staff will always be difficult [39]. Study leadership often gets credit and recognition for work done largely by people whose contributions may remain unknown to the scientific community. One way to alleviate this problem is to appoint as many capable junior staff as possible to subcommittees. Such staff should also be encouraged to develop studies ancillary to the main trial. This approach will enable them to claim authorship for their own work while using the basic structure of the trial to get access to participants and supporting data. Such ancillary studies may be performed on only a subgroup of participants and may not necessarily be related to the trial as a whole. Care must be taken to ensure that they do not interfere with the main effort, either through unblinding, by harming the participants, or by causing the participants to leave the trial. Sackett and Naylor discuss the issues for and against allowing publication of ancillary studies before the main trial is completed [40].

Globalization of Trials

As noted earlier, many multicenter clinical trials are international; there are several reasons for this. One, it provides greater numbers of potential participants, allowing for quicker accrual. Two, the broader populations may allow for wider generalization of results. It is not simply people from one country with one medical care system who are enrolled; the data from the trial apply to many sorts of people with very different medical systems. Three, it may be easier and less expensive to screen people in some regions. Even in NIH-sponsored trials, an increasing proportion of participants are being enrolled internationally, largely due to inability to enroll enough patients at centers in the United States [41] (see Fig. 21.1).

Fig. 21.1

International enrollment in NIH-sponsored randomized trials of coronary disease [41]

There are, however, limitations and concerns with globalization of trials. As discussed in Chap. 2, the ethics of enrolling participants from underdeveloped countries or areas can be problematic [42]. It is unethical to enter people into a trial simply to save money, or because the regulatory oversight is less rigorous, when there is little likelihood that the population will benefit from or have access to the trial intervention. Logistics of implementing international trials may be daunting. In addition to multi-language communication, there is the issue of translating forms and questionnaires. Not all forms, particularly those that have been validated in certain groups, may be usable in very different communities and cultures. Transporting drugs and other materials across borders may not be simple. In addition, each country has its own regulatory structure that must be negotiated.

Some countries may present particular challenges in regulatory approval, such as China where the process may take over a year for drug trials. In India, concerns over unethical trial practices have led to laws requiring trial sponsors to cover medical costs of trial-associated adverse outcomes and to requirements to video record the informed consent process. These regulatory and legal requirements resulted in unwillingness to include Indian sites in many trials. At least 35 NIH trials were put on hold in India in 2013, although many subsequently resumed [43, 44].

Interpretation of regionally diverse results may be questioned. Are the overall results relevant to all countries? Does the culture, social structure, or medical care system (including concomitant medications and other treatment) affect the outcome? Does each trial participant need minimal standard background care? If so, this must be specified in advance in the protocol. An example of a trial that examined effect by geography is the Platelet IIb/IIIa in Unstable angina: Receptor Suppression Using Integrilin Therapy (PURSUIT) trial [45]. Relative reductions in the primary response variable (a combination of death or myocardial infarction) varied among geographic regions. In trials of beta blockers in heart failure, there appears to have been a consistently lesser treatment effect in the United States than in other countries, for unclear reasons [46] (see Fig. 21.2).

Fig. 21.2

Effects of beta blockers on all-cause mortality in major heart failure trials [46]. Panel (a) is overall, panel (b) contrasts the United States and rest of world (ROW), with p values in panel (b) for effects in ROW, and NS referring to non-significant p-value for the overall effect in the U.S.

The Treatment of Preserved Cardiac Function Heart Failure With an Aldosterone Antagonist (TOPCAT) trial included about half of patients from Russia and Georgia and half from the Americas. The populations in these regions differed at baseline, with more of the patients in Russia and Georgia being characterized by prior myocardial infarction and prior heart failure hospitalization [47]. There was a four-fold higher rate of the primary outcome of cardiovascular death, aborted cardiac arrest, or heart failure hospitalization in Russia and Georgia than in the Americas, and the treatment benefit observed with spironolactone in the Americas was not seen in Russia and Georgia. Spironolactone did not have the degree of effect on laboratory values (potassium and creatinine) in Georgia and Russia that it did in the Americas [48]. Related regional differences in composition and outcome of populations with heart failure with preserved ejection fraction have been observed in other clinical trials [49]. These findings suggest that the diagnosis, management, outcomes, and response to therapy of heart failure with preserved ejection fraction may be different in different geographic regions. This regional heterogeneity will in turn impact on the results of clinical trials.

The Platelet inhibition and Patient Outcome (PLATO) trial, which studied ticagrelor versus clopidogrel after acute coronary syndromes, provides an even more striking example of heterogeneity of treatment effect according to country [50]. Overall, there was a 16% relative risk reduction in the primary composite outcome. In the United States, which included 8% of the patients, the hazard ratio was 1.27, and in other countries, 0.81, with a p value for the interaction of 0.0095 [51]. There is also evidence that it may be related to, and even explained by, the higher dose aspirin used in the United States [51].

In these examples, chance still may be the most likely explanation. However, investigators need to consider, in advance, whether combining results from geographically and culturally different sites is appropriate. In any case, if a robust and consistent treatment effect is desired to be demonstrated in the United States population, or in any specific population, enrolling a sufficiently large portion from that population is important. Vickers et al. [52] found that some countries tended to produce results more favorable to the new intervention than other countries, though publication bias was the likely reason.

Large, Simple Trials

Large, simple trials [53] are a subset of multicenter trials that typically involve a large number of participating centers, many of which are non-academic institutions representative of general practice. Education, training, and standardization may need to be more focused and streamlined compared with other trial models. For example, in streamlined trials background care may be left to the caring physician such that standard of care is the goal, although for many trials, encouraging high quality standard of care may be important for the results to be accepted as relevant. Clinician-investigators need to understand the basic concepts and intent of clinical trials and how the rules of research, which may sometimes seem arbitrary, [54] differ from the way they practice medicine (See Chap. 2). The reliance on hard endpoints such as all-cause mortality, and limited data collection, tends to reduce the need for elaborate quality control procedures.

Successful conduct of streamlined trials has become more difficult with more complex and heterogeneous regional regulatory requirements, which have caused large trials to be very expensive. The expense related to complexity and various barriers that do not result in improved quality has far reaching consequences, including resulting in an inability to conduct many trials that are necessary to guide clinical care. In response to these barriers, recommendations have been made to simplify procedures for large, simple trials [55–57].

The U.S. Food and Drug Administration (FDA), in partnership with Duke University in the Clinical Trials Transformation Initiative (CTTI), [58] has made a concerted effort to provide guidance to promote streamlining when appropriate. For example, in December 2012, a guidance was issued for simplified adverse event reporting in large, simple trials [59] and in August 2013, another guidance was issued for risk-based monitoring of trial conduct and data [60].

There are examples in which randomized clinical trials have been successfully conducted on the platform of clinical registries [61], such as the Thrombus Aspiration during ST-segment Elevation myocardial infarction (TASTE) trial that randomized over 7000 patients in less than 3 years (80% of all eligible acute myocardial infarctions in Sweden during the enrollment period) to thrombus aspiration or control for an estimated total US$300,000 marginal cost [62] (see Chap. 10, Fig. 10.​4). Another pragmatic trial, INforming Fresh versus Old Red cell Management (INFORM), is planning to randomize 31,497 patients undergoing blood transfusion to freshest versus standard (older) blood for transfusion at five medical centers in Canada, Australia, and the United States as of December 2014. In this trial, consent is waived and in-hospital data are collected using the electronic health record such that the cost is a fraction of what would be typical for a trial of this size [63].

Another example of a streamlined approach to integrating clinical trials and clinical practice comes from the NIH Health Care Systems Research Collaboratory. Launched in 2006, this program supports demonstration projects in which health care organizations partnered with researchers conduct pragmatic clinical trials in everyday health care settings. One such project was The Randomized Evaluation of Decolonization versus Universal Clearance to Eliminate MRSA (REDUCE MRSA), a cluster randomized trial of 43 hospitals (including 74 intensive care units and 74,256 patients) testing whether daily antiseptic baths and a nasal antibiotic were more effective than other procedures to decolonize patients to prevent staphylococcal infections in intensive care units [64]. The Collaboratory group has outlined key steps to develop successful partnerships between health care systems and researchers to conduct pragmatic clinical trials that address important gaps in knowledge to improve patient care. These steps include building partnerships, defining the important questions, assessing feasibility, involving stakeholders in the design, and implementing workflow [65].

References

1.

Fleiss JL. Multicentre clinical trials: Bradford Hill’s contributions and some subsequent developments. Stat Med 1982;1:353–359.

2.

Greenberg BG. Conduct of cooperative field and clinical trials. Am Stat 1959;13:13–28.

3.

Klimt CR. Principles of multi-center clinical studies. In Boissel JP, Klimt CR. Multi-center Controlled Trials. Principles and Problems. Paris: INSERM, 1979.

4.

Levin WC, Fink DJ, Porter S, et al. Cooperative clinical investigation: a modality of medical science. JAMA 1974;227:1295–1296.

5.

Coronary Drug Project Research Group. PL Canner (ed.). The Coronary Drug Project: methods and lessons of a multicenter clinical trial. Control Clin Trials 1983;4:273–541.

6.

ISIS-1 (Second International Study of lnfarct Survival) Collaborative Group. Randomised trial of intravenous atenolol among 16027 cases of suspected acute myocardial infarction: ISIS-1. Lancet 1986;328:57–66.

7.

ISIS-2 (Second International Study of lnfarct Survival) Collaborative Group. Randomised trial of intravenous streptokinase, oral aspirin, both, or neither among 17 187 cases of suspected acute myocardial infarction: ISIS-2. Lancet 1988;332:349–360.

8.

Schulz S, Mehilli J, Schömig A, Kastrati A. ISAR—a story of trials with impact on practice. Circ J 2010;74:1771–1778.

9.

Schulz S, Richardt G, Laugwitz KL, et al. Comparison of prasugrel and bivalirudin vs clopidogrel and heparin in patients with ST-segment elevation myocardial infarction: Design and rationale of the Bavarian Reperfusion Alternatives Evaluation (BRAVE) 4 trial. Clin Cardiol 2014;37:270–276.

10.

The GUSTO Investigators. An international randomized trial comparing four thrombolytic strategies for acute myocardial infarction. N Engl J Med 1993;329:673–682.

11.

Women’s Health Initiative. Available at http://​www.​nhlbi.​nih.​gov/​whi/​. Accessed January 14, 2015.

12.

Rossouw JE, Anderson GL, Prentice RL, et al. Risks and benefits of estrogen plus progestin in healthy postmenopausal women: principal results from the Women’s Health Initiative randomized controlled trial. JAMA 2002;288:321–333.

13.

Roth JA, Etzioni R, Waters TM, et al. Economic return from the Women’s Health Initiative estrogen plus progestin clinical trial: A modeling study. Ann Intern Med 2014;160:594–602.

14.

HPS2-THRIVE Collaborative Group, Landray MJ, Haynes R, et al. Effects of extended-release niacin with laropiprant in high-risk patients. N Engl J Med 2014; 371:203–212.

15.

National Cancer Institute National Clinical Trials Network. Available at http://​www.​cancer.​gov/​clinicaltrials/​nctn. Accessed January 14, 2015.

16.

HIV/AIDS Clinical Trials Networks. Available at http://​www.​niaid.​nih.​gov/​about/​organization/​daids/​Networks/​Pages/​daidsnetworks.​aspx. Accessed January 14, 2015.

17.

Gheorghe A, Roberts TE, Ives JC, Fletcher BR, Calvert M. Centre selection for clinical trials and the generalisability of results: a mixed methods study. PLoS One 2013;8:e56560.

18.

Katz A, Cohn G, Mashal A, Cristal N. Thrombolytic therapy for acute myocardial infarction in a 110-year-old man. Am J Cardiol 1993;71:1122–1123.

19.

ALLHAT Officers and Coordinators for the ALLHAT Collaborative Research Group. Major outcomes in high-risk hypertensive patients randomized to angiotensin-converting enzyme inhibitor or calcium channel blocker vs diuretic. The Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT). JAMA 2002;288:2981–2997.

20.

Croke G. Recruitment for the National Cooperative Gallstone Study. Clin Pharmacol Ther 1979;25:691–694.

21.

Shea S, Bigger JT, Campion J, et al. Enrollment in clinical trials: institutional factors affecting enrollment in the Cardiac Arrhythmia Suppression Trial (CAST). Control Clin Trials 1992;13:466–486.

22.

Meinert CL. Clinical Trials. Design, Conduct and Analysis. Second Edition. New York: Oxford University Press, 2012.

23.

Fisher MR, Roecker EB, DeMets DL. The role of an independent statistical analysis center in the industry-modified National Institutes of Health model. Drug Inf J 2001;35:115–129.

24.

Byington RP, for the Beta-Blocker Heart Attack Trial Research Group. Beta-Blocker Heart Attack Trial: design, methods, and baseline results. Control Clin Trials 1984;5:382–437.

25.

Meinert CL. Organization of multicenter clinical trials. Control Clin Trials 1981;1:305–312.

26.

Carbone PP, Tormey DC. Organizing multicenter trials: lessons from the cooperative oncology groups. Prev Med 1991;20:162–169.

27.

Friedman L, DeMets D. The data monitoring committee: how it operates and why. IRB 1981;3:6–8.

28.

Fleming TR, DeMets DL. Monitoring of clinical trials: issues and recommendations. Control Clin Trials 1993;14:183–197.

29.

Angell M, Kassirer JP. Setting the research straight in the breast-cancer trials. N Engl J Med 1994;330:1448–1450.

30.

Cohen J. Clinical trial monitoring: hit or miss? Science 1994; 264;1534–1537.

31.

DeMets DL, Califf RM. A historical perspective on clinical trials innovation and leadership: where have the academics gone? JAMA 2011;305:713–714.

32.

International Study of Comparative Health Effectiveness with Medical and Invasive Approaches (ISCHEMIA) trial. Available at https://​www.​ischemiatrial.​org/​. Accessed January 14, 2015.

33.

Eisenstein EL, Lemons PW II, Tardiff BE, et al. Reducing the costs of phase III cardiovascular clinical trials. Am Heart J 2005;149:482–488.

34.

Kassirer JP, Angell M. On authorship and acknowledgments. N Engl J Med 1991;325:1510–1512.

35.

Goldberg MF. Changes in the archives. Arch Ophthalmol 1993;111:39–40.

36.

Meinert CL. In defense of the corporate author for multicenter trials. Control Clin Trials 1993;14:255–260.

37.

Winston DJ, Ho WG, Gale RP. Prophylactic granulocyte transfusions during chemotherapy of acute nonlymphocytic leukemia. Ann Intern Med 1981;94:616–622.

38.

Strauss RG, Connett JE, Gale RP, et al. A controlled trial of prophylactic granulocyte transfusions during initial induction chemotherapy for acute myelogenous leukemia. N Engl J Med 1981; 305:597–603.

39.

Remington RD. Problems of university-based scientists associated with clinical trials. Clin Pharmacol Ther 1979; 25:662–665.

40.

Sackett DL, Naylor CD. Should there be early publication of ancillary studies prior to the first primary report of an unblinded randomized clinical trial? J Clin Epidemiol 1993;46:395–402.

41.

Kim ES, Carrigan TP, Menon V. International participation in cardiovascular randomized controlled trials sponsored by the National Heart, Lung, and Blood Institute. J Am Coll Cardiol 2011;58:671–676.

42.

Glickman SW, McHutchison JG, Peterson ED, et al. Ethical and scientific implications of the globalization of clinical research. N Engl J Med 2009;360:816–823.

43.

Kakkar AK. India puts informed consent on camera. Science 2014;344:150–151.

44.

Reardon S. NIH makes wary return to India. Nature 2014;506:143.

45.

Akkerhuis KM, Deckers JW, Boersma E, et al, for the PURSUIT Investigators. Geographic variability in outcomes within an international trial of glycoprotein IIb/IIIa inhibition in patients with acute coronary syndromes: results from PURSUIT. Eur Heart J 2000;21:371–381.

46.

O’Connor CM, Fiuzat M, Swedberg K, et al. Influence of global region on outcomes in heart failure β-blocker trials. J Am Coll Cardiol 2011;58:915–922.

47.

Pitt B, Pfeffer MA, Assmann SF, et al. Spironolactone for heart failure with preserved ejection fraction. N Engl J Med 2014;370:1383–1392.

48.

Pfeffer MA, Claggett B, Assmann SF, et al. Regional variation in patients and outcomes in the treatment of preserved cardiac function heart failure with an aldosterone antagonist (TOPCAT) trial. Circulation 2015;131:34–42.

49.

Kristensen SL, Køber L, Jhund PS, et al. International geographic variation in event rates in trials of heart failure with preserved and reduced ejection fraction. Circulation 2015;131:43–53.

50.

Mahaffey KW, Wojdyla DM, Carroll K, et al. Ticagrelor compared with clopidogrel by geographic region in the Platelet Inhibition and Patient Outcomes (PLATO) trial. Circulation 2011;124:544–554.

51.

Carroll KJ, Fleming TR. Statistical evaluation and analysis of regional interactions: The PLATO Trial Case Study. Stat Biopharm Res 2013;5:91–101.

52.

Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials 1998;19:159–166.

53.

Yusuf S, Collins R, Peto R. Why do we need some large, simple randomized trials? Stat Med 1984;3:409–422.

54.

Sugarman J, Califf RM. Ethics and regulatory complexities for pragmatic clinical trials. JAMA 2014;311:2381–2382.

55.

Yusuf S, Bosch J, Devereaux PJ, et al. Sensible guidelines for the conduct of large randomized trials. Clin Trials 2008;5:38–39.

56.

Reith C, Landray M, Devereaux PJ, et al. Randomized clinical trials—removing unnecessary obstacles. N Engl J Med 2013;369:1061–1065.

57.

Eapen ZJ, Lauer MS, Temple RJ. The imperative of overcoming barriers to the conduct of large, simple trials. JAMA 2014;311:1397–1398.

58.

Clinical Trials Transformation Initiative (CTTI). Available at http://​www.​ctti-clinicaltrials.​org/​. Accessed January 14, 2015.

59.

Guidance for Industry and Investigators. Safety Reporting Requirements for INDs and BA/BE Studies. Available at http://​www.​fda.​gov/​downloads/​Drugs/​GuidanceComplian​ceRegulatoryInfo​rmation/​Guidances/​UCM227351.​pdf. Accessed January 14, 2015.

60.

Guidance for Industry. Oversight of Clinical Investigations—A Risk-Based Approach to Monitoring. Available at http://​www.​fda.​gov/​downloads/​Drugs/​…/​Guidances/​UCM269919.​pdf. Accessed January 14, 2015.

61.

Lauer MS, D’Agostino RB Sr. The randomized registry trial—the next disruptive technology in clinical research? N Engl J Med 2013;369:1579–1581.

62.

Fröbert O, Lagerqvist B, Olivecrona GK, et. al. Thrombus aspiration during ST-segment elevation myocardial infarction. N Engl J Med 2013;369:1587–1597.

63.

INforming Fresh versus Old Red cell Management (INFORM). Available at http://​www.​controlled-trials.​com/​ISRCTN08118744. Accessed January 14, 2015.

64.

Huang SS, Septimus E, Kleinman K, et al. Targeted versus universal decolonization to prevent ICU infection. N Engl J Med 2013; 368:2255–2265.

65.

Johnson KE, Tachibana C, Coronado GD, et al. A guide to research partnerships for pragmatic clinical trials. BMJ 2014;349:g6826.

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_22

22. Regulatory Issues

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

The purpose of a clinical trial is to assess reliably the benefits and harms of an intervention. In this book, we have focused on good clinical trial design, conduct, monitoring, and analysis principles. For many clinical trials, including those involving new drugs, devices, or biologics, or new indications for existing interventions, there are national and local regulations that must be followed in order to conduct clinical research, including the types of trials we have discussed. Furthermore, in order for an industry sponsor to market a medical product, regulatory agency approval is required in the U.S. and much of the rest of the world. The primary goal of this chapter is not to summarize all of the regulations relating to medical products; that is beyond the scope of this book. Rather, it is to focus on those laws, regulations and policies that bear on the design, conduct, and reporting of clinical trials. Even then, it will be highly selective, limited to those aspects that we think are most relevant and we will concentrate on U.S. laws, policies and regulations.

In the U.S., the Food and Drug Administration is the agency that reviews products for marketing and use to prevent disease, diagnose disease or treat individuals or animals, regardless whether the product was developed by industry or other research institutions. The FDA is a large organization composed of seven Centers: Center for Drug Evaluation & Research (CDER), Center for Devices & Radiological Health (CDRH), Center for Biologics Evaluation and Research (CBER), Center for Food Safety and Applied Nutrition, Center for Toxicological Research, Center for Veterinary Medicine and Center for Tobacco Products [1]. (See Fig. 22.1.) Most of these Centers have divisions that reflect disease types such as cardiovascular and renal diseases, or types of interventions such as cardiovascular devices or surgical devices. While each Center abides by the laws under which it was created and follows the regulations written to describe its standards and procedures, there are important differences in the kinds of study designs (control groups, end points, blinding, etc.) that reflect the specific disease area or product type.

Fig. 22.1

FDA organizational chart 2013

In Europe, the European Medicines Agency (EMA) has regulations that apply to those countries under its purview [2]. The development of the International Conference on Harmonisation (ICH) guidance documents [3] was aimed to make clinical trial standards more comparable internationally. But regulatory agency rules and guidelines still differ among countries, and these differences may contribute to different approval decisions [4]. Regulations and guidelines in Europe have changed over time [5]. Importantly, rules of conduct differ in the various countries in which a multinational trial might be conducted. Even within a country with a common standard, differences in judgment in applying those standards may exist. Thus, another purpose of this chapter is to provide links and other sites to which investigators can go for fuller and more current information and assistance.

The differences can be challenging for a new investigator to navigate. Nevertheless, investigators must become knowledgeable if they want to develop a successful clinical trial program.

Fundamental Point

When designing and conducting a clinical trial, investigators must know and follow national, state, and institutional regulations that are designed to protect research integrity and participant safety.

Background

Overview

Research under the Code of Federal Regulations is defined as “a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge” [6]. Clearly, clinical trials fall into that definition.

Clinical research regulations (or rules) and guidelines in the U.S. are created at the federal level by the Department of Health and Human Services, the Office of Human Research Protection and the Food and Drug Administration (FDA) and include several guidance documents [6–9]. In addition there are local requirements from universities and other research institutions, in response to some of the federal guidelines. Clinical researchers must be familiar and be in compliance with these regulations and guidances. Finally, the FDA requires investigators and trial sponsors to register all trials with ClinicalTrials.gov [10], providing basic information about the design of the trial within 21 days after the first participant is enrolled, and within 1 year after trial completion to submit the overall results to the same web site [11], as discussed below.

Other chapters in the book cover regulatory issues as they concern topics discussed in those chapters (Table 22.1). Chapter 1 discusses clinical trial phases. Chapter 2 contains discussions of trials in emergency settings and studies that enroll vulnerable populations, which have special regulatory requirements, as well as requirements for Institutional Review Boards that are mandated in the Code of Federal Regulations [6], sometimes referred to as the Common Rule. These will not be repeated here. Noninferiority trials, the use of adaptive designs, and cross-over trials, all of which might involve input from regulatory agencies, are discussed in Chap. 5. In this chapter, we will focus on pretrial requirements, trial conduct, and posttrial requirements.

Table 22.1

List of regulatory issues discussed in other chapters

Pages

Chapter 1

Trial Phases

4–10

Chapter 2

Ethics Committees

25, 28–29, 31–34, 36, 39

Informed Consent

34–37, Table 2.​2

Trials in Emergency Settings

36

Trials in Vulnerable Populations

37

Chapter 3

Use of Surrogate Outcomes

62–65

Chapter 5

Cross-Over Trials

102–103

Noninferiority Trials

109–113

Adaptive Designs

114–115

Chapter 7

Blinding of Nonpharmacologic Trials

154

Chapter 9

Pharmacogenetic Markers

206

Chapter 11

Audits

248–250

Chapter 12

Assessment of Harm

256

Boxed Warnings and Drug Approval Withdrawal

258

Classification of Adverse Events

260

Reporting Adverse Events

266

Recommendations for Assessing and Reporting Harm

273

Chapter 13

Health Related Quality of Life/Patient-Reported Outcomes

279

Chapter 16

Data Monitoring Committees

345–350, 367

Chapter 19

Data Cleanup and Verification

469

Storage of Data and Materials

470–471

Chapter 20

Data Sharing

493–494

Chapter 21

Globalization of Trials

511–514

Site Investigator Responsibilities

510, Table 21.​1

History

Regulation of drugs, devices, and other medical products has a long history [12, 13]. In many countries, including the U.S., various laws and amendments require that new products (i.e., those not yet marketed) such as drugs, biologics or devices be proven safe and effective before they are approved for marketing. As medical practice and our understanding of how interventions work improves, and as a result of several egregious events [14], these regulations have evolved over the years. Following the Pure Food and Drug Act of 1906, a key event in the U.S. was the passage of the Food, Drug, and Cosmetic Act of 1938 [15]. Among other things, this act required that new drugs be shown safe and authorized standards of quality. In 1962, the Kefauver-Harris Drug Amendments required that the effectiveness of products be shown before marketing [16]. They said that the required “substantial evidence” of effectiveness could be demonstrated only on the basis of “adequate and well-controlled trials.” The “s” at the end of “trials” is important as it implies that more than one trial is needed. The Medical Device Amendment in 1976 provided guidance on how devices were classified for regulation and approval [17]. The Safe Medical Device Act (1990) further expanded the role of the FDA [18].

In recent years, there have been rules that allow for making drugs available as rapidly as possible, including four approaches: fast track, breakthrough therapy, accelerated approval, and priority review [19]. Fast track is aimed at expediting review of drugs filling unmet clinical needs for serious conditions. Breakthrough therapy is a process for drugs that have early evidence of improvement over available treatments for serious conditions. Accelerated approval is for unmet clinical needs for serious conditions using surrogate endpoints with a requirement for further studies. The priority review designation provides for an FDA goal of regulatory action within 6 months (compared to 10 months under standard review), based on anticipated significant improvements in safety and/or efficacy for serious conditions.

Regulatory Requirements

Table 22.2 lists key actions and responsibilities required of investigators (both lead and other) conducting trials that fall under the purview of the FDA.

Table 22.2

Actions and responsibilities of investigators conducting clinical trials under auspices of FDA

Trial

Site

Leadership

Investigators

Pretrial

 Ethics Training

  Principles of Research

x

x

  IRB Requirements

x

x

  Informed Consent Process

x

x

 Knowledge of Basic Regulations

  45 CFR 46 (Common Rule)

x

x

  21 CFR 50 (FDA Regulations)

x

x

  45 CFR 160 (Privacy Act)

x

x

 IND or IDE Completion

  Pre-clinical materials and references

x

  Final protocol (allowing FDA up to 30 days for review)

x

  Information Showing Competence of Investigator(s)

x

x

  Information Showing Adequacy of Facilities

x

x

  Registration with ClinicalTrials.gov

x

 IRB Approval

x

x

Conduct

 Site Monitoring

x

 Data Monitoring

x

 Other Quality Assurance Activities

x

 Reporting to IRB(s) and FDA

  Protocol Modifications

x

  Investigator Changes

x

  Safety Reports

   Serious Adverse Events

x

x

   Routine

x

Posttrial

 Submission of data and documents to FDA (if seeking product approval)

  Protocol

x

  Completed Case Report Forms

x

  SAE Reports

x

  Product Accountability

x

  Presentation to Advisory Committee

x

  Possible Conduct of Post-Approval Studies

x

x

 Publication of Trial Results

x

x

 Timely Submission of Data to ClinicalTrials.gov

x

Although the principal investigator (PI) is traditionally considered to be the individual designated as being responsible for the clinical trial, from a regulatory perspective, it is the sponsor who has primary responsibility. “The sponsor may be an individual or pharmaceutical company, governmental agency, academic institution, private organization, or other organization. The sponsor does not actually conduct the investigation unless the sponsor is a sponsor-investigator” [20]. Thus, being PI on a grant that funds the clinical trial is not the same as being a sponsor. The sponsor is responsible for seeing that all of the documentation, approvals, and reporting requirements, including under the purview of the IRB, are fulfilled. The same individual often serves in both capacities, but they need not be the same. As listed at the end of this chapter, many guidance documents are available on the web to help familiarize investigators beyond the level of detail presented here. The National Institutes of Health and many universities require clinical researchers to take courses or workshops (often online) on research ethics and regulations, and many research institutions provide them. See Chap. 2 for specific resources.

The U.S. Health Insurance Portability and Accountability Act (HIPAA), passed in 1996, includes the Privacy Rule that protects the privacy of individually identifiable health information [21]. The HIPAA Privacy Rule gives conditions under which protected health information (PHI) may be used or shared with individuals or what is referred to as covered entities for conducting medical research. Protected health information refers to patient names, date of birth or other identifying dates, telephone numbers, email addresses, Social Security numbers, medical record numbers, photographs or any other identifying information. The HIPAA establishes conditions under which PHI data collected for research may be used or disclosed for research purposes, including who may obtain such information. Trial participants must be informed of uses and any disclosures of their medical information and disclosure of PHI data requires the permission of the individual or special approval procedures. If clinical trial data are shared outside of the trial that collected the data, they must be de-identified [22]. These HIPAA requirements can bring challenges in carrying out clinical trials. The investigator is responsible for doing this correctly. Often, these issues can be addressed in the informed consent process if anticipated.

Trial Phases

Clinical trial phases are discussed in Chap. 1. It should be noted that traditionally it was the drug trials that were divided into phases I, II, III, and IV [23]. Guidelines for early phase studies for biologics have also been developed [24]. These take into account the special needs of development and testing of cellular and gene therapy products and other biologics.

Before marketing of a new product or for a new indication of an already approved product, late phase trials are typically conducted. Sometimes, though, new products or new indications may be approved on the basis of trials that do not use clinical outcomes but rather an intermediate outcome as a surrogate endpoint With devices, the situation is even more variable, as modifications of approved devices (through the 510 (k) process [25]) may not require any clinical trials when a suitable combination of preclinical data (i.e., bench testing, animal model, and computational modeling) can be demonstrated to show substantial equivalence to the predicate device.

Pretrial Requirements

What kinds of trials do and do not require regulatory agency approval? Trials that involve new drugs, devices, and biologics that are not marketed require approval prior to conduct. Interventions that are already approved for the indication that is being studied in the trial do not generally need to be submitted for FDA regulatory review. For example, a drug may have been approved on the basis of a surrogate outcome, but an investigator now wants to evaluate it using a clinical outcome. Though, if the trial is to test a new indication or to evaluate the intervention administered in a different way or using a different dose, regulatory agency approval would typically be required [26, 27]. If new information is to be generated with the intent of including that information in the drug label or in advertising, an Investigational New Drug Application (IND) is needed [28]. Non-drug and non-device trials, where the intervention might consist of training, education, or surgical procedures do not, as a rule, require prior approval by the FDA, although other countries may require national regulatory approval even for these trials.

The FDA and the EMA websites include current clinical trials guidance documents, as well as forms that need to be completed for trials of drugs or biologics and devices [2, 9]. All investigators considering conducting clinical trials that might fall under FDA or EMA guidance should consult these materials before designing the trial. The version of the ICH document E6 (Good Clinical Practice: Consolidated Guidance) [29] that was amended in June, 2014 contains considerable information concerning the design and conduct of clinical trials and is generally consistent with the fundamental principles outlined in this book. While this document was created with a focus on all phases of pharmaceutical trials, and it has features that are overly complex for many large, simple or pragmatic trials, it does allow for some flexibility [30].

In advance, it is often useful and recommended to meet with staff of regulatory agencies to discuss the planned protocol and the proposed data to be collected, especially for phase III studies. This is especially true in an area where scientific knowledge is rapidly changing or if other than a standard design and outcomes are being considered. For example, if any of the adaptive designs will be used (see Chap. 5), it may be advisable to solicit advice from the FDA. Similarly, if the outcomes that will be used are not ones that are generally accepted for the condition being studied, discussions with regulatory agency staff are essential. The FDA may agree that some of the proposed data collection is not necessary for their approval process and offers an opportunity for some reduction in effort and cost. Additionally, studies in children or other vulnerable populations, and trials in emergency settings have special regulations and requirements of which investigators need to be aware.

The draft protocol and the statistical analysis plan along with the data monitoring charter and the monitoring plan, if appropriate, should be submitted to agencies before the trial begins. For complicated Bayesian or adaptive designs the actual computer code may be requested by a regulatory agency to independently assess accuracy.

Conduct

Protocol amendments for a trial being conducted under an Investigative New Drug (IND) application should be submitted prior to implementation. Also, after initiation of the trial, investigators and/or sponsors must report adverse events to the regulatory agencies. Generally, this is as an annual report. However, serious adverse events, particularly those that are unexpected or life-threatening, need to be reported in a more prompt manner. For FDA definitions of serious, life-threatening, and unexpected, and what actions need to be taken, see Chap. 12. As a rule, those that are unexpected (e.g., not related to the condition being studied nor in the drug investigator brochure or the package insert) need to be submitted in a timely way as safety reports. Historically, investigators reported to the trial sponsor any and all adverse events they discovered with the trial participant, and in turn these reports were sent to the FDA and other regulatory agencies as well as to all investigators in the trial worldwide, who in turn, sent them to their ethics committees. This resulted in a flood of individual adverse event reports that by themselves were largely uninterpretable. It has been recognized that this extensive adverse event reporting is not only unhelpful, but can be harmful to trial quality, since it diverts limited resources from aspects of trial conduct that are more important for quality.

As a result, in 2011 the FDA issued a new Investigative New Drug (IND) Rule for drugs that tried to reduce the number of adverse event reports that were not informative [31]. In the revision, the sponsor of the trial is to review the investigator reported events and determine if these events are Serious Unexpected Suspected Adverse Reactions (SUSARs) before they are reported to the FDA. Serious adverse events (SAEs), defined as events that are fatal, life threatening, require hospitalization, or result in permanent injury, must be reported in a more expedited fashion (e.g., less than 15 days from identification). Unexpected refers to events that are not listed in the Investigator Brochure or the package insert or other relevant documents regarding the product. A suspected adverse reaction is an event that has a reasonable probability of being caused by the intervention and is otherwise uncommon in the treated population. An event may be a SUSAR if it is: 1) a single occurrence of an uncommon serious adverse event that very rarely occurs spontaneously (e.g., Stevens-Johnson syndrome), 2) one or more occurrences of event that does not commonly occur (e.g., tendon rupture), or 3) an aggregate analysis of a specific event in a clinical trial observed more frequently on drug than control [31]. Implementing this new IND rule has been challenging because sponsors have been reluctant to unblind even small numbers of participants with these SAEs while the FDA has said that the needed assessment will generally require unblinding of certain parties within the sponsor organization in a way that will not damage the study integrity [32]. Those parties who are unblinded should not be those involved in the conduct of the trial and they should not be investigators.

It may not be necessary to collect adverse events that are non-serious and that do not result in discontinuation of study drug after data have been collected on a certain number of participants(e.g., 2,000 to 5,000). Whether to collect all adverse events for a non-approved drug needs to be negotiated with the FDA before the trial begins. Serious but expected adverse events (related to the drug and/or expected in the course of the disease) may be collected on the case report form, and reported to the FDA in a systematic way, again with prior agreement of the agency.

A certain amount of quality assurance by the study sponsor is essential (see Chap. 11). Obviously, those that ensure proper assignment to intervention and control (including the randomization process), appropriate intervention, and outcome assessment are important, as are those that guarantee ethical standards (e.g., informed consent). Ongoing measurement, feedback, and improvement of these parameters are necessary during the conduct of the trial. Corrective action can be taken so as to prevent continuation of data collection problems and poor application of the protocol. Source data verification of all variables, and using this information simply at the end of the trial for documentation, is usually not helpful because the trial is over and it is too late to take corrective action. However, regulatory agencies reserve the authority to check on data that have been collected, including by means of visits to clinic sites. Investigators should be prepared for such visits, particularly at the end of trials that are viewed as yielding practice-changing outcomes. These visits may also review documentation of informed consent and study drug reconciliation, accounting for the amount of drug received and the amount dispensed according to the protocol. Standards for computerized data systems and record retention are included in guidance documents [33]. It should be emphasized that site visits may be of three sorts; routine, structured, and for cause. See Chap. 11 for a discussion of the kinds of audits.

Often, however, the effort spent on quality assurance is beyond what is needed for important aspects of trial quality and required by regulatory agencies [34–37]. Having unbiased assessment of key primary outcomes without an unusually large percentage of missing data will do far more to promote good quality clinical research than spending time and resources ensuring that secondary and tertiary measures are perfectly performed. Additionally, site visits to clinics for data verification are often unnecessary, as most key monitoring can be done centrally. There may be other reasons for site visits, though. For example, they help to assure appropriate training of research personnel and adequate understanding of the protocol and the informed consent process. Regulatory agencies have given mixed messages regarding the kinds and amount of essential quality assurance. Therefore, many clinical trial sponsors, especially those from pharmaceutical and device companies, have typically engaged in exhaustive quality assurance. Yet the results of many trials conducted by others (e.g., the National Institutes of Health) have been accepted by regulatory agencies despite less extensive quality assurance. Central and “risk based” monitoring is within the FDA guidelines [38] and should be actively discussed with the agency in early phase meetings to determine if appropriate.

The U.S. law requires that clinical trials conducted in emergency settings, when informed consent is unobtainable, have a data monitoring committee [39, 40]. With that one exception, data monitoring committees are not required by law. FDA guidelines, however, discuss the importance of an independent data monitoring committee when the trial outcomes entail mortality or major morbidity, when there are major risks to the participants, or when having such a committee will “help assure the scientific validity of the trial” [40]. There is considerable emphasis on the independence of the committee members and on keeping the trial sponsor uninformed of interim data by intervention assignment. The European Medicines Agency has issued similar guidelines [41]. The International Conference on Harmonisation (ICH) [42] and the World Health Organization [43] also provide guidelines that are generally consistent with those of the FDA. For a fuller discussion of regulatory guidelines and data monitoring committees, see Chap. 16, and Ellenberg, Fleming, DeMets [44].

Interventions: Drugs

The classic structure of clinical trials, with its phases, placebo control, and blinding, derive from trials of drugs. Most regulatory agencies require that new drugs or drugs being tested in new settings or for new indications undergo the kind of clinical trials described in this book. Obviously, depending on the situation, the comparison may be another drug already proven to be beneficial or accepted as standard therapy, rather than placebo (unless the placebo is on top of standard therapy), and the trial could be designed either as a superiority study or as a noninferiority trial. In some circumstances (see Chap. 5), crossover or other special designs might be used.

Approval of drugs generally falls into the responsibility of the FDA’s Center for Drug Evaluation and Research, better known as CDER. (This is somewhat of an oversimplification since some biologics can be viewed as drugs but are reviewed by the Center for Biologics Evaluation and Research.) Within CDER, there are many Divisions to handle drugs for different disease entities. Divisions may use external advisory committees to assist them in their evaluations. Criteria provided in the FDA and ICH guidelines are applied in the approval process for level of evidence.

Prior to initiating the evaluation of a new drug in humans, an Investigational New Drug (IND) application must be submitted [26]. One basis for this requirement is the federal law that requires such submission before a drug can be transported across state lines for research purposes. During early phase pre-clinical development, sponsor and investigators are attempting to establish some evidence of favorable drug activity and that it is reasonably safe to administer to humans for initial testing [23]. This generally means that the molecule has been screened for pharmacologic activity and toxicity in animal models. As the drug development progresses under the IND through the various phases, typically including two phase III trials, sponsors will submit all of their data as part of a New Drug Application (NDA) for approval for sale and marketing in the U.S.

Safety standards are not different for expedited review, accelerated approval or regular approval, but the efficacy requirements are different, as accelerated approval is based on an intermediate marker used as a surrogate outcome, with requirements for further studies using clinical outcomes. Post-market safety issues after approval of a drug that may later turn out to be not beneficial, or even harmful, can arise with any of the approval strategies. In 2012, the FDA approved the use of ponatinib for chronic myeloid leukemia, under an accelerated approval pathway, on the basis of hematologic and cytogentic reponses as the primary outcome [45]. Subsequently, increases in cardiovascular, cerebrovascular, and peripheral vascular thromboses were observed. This led the FDA to at first suspend, and then allow limited marketing of the drug. The FDA approved bedaquiline for drug-resistant tuberculosis on an accelerated basis for a serious unmet need using results from a trial showing greater conversion of sputum culture from positive to negative [46, 47]. This was done despite more deaths in the bedaquiline group than the placebo group (10 out of 79 vs. 2 out of 81), in part because of the urgent need for effective anti-tuberculosis drugs. A drug that seemed to clear sputum seems unlikely to increase death from tuberculosis and renders the patient noncontagious. The fact that many of the deaths did not seem to be drug related and occurred long after the patients were off treatment were also determining factors in the accelerated approval [48]. However, as with any accelerated approval based on a surrogate, the FDA required that a confirmatory trial be conducted.

Interventions: Devices

Approval of devices in the U.S. has usually not required the same kind of evidence as approval of drugs since regulations for device approval were developed separately and at different times [17, 18]. In many cases the mechanism of action and performance of a device can be adequately assessed without a large clinical trial. Examples might be diagnostic coronary catheters or electrocardiographic machines where proper preclinical bench testing is what is needed to assess adequate performance. On the other hand, with the burgeoning importance of device technology, well designed and performed clinical trials to assess safety and effectiveness may be needed to properly assess devices. Recent examples include drug-eluting stents and percutaneous aortic valve devices. Device review and approval falls under the FDA’s Center for Device and Radiological Health (or CDRH). Similar to CDER, CDRH has internal divisions based on disease or device type (e.g., cardiovascular devices or surgical devices) and also use external independent advisory committees to assist in the review and approval process.

Medical device development often has important differences from drug development. Drugs do not often change over time but devices are continually being changed (improved or modified) based on bench or clinical performance. With complex devices the performance of a device, and subsequent clinical results, may be operator dependent, which is not typically the case for drugs. Device development may be led by visualization of the performance of the device that is clearer than predicting the action and effects of a drug. FDA device laws and regulations traditionally required only one trial where drugs typically require a minimum of two trials although there is now some flexibility in that due to the 1997 FDA modernization act [49].

Unlike drug regulation which utilizes a reasonable uniform pathway for regulation, devices are classified into one of three categories or classes that affect the standards for approval and the approval process [50]. These classes are based on the level of control necessary to establish safety and effectiveness. Class I devices have minimal risk and are defined as those not intended to support or maintain life and may not present any risk (e.g., surgical gloves). Class II devices have moderate risk and are designed to perform as indicated without causing harm or injury (e.g., infusion pumps, diagnostic catheters, guidewires). Class III devices are high risk and generally support or sustain human life or present a potential for an unreasonable risk of illness or injury (e.g., pacemakers, defibrillators, heart valves). These devices require FDA approval of a premarket approval application (PMA) Due to the complexity of these devices, extensive preclinical and clinical testing are often required prior to approval, making this application process in many ways similar to the standard drug approval process [51]. A 510 (k) premarket notification [52] allows the FDA to evaluate whether the proposed device is essentially equivalent to a predicate device already cleared via a 510 (k). This might be the case for a modified model of the original predicate or a competitor’s “equivalent” model. An investigative device exemption (IDE) is much like an IND for drugs in that it gives the manufacturer permission to conduct trials on the device, usually in preparation for a PMA submission [53, 54].

A PMA device is considered safe when, based on valid scientific evidence, the probable benefits outweigh the probable risks as long as the device is used according to conditions for which it was intended. A device is considered effective if the benefits are clinically significant. As with drugs, there are no perfect intermediate outcomes to be used as surrogates for clinical outcomes in trials of devices, but CDRH must often rely on them. Moreover, many important PMA devices are chronic implants where significant device failures may occur after the intermediate time point assessed in a usual FDA device approval trial. As a result, CDRH relies heavily on a “total product life cycle” regulatory strategy where post market approval studies are an important and necessary part of device assessment.

All the principles of good clinical trial development and conduct discussed in this text are relevant for planning an appropriate device development strategy. It is therefore often the case that when assessing new types of medical device technologies, a randomized clinical trial will be required for demonstration of safety and effectiveness. However, when considering the often small to moderate iterations that occur over time for a particular device and/or the maturation in basic device design that often occurs for a given device area, it may be reasonable to consider other trial designs (i.e. nonrandomized) as being appropriate for demonstration of safety and effectiveness. The usual cautions apply to the use of nonrandomized designs. Reference to the FDA CDRH guidance document on Clinical Trial Design of Device Trials as well as consultation is therefore recommended [55].

Given that many devices are no longer totally or predominantly external, but may be implanted and remain in the body for some years (e.g., pacemakers, defibrillators, stents), concerns about adverse events occurring long after the device implantation also need to be considered. The optimal system for medical device development and regulatory approval remains controversial. Unlike the U.S., the European System of Regulation does not require demonstration of clinical effectiveness for high risk devices prior to approval. On the other hand Dhruva [56] and Redberg [57] using a drug-centric series of metrics suggested that there were possibly major problems associated with the current U.S. device approval system. Dhruva et al. [56] assessed what kinds of studies were conducted to support approval of 78 high-risk cardiac devices. Of the 123 studies, only 33 were randomized clinical trials. Only about half of the primary outcomes were compared with controls and almost a third of these were retrospective. Almost 90% of the primary outcomes were surrogate measures such as lesion revascularization or lead implant success.

Redberg [57] has suggested that lack of sham controls is often a major weakness of device clinical studies. The placebo effect can be so great that seemingly large benefits may not reflect a true intervention effect. She cites the apparent benefit for treating hypertension from radiofrequency ablation of renal artery nerves. Only when an FDA required trial using sham treatment for the control group was conducted, was lack of benefit (versus the blinded control) observed [58]. A similar situation occurred with laser transmyocardial revascularization, in which open-label trials of using lasers to create myocardial channels resulted in substantial improvement in angina. When a sham-controlled trial was conducted, the sham procedure was equally effective [59]. Much earlier, devices such as intermittent positive pressure breathing (IPPB) for patients with advanced chronic obstructive pulmonary disease were in wide spread use before a clinical trial with a control group demonstrated no clinical benefit [60]. Their use was based on the ability of the device to deliver a treatment deep into the lungs using the pressure gradient.

Objections to requiring conduct of clinical trials with control arms and clinically important outcomes include the fact that many devices have frequent modifications so that a formal clinical trial for each modification would not be feasible. In addition, many device manufacturers are small companies that do not have the human and financial resources to conduct large trials. Requiring a late phase trial for every device and every modification would not be feasible for them or even for large companies.

Rome et al. [61] looked at FDA approval of cardiac implantable electronic devices originally and as supplements. The authors found that from 1979 to 2012, the FDA approved 77 original devices with an average of 2.6 supplements per device that involved design or other major modifications. For those approved supplements that involved major design changes from 2010 to 2012, less than a quarter (15 of 64) provided clinical data.

Yet randomized clinical trials of devices compared to best standard of care, using objective and clinically meaningful outcomes, are feasible. The Comparison of Medical Therapy, Pacing, and Defibrillation in Heart Failure (COMPANION) trial [62] compared a pacemaker alone and a combination of pacemaker and defibrillator against best standard of care in a New York Heart Association class III or IV heart failure population. The primary outcome was all-cause mortality plus all cause hospitalization. Over 1,500 patients were randomized in a 2:2:1 ratio. For the primary outcome, the two intervention arms were statistically and clinically superior to standard of care (approximately 20% relative reductions in events). For the secondary outcome of all cause mortality and cardiovascular hospitalization, approximately 30% reductions were observed in each device arm. The pacemaker-defibrillator arm had a 43% relative reduction and the pacemaker alone arm a 24% reduction in mortality, both being highly significant statistically.

Despite the argument that increasing clinical trial requirements for new or improved devices would discourage and lessen investment in device development and innovation, we think that many medical devices could and should be evaluated according to the fundamentals presented in the previous chapters.

Interventions: Biologics

From a legal and regulatory perspective, biologics products, which replicate natural substances in human bodies such as enzymes, antibodies, or hormones, are generally similar to other drugs. Some, such as vaccines and blood products, are regulated by the Center for Biologics Evaluation and Research (CBER), whereas others, such as anti-TNF agents, are regulated by CDER. Although they are regulated under different acts, the standard requiring that biologic agents be “safe, pure, and potent” is considered to be equivalent to other drugs being “safe and effective” [63]. However, there are some differences (Siegel J, personal communication). First, most biologics are immunogenic. Immunogenicity affects pharmacokinetics, safety, and efficacy. Immunogenic toxicity can be very prominent for some biologics and very serious. Second, biologics have high specificity and are much less likely to have off-target effects than other drugs. Toxicity, therefore, may be more predictable. Off-target effects (e.g., on liver, cardiac rhythm, bone marrow) are less common, although still may occur. Third, biologics usually do not compete with other drugs for clearance so interactions tend to be less common. Biologics, though, can induce liver enzyme production, and can have interactions related to their pharmacodynamics, rather than their metabolism. Fourth, manufacturing biologics in a consistent manner is more difficult than with other drugs, as is characterizing them. Thus, investigators in phase III trials have a strong incentive to use the same commercial process that would be used after approval in preparing the biologic materials to minimize the challenge of showing that the commercial material is comparable to the clinical trial material.

Finally, it has been argued that producing generic versions is far more difficult. Many think that because it is far more difficult to demonstrate that two biologic products are identical, the term “biosimilar” is preferable to generic [64–66]. This has implications for how much clinical data is required for each, presumably comparable, product. Generic drugs, by definition, have the same active pharmaceutical ingredient as the reference product. As a result, they can be developed by referencing data from an approved product without clinical testing other than bioavailability. Due to limitations in the ability to manufacture and characterize biologics, one cannot ensure that two products are the same, and, even if they were, one cannot know that. Therefore, biologic generics are not technically possible at present. Due to this broadly accepted fact, unlike for small molecules, there is no law or regulatory pathway to have a generic biologic in U.S., in EU, or much of the rest of the world. As one cannot have abbreviated development pathways based upon the same approved product indication (as for generics), abbreviated pathways were created (part of the Affordable Care Act in the U.S.) allowing the referencing of an approved product where a high degree of similarity has been shown, i.e. biosimilars pathways. But similarity as opposed to sameness leaves more potential for clinically meaningful differences, so biosimilars pathways envision that usually some amount of clinical testing will be required to rule out such differences.

Post-trial Requirements

At the end of the clinical trial, regulatory agencies will expect a complete submission of the data in a format that is acceptable to the agency [67, 68]. The ICH document E6 [29] lists the documents that are considered important for the regulatory agencies to have on file, although this list is designed for what is needed for approval of new drugs and may not be applicable to pragmatic trials. These documents include the protocol and any amendments, informed consent materials, sponsor-investigator financial and other arrangements, ethics review committee approval, master randomization list, enrollment logs, source documents, completed case report forms, serious adverse event reports, investigator brochures and any updates, and product accountability.

The following discussion refers primarily to U.S. FDA requirements. However, similar requirements exist in other countries. Sponsors and/or investigators may face three regulatory issues. First, if the trial shows efficacy of an intervention and regulatory approval is sought, documentation in support of the efficacy claim has to be submitted to the FDA, generally through the sponsor or the manufacturer. Second, if the regulatory agency decides to bring the case to a public FDA Advisory Committee meeting for recommendations, the investigators may be called upon to present the trial findings to and answer questions by the committee. Third, FDA approval for marketing of the intervention in the U.S. may require additional post-marketing clinical investigations.

In January of 2015, the Institute of Medicine (IOM) released a report on “Sharing Clinical Trial Data: Maximizing Benefits, Minimizing Risk” which calls for sharing of patient level de-identified data after study results have been published or after trials have been submitted for regulatory review [69]. While these are currently recommendations by the IOM, they are likely to also be adopted by many sponsors of clinical trials.

Documents for FDA submission

Many regulatory provisions govern the types of documents that need to be submitted in connection with a clinical trial that shows efficacy of an intervention and supports a marketing application. The documents discussed below are not an exhaustive list of those that must be submitted. In addition, from a scientific standpoint, the extent of documentation necessary depends on the particular study, the types of data involved, and the other evidence available to support the effectiveness claim. The FDA guidance on Providing Clinical Evidence of Effectiveness for Human Drug and Biological Products [70] provides some general considerations on the documentation of the quality of evidence supporting an effectiveness claim. It notes that when submitting the requisite quantity of data to support approval of a new product or new use of an approved product, regulations state that sponsors must also document that the studies were adequately designed and well conducted. To demonstrate that a trial supporting an effectiveness claim is adequate and well-controlled, extensive documentation of trial planning, protocols, conduct, and data handling is usually submitted to the FDA, and detailed participant records are available at the clinical sites. Providing written standard operating procedures and statistical analysis plans (as well as the charter for a data monitoring committee, if one was used), and the interim reports the committee reviewed along with minutes of those meetings are also part of the documentation. Documentation tends to be very extensive for sponsors and investigators.

Advisory Committee Meeting

Advisory committees, which are convened when the FDA desires external advice around a drug or device approval, provide independent advice to the FDA on a range of issues, including those relating to a specific drug, device or biological product [71]. FDA regulations and guidelines concerning advisory committees do not address what is expected from clinical investigators in the preparation for advisory committee meetings since most of that responsibility rests with the trial sponsor. It is possible that in practice a sponsor-applicant may seek assistance from the trial investigators when preparing these background materials. This assistance could include oral presentations of the trial results or an overall summary of the product’s benefits and harms at the advisory committee meeting as well as written summaries of the trial design, conduct, and results. The summary information often includes, among other items, clinical pharmacology and dosing evaluations, clinical efficacy data, and clinical safety data.

Post-approval Issues and Postmarketing Investigations

Postmarketing clinical investigations are subject to many statutory and regulatory requirements if they have the potential or intent to support a product label change. They are often conducted under an IND and therefore may be subject to the IND requirements. Postmarketing reporting of adverse events and submissions of an annual report are also generally expected. When the product has been approved on the basis of a clinical trial, postmarketing clinical investigations may be required if there is new safety information or there is need to verify clinical benefit. Postmarketing clinical investigations can be requested, as agreed-upon in postmarketing commitments, if needed to further evaluate efficacy and safety in a product that has undergone traditional approval.

Certain postmarketing clinical investigations must be registered and have results submitted to ClinicalTrials.gov in a timely fashion. There may be financial penalties or fines for not complying with these requirements. For drugs and biological products, the trial registration and results submission would be required for any “applicable drug clinical trial,” which, in general, means a controlled clinical investigation, other than a phase I clinical investigation. Despite the requirement to submit data to ClinicalTrials.gov, many trials either had delays in publication or no publication [72]. One study [73] showed that industry-sponsored trials had a much higher compliance in entering results into ClinicalTrials.gov than non-industry trials, although in neither case did the majority submit results within the mandated 1 year. Low rates of data submission were also found by Jones et al. [74]. Another study suggested that the high data submission rates to ClinicalTrials.gov were limited to late-phase trials [75].

The generally low rates of data submission are perhaps due to inadequate understanding of the requirements by the investigators. It is important that all investigators know that trials conducted under FDA regulations or with funding from the National Institutes of Health need to have the data submitted to ClinicalTrials.gov. A proposal in 2014 to expand the number of trials funded by the NIH that are required to have their data entered into ClinicalTrials.gov [76] should be noted.

For many years, regulatory agencies have solicited reports of adverse events that have occurred in clinical practice, after a drug or device has been approved and is marketed. As discussed in Chap. 12, even large trials of long duration may miss important adverse effects. Only after something has been used in many different people for sometimes years will some adverse effect be identified. Follow-up is even more important for products that are approved and marketed without trials that monitor important clinical outcomes. For example, for drugs approved on the basis of surrogate outcomes, and when the trials were therefore either too small or too short to obtain sufficient numbers of clinical events, post-marketing reports become extremely important. However, given the lack of a rigorous control group, postmarketing reports can be misleading [77]. Pressures and incentives to approve products more quickly, particularly for life-threatening conditions for which there are few if any treatment options, has led to regulatory changes for so-called “breakthrough drugs” [19, 78, 79]. While the law does not allow a different standard for use of intermediate markers as surrogates with rare diseases than for other more common diseases, still requiring a confirmatory clinical outcome trial, the use of surrogates or biomarkers may be necessary with rare diseases for which a sufficiently large trial with a clinical outcome is difficult or even impossible [80, 81].

For products that have had accelerated approval, whether drugs, devices, or biologics, the pre-approval clinical information is far more limited than for products that have undergone clinical outcome trials. Part of the accelerated approval process calls for additional post-approval clinical assessment, including adverse event monitoring [19]. Sometimes, actual clinical trials are required after accelerated approval. As discussed in the section on drug interventions, the FDA approved bedaquiline for a serious unmet need using the first of a new class of drugs to treat drug-resistant tuberculosis [47]. This was done on the basis of a phase IIb placebo-controlled trial on an expedited approval path, with the requirement that a confirmatory trial would be conducted, though not completed until 2022 [79]. Approval decisions are often difficult and controversial. There are added complications when expedited approval is used, involving the weighing of many factors. As in the case with bedaquiline, advisory committees and regulatory agencies need to use considerable judgment, balancing early access to the benefits of important therapeutic early interventions against possible longer term harms.

More rapid approval of drugs may lead to identification of more adverse effects after marketing. Frank et al. [82] noted an association between a greater number of “boxed” warnings in drug labeling and actual drug withdrawal for safety concerns and acceleration of FDA drug approval. However, association does not demonstrate causation and even if an association reflects causation, the balance between any benefits from faster approval and the harms discovered later is unknown.

An FDA web page on postmarketing requirements and commitments may provide useful information [83].

Key Links

It is essential that all investigators conducting trials that are, or might be, subject to regulatory approval, keep aware of current regulations and guidances. Key links are shown here:

International Conference on Harmonisation

ICH Official Web Site: http://​www.​ich.​org/​

Efficacy Guidelines: http://​www.​ich.​org/​products/​guidelines/​efficacy/​article/​efficacy-guidelines.​html

Safety Guidelines: http://​www.​ich.​org/​products/​guidelines/​safety/​article/​safety-guidelines.​html

MedDRA: http://​www.​ich.​org/​products/​meddra.​html

U.S. Food and Drug Administration

FDA Home Page: http://​www.​fda.​gov/​

FDA Guidances: http://​www.​fda.​gov/​Drugs/​GuidanceComplian​ceRegulatoryInfo​rmation/​Guidances/​default.​htm

Conducting Clinical Trials in Drugs: http://​www.​fda.​gov/​drugs/​developmentappro​valprocess/​conductingclinic​altrials/​default.​htm

http://​www.​fda.​gov/​Drugs/​DevelopmentAppro​valProcess/​HowDrugsareDevel​opedandApproved/​ApprovalApplicat​ions/​InvestigationalN​ewDrugINDApplica​tion/​ucm071098.​htm#form1571

Conducting Clinical Trials in Devices: http://​www.​fda.​gov/​downloads/​MedicalDevices/​DeviceRegulation​andGuidance/​GuidanceDocument​s/​UCM373766.​pdf

Conducting Clinical Trials in Gene Therapy:

Early phase clinical trials.

http://​www.​fda.​gov/​downloads/​Biologi…/​UCM359073.​pdf

Observing subjects for delayed adverse events. http://​www.​fda.​gov/​BiologicsBloodVa​ccines/​GuidanceComplian​ceRegulatoryInfo​rmation/​Guidances/​CellularandGeneT​herapy/​ucm072957.​htm

European Medicines Agency

EMA Home Page: http://​maintenance.​ema.​europa.​eu/​

Clinical Trials in Human Medicines: http://​www.​ema.​europa.​eu/​ema/​index.​jsp?​curl=​pages/​special_​topics/​general/​general_​content_​000489.​jsp&​mid=​WC0b01ac05806067​6f

Health Canada

Health Canada Home Page: http://​www.​hc-sc.​gc.​ca/​index-eng.​php

Guidance Document for Clinical Trials Sponsors: clinical trial applications: http://​www.​hc-sc.​gc.​ca/​dhp-mps/​prodpharma/​applic-demande/​guide-ld/​clini/​ctdcta_​ctddec-eng.​php

Pharmaceuticals and Medical Devices Agency, Japan

Home Page: http://​www.​pmda.​go.​jp/​english/​

Ministerial Ordinance on Good Clinical Practice for Drugs. http://​www.​pmda.​go.​jp/​english/​service/​pdf/​ministerial/​20130329No_​28.​pdf

Bioethics Resources

http://​bioethics.​od.​nih.​gov/​IRB.​html

References

1.

U.S. Food and Drug Administration. http://​www.​fda.​gov/​default.​htm

2.

European Medicines Agency: Clinical trials in human medicines. http://​www.​ema.​europa.​eu/​ema/​index.​jsp?​curl=​pages/​special_​topics/​general/​general_​content_​000489.​jsp&​mid=​WC0b01ac05806067​6f

3.

International Conference on Harmonisation. http://​www.​ich.​org/​

4.

Wolfe SM. When EMA and FDA decisions conflict: differences in patients or in regulation? BMJ 2013;347:f5140.

5.

Gøtzsche PC. Deficiencies in proposed new EU regulation of clinical trials. BMJ 2012;345:e8522 doi.

6.

Code of Federal Regulations (45 CFR 46). http://​www.​hhs.​gov/​ohrp/​humansubjects/​guidance/​45cfr46.​html#46.​102

7.

Code of Federal Regulations (21 CFR 50). http://​www.​accessdata.​fda.​gov/​scripts/​cdrh/​cfdocs/​cfcfr/​CFRSearch.​cfm?​CFRPart=​50

8.

Office for Human Research Protections (OHRP). http://​www.​hhs.​gov/​ohrp/​

9.

Food and Drug Administration: Clinical trial guidance documents. http://​www.​fda.​gov/​RegulatoryInform​ation/​Guidances/​ucm122046.​htm.

10.

ClinicalTrials.gov. https://​clinicaltrials.​gov/​

11.

FDA: Certifications to accompany drug, biological product, and device applications/submissions. http://​www.​fda.​gov/​RegulatoryInform​ation/​Guidances/​ucm125335.​htm

12.

Milestones in U.S. Food and Drug Law History. http://​www.​fda.​gov/​AboutFDA/​WhatWeDo/​History/​Milestones/​ucm128305.​htm.

13.

About FDA. Promoting safe and effective drugs for 100 years. http://​www.​fda.​gov/​AboutFDA/​WhatWeDo/​History/​ProductRegulatio​n/​PromotingSafeand​EffectiveDrugsfo​r100Years/​default.​htm

14.

Beecher HK. Ethics and clinical research. N Engl J Med 1966;274:1354–1360.CrossRef

15.

The 1938 Food, Drug, and Cosmetic Act. http://​www.​fda.​gov/​aboutfda/​whatwedo/​history/​productregulatio​n/​ucm132818.​htm.

16.

Greene JA, Podolsky SH. Reform, regulation, and pharmaceuticals — the Kefauver-Harris amendments at 50. N Engl J Med 2012;367:1481–1483.CrossRef

17.

FDA: PMA Approvals. http://​www.​fda.​gov/​medicaldevices/​productsandmedic​alprocedures/​deviceapprovalsa​ndclearances/​pmaapprovals/​default.​htm

18.

FDA: Medical device reporting (MDR). http://​www.​fda.​gov/​MedicalDevices/​Safety/​ReportaProblem/​default.​htm

19.

FDA: Guidance for Industry: expedited programs for serious conditions—drugs and biologics.http://​www.​fda.​gov/​downloads/​Drugs/​GuidanceComplian​ceRegulatoryInfo​rmation/​Guidances/​UCM358301.​pdf

20.

Code of Federal Regulations Title 21. http://​www.​accessdata.​fda.​gov/​scripts/​cdrh/​cfdocs/​cfcfr/​CFRSearch.​cfm?​fr=​312.​3

21.

Code of Federal Regulations (45 CFR 160). http://​www.​gpo.​gov/​fdsys/​pkg/​CFR-2007-title45-vol1/pdf/CFR-2007-title45-vol1-part160.pdf22.

22.

Code of Federal Regulations (45 CFR 164). http://​www.​hhs.​gov/​ocr/​privacy/​hipaa/​administrative/​privacyrule/​

23.

Food and Drug Administration: Guidance for industry: GCMP for phase I investigational drugs. http://​www.​fda.​gov/​downloads/​drugs/​guidancecomplian​ceregulatoryinfo​rmation/​guidances/​ucm070273.​pdf

24.

Food and Drug Administration: Considerations for the early phase clinical trials of cellular and gene therapy products: draft guidance. http://​www.​fda.​gov/​downloads/​biologicsbloodva​ccines/​guidancecomplian​ceregulatoryinfo​rmation/​guidances/​cellularandgenet​herapy/​ucm359073.​pdf

25.

FDA: Medial Devices. The new 510(k) paradigm—alternate approaches to demonstrating substantial equivalence in premarket notifications—final guidance. http://​www.​fda.​gov/​medicaldevices/​deviceregulation​andguidance/​guidancedocument​s/​ucm080187.​htm

26.

Food and Drug Administration: IND forms and instructions. http://​www.​fda.​gov/​drugs/​developmentappro​valprocess/​howdrugsaredevel​opedandapproved/​approvalapplicat​ions/​investigationaln​ewdrugindapplica​tion/​ucm071073.​htm

27.

FDA: “Off-label” and investigational use of marketed drugs, biologics, and medical devices—information sheet. http://​www.​fda.​gov/​RegulatoryInform​ation/​Guidances/​ucm126486.​htm

28.

FDA: Investigational New Drug (IND) Application. http://​www.​fda.​gov/​Drugs/​DevelopmentAppro​valProcess/​HowDrugsareDevel​opedandApproved/​ApprovalApplicat​ions/​InvestigationalN​ewDrugINDApplica​tion/​default.​htm

29.

ICH Guidance for Industry: E6 Good Clinical Practice: Consolidated Guidance. http://​www.​fda.​gov/​downloads/​Drugs/​GuidanceComplian​ceRegulatoryInfo​rmation/​Guidances/​UCM073122.​pdf

30.

Reith, C, Landray, M, Devereaux, PJ, et al. Randomized clinical trials -- removing unnecessary obstacles. N Engl J Med 2013;369:1061–1065

31.

Food and Drug Administration: Guidance for industry and investigators: safety reporting requirements for INDs and BA/BE studies. http://​www.​fda.​gov/​downloads/​drugs/​guidancecomplian​ceregulatoryinfo​rmation/​guidances/​ucm227351.​pdf

32.

Wittes J, Crowe B, Chuang-Stein C, et al, for the Biostatistics Working Group of the CTTI-IND Safety Reporting Team. The FDA’s Final Rule on Expedited Safety Reporting: statistical considerations. Stat Biopharm Res 2015;7(5) In Press.

33.

FDA: Guidance for industry: computerized systems used in clinical trials. http://​www.​fda.​gov/​downloads/​iceci/​enforcementactio​ns/​bioresearchmonit​oring/​ucm133749.​pdf

34.

Yusuf S, Bosch J, Devereaux PJ, et al. Sensible guidelines for the conduct of large randomized trials. Clin Trials 2008;5:38–39.CrossRef

35.

Duley L, Antman K, Arena J, et al. Specific barriers to the conduct of randomized trials. Clin Trials 2008;5:40–48.CrossRef

36.

Baigent C, Harrell FE, Buyse M, et al. Ensuring trial validity by data quality assurance and diversification of monitoring methods. Clin Trials 2008;5:49–55.CrossRef

37.

Kramer JM, Smith PB, Califf RM. Impediments to clinical research in the United States. Clin Pharmacol Ther 2012;91:535–541.CrossRef

38.

Guidance for Industry: Oversight of clinical investigations—a risk-based approach to monitoring. http://​www.​fda.​gov/​downloads/​Drugs/​Guidances/​UCM269919.​pdf

39.

FDA: Exception from informed consent for studies conducted in emergency settings: regulatory language and excerpts from preamble-information sheet: guidance for institutional review boards and clinical investigators. http://​www.​fda.​gov/​regulatoryinform​ation/​guidances/​ucm126482.​htm

40.

FDA: Guidance for clinical trial sponsors: establishment and operation of clinical trial data monitoring committees. http://​www.​fda.​gov/​downloads/​regulatoryinform​ation/​guidances/​ucm127073.​pdf

41.

European Medicines Agency: Guideline on data monitoring committees. http://​www.​ema.​europa.​eu/​docs/​en_​GB/​document_​library/​Scientific_​guideline/​2009/​09/​WC500003635.​pdf

42.

ICH Harmonised Tripartite Guideline: Statistical principles for clinical trials: E9. http://​www.​ich.​org/​fileadmin/​Public_​Web_​Site/​ICH_​Products/​Guidelines/​Efficacy/​E9/​Step4/​E9_​Guideline.​pdf

43.

WHO: Operational guidelines for the establishment and functioning of data and safety monitoring boards. http://​www.​who.​int/​tdr/​publications/​documents/​operational-guidelines.​pdf?​ua=​1

44.

Ellenberg SS, Fleming TR, DeMets DL. Data monitoring committees in clinical trials: a practical perspective. West Sussex: John Wiley and Sons, 2003.

45.

Prasad V, Mailankody S. Viewpoint: the accelerated approval of oncologic drugs: lessons from ponatinib. JAMA 2014;311:353–354.CrossRef

46.

Diacon AH, Pym A, Grobusch MP, et al. Multidrug-resistant tuberculosis and culture conversion with bedaquiline. N Engl J Med 2014;371:723–732.CrossRef

47.

Cox E, Laessig K. FDA approval of bedaquiline—the benefit-risk balance for drug-resistant tuberculosis. N Engl J Med 2014;371:689–691.CrossRef

48.

Mahajan R, Bedaquiline: First FDA-approved tuberculosis drug in 40 years, Int J Basic Applied Med Res 2013;3:1–2.CrossRef

49.

FDA: Food and Drug Administration Modernization Act (FDAMA) of 1997. http://​www.​fda.​gov/​RegulatoryInform​ation/​Legislation/​FederalFoodDruga​ndCosmeticActFDC​Act/​SignificantAmend​mentstotheFDCAct​/​FDAMA/​default.​htm

50.

Code of Federal Regulations (21 CFR 860). http://​www.​accessdata.​fda.​gov/​scripts/​cdrh/​cfdocs/​cfcfr/​CFRSearch.​cfm?​CFRPart=​860

51.

FDA: 510 (k) Clearances. http://​www.​fda.​gov/​MedicalDevices/​ProductsandMedic​alProcedures/​DeviceApprovalsa​ndClearances/​510kClearances/​

52.

FDA: Premarket Approval (PMA). http://​www.​fda.​gov/​Medicaldevices/​Deviceregulation​andguidance/​Howtomarketyourd​evice/​Premarketsubmiss​ions/​Premarketapprova​lpma/​Default.​Htm

53.

Food and Drug Administration: Investigational Device Exemption (IDE). http://​www.​fda.​gov/​aboutfda/​centersoffices/​officeofmedicalp​roductsandtobacc​o/​cdrh/​cdrhtransparency​/​ucm205697.​htm

54.

FDA: Guidance for industry and FDA staff: modifications to devices subject to premarket approval (PMA)—the PMA supplement decision-making process. http://​www.​fda.​gov/​downloads/​MedicalDevices/​DeviceRegulation​andGuidance/​GuidanceDocument​s/​UCM270606.​pdf

55.

FDA: Design considerations for pivotal clinical investigations for medical devices: guidance for industry, clinical investigators, institutional review boards and Food and Drug Administration staff. http://​www.​fda.​gov/​MedicalDevices/​DeviceRegulation​andGuidance/​GuidanceDocument​s/​ucm373750.​htm

56.

Dhruva SS, Bero LA, Redberg RF. Strength of study evidence examined by the FDA in premarket approval of cardiovascular devices. JAMA 2009; 302:2679–2685.CrossRef

57.

Redberg RF. Sham controls in medical device trials. N Engl J Med 2014;371:892–893.CrossRef

58.

Bhatt DL, Kandzari DE, O’Neill WW, et al. A controlled trial of renal denervation for resistant hypertension. N Engl J Med 2014;370:1393–1401.CrossRef

59.

Leon MB, Kornowski R, Downey WE, et al. A blinded, randomized, placebo-controlled trial of percutaneous laser myocardial revascularization to improve angina symptoms in patients with severe coronary disease. J Am Coll Cardiol. 2005;46:1812–9.CrossRef

60.

The Intermittent Positive Pressure Breathing Trial Group. Intermittent positive pressure breathing therapy of chronic obstructive pulmonary disease. A clinical trial. Ann Intern Med 1983;99:612–620.

61.

Rome BN, Kramer DB, Kesselheim AS. FDA approval of cardiac implantable electronic devices via original and supplement premarket approval pathways, 1979-2012. JAMA 2014;311:385–391.CrossRef

62.

Bristow MR, Saxon LA, Boehmer J, et al. Cardiac-resynchronization therapy with or without an implantable defibrillator in advanced chronic heart failure. N Engl J Med 2004;350:2140–2150.CrossRef

63.

Code of Federal Regulations (21 CFR 600). http://​www.​accessdata.​fda.​gov/​scripts/​cdrh/​cfdocs/​cfcfr/​CFRSearch.​cfm?​CFRPart=​600

64.

FDA: Biosimilars guidances. http://​www.​fda.​gov/​Drugs/​GuidanceComplian​ceRegulatoryInfo​rmation/​Guidances/​ucm290967.​htm

65.

FDA: Guidance for Industry: clinical pharmacology data to support a demonstration of biosimilarity to a reference product (draft guidance). http://​www.​fda.​gov/​downloads/​Drugs/​GuidanceComplian​ceRegulatoryInfo​rmation/​Guidances/​UCM397017.​pdf

66.

ICH guidance on biosimilars: http://​www.​ich.​org/​products/​guidelines/​quality/​article/​quality-guidelines.​html

67.

FDA: Study data submissions to CBER. http://​www.​fda.​gov/​biologicsbloodva​ccines/​developmentappro​valprocess/​ucm209137.​htm

68.

FDA: Guidance for industry: providing submissions in electronic format—summary level clinical site data for CDER’s inspection planning: draft guidance. http://​www.​fda.​gov/​downloads/​drugs/​developmentappro​valprocess/​formssubmissionr​equirements/​ucm332468.​pdf

69.

Institute of Medicine of the National Academies. Sharing Clinical Trial Data: Maximizing Benefits, Minimizing Risk. http://​www.​iom.​edu/​Reports/​2015/​Sharing-Clinical-Trial-Data.​aspx

70.

FDA: Guidance for industry: providing clinical evidence of effectiveness for human drug and biological products. http://​www.​fda.​gov/​downloads/​drugs/​guidancecomplian​ceregulatoryinfo​rmation/​guidances/​ucm078749.​pdf

71.

FDA: Guidance for the public and FDA staff on convening advisory committee meetings: draft guidance. http://​www.​fda.​gov/​downloads/​regulatoryinform​ation/​guidances/​ucm125651.​pdf

72.

Riveros C, Deschartres, Perrodeau E, et al. Timing and completeness of trial results posted at ClinicalTrials.gov and published in journals. PLOS Medicine 2013;DOI:10 1371/journal pmed.1001566

73.

Prayle AP, Hurley MN, Smyth AR. Compliance with mandatory reporting of clinical trial results on ClinicalTrials.gov: cross sectional study. BMJ 2011;344:d7373 doi: 10.1136/bmj.d7373CrossRef

74.

Jones CW, Handler L, Crowell KE, et al. Non-publication of large randomized clinical trials: cross sectional analysis. BMJ 2013;347:f6104 doi: 10.1136/bmj.f6104CrossRef

75.

Saito J, Gill CJ. How frequently do the results from completed US clinical trials enter the public domain?—a statistical analysis of the ClinicalTrials.gov database. PLoS ONE 2014;9:e101826. doi:10.1371/journal.pone.0101826CrossRef

76.

Summary of HHS/NIH proposals to enhance transparency of clinical trial results. http://​www.​nih.​gov/​news/​health/​nov2014/​od-19_​summary.​htm

77.

FDA Drug Safety Communication: FDA Study of Medicare patients finds risks lower for stroke and death but higher for gastrointestinal bleeding with Pradaxa (dabigatran) compared to warfarin. http://​www.​fda.​gov/​Drugs/​DrugSafety/​ucm396470.​htm

78.

The European Medicines Agency Roadmap to 2010: Preparing the Group for the Future. http://​www.​ema.​europa.​eu/​docs/​en_​GB/​document_​library/​Report/​2009/​10/​WC500004903.​pdf

79.

Darrow JJ, Avorn J, Kesselheim AS. New FDA breakthrough-drug category—implications for patients. N Engl J Med 2014;370:1252–1258.CrossRef

80.

Pariser A. Small clinical trials. http://​www.​fda.​gov/​downloads/​newsevents/​meetingsconferen​cesworkshops/​ucm415213.​pdf.

81.

Coté TR, Kui X, Pariser AR. Comment: Accelerating orphan drug development. Nat Rev Drug Discov 2010;9:901–902.CrossRef

82.

Frank C, Himmelstein DU, Woolhandler S, et al. Era of faster FDA drug approval has also seen increased black-box warnings and market withdrawals. Health Affairs 2014;33:1453–1459.CrossRef

83.

FDA: Postmarketing requirements and commitments: introduction. http://​www.​fda.​gov/​Drugs/​GuidanceComplian​ceRegulatoryInfo​rmation/​Post-marketingPhaseIV​Commitments/​

© Springer International Publishing Switzerland 2015

Lawrence M. Friedman, Curt D. Furberg, David L. DeMets, David M. Reboussin and Christopher B. GrangerFundamentals of Clinical Trials10.1007/978-3-319-18539-2_23

Erratum

Erratum

Lawrence M. Friedman1 , Curt D. Furberg², David L. DeMets³, David M. Reboussin⁴ and Christopher B. Granger⁵

(1)

North Bethesda, MD, USA

(2)

Division of Public Health Sciences, Wake Forest School of Medicine, Winston-Salem, NC, USA

(3)

Department Biostatistics and Medical Informatics, University of Wisconsin, Madison, WI, USA

(4)

Department of Biostatistics, Wake Forest School of Medicine, Winston-Salem, NC, USA

(5)

Department of Medicine, Duke University, Durham, NC, USA

The updated original online version for this chapter can be found at 10.​1007/​978-3-319-18539-2_​17

In the original publication on page vi, there is a typographical error in the print and online versions of this book. “Principal” was incorrectly spelled as “Principle.”

Corrections to chapter 17, page 381, follow, and these changes have been updated in the book.

Chapter 17

Statistical Methods Used in Interim Monitoring

p 381

Many different spending functions can be specified. The O’Brien–Fleming α ₁(t*) and Pocock α ₂ (t*) type spending functions are specified as follows:

The spending function α₃(t*) spends alpha uniformly during the trial for θ = 1, at a rate somewhat between α₁(t*) and α₂(t*). Other spending functions have also been defined [75, 76].

Index

A

Accidental bias.

SeeBias, accidental

Adaptive design.

SeeDesign, adaptive

Adaptive randomization.

SeeRandomization, adaptive

Adherence

effects on adverse events

definition

design factors

discussing with participants

maintaining

monitoring

planning

protocol adherence

run-in

withdrawal

Adverse effects/adverse events

analysis

ascertainment

assessment and reporting

avoiding

classification

CONSORT statement

data collection

definition

design issues

effects on adherence

factorial designs

genetic subgroups

identification

meta-analysis

monitoring

missing data

post-marketing

predicted by baseline measures

regulatory

reporting

serious

unexpected

withdrawal studies

Alpha spending functions.

SeeSequential testing

Alternative hypothesis.

SeeHypotheses

Analysis

intention-to-treat

on-treatment

per protocol

trend adaptive designs

Ancillary

Ascertainment

adverse events

bias

health related quality of life

low adherence effects on

outcomes

post-study

Audits

B

Baseline data

comparability

definition

measurement

testing for imbalance

uses

Bayesian

data monitoring

methods

Belmont Report.

SeeEthics

Bias

accidental

allocation

ascertainment

assessment

estimation

experimental

interviewer

investigator

publication

selection

Biased coin randomization.

SeeRandomization, adaptive

Biomarkers

genetic

outcomes

personalized medicine

run-in period

screening for

subgroup

surrogate outcomes

Blinding

assessment

double-blind design

double-dummy

open trial

protecting

reporting

single-blind design

triple-blind design

unblinded (open) design

unblinding

Blindness

Blocked randomization.

SeeRandomization, blocked

Bonferroni.

SeeMultiple testing

C

Case-control studies.

SeeDesign

Censoring.

See also Under survival

bias

comparing survival curves

definition

information loss

informative

Kaplan-Meier

nonadherence

sequential methods

Classical sequential monitoring.

SeeSequential testing

Clinical equipoise.

SeeEquipoise

Close-out

transfer of care

visit

Common Rule.

SeeEthics

Comparison of survival curves.

SeeSurvival

Compensatory treatment

Competing event issues

Compliance.

SeeAdherence

Concomitant treatment

Concurrent control studies.

SeeDesign

Conditional power.

SeeSequential testing

Conflict of interest

CONSORT statement

Continuous response.

SeeResponse variables

Control group

concurrent, non-randomized

historical

placebo

randomized

Covariate adjustment

using baseline data

using surrogates

Covariate-intervention interaction

Cross-over.

SeeDesign

Cutler-Ederer.

SeeSurvival, Cutler-Ederer

D

Data analysis

Conflicts of interest

Issues

Presentation

Randomization process

Data and safety monitoring board.

SeeData monitoring committee

Databases

dissemination

electronic health

finalization

historical controls

Data collection

data entry

enhancing quality

essential data

forms

See(Forms)

guidelines

problems

verification

Data, missing.

SeeMissing data

Data monitoring committee

decision process

meetings

membership

Data sharing

See alsoDissemination

Declaration of Helsinki.

SeeEthics

Design

adaptive

response adaptive

sample size

trend adaptive

case-control studies

concurrent control studies

cross-over

double-blind

factorial

hybrid

large simple trials

Dichotomous response.

SeeResponse

Dissemination

Dose-escalation studies

Double-blind design.

SeeDesign

Drop-in

Drop-out

Drugs

coding

matching

quality control

E

Effectiveness trial

Efficacy trial

Equipoise

clinical

Equivalence study

Ethics

Belmont Report

clinical equipoise

common rule

conflict of interest

data monitoring

Declaration of Helsinki

developing countries

emergency settings

financial incentives

guidelines

informed consent

Nuremberg Code

privacy and confidentiality

publication bias

surrogate consent

Ethics review committee

Exclusion

from analysis

from follow-up

ineligibility

missing data

nonadherence

Expected number of events

Exponential distribution

See alsoSurvival

Extension of follow-up

F

Factorial design.

SeeDesign, factorial

Falsification of data

Fixed allocation

Follow-up, poststudy

Forms

development

monitoring

pretesting

G

Ghost authorship

Greenwood’s variance formula

Group allocation

Group sequential

alpha spending functions

asymmetric boundaries

Haybittle–Peto

Lan-DeMets alpha spending functions

methods for monitoring

monitoring applications

O’Brien–Fleming

Pocock

Guest authorship

H

Haybittle–Peto group sequential method.

SeeGroup sequential, Haybittle–Peto

Health-related quality of life

administration

clinical significance

components

definition

design

interpretation

methodology

outcomes

pretesting

primary dimension

selection of instruments

uses

utility measures

Historical control studies

Hochberg procedure for multiple testing.

SeeMultiple testing

Holm procedure for multiple testing.

SeeMultiple testing

Hybrid design.

SeeDesign, hybrid

Hypotheses

alternative

null

one-sided vs. two-sided

I

Imbalance

baseline

covariate adjustment

randomization

safety events

subgroups

Imputation.

SeeMissing data

Industry modified NIH clinical trials model

Informative censoring

Informed consent

changes

closeout

ethics

genetics

privacy and confidentiality

quality monitoring

regulatory issues

study population

Institutional Review Board (IRB)

Intention-to-treat (ITT)

Interim events

IRB.

SeeInstitutional Review Board (IRB)

IVRS, IWRS.

SeeRandomization

K

Kaplan-Meier

See also Under survival

L

Lan-DeMets alpha spending functions.

SeeGroup sequential, Lan-DeMets alpha spending functions

Large simple trials.

SeeDesign, Large simple trials

Last observation carried forward.

SeeMissing data, Last observation carried forward

Levels of significance

M

Mantel Haenszel statistic

survival analysis

two by two tables

Manual of procedures

Margin of indifference

Margin of noninferiority

Masking.

SeeBlinding

Maximally tolerated dose

Measurement error

Medications.

SeeDrugs

Meta-analysis

issues

methods

Minimization allocation

Missing data

imputation

last observation carried forward

missing at random (MAR)

missing completely at random (MCAR)

Monitoring

Committee

interim

procedures

quality

response variables

sequential

Multicenter trials

conduct

globalization

organization

reasons for

recruitment

Multiple testing

Bonferroni

Hochberg

Holm

N

Natural history

NIH clinical trials model

Nonadherence

analysis

impact on sample size

Noninferiority

analysis

assay sensitivity

constancy assumption

control group

efficacy imputation

interpretation

margin of indifference

margins

Null hypothesis

Nuremberg Code.

SeeEthics

O

O’Brien–Fleming.

SeeGroup sequential, O'Brien–Fleming

Observational studies

One-sided vs. two-sided hypothesis.

See Hypotheses, one-sided vs. two-sided

Open trial.

SeeBlinding

P

Per protocol analysis.

SeeAnalysis, Per protocol

Pharmacogenetics

Phases of trials

early phase

late phase

phase I

phase II

phase III

phase IV

Pilot study

Placebo control

Play-the-winner

Pocock group sequential method.

SeeGroup sequential

Pocock–Simon randomization.

SeeRandomization

Population

definition

eligibility criteria

external validity

generalization

heterogeneity

homogeneity

large, simple trials

recruitment

representativeness

selection

Power.

SeeSample size

Pragmatic trial

Privacy

Protocol

adherence

SeeAdherence

adverse events prespecified

baseline factors prespecified

closeout

contents

defining study population

guidelines and registries

health related quality of life

modification

multicenter trial

Per protocol

SeeAnalysis, per protocol

pilot

regulatory

subgroups prespecified

violation

Publication bias

Q

Quality control

certification

monitoring

Quality of life.

SeeHealth-related quality of life

Question

adverse event

ancillary

natural history

primary

secondary

R

Randomization

adaptive

baseline adaptive

biased coin (adaptive?)

blocked

fixed allocation

group allocation

IVRS, IWRS

mechanics

Pocock–Simon method

response adaptive

simple

two-armed bandit

urn design

Randomized control trials

Recruitment

approaches to lagging recruitment

conduct

contingency plan

ethics

logs

monitoring

planning

problems

reasons for participating

recycling subjects

selection of study sample

sources

Registration of trials

Regression toward the mean

Repeated confidence intervals.

SeeSequential testing

Repeated testing for significance

See alsoMultiple testing

Reporting results

analysis

guidelines

interpretation

publication bias

Response adaptive randomization.

SeeRandomization, response adaptive

Response variables

assessment

combined (composite)

continuous

dichotomous

discrete

monitoring

surrogate

Results, dissemination

S

Safety

advantages

analysis

Committee

SeeMonitoring committee

early termination

eligibility

ethical issues

hampered by blinding

importance to primary question

meta-analysis

monitoring

regulatory considerations

reporting

Sample size

cluster designs

continuous outcome for repeated measures

continuous outcome for two independent samples

continuous outcome paired data

dichotomous outcome for paired data

dichotomous outcome for two independent samples

multiple outcomes

nonadherence adjustment

noninferiority designs

parameter estimation

time to failure outcome

Screening

effect

eligibility

prescreening

recruitment

visit

Selection bias.

SeeBias, selection

Sequential testing

alpha spending functions

classical

conditional power

repeated confidence intervals

statistical methods

Simple randomization.

SeeRandomization, simple

Single-blind design.

SeeBlinding

Statistical methods for data monitoring.

SeeSequential testing, statistical methods

Statistical power

Storage of study material

Stratification

of analysis

of randomization

Subgroup

analysis

definition

Superiority trials

Surrogate consent

Surrogate response variable

Survival

analysis methods

comparison of survival curves

Cox Proportional Hazards model

Mantel-Haenszel

medians

point by point

weighted rank tests

Wilcoxon–Gehan

curve estimation

Cutler-Ederer

exponential distribution

Greenwood’s variance formula

informative censoring

Kaplan-Meier

time to failure, time to event

T

Termination of trial

examples

planning

procedures

Therapeutic misconception

Time to failure response

See alsoSurvival

Training

ethics

health related quality of life

interviewer

multicenter trial

pretesting

quality assurance

Triple-blind design.

SeeBlindness

Two-armed bandit randomization.

SeeRandomization

Type I error

Type II error

U

Unblinded design.

SeeBlinding

Unblinding,

ancillary studies

closeout

drug coding

emergency

inadvertent

monitoring committee

official

purposeful

reporting

serious adverse events

Urn design randomization.

SeeRandomization

V

Variability

cluster

estimates

measurement

quality control

reduction

sample size parameter

unequal allocation

W

Withdrawal of subjects

See alsoExclusion

Withdrawal studies