Quasi-Experimentation

Overview

1.1 Introduction

1.2 The Definition of Quasi-Experiment

1.3 Why Study Quasi-Experiments

1.4 Overview of the Volume

1.5 Conclusions

1.6 Suggested Reading

2. Cause and Effect

Overview

2.1 Introduction

2.2 Practical Comparisons and Confounds

2.3 The Counterfactual Definition

2.4 The Stable-Unit-Treatment-Value Assumption (SUTVA)

2.5 The Causal Question Being Addressed

2.6 Conventions

2.7 Conclusions

2.8 Suggested Reading

3. Threats to Validity

Overview

3.1 Introduction

3.2 The Size of an Effect

3.3 Construct Validity

3.4 Internal Validity

3.5 Statistical Conclusion Validity

3.6 External Validity

3.7 Trade-offs among Types of Validity

3.8 A Focus on Internal and Statistical Conclusion Validity

3.9 Conclusions

3.10 Suggested Reading

4. Randomized Experiments

Overview

4.1 Introduction

4.2 Between-Groups Randomized Experiments

4.3 Examples of Randomized Experiments Conducted in the Field

4.4 Selection Differences

4.5 Analysis of Data from the Posttest-Only Randomized Experiment

4.6 Analysis of Data from the Pretest–Posttest Randomized Experiment

4.7 Noncompliance with Treatment Assignment

4.8 Missing Data and Attrition

4.9 Cluster-Randomized Experiments

4.10 Other Threats to Validity in Randomized Experiments

4.11 Strengths and Weaknesses

4.12 Conclusions

4.13 Suggested Reading

5. One-Group Posttest-Only Designs

Overview

5.1 Introduction

5.2 Examples of One-Group Posttest-Only Designs

5.3 Strengths and Weaknesses

5.4 Conclusions

5.5 Suggested Reading

6. Pretest–Posttest Designs

Overview

6.1 Introduction

6.2 Examples of Pretest–Posttest Designs

6.3 Threats to Internal Validity

6.4 Design Variations

6.5 Strengths and Weaknesses

6.6 Conclusions

6.7 Suggested Reading

7. Nonequivalent Group Designs

Overview

7.1 Introduction

7.2 Two Basic Nonequivalent Group Designs

7.3 Change-Score Analysis

7.4 Analysis of Covariance

7.5 Matching and Blocking

7.6 Propensity Scores

7.7 Instrumental Variables

7.8 Selection Models

7.9 Sensitivity Analyses and Tests of Ignorability

7.10 Other Threats to Internal Validity besides Selection Differences

7.11 Alternative Nonequivalent Group Designs

7.12 Empirical Evaluations and Best Practices

7.13 Strengths and Weaknesses

7.14 Conclusions

7.15 Suggested Reading

8. Regression Discontinuity Designs

Overview

8.1 Introduction

8.2 The Quantitative Assignment Variable

8.3 Statistical Analysis

8.4 Fuzzy Regression Discontinuity

8.5 Threats to Internal Validity

8.6 Supplemented Designs

8.7 Cluster Regression Discontinuity Designs

8.8 Strengths and Weaknesses

8.9 Conclusions

8.10 Suggested Reading

9. Interrupted Time-Series Designs

Overview

9.1 Introduction

9.2 The Temporal Pattern of the Treatment Effect

9.3 Two Versions of the Design

9.4 The Statistical Analysis of Data When N = 1

9.5 The Statistical Analysis of Data When N Is Large

9.6 Threats to Internal Validity

9.7 Design Supplements I: Multiple Interventions

9.8 Design Supplements II: Basic Comparative ITS Designs

9.9 Design Supplements III: Comparative ITS Designs with Multiple Treatments

9.10 Single-Case Designs

9.11 Strengths and Weaknesses

9.12 Conclusions

9.13 Suggested Reading

10. A Typology of Comparisons

Overview

10.1 Introduction

10.2 The Principle of Parallelism

10.3 Comparisons across Participants

10.4 Comparisons across Times

10.5 Comparisons across Settings

10.6 Comparisons across Outcome Measures

10.7 Within- and Between-Subject Designs

10.8 A Typology of Comparisons

10.9 Random Assignment to Treatment Conditions

10.10 Assignment to Treatment Conditions Based on an Explicit Quantitative Ordering

10.11 Nonequivalent Assignment to Treatment Conditions

10.12 Credibility and Ease of Implementation

10.13 The Most Commonly Used Comparisons

10.14 Conclusions

10.15 Suggested Reading

11. Methods of Design Elaboration

Overview

11.1 Introduction

11.2 Three Methods of Design Elaboration

11.3 The Four Size-of-Effect Factors as Sources for the Two Estimates in Design Elaboration

11.4 Conclusions

11.5 Suggested Reading

12. Unfocused Design Elaboration and Pattern Matching

Overview

12.1 Introduction

12.2 Four Examples of Unfocused Design Elaboration

12.3 Pattern Matching

12.4 Conclusions

12.5 Suggested Reading

13. Principles of Design and Analysis for Estimating Effects

Overview

13.1 Introduction

13.2 Design Trumps Statistics

13.3 Customized Designs

13.4 Threats to Validity

13.5 The Principle of Parallelism

13.6 The Typology of Simple Comparisons

13.7 Pattern Matching and Design Elaborations

13.8 Size of Effects

13.9 Bracketing Estimates of Effects

13.10 Critical Multiplism

13.11 Mediation

13.12 Moderation

13.13 Implementation

13.14 Qualitative Research Methods

13.15 Honest and Open Reporting of Results

13.16 Conclusions

13.17 Suggested Reading

Appendix: The Problems of Overdetermination and Preemption

A.1 The Problem of Overdetermination

A.2 The Problem of Preemption

References

Glossary

Author Index

Subject Index

About the Author