Data Analysis for Experimental Design

1.1 Introduction

1.2 Observations and Variables

1.3 Behavioral Variables

1.4 Stimulus Variables

1.5 Individual Difference Variables

1.6 Discrete and Continuous Variables

1.7 Levels of Measurement

1.8 Summarizing Observations in Research

1.9 Questions and Problems

2. Principles of Experimental Design

2.1 The Farmer from Whidbey Island

2.2 The Experiment

2.3 The Question of Interest

2.4 Sample Space and Probability

2.5 Simulation of the Experiment

2.6 Permutations

2.7 Combinations

2.8 Probabilities of Possible Outcomes

2.9 A Sample Space for the Experiment

2.10 Testing a Null Hypothesis

2.11 Type I and Type II Errors

2.12 Experimental Controls

2.13 The Importance of Randomization

2.14 A Variation in Design

2.15 Summary

2.16 Questions and Problems

3. The Standard Normal Distribution: An Amazing Approximation

3.1 Introduction

3.2 Binomial Populations and Binomial Variables

3.3 Mean of a Population

3.4 Variance and Standard Deviation of a Population

3.5 The Average of a Sum and the Variance of a Sum

3.6 The Average and Variance of Repeated Samples

3.7 The Second Experiment with the Farmer: µT and sT

3.8 Representing Probabilities by Areas

3.9 The Standard Normal Distribution

3.10 The Second Experiment with the Farmer: A Normal Distribution Test

3.11 The First Experiment with the Farmer: A Normal Distribution Test

3.12 Examples of Binomial Models

3.13 Populations That Have Several Possible Values

3.14 The Distribution of the Sum from a Uniform Distribution

3.15 The Distribution of the Sum T from a U-Shaped Population

3.16 The Distribution of the Sum T from a Skewed Population

3.17 Summary and Sermon

3.18 Questions and Problems

4. Tests for Means from Random Samples

4.1 Transforming a Sample Mean into a Standard Normal Variable

4.2 The Variance and Standard Error of the Mean When the Population Variance s2 Is Known

4.3 The Variance and Standard Error of the Mean When Population s2 Is Unknown

4.4 The t Distribution and the One-Sample t Test

4.5 Confidence Interval for a Mean

4.6 Standard Error of the Difference between Two Means

4.7 Confidence Interval for a Difference between Two Means

4.8 Test of Significance for a Difference between Two Means: The Two-Sample t Test

4.9 Using a Computer Program

4.10 Returning to the Farmer Example in Chapter 2

4.11 Effect Size for a Difference between Two Independent Means

4.12 The Null Hypothesis and Alternatives

4.13 The Power of the t Test against a Specified Alternative

4.14 Estimating the Number of Observations Needed in Comparing Two Treatment Means

4.15 Random Assignments of Participants

4.16 Attrition in Behavioral Science Experiments

4.17 Summary

4.18 Questions and Problems

5. Homogeneity and Normality Assumptions

5.1 Introduction

5.2 Testing Two Variances: The F Distribution

5.3 An Example of Testing the Homogeneity of Two Variances

5.4 Caveats

5.5 Boxplots

5.6 A t Test for Two Independent Means When the Population Variances Are Not Equal

5.7 Nonrandom Assignment of Subjects

5.8 Treatments That Operate Differentially on Individual Difference Variables

5.9 Nonadditivity of a Treatment Effect

5.10 Transformations of Raw Data

5.11 Normality

5.12 Summary

5.13 Questions and Problems

6. The Analysis of Variance: One Between-Subjects Factor

6.1 Introduction

6.2 Notation for a One-Way Between-Subjects Design

6.3 Sums of Squares for the One-Way Between-Subjects Design

6.4 One-Way Between-Subjects Design: An Example

6.5 Test of Significance for a One-Way Between-Subjects Design

6.6 Weighted Means Analysis with Unequal n's

6.7 Summary

6.8 Questions and Problems

7. Pairwise Comparisons

7.1 Introduction

7.2 A One-Way Between-Subjects Experiment with 4 Treatments

7.3 Protection Levels and the Bonferroni Significant Difference (BSD) Test

7.4 Fisher's Significant Difference (FSD) Test

7.5 The Tukey Significant Difference (TSD) Test

7.6 Scheffé's Significant Difference (SSD) Test

7.7 The Four Methods: General Considerations

7.8 Questions and Problems

8. Orthogonal, Planned and Unplanned Comparisons

8.1 Introduction

8.2 Comparisons on Treatment Means

8.3 Standard Error of a Comparison

8.4 The t Test of Significance for a Comparison

8.5 Orthogonal Comparisons

8.6 Choosing a Set of Orthogonal Comparisons

8.7 Protection Levels with Orthogonal Comparisons

8.8 Treatments as Values of an Ordered Variable

8.9 Coefficients for Orthogonal Polynomials

8.10 Tests of Significance for Trend Comparisons

8.11 The Relation between a Set of Orthogonal Comparisons and the Treatment Sum of Squares

8.12 Tests of Significance for Planned Comparisons

8.13 Effect Size for Comparisons

8.14 The Equality of Variance Assumption

8.15 Unequal Sample Size

8.16 Unplanned Comparisons

8.17 Summary

8.18 Questions and Problems

9. The 2k Between-Subjects Factorial Experiment

9.1 Introduction

9.2 An Example of a 23 Factorial Experiment

9.3 Assumption of Homogeneity of Variance

9.4 Factorial Data as a One-Way Between-Subjects Design

9.5 Partitioning the Treatment Sum of Squares

9.6 Summary of the Analysis of Variance

9.7 Graphs That Depict the Interactions

9.8 Other 2k Factorial Experiments

9.9 Notation and Sums of Squares for a Factorial Experiment

9.10 Summary

9.11 Questions and Problems

10. Between-Subjects Factorial Experiments: Factors with More Than Two Levels

10.1 Introduction

10.2 An Example of a 4 x 3 x 2 Factorial Experiment

10.3 Partitioning the Sum of Squares into Main Effects and Interactions

10.4 Orthogonal Partitioning for Main Effects

10.5 Orthogonal Partitioning for Interactions

10.6 Effect Size for Comparisons in a Factorial Design

10.7 Performing Multiple Tests

10.8 The Structural Model and Nomenclature

10.9 Summary

10.10 Questions and Problems

11. Between-Subjects Factorial Experiments: Further Considerations

11.1 The Scheffé Test for Comparisons

11.2 Pairwise Comparisons in Factorial Designs

11.3 Unequal Sample Sizes in a Factorial Design

11.4 Individual Difference Factors

11.5 Control Variables

11.6 Random-Effect Factors

11.7 Nested Factors

11.8 Homogeneity of Variance

11.9 Summary

11.10 Questions and Problems

12. Within-Subjects Factors: One-Way and 2k Factorial Designs

12.1 Introduction

12.2 Example: One-Way ANOVA with a Within-Subjects Factor

12.3 Trend Analysis on One-Way Within-Subjects Designs

12.4 Assumptions and Effect Size Measures

12.5 2k Factorial Designs: All Within-Subjects Factors

12.6 Multiple Tests

12.7 Design Considerations with Within-Subjects Designs

12.8 Scheffé Test for Within-Subjects Factors

12.9 SPSS Syntax

12.10 Multilevel Approach to Within-Subjects Designs

12.11 Summary

12.12 Questions and Problems

13. Within-Subjects Factors: General Designs

13.1 Introduction

13.2 General Within-Subjects Factorial Design

13.3 Designs Containing Both Within-Subjects and Between-Subjects Factors

13.4 Omnibus Tests

13.5 Summary

13.6 Questions and Problems

14. Contrasts on Binomial Data: Between-Subjects Designs

14.1 Introduction

14.2 Preliminaries

14.3 Four Examples of Wald Tests

14.4 Other Statistical Tests for Comparisons on Proportions

14.5 Numerical Examples

14.6 How Do These Tests Differ and What Do They Test?

14.7 Summary

14.8 Questions and Problems

15. Debriefing

15.1 Introduction

15.2 Descriptive Statistics and Plotting Data

15.3 Presenting Your Results

15.4 Nonparametric Statistical Tests

15.5 Nonexperimental Controls

15.6 Questions and Problems

Appendix A. The Method of Least Squares

Appendix B. Statistical Tables