Experimental Designs and Sampling in Research: A Comprehensive Guide, Lecture notes of Social Statistics and Data Analysis

Download Experimental Designs and Sampling in Research: A Comprehensive Guide and more Lecture notes Social Statistics and Data Analysis in PDF only on Docsity!

The Logic of Research: Experimental

Designs

Early designs assessed changes in a

D.V. due to an I.V.

Pre-experimental designs were

agricultural so researchers ignored

threats to internal validity.

1. One-Group Posttest Design (One-shot case

study)

 No pretesting & control group makes it vulnerable to threats to internal validity; cannot be sure that the independent (treatment) variable (IV) has caused changes in the dependent variable (DV).

2. Static-Group Comparison

 Examines change in (DV) after (IV) is introduced for one group, and changes in DV for 2nd^ group not exposed to IV. Significant difference in group observations (O1 - O2) is evidence of the effect of the IV.

 No random assignment / pretesting means the design is vulnerable to selection bias (mortality). Meaningful comparisons are only possible if groups are comparable !!

Minimize threats to internal validity

by doing the study fast, use a

isolated study group, avoid

measuring complex variables &

relationships.

D1 = O2-O

D2 = O4-O

If D1 >or < D then X is the cause.

The Pretest-Posttest Control Group Design

(a.k.a. The Classic Experimental Design

 Controls for most threats to internal validity by using a comparable control group that does not receive exposure to the treatment variable.

 Still vulnerable to bias from testing and experimental mortality.

D1=O2-O (Pretesting + Treatment)

D2=O4-O (Pretesting Alone)

D3=O5-O (Treatment Alone)

 Randomization is key to a true experimental design. We want comparable groups by eliminating any differences between them that could provide alternative explanations for any differences we observe in the DV after exposure to the IV or treatment variable.

 Random assignment of subjects to the treatment and control group(s) achieves this…sometimes augmented by matching.

Non-Equivalent Control Group Design

 Uses a “comparison group” rather than a true control group.

 Example: Psychosocial effects of introducing pets to nursing homes. Cannot randomly assign seniors to nursing homes to form a true control group. Could find comparable nursing homes.

 Introduce pets to one nursing home and use a comparison nursing home (where no pets are allowed) to assess the effect of pets on seniors.

Another poll ruins election suspense….

 On January 22, 2006 the research firm SES sampled 1,200 Canadians on their voting intentions in the upcoming Federal election. These were the results of the poll:  Conservative 36.4%  Liberal 30.1%  NDP 17.4%  Bloc Quebecois 10.6%  Green/Other 5.6%

 The company claimed that these estimates were accurate to within +/- 3 percentage points 19 times out of 20.

The next day (January 23, 2006) several million Canadian voters cast their votes as follows….  Conservative 36.3%  Liberal 30.2%  NDP 17.5%  Bloc Quebecois 10.5%  Green/Other 5.5%  Q. How could SES Research, with a tiny sample of 1200 predict with amazing accuracy the voting behavior of several million Canadians…and ruin much of the suspense on election night?

 A. With careful sampling techniques!!!