Research Design(2)

Research Design:

Research Design 17.871 Spring 2003

General Comments:

General Comments The road map of political science Different ways of doing political science research Major components of research designs Designing research to ferret out causal relationships Social science vs. natural science/engineering

The Road Map:

The Road Map Philosophy Normative Positive Causal Correlational Descriptive Theoretical Empirical

Different Ways of Doing Empirical Research:

Different Ways of Doing Empirical Research Interpretive Small- n case study Haphazard Structured Large- n statistical analysis Interactions among these ways

Major Components of Research Designs:

Major Components of Research Designs Research question Theory Data

Research Question:

Research Question Important Not too general Not too specific Just right Contribute to literature How to tell: Social Sciences Citation Index http://libraries.mit.edu/get/webofsci E.g.: effect of redistricting on congressional election results Search for Cox & Katz, “The Reapportionment Revolution and Bias in U.S. Congressional Elections,” AJPS 1999

Theory:

Theory Definition: A general statement of a proposition that argues why events occur as they do and/or predicts future outcomes as a function of prior conditions General/concrete trade-off Desirable qualities of theories Falsification (Karl Popper) Parsimony (Occam’s razor)

Data:

Data Terms Cases Observations Variables Dependent variables Independent variables Units of analysis Mapping between the abstract and concrete (we’ll come back to this) Measures Indicators

Causality:

Causality Definition of causality Problems in causal research Side trip to Campbell and Stanley

Definitions of Causality:

Definitions of Causality Logical A causes B if the “presence” of A is a sufficient condition for B . Experiential A causes B if B occurs following the “exogenous” introduction of A When does exogeneity occur? Positive example: “ethnic” names on resumes Negative example: campaign spending

The Biggest Problem in Causal Research:

The Biggest Problem in Causal Research Establishing the exogeneity of “causes”

How to Establish Causality:

How to Establish Causality Donald Campbell and Julian Stanley, Experimental and Quasi-Experimental Designs for Research (1963)

Design types:

Design types One-shot case study One-group pre-test/post-test Static group comparison Pre-test/post-test with control group Solomon four-group design Post-test only experiment [Running example: racial discrimination in resumes]

One-shot Case Study:

One-shot Case Study Summary: X O or O X Journalism Common sense “of no scientific value”

One-group Pre-test/Post-test:

One-group Pre-test/Post-test Summary: O X O Better than nothing Standard way of doing most research Big problems No comparison group No random assignment Encourages “samples of convenience”

Static group comparison:

Static group comparison Summary: X O 1 ----------- O 2 This is most cross-sectional & correlational analysis Problems Selection into the two groups No pre-“treatment” measurement

Pre-test/Post-test Control Group:

Pre-test/Post-test Control Group Summary: R O 1 T X O 2 T -------------------------------- R O 1 C O 2 C Effect of treatment: [O 2 T – O 1 T ] – [O 2 C – O 1 C ] This is the classic randomized experiment Problem: “Hawthorne effect”

Solomon Four-Group Design:

Solomon Four-Group Design Summary: R O X O R O O R X O R O Allows you to control for the effect of the experiment itself

Post-test only experiment:

Post-test only experiment Summary: R X O R O No prior observation (assume O 1 T = O 1 C ) Classical scientific and agricultural experimentalism

Where do standard political science studies fall among the Stanley/Campbell designs?:

Where do standard political science studies fall among the Stanley/Campbell designs? One-shot case study Little scientific value, but may be descriptively useful One-group pre-test/post-test Often used in policy analysis Only justified as a “best design” if there are ethical or other constraints Static group comparison Correlational studies by far the most common “scientific” social science research Pre-test/post-test with control group “Real” experiments uncommon, but growing in frequency “Quasi-experiments” growing more rapidly Solomon four-group design Don’t recall ever seeing this Post-test only experiment Leads to weaker statistical tests

Social Science vs. Natural Science and Engineering:

Social Science vs. Natural Science and Engineering Reductionism Degree of reductionism Implications Measures of association weak Aggregates often better predictors Why we have statistics