EDRM 600 Chapter 10 Summary

Experimental Designs: 

Experimental Designs Presented by David Knott Angela Neve Rebekah Carroll & Vicki Sommerfeldt

Brief History: 

Brief History Began as psychological experiments in late 19th & early 20th centuries How to Conduct an Experiment by McCall est. randomized assignment Campbell & Standley specified 15 types since then, more sophisticated designs & advanced statistical procedures

Slide 3: 

Idea or Procedure (Independent Variable) Outcome (Dependent Variable) Does the procedure influence the outcome? (Experiment)

Why choose experimental designs?: 

Why choose experimental designs? To establish possible cause and effect between an idea/practice/procedure (independent variable) and an outcome (dependent variable)

Intro to our example study : 

Intro to our example study The Effects of Improvisational Music Therapy on Joint Attention Behaviors in Autistic Children: A Randomized Controlled Study Repeated measures comparison design Dependent variable - joint attention Independent variables - included treatment condition (12 weekly 30m improvisational music therapy sessions and 12 weekly 30m play sessions with toys. Measures: PDDBI & ESCS, videotaped sessions, multiple observers/raters, statistical procedures Results: IMP more effective in facilitating joint attention behaviors and non-verbal social communication skills than play with toys.

Control: 

Control Steps researcher takes to reduce extraneous influences (influences other than the treatment effect) between the cause and effect

Key Characteristics of Experiments: 

Key Characteristics of Experiments Random assignment or Equating the groups Assigning individuals to groups so that individual differences are distributed equally Experimental design without randomization is called quasi-experimental and is a good compromise for settings where randomization is unethical or not possible (health care or education for ex.)

Control over extraneous variables: 

Control over extraneous variables Pretest & posttests Covariates (ex. p.299, figure 10.1) Matching of participants Homogenous samples Blocking variables

Manipulating Treatment Conditions : 

Manipulating Treatment Conditions Treatment variables Conditions Intervening in the treatment conditions

Outcome measures: 

Outcome measures Dependent variable Response, criterion or posttest

Our Music Study Example : 

Our Music Study Example Independent Variables Dependent Variables Age (cannot manipulate) Joint Attention Gender (cannot manipulate) 3. Autism diagnosis (cannot manipulate) 4. Types of treatment (can manipulate) Play Therapy (control) Music Therapy (experimental)

Group Comparisons: 

Group Comparisons The process of obtaining scores for individuals/groups on the dependent variable Compare the means and variance both within the group and between groups

Threats to Validity: 

Threats to Validity Specific reasons why we could be wrong when we make an inference in an experiment Internal validity: Problems in drawing correct inferences about whether one treatment variable causes the variation in another variable and the outcome reflects a causal relationship External validity: Problems in drawing correct inferences from the sample data to other persons, settings, treatment variables, and measures

Threats to Internal Validity: 

Threats to Internal Validity Having to do with participants History Maturation Regression Selection Mortality Interactions with selection

Threats to Internal Validity: 

Threats to Internal Validity Having to do with Treatments/Procedures Diffusion Equalization Compensatory rivalry Resentful demoralization Testing Instruments

Threats to External Validity: 

Threats to External Validity Interaction of: Selection/treatment Setting and treatment History and treatment

Slide 17: 

Between Group Designs Within Group or Individual Designs True Experiment Quasi-Experiment Factorial Time Series Repeated Measures Single Subject Random assignment? Yes No May be used No No No Number of groups/ individuals compared? Two or more Two or more Two or more One group One group One individual studied at a time Number of interventions used? One or more interventions One or more interventions One or more interventions One or more interventions Two or more interventions One or more interventions Number of times the dependent variables measured/ observed? Once Once Once After each intervention After each intervention Multiple points Controls typically used? Pretest, matching, blocking, covariates Pretest, matching, blocking, covariates Pretest, matching, blocking, covariates Group becomes its own controls Covariates Individuals become their own control

Slide 18: 

Within Group or Individual Designs Time Series Repeated Measures Single Subject Random assignment? No No No Number of groups/ individuals compared? One group One group One individual studied at a time Number of interventions used? One or more interventions Two or more interventions One or more interventions Number of times the dependent variables measured/ observed? After each intervention After each intervention Multiple points Controls typically used? Group becomes its own controls Covariates Individuals become their own control

Slide 19: 

Within Group or Individual Designs Time Series Repeated Measures Single Subject Random assignment? No No No Number of groups/ individuals compared? One group One group One individual studied at a time Number of interventions used? One or more interventions Two or more interventions One or more interventions Number of times the dependent variables measured/ observed? After each intervention After each intervention Multiple points Controls typically used? Group becomes its own controls Covariates Individuals become their own control

Slide 21: 

Within Group or Individual Designs Time Series Repeated Measures Single Subject Random assignment? No No No Number of groups/ individuals compared? One group One group One individual studied at a time Number of interventions used? One or more interventions Two or more interventions One or more interventions Number of times the dependent variables measured/ observed? After each intervention After each intervention Multiple points Controls typically used? Group becomes its own controls Covariates Individuals become their own control

Slide 23: 

Within Group or Individual Designs Time Series Repeated Measures Single Subject Random assignment? No No No Number of groups/ individuals compared? One group One group One individual studied at a time Number of interventions used? One or more interventions Two or more interventions One or more interventions Number of times the dependent variables measured/ observed? After each intervention After each intervention Multiple points Controls typically used? Group becomes its own controls Covariates Individuals become their own control

Slide 25: 

Within Group or Individual Designs Time Series Repeated Measures Single Subject Random assignment? No No No Number of groups/ individuals compared? One group One group One individual studied at a time Number of interventions used? One or more interventions Two or more interventions One or more interventions Number of times the dependent variables measured/ observed? After each intervention After each intervention Multiple points Controls typically used? Group becomes its own controls Covariates Individuals become their own control

Slide 27: 

Within Group or Individual Designs Time Series Repeated Measures Single Subject Random assignment? No No No Number of groups/ individuals compared? One group One group One individual studied at a time Number of interventions used? One or more interventions Two or more interventions One or more interventions Number of times the dependent variables measured/ observed? After each intervention After each intervention Multiple points Controls typically used? Group becomes its own controls Covariates Individuals become their own control

Slide 28: 

A: 2 Baseline Sessions “What is your home telephone number?” B: 5 Treatment Sessions After the music therapy treatment, “What is your home telephone number?”

Slide 30: 

Within Group or Individual Designs Time Series Repeated Measures Single Subject Random assignment? No No No Number of groups/ individuals compared? One group One group One individual studied at a time Number of interventions used? One or more interventions Two or more interventions One or more interventions Number of times the dependent variables measured/ observed? After each intervention After each intervention Multiple points Controls typically used? Group becomes its own controls Covariates Individuals become their own control

Slide 31: 

Within Group or Individual Designs Time Series Repeated Measures Single Subject Random assignment? No No No Number of groups/ individuals compared? One group One group One individual studied at a time Number of interventions used? One or more interventions Two or more interventions One or more interventions Number of times the dependent variables measured/ observed? After each intervention After each intervention Multiple points Controls typically used? Group becomes its own controls Covariates Individuals become their own control

Slide 32: 

Within Group or Individual Designs Time Series Repeated Measures Single Subject Random assignment? No No No Number of groups/ individuals compared? One group One group One individual studied at a time Number of interventions used? One or more interventions Two or more interventions One or more interventions Number of times the dependent variables measured/ observed? After each intervention After each intervention Multiple points Controls typically used? Group becomes its own controls Covariates Individuals become their own control

Slide 33: 

Within Group or Individual Designs Time Series Repeated Measures Single Subject Random assignment? No No No Number of groups/ individuals compared? One group One group One individual studied at a time Number of interventions used? One or more interventions Two or more interventions One or more interventions Number of times the dependent variables measured/ observed? After each intervention After each intervention Multiple points Controls typically used? Group becomes its own controls Covariates Individuals become their own control

Types of Experimental Designs: 

Types of Experimental Designs Most Common Designs Between-Group Designs True Experiments Quasi-experiments Factorial Designs

Major Characteristics for Design Identification: 

Major Characteristics for Design Identification Random assignment of participants to groups Number of groups or individuals being compared Number of interventions used by the researcher Number of times the dependent variable is measured or observed Control of extraneous variables

Threats to Internal Validity: 

Threats to Internal Validity To Participants History Maturation Regression Selection Mortality Interaction of selection and maturation, history, and instrumentation To Procedures Testing Instrumentation Threats to internal validity vary according to experimental design

Between-Group Designs: True Experiments: 

Between-Group Designs: True Experiments Characteristics of True Experiments Include random assignment of participants Two or more groups One or more interventions used by researcher Dependent variable is measured or observed one time Pretest, matching, blocking, & covariates are controls typically used

Between-Group Designs: True Experiments: 

Between-Group Designs: True Experiments Threats to Internal Validity All threats to validity re: participants are controlled Randomization of groups minimizes possibility of internal threats to participants Treatment threats such as diffusion, rivalry, resentful demoralization, and compensatory equalization are possibilities Due to 2+ groups exist If true experiments include only a posttest, threats to testing, instrumentation, and regression are reduced because no pretest

True Experiments: Relating to the Sample Study: 

True Experiments: Relating to the Sample Study Need to randomly assign participants to control or experimental group Interested in effect of improvisational music therapy intervention on joint attention in children with autism Specifically Control group Experimental/Treatment group for improvisational music therapy intervention

Between-Group Designs: Quasi-Experiments: 

Between-Group Designs: Quasi-Experiments Characteristics of Quasi-Experiments Include assignment but no random assignment of participants to groups Two or more groups One or more interventions used by researcher Dependent variable is measured or observed one time Pretest, matching, blocking, & covariates are controls typically used

Between-Group Designs: Quasi-Experiments: 

Between-Group Designs: Quasi-Experiments Major difference between True Experiments and Quasi-Experiments No random assignment because researchers may have to use groups available Not able to artificially create groups

Between-Group Designs: Quasi-Experiments: 

Between-Group Designs: Quasi-Experiments Threats to Internal Validity History, Maturation, Regression, Selection, Mortality, and Interaction of selection, maturation, history, and instrumentation are all potential threats Individuals assigned to the 2 groups may have selection factors that go uncontrolled in experiment Treatment threats present may be present when comparing two groups

Quasi-Experiments: Relating to our Sample Study: 

Quasi-Experiments: Relating to our Sample Study No random assignment of participants to group Still have control group and experimental group Music therapy intervention Control group-no intervention

Table: Between-Group Designs: 

Table: Between-Group Designs

Between-Group Designs: Factorial Designs: 

Between-Group Designs: Factorial Designs Characteristics of Factorial Designs Random assignment may be used Two or more groups Two or more interventions used by researcher Dependent variable is measured or observed one time Pretest, matching, blocking, & covariates are controls typically used

Between-Group Designs: Factorial Designs: 

Between-Group Designs: Factorial Designs Purpose of design is to study independent and simultaneous effects of 2+ independent treatment variables Advantage-high level control

Between-Group Designs: Factorial Designs: 

Between-Group Designs: Factorial Designs Threats to Internal Validity As in True Experiment, if random assignment internal threats to participants are controlled As in Quasi-Experiments, if no random assignment internal threats to participants may exist Similar threats to procedures exist in Factorial Designs as in Quasi-Experiments & True Experiments

Between-Group Designs: Factorial Designs: 

Between-Group Designs: Factorial Designs Multiple independent variables = more complex statistical procedures and results More participants needed in each group for tests Typically include, at most, 3 independent variables

Factorial Design: Process of Conducting: 

Factorial Design: Process of Conducting Researcher identifies research question including 2+ independent variables & 1 dependent variable To answer, researcher identifies levels of each independent variable # of levels determines how refer to factorial design

Factorial Design: Process of Conducting: 

Factorial Design: Process of Conducting Participants assigned to each group (6 groups in this example) Posttest used to measure Scores organized into cells representing each group Represent mean scores Find if statistically significant & determine whether scores support/reject null hypothesis

Factorial Design: Relating to our sample study: 

Factorial Design: Relating to our sample study 3, 20-minute sessions 2, 30-minute sessions 1, 60-minute session Music Therapy Intervention Group 1-Joint attention Group 2-Joint attention Group 3-Joint attention Play Intervention Group 4-Joint attention Group 5-Joint attention Group 6-Joint attention Sample study levels- Dependent variable-Joint attention Independent variables Factor 1-Type of intervention Level 1-Music Therapy Intervention Level 2-Play Intervention Factor 2-S ession length/presentation Level 1-3, 20-minute sessions Level 2-2, 30-minute sessions Level 3-1, 60-minute session

Slide 52: 

Now head on over to the discussion board and ask us any questions you may still have lingering!