logging in or signing up Reliability and Validity dbuonline Download Post to : URL : Related Presentations : Let's Connect Share Add to Flag Embed Email Send to Blogs and Networks Add to Channel Copy embed code: Embed: Flash iPad Dynamic Copy Does not support media & animations Automatically changes to Flash or non-Flash embed WordPress Embed Customize Embed URL: Copy Thumbnail: Copy The presentation is successfully added In Your Favorites. Views: 351 Category: Entertainment License: All Rights Reserved Like it (0) Dislike it (0) Added: February 12, 2013 This Presentation is Public Favorites: 0 Presentation Description No description available. Comments Posting comment... Premium member Presentation Transcript Reliability and Validity: Reliability and Validity THE GREAT TRIAD : Reliability – Does the instrument measure accurately and consistently? Validity- Does the instrument measure what it says it measures? Objectivity- Is the instrument immune to the personal attitudes and opinions of the researcher? THE GREAT TRIAD RELIABILITY : RELIABILITY The degree to which test scores are free from errors of measurement. The degree of consistency with which an instrument measures what it is measuring. RELIABILITY AND ERROR : Error Variance Systematic Error – there is a system (methods are planned, orderly, and methodical) Unsystematic Error – there is a lack of a system (occurrences are presumed to be random) RELIABILITY AND ERROR 3 IMPORTANT MEASURES OF RELIABILITY : 1. Coefficient of Stability How stable is it over time? 2. Coefficient of Internal Consistency Is the test internally consistent? Coefficient of Equivalence Is the test the same/different than others? 3 IMPORTANT MEASURES OF RELIABILITY MEASURES OF RELIABILITY: Test-Retest Reliability (Stability) Alternate, Parallel, or Equivalent Forms Reliability (Equivalence) Split-Half or Odd-Even Reliability (Consistency) Cronbach’s Coefficient Alpha Kuder -Richardson Interrater Reliability MEASURES OF RELIABILITYStandard Error of Measurement: AKA: SEM Every score has 2 parts: True Score + Error The SEM estimates the error and places a band around the given score. The true score will fall within the range. This is referred to as a confidence band. Standard Error of MeasurementSEM Practice: If the SEM is 6 and the person’s score was 90, we would say that his true score would be between 84 and 96. 90 – 6 = 84, 90 + 6 = 96. If the score was 112 and the SEM was 5, what would the Confidence band be? 107 – 117. SEM PracticeConcerns: High coefficients of stability does not mean that internal consistency is high. Reliability coefficients apply to data and not to instruments. Reliability estimates consistency ConcernsVALIDITY: VALIDITY The degree to which all of the accumulated evidence supports the intended interpretation of test scores for the intended purpose The ability of research instruments to measure what they say they measureTYPES OF VALIDITY: Content Validity Face Validity Criterion-Related Validity Concurrent Validity Predictive Validity TYPES OF VALIDITY TYPES OF VALIDITY : TYPES OF VALIDITY Construct Validity Factor Analysis Meta-analysis Convergent Validity Discriminant ValidityInternal Invalidity: Ask the question, “Are the measurements influenced only by the treatment or are there other influences which change it?” Concerned with the DV Internal Invalidity Internal Invalidity: History Maturation Testing Instrumentation Regression Selection Interaction Mortality John Henry Effect Internal InvalidityAdditional Problems: Hawthorne Effect – If subjects know they are part of an experiment – or if they are given more attention because of the experiment – their performance sometimes improves. Halo Effect – The halo effect occurs when a trait that is not being evaluated influences a researcher’s rating on another trait. Rosenthal Effect – The experimenter’s beliefs about the individual may cause the individual to be treated in a special way so that the individual begins to fulfill the experimenter’s expectations. Reactive Effect – The very presence of the researcher can influence behavior. Additional Problems External Invalidity: Ask the question, “How confidently can I generalize my experimental findings to the world?” Concerned with the IV External Invalidity External Invalidity: Effects of Testing Treatment and Subject Testing and Subject Multiple Treatments External InvalidityIMPORTANT!!!: A Test Must be Reliable in Order to be Valid! A Test Can be Reliable Even if it Isn’t Valid! NCE hint: Validity is more important than reliability. IMPORTANT!!!Observation Obstacles: Personal Interest Early Decision Personal Characteristics (Being human) Observation Obstacles Measurement Types: Nominal Measurement Variables which are categorized into discrete groups. Ordinal Measurement Variables which are rank ordered. Interval Measurement In order to measure distance between data points, we need a scale of equal, fixed gradation. No meaningful zero! Ratio Measurement The same as interval measurements except... With a meaningful zero! Measurement TypesPowerPoint Presentation: Name Attitude Score Test Score Test Rank Test Grade Gender Chris 80 92 2 A M Macy 55 81 4 B F Mike 90 96 1 A M Tina 84 85 3 B FPowerPoint Presentation: Name Attitude Score Test Score Test Rank Test Grade Gender Chris 80 92 2 A M Macy 55 81 4 B F Mike 90 96 1 A M Tina 84 85 3 B F You do not have the permission to view this presentation. In order to view it, please contact the author of the presentation.
Reliability and Validity dbuonline Download Post to : URL : Related Presentations : Let's Connect Share Add to Flag Embed Email Send to Blogs and Networks Add to Channel Copy embed code: Embed: Flash iPad Dynamic Copy Does not support media & animations Automatically changes to Flash or non-Flash embed WordPress Embed Customize Embed URL: Copy Thumbnail: Copy The presentation is successfully added In Your Favorites. Views: 351 Category: Entertainment License: All Rights Reserved Like it (0) Dislike it (0) Added: February 12, 2013 This Presentation is Public Favorites: 0 Presentation Description No description available. Comments Posting comment... Premium member Presentation Transcript Reliability and Validity: Reliability and Validity THE GREAT TRIAD : Reliability – Does the instrument measure accurately and consistently? Validity- Does the instrument measure what it says it measures? Objectivity- Is the instrument immune to the personal attitudes and opinions of the researcher? THE GREAT TRIAD RELIABILITY : RELIABILITY The degree to which test scores are free from errors of measurement. The degree of consistency with which an instrument measures what it is measuring. RELIABILITY AND ERROR : Error Variance Systematic Error – there is a system (methods are planned, orderly, and methodical) Unsystematic Error – there is a lack of a system (occurrences are presumed to be random) RELIABILITY AND ERROR 3 IMPORTANT MEASURES OF RELIABILITY : 1. Coefficient of Stability How stable is it over time? 2. Coefficient of Internal Consistency Is the test internally consistent? Coefficient of Equivalence Is the test the same/different than others? 3 IMPORTANT MEASURES OF RELIABILITY MEASURES OF RELIABILITY: Test-Retest Reliability (Stability) Alternate, Parallel, or Equivalent Forms Reliability (Equivalence) Split-Half or Odd-Even Reliability (Consistency) Cronbach’s Coefficient Alpha Kuder -Richardson Interrater Reliability MEASURES OF RELIABILITYStandard Error of Measurement: AKA: SEM Every score has 2 parts: True Score + Error The SEM estimates the error and places a band around the given score. The true score will fall within the range. This is referred to as a confidence band. Standard Error of MeasurementSEM Practice: If the SEM is 6 and the person’s score was 90, we would say that his true score would be between 84 and 96. 90 – 6 = 84, 90 + 6 = 96. If the score was 112 and the SEM was 5, what would the Confidence band be? 107 – 117. SEM PracticeConcerns: High coefficients of stability does not mean that internal consistency is high. Reliability coefficients apply to data and not to instruments. Reliability estimates consistency ConcernsVALIDITY: VALIDITY The degree to which all of the accumulated evidence supports the intended interpretation of test scores for the intended purpose The ability of research instruments to measure what they say they measureTYPES OF VALIDITY: Content Validity Face Validity Criterion-Related Validity Concurrent Validity Predictive Validity TYPES OF VALIDITY TYPES OF VALIDITY : TYPES OF VALIDITY Construct Validity Factor Analysis Meta-analysis Convergent Validity Discriminant ValidityInternal Invalidity: Ask the question, “Are the measurements influenced only by the treatment or are there other influences which change it?” Concerned with the DV Internal Invalidity Internal Invalidity: History Maturation Testing Instrumentation Regression Selection Interaction Mortality John Henry Effect Internal InvalidityAdditional Problems: Hawthorne Effect – If subjects know they are part of an experiment – or if they are given more attention because of the experiment – their performance sometimes improves. Halo Effect – The halo effect occurs when a trait that is not being evaluated influences a researcher’s rating on another trait. Rosenthal Effect – The experimenter’s beliefs about the individual may cause the individual to be treated in a special way so that the individual begins to fulfill the experimenter’s expectations. Reactive Effect – The very presence of the researcher can influence behavior. Additional Problems External Invalidity: Ask the question, “How confidently can I generalize my experimental findings to the world?” Concerned with the IV External Invalidity External Invalidity: Effects of Testing Treatment and Subject Testing and Subject Multiple Treatments External InvalidityIMPORTANT!!!: A Test Must be Reliable in Order to be Valid! A Test Can be Reliable Even if it Isn’t Valid! NCE hint: Validity is more important than reliability. IMPORTANT!!!Observation Obstacles: Personal Interest Early Decision Personal Characteristics (Being human) Observation Obstacles Measurement Types: Nominal Measurement Variables which are categorized into discrete groups. Ordinal Measurement Variables which are rank ordered. Interval Measurement In order to measure distance between data points, we need a scale of equal, fixed gradation. No meaningful zero! Ratio Measurement The same as interval measurements except... With a meaningful zero! Measurement TypesPowerPoint Presentation: Name Attitude Score Test Score Test Rank Test Grade Gender Chris 80 92 2 A M Macy 55 81 4 B F Mike 90 96 1 A M Tina 84 85 3 B FPowerPoint Presentation: Name Attitude Score Test Score Test Rank Test Grade Gender Chris 80 92 2 A M Macy 55 81 4 B F Mike 90 96 1 A M Tina 84 85 3 B F