Class 6: Review: Psychological Testing : Class 6: Review: Psychological Testing An example of anthropological research
Psychological Testing
Intro: Categories of tests
History of psych testing
Q: why testing?
Class 7: Psychological Testing : 2 Class 7: Psychological Testing Presentation
Classical test theory:
Bias, Reliability, Validity
Basic descriptive statistics for psychological tests
McWilliams on assessment reports
Test Bias : 3 Test Bias 1. Test bias:
When scores are influenced by factors other than what the test claims to measure. *
If there is bias in the test, the result:
Under- or over-estimating the quality measured (e.g., IQ).
E.g., IQ results that show race or gender or class differences.
Is the test a valid measure of ability, or is the test biased?
Test Bias vs. Test Fairness : Test Bias vs. Test Fairness 2. Test fairness:
Is the test put to uses that are socially/morally appropriate?
Even if the test is valid, is its use justified in being used in furthering values chosen?
Very easy for tests to be used unfairly (as well as to be invalid).
SAT test: an example : 5 SAT test: an example SAT-I (general reasoning test) systematically under-estimates the academic potential of ?
Revised recently
But some causes of the score gaps remain in place:
the test's multiple-choice format,
highly-speeded pace, and
rewards for strategic guessing
Gender gap reduced by essay question *
While score gap for minorities will likely grow *
Bias? The IQ gap * : Bias? The IQ gap * Differ 15 IQ points
less when social class and status considered
Why?
Geo. Farkas: “talk deficit”:
differences in verbal exchanges between cultures*
Farkas: another factor: “oppositional culture”:
doing well in school is “negatively sanctioned as ‘acting White’.”
“[I]t’s unlikely that a test of anything is fair to all cultures under all circumstances”
A dilemma of assessment : A dilemma of assessment Obtaining “scores” inexact and potentially depersonalizing
The person on the street often believes tests scores are exact, represent “real” qualities (vs. theoretical).
The alternatives of no testing as bad:
How does one make decisions about which persons to help?
Some of the early issues…
Classical Testing Theory: 1. Reliability : Classical Testing Theory: 1. Reliability Basic definition: consistency: does a test result in the same score (approximately) when readministered to the same individual.
Without reliability, a test can’t be used (there are exceptions to this rule).
Whatever it purports to measure can’t be genuinely measured.
But it doesn’t have to be precise *
Practical implications
Reliability measurement : Reliability measurement Error score example: Public opinion polls’ “margin of error”
Reliability is typically measured by “correlation”:
how well do two samples of the same behavior or testing (e.g., multiple versions of a test) yield the same scores across a group of people?
What are some ways to establish reliability?
Slide 11: Parallel
Forms
2 versions of test
give same score
Classical Testing Theory: 2. Validity : 12 Classical Testing Theory: 2. Validity Basic dn: How well does a test measure what it claims to evaluate? *
Does language match our experience?
Or: How does one know if/when psychological constructs exist? ***
Reliability (consistency) is necessary for validity (actually measuring a construct accurately)
Several types
Examples of types of validity : Examples of types of validity 1. Content validity:
Does the content of a test parallel the content of the course/training?*
2. Criterion validity:
Does the test relate to the performance it is predicting? **
3. Construct validity:
Does the test assess the ability (construct) it claims to examine? ***
4. Consequential validity:
What are the consequences of using or interpreting tests?
Validity of Selection Procedures : 14 Validity of Selection Procedures
Valid? Reliable? : Valid? Reliable?
Slide 16: 16 NY Times: “Valid” New Divisions?
Validity-Final thoughts : 17 Validity-Final thoughts Both good reliability and validity are necessary if there is to be a meaningful test.
Validity on the same test may vary *
Validity generalization **
“Meta-analysis”
Reliability vs. Validity: example : Reliability vs. Validity: example Menand, L. (2010) “Head Case; can psychiatry be a science?” New Yorker, Mar. 1, 2010, 68-74. *
Before DSM-III (1980), there was poor agreement in diagnosing the same patient.
To improve reliability, the DSM–III created a list of symptoms to be met to dx
Menand: the DSM did not address another problem:
did the psychological dxs correspond to real pathology (illnesses)?
Closing thoughts
Appendix: Descriptive statistics : 19 Appendix: Descriptive statistics Ways to characterize distributions of data:
1. Averages (measures of central tendency)
2. Variability (measures of distribution of scores--scatter)
3. The Normal curve (shape of distribution of scores)
Or to put it slightly differently…
Slide 20: 20
Scores and Norms : 21 21 Scores and Norms Raw vs. transformed scores
“Raw” scores:
refer to the number obtained on a test. *
One transformation:
Develop “norms” from observing scores of many people on the same test: **
Where does the person “fall” relative to others?
Transformed Scores* : 22 22 Transformed Scores* 1. Percentile:
the score where a % of the scores fall below that score.
Norm-based tests
2. Percentage correct:
the number of questions answered correctly.
Criterion-based tests
Types of averages (measures of central tendency) : 23 23 Types of averages (measures of central tendency) Most typical score. The “representative” value of a distribution of #s or scores.
Mean (µ):
Most common meaning of “average”:
sum all the scores up and divide by the number of scores.
Median: The middle score when scores are arranged in order.
Mode: The most frequently occurring score
Mean, … : Mean, …
Median, or Mode? : Median, or Mode?
2. Variability (measures of distribution of scores--scatter)* : 28 2. Variability (measures of distribution of scores--scatter)* a. Range:
the distance between the high and low scores. *
b. Standard deviation (s; SD; σ):
a measure of typical (average) variability from the mean. *
Standard error of measurement (SEM):*
the typical amount that a score may vary in taking the same test over repeated administrations.
3. The Normal curve : 29 29 3. The Normal curve Vertical axis: frequency of scores occurring.
Horizontal axis: the range of scores/data points: e.g., scores on a test, weight.
Normal Curve: Properties : 30 Normal Curve: Properties Mathematically derived *
Occurs in nature *
Some features: the mean, median, and mode score are the same in the “normal” curve.
Predictions possible*
Variations exist: E.g.: *
Raw scores transformed
Some characteristics of the normal curve : 31 Some characteristics of the normal curve
Calvin and Hobbs reaction to statistics? : 33 Calvin and Hobbs reaction to statistics?
How to write your assessment? : How to write your assessment?
Slide 35: Case Formulation
Case Formulation: Chpt. 1* : Case Formulation: Chpt. 1* Formulation
DSM diagnosis
Case formulation vs. DSM discrete dxs
Therapist’s needs
Therapy goals in a case formulation (McWilliams) : Therapy goals in a case formulation (McWilliams) 1. Symptom relief
2. Insight
3. Agency
4. Identity
5. Self-esteem
6. Recognizing/handling feelings
7. Ego strength, self-cohesion
8. Love, work, mature dependency
9. Pleasure, serenity
Slide 38: End of Class 7, Summer 2010 38
Exercise : Exercise Groups of 3
1st 5 minutes:
Person 1: Role play a client with exaggerated pathology/characteristics (ham it up)
Person 2: Interviewer looking for psychological inner experience of client
Next 5 minutes
Person 3: Observer: report on what s/he found of the qualities we’ve just listed;
what seems to be the varieties of subjective experiences in various realms?
Check w each other to see more?
Corrections?
Next 10 minutes: report to the class about the exercise.
Integrating McWilliams’ formulation into a case report : Integrating McWilliams’ formulation into a case report Demographics—
age,
work/school,
love,
key relations (parenting, partners/friends, culture),
Hx (history)
Behavioral observations
Symptoms—Presenting problems
Mental Status Exam
Case report (continued) : Case report (continued) Defenses (added from McWilliams)
Insight (unconscious factors)
Agency (and lack of it)
Identity issues
Self-esteem
Affect/feelings
Transference, Countertransference (added)
Ego strength/self-cohesion
Love and sexuality, work, mature dependency (relational style)
Pleasure, serenity
Case report (completed) : Case report (completed) DSM Dx (diagnosis)
Testing and therapy recommendations
Summary and conclusions