Slide1: Methods for assessing the outcomes of rehabilitation: theory and practice Harvey Dillon National Acoustic Laboratories (NAL) CRC for Cochlear Implant and Hearing Aid Innovation Hearing International Pattaya, January, 2002


Talk outline: Talk outline Applications of outcome measures Types of outcomes measures When to measure outcomes Talk can be downloaded from www.nal.gov.au (from next week)


Definition of Outcomes: Definition of Outcomes A measure of the changes in a client’s life as a result of delivering a service and/or fitting devices Outcomes versus inputs Inputs: what the service does to or for a client. Outcomes: what changes in the life of the client.


Need for outcome measures: Need for outcome measures Which device for which clients Determine when to finish the program Systematically study device effectiveness Know what to teach Know which devices to pay more for Know which devices to pay more for Know what to spend time doing Verify effectiveness of new procedures Know what procedures to teach Know what procedures to pay for Benefit from time well spent Benefit from choosing an effective provider Government/insurance money spent most effectively Get best value for money Monitor improvements / deterioration in average outcome quality Develop realistic expectations and goals


Types of outcome measures: Types of outcome measures


Types of outcome measures: Types of outcome measures Broad type: Objective versus self-report Quantity measured: Benefit, satisfaction, aid usage, generic quality-of-life Calculation: Change versus state Format: Standardized versus individualized Administration: Clinician versus anonymous


Broad types of measures: Broad types of measures Speech Tests (before and after) + Objective Time consuming Depend on conditions (Totally speech oriented) Observation of clients Impractical Client Report From client’s perspective All inclusive


Broad types of measures: Broad types of measures Speech Tests (before and after) + Objective Time consuming Depend on conditions Totally speech oriented (awareness, localization) Observation of clients Impractical Client Report From client’s perspective Specific or all inclusive Not Insertion Gain


Quantity measured: Quantity measured Benefit Impairment Disability (or activity limitation) Impairment


HAPI / SHAPIEWalden, Demorest & Hepler, 1984Dillon, 1994: HAPI / SHAPIE Walden, Demorest & Hepler, 1984 Dillon, 1994 1. You are watching TV and there are distracting noises such as others talking. ____ ___ ___ ___ ___ 2. You are at home in conversation with a member of your family who is in another room ___ ___ ___ ___ ___


Quantity measured: Quantity measured Benefit Impairment Disability (or activity limitation) Handicap (or participation limitation)


Hearing Handicap Inventory for the Elderly (HHIE)Ventry & Weinstein, 1982: YES SOME- NO (or TIMES (or usually) rarely) 1. Does a hearing problem cause you ____ ____ ____ to use the phone less often than you would like? 2. Does a hearing problem cause you ____ ____ ____ to feel embarrassed when meeting new people? 3. Does a hearing problem cause you ____ ____ ____ to avoid groups of people? Hearing Handicap Inventory for the Elderly (HHIE) Ventry & Weinstein, 1982


Quantity measured: Quantity measured Benefit Impairment Disability (or activity limitation) Handicap (or participation limitation) Satisfaction


Measuring overall satisfaction: Measuring overall satisfaction 1. Hearing Aid: Please mark, on the scale below, how satisfied you are overall with your hearing aid. Zero would mean that you are not at all satisfied; 100 would mean that you are totally satisfied. +---------+---------+---------+---------+---------+ 0 20 40 60 80 100


Measuring overall satisfaction: Measuring overall satisfaction 2. Service provided: Overall, how satisfied you with the rehabilitation service you have received ? Extremely satisfied Satisfied Neither satisfied nor dissatisfied Dissatisfied Extremely dissatisfied


Measuring satisfaction in detail: Measuring satisfaction in detail Satisfaction with Amplification in Daily Life (The SADL)


Quantity measured: Quantity measured Benefit Impairment Disability (or activity limitation) Handicap (or participation limitation) Satisfaction Device usage


Device usage: Device usage On average, how many hours per day do you wear your hearing aids: More than 8 hours 4 to 8 hours 1 to 4 hours < 1 hour Never For what proportion of the time that you need hearing aids do you actually wear them?


Quantity measured: Quantity measured Benefit Impairment Disability (or activity limitation) Handicap (or participation limitation) Satisfaction Device usage Quality of life SF 36 Sickness Impact Profile


Calculation method: Calculation method Change measures: How much better do you hear when aided? State measures: How well do you hear when unaided? How well do you hear when aided? Tells us about the starting and finishing states as well as the difference A more accurate estimate of the difference


Calculation method: Calculation method Change method: e.g. HAPI / SHAPIE You are watching TV and there are distracting noises such as others talking. ____ ___ ___ ___ ___ 4 3 2 1 -


Calculation method: Calculation method State method: e.g. APHAB (Cox & Alexander, 1995) I have difficulty hearing a conversation when I’m with one of my family at home. ___ ___ __ ___ ___ ___ ___ 1 2 3 4 5 6 7


Calculation method: Calculation method Change versus state method Glasgow Hearing Aid Benefit Profile (GHABP) (Gatehouse, 1997) Change measure more sensitive than state measure But change alone can be inadequate starting or finishing position.


Format: standardized versus individualized: Format: standardized versus individualized Questionnaires usually standardized in format + same questions for everyone + easy to score + easy to compare results across people + easy to combine results across people can be tedious can contain irrelevant items difficult to integrate with rehab process Questionnaires can be individualized e.g. the Client Oriented Scale of Improvement (COSI) individualized concept from Stephens (1980)


COSI: COSI The NAL Client Oriented Scale of Improvement


Client Oriented Scale of Improvement (COSI): Client Oriented Scale of Improvement (COSI) At interview Where / when would you like better hearing? At final appointment, for each situation: How much improvement have you noticed? How well can you now hear?


Things clinicians like about COSI: Things clinicians like about COSI


Things clinicians dislike about COSI: Things clinicians dislike about COSI


Some Anecdotal Observations: Some Anecdotal Observations The COSI helps build rapport Clients feel they are being listened to, and that clinician remains interested in their problems Goal/needs gives direction regarding aid fitting and other services The COSI clarifies clients needs to themselves and requires acknowledgment of disability Simple documentation helps case continuity


Administration: Administration Administered by clinician + Pre-measures may influence rehab program + Can act on results immediately Responses may be less honest Self administered + Less clinician time needed More chance of mis-interpretation (e.g. HHIE critical differences doubling)


When to measure outcomes: When to measure outcomes Three to four weeks after fitting Six to seven weeks after fitting Three months after fitting Six months after fitting One year after fitting


Slide34: The importance of a good fitting …. and of checking HAUQ questionnaire


Problems reported at 3 months: Problems reported at 3 months


Conclusions: Conclusions Do measure outcomes - replace anecdotes with systematic observations, biases with supportable facts. Include dimensions of benefit, satisfaction, and usage e.g. International Outcomes Inventory - Hearing Aids (IOI-HA) Assess problems as well Use a measure that also contributes to the outcome e.g. Client Oriented Scale of Improvement (COSI) Be ready to argue about outcomes for hearing-impaired people with people who want to argue about dollars and cents (or Bahts)


For further information …… : For further information …… 1. Download presentation: www.nal.gov.au 2. Chapter 13: Hearing Aids (Dillon, 2001; Boomerang Press / Thieme Publishing). 3. Entire book: Noble, W (1998). Self-assessment of hearing and related functions. Whurr. 4. Special issue: Ear & Hearing, Vol 21, Number 4 (2000), edited by Robyn Cox. 5. International Journal of Audiology: First issue, just out.


Slide38: The end