logging in or signing up CASRO2007 Cell Phones for Data Collection Donato Download Post to : URL : Related Presentations : Share Add to Flag Embed Email Send to Blogs and Networks Add to Channel Uploaded from authorPOINTLite Insert YouTube videos in PowerPont slides with aS Desktop Copy embed code: Embed: Flash iPad Dynamic Copy Does not support media & animations Automatically changes to Flash or non-Flash embed WordPress Embed Customize Embed URL: Copy Thumbnail: Copy The presentation is successfully added In Your Favorites. Views: 374 Category: Education License: All Rights Reserved Like it (0) Dislike it (0) Added: January 31, 2008 This Presentation is Public Favorites: 0 Presentation Description No description available. Comments Posting comment... Premium member Presentation Transcript Cell Phones for Data Collection: Costs and Challenges: Cell Phones for Data Collection: Costs and Challenges Michael Link1, Michael Battaglia2, Martin Frankel3, Larry Osborn4, and Ali Mokdad5 1 Nielsen Media Research 2 Abt Associates Inc. 3 Baruch College, City University of New York 4 Knowledge Networks 5 Centers for Disease Control and Prevention The Plague of Cell Phones!!!: The Plague of Cell Phones!!!Cell phones and telephone surveys: Cell phones and telephone surveys Reliance on cell phones increasing (July-December NHIS): 57.1% of households have a working cell phone 11.6% of households (11.8% of adults) are cell phone only 25.4% for age 18-24, 29.1% for age 25-29, 54.0% of unrelated adult households w/o children, 26.4% for renters Result: increased potential for noncoverage bias Cell Phone Summit 2005, TSM II 2006, AAPOR 2007 – special issue of POQ Cell phones and telephone surveys: Cell phones and telephone surveys Conducting surveys via cell phones can be operationally challenging Cell phone frame may not be that efficient Geographic specificity is a problem Cannot use autodialers/predictive dialers Charges for incoming calls/minutes used Safety concerns Potential mode effects / measurement errors Level of cognitive engagement Behavioral Risk Factor Surveillance System (BRFSS): Behavioral Risk Factor Surveillance System (BRFSS) Monthly state-based landline RDD survey of health issues and related risk factors 50 states, District of Columbia, Puerto Rico, Guam, and Virgin Islands 350,000+ adult interviews conducted in 2006 Significant declines in participation overall, particularly among younger adults and males 2007 BRFSS cell phone pilot : 2007 BRFSS cell phone pilot Conducted in Georgia, New Mexico, & Pennsylvania Target: 200 cell & landline / 200 cell-only adults (per state) 1,200 total interviews Abbreviated BRFSS core interview: 66 questions 15-17 minutes (on average) Incentives: $10 post-paid incentive for completing the detailed interview $1 for completing the screenerSample design: Sample design Marketing Systems Group (MSG): All designated cellular 1,000 banks Implicit stratification by area code and exchange Equal probability sample of telephone numbers Survey Sampling Inc. (SSI): All 100 series banks designated as cellular “Mixed use” (landline / cell phone) banks containing zero residential directory-listed numbers Implicit stratification by FIPS, carrier, & 100-block Systematic random sample of 100 blocks Randomly generate last 2 digits of numberScreening questions: Screening questions Introduction Confirmed telephone number Is this a cellular telephone? Are you 18 years of age or older? Are you a resident of (state)? “Do you also have a landline telephone that is used to make and receive calls?” Yes – took subsample of respondents No – took all respondentsSurvey Participation Rates: Survey Participation RatesCalculation of rates: Calculation of rates Used detailed disposition codes modeled after Callegaro et al (2007) with some modifications/additions Included “ring, no answer” and “voice mail” as working residential numbers Only cases confirmed by company message as being not in service were excluded Used AAPOR response rate guidelines Calculated separate rates for: Screening for eligible respondent Completion of interviewParticipation rates: Participation rates Interview rate by landline access: Interview rate by landline access 64.3 64.9 57.7 66.1 66.7 64.1 70.2 64.8Percent working cell numberby sample vendor: Percent working cell number by sample vendor 63.3 66.1 61.2 65.3 70.5 74.5Comparison of respondent demographics: Comparison of respondent demographicsLandline and Cell phonepopulations and frames: Landline and Cell phone populations and frames CELL PHONE LANDLINE A B C Percent male: 46.6 51.1 38.2 37.9 Landline survey Cell phone survey Percent male State equalized design weight appliedPercent 18-34 years: 24.0 51.4 19.6 14.5 Landline survey Cell phone survey Percent 18-34 years State equalized design weight appliedPercent Hispanic: 15.2 21.4 12.2 16.8 Landline survey Cell phone survey Percent Hispanic State equalized design weight appliedPercent black: 15.0 15.8 7.5 9.3 Landline survey Cell phone survey Percent black State equalized design weight appliedPercent high school or less education: 39.8 48.5 33.6 60.3 Landline survey Cell phone survey Percent high school or less education State equalized design weight appliedPercent married: 62.0 32.0 69.8 49.5 Landline survey Cell phone survey Percent married State equalized design weight appliedSummary of significant differencesacross demographic subgroups: Summary of significant differences across demographic subgroups Cell only v.s. cell & landline adults (from cell phone survey): Significant differences for 12 of 24 subgroups examined Particularly age, employment status & marital status Cell & landline adults (cell phone survey v.s. landline survey): Significant differences for 11 of 24 subgroups examined Particularly sex, race, marital status, and children in householdComparison of key survey estimates: Comparison of key survey estimatesPercent any kind of health care coverage: 86.0 70.1 89.0 78.7 Landline survey Cell phone survey Percent any kind of health care coverage State equalized design weight appliedPercent not received care due to cost barrier: 16.3 24.9 10.2 20.4 Landline survey Cell phone survey Percent not received care due to cost barrier State equalized design weight appliedPercent currently smoke cigarettes: 19.7 31.1 17.3 24.8 Landline survey Cell phone survey Percent currently smoke cigarettes State equalized design weight appliedPercent ever tested for HIV: 43.6 54.2 36.6 37.5 Landline survey Cell phone survey Percent ever tested for HIV State equalized design weight appliedPercent binge drink past 30 days: 13.0 23.5 21.1 11.0 Landline survey Cell phone survey Percent binge drink past 30 days State equalized design weight appliedComparison of Survey Costs: Comparison of Survey CostsCost per Interview: Cost per Interview Data collection costs only Level of effort: RDD = 7.4 calls/case Cell = 3.2 calls / case Response rate: RDD = 38% Cell = 26% Interview length: RDD = 25 minutes Cell = 12 minutes $60 $74 $196What have we learned?: What have we learned? Group with both landline & cell phone differ across landline and cell phone surveys Mode effect? Response/nonresponse effect? Frame effect? This is an important issue when we try to combine landline & cell phone surveys Cell phone only group differs significantly from landline group on some health variables, but not others Risk behaviors seem most problematicWhat have we learned?: What have we learned? Cell phone & landline usage varies significantly across states Makes use of national estimates from the NHIS for post survey adjustment problematic Compared to landline surveys, cell phone surveys: Have lower rates of response at the screener stage But similar rates at the interview stage Working residential rates lower, but not as bad as expected Are considerably more expensive, especially if we decide to screen for cell-only adults Future directions: Future directions Problem is not going away, but will continue to worsen Focus on combining estimates from cell and landline frames (Frankel and Battaglia are working on this for the BRFSS) Cell only households not the only problem Need more focus on primary cell users Best to reach by landline or cell frame? Mode effects – what measurement issues are raised by interviewing via cell phones? You do not have the permission to view this presentation. In order to view it, please contact the author of the presentation.