combining cellphone and landline samples AAPOR2007

Slide1: 

COMBINING CELL PHONE AND LANDLINE SAMPLES DUAL FRAME APPROACH Randal ZuWallack

Cell Phone Pilot: 

Cell Phone Pilot About 1900 interviews across 6 states (CT, FL, MA, MT, NJ, TX) from Oct 2006 through Feb 2007 Contributors: James Dayton, MBA Daniel Gundersen, MA Cristine Delnevo, PhD, MPH Zi Zhang, MD, MPH Lori Westphal, PhD, MPH Michelle L. Cook, MPH Susan J Cummings, BSN, CPHQ Diane Aye, MPH, PhD Randal ZuWallack, MS Naomi Freedner, MPH

Slide3: 

Cell Sample and BRFSS: 

Cell Sample and BRFSS Dual-frame approach: measure the overlap 3 Groups: Adults in landline households with no cell phone Adults in landline households with a cell phone Adults in non-landline households with a cell phone (cell only)

Dual Frame: 

Dual Frame Select 2 independent samples Traditional 2-stage landline RDD Cell phone RDD Traditional dual frame estimators consist of combining the three components, where the two estimates for the overlapping group are averaged.

Issues: 

Issues How do we weight the cell phone data? Traditional selection probabilities Nonresponse Tricky with CATI since we have so many unknowns Knowing the size of the three population components makes everything easier: Poststratification CPS 2004 Supplement? NHIS? Small areas – states, counties, etc.

Two approaches: 

Two approaches 1. Selection probabilities: combine the samples by calculating the probability of being selected in one or the other sample. 2. Relative shares: estimate the size of each group using estimated ratios measured for each survey

Selection Probabilities: 

Selection Probabilities Adult probability of selection via cell: Adult probability of selection via landline: Adult probability of selection via cell or landline:

Selection Probabilities: 

Selection Probabilities Pr(C) is 0 for adults with no cell phones so the selection probability only reflects the landline component: Pr(L) is 0 for adults in households with no landline so the selection probability only reflects the cell component: For adults with a cell phone and in a household with a landline, we need information about the household (PHONESL and ADULTSL) and about their cell phones (PHONESC).

Establishing overlap: 

Establishing overlap From the cell phone survey, we need to establish: How many cell phones Whether the respondent lives in a landline household and, if yes: How many adults in the household, and How many telephone lines in the household From the landline survey, we need to establish: Whether the respondent has a cell phone and, if yes: How many cell phones used regularly, and How often they have the cell on

Weighting: 

Weighting Design weight = inverse of adult probability of selection via cell or landline: Poststratify and ratio adjust to general population

Relative Shares: 

Relative Shares We know the total population (T) for geographic area, but we don’t know how the population is divided between the three groups. A + B + C = T

Relative Shares: 

Relative Shares Rewrite components relative to B: A/B + 1 + C/B = T/B B = T/(A/B + 1 + C/B) We can estimate A/B from the landline sample, a/b a = landline sample without a cell phone b = landline sample with a cell phone We can estimate C/B from the cell sample, c/b c = cell only b = cell sample living in a household with a landline

Size Estimation: 

Size Estimation = T/(a/b + 1 + c/b) = (a/b) = (c/b) Weight each sample separately to the estimated sizes and average the overlapping group with a composite weight Poststratify and ratio adjust to general population

Nonresponse: 

Nonresponse Both methods break down with differential nonresponse, but in different ways Conducted a simulation to evaluate the impact of nonresponse. Feb 2004 CPS Cell Phone Supplement A: Adults with landline but no cell phone B: Adults with landline and cell phone C: Adults with cell phone but no landline SRS of adults in A+B SRS of adults in B+C 1000 sets of 2 samples 5 situations simulating group nonresponse

Simulation Scenarios: 

Simulation Scenarios No differential response – RRs equal across groups Landline RR 25% higher than Cell RR Cell RR 25% higher than Landline RR Landline RR for adults with cells is 25% higher than RR for adults with landline only and Cell RR for adults with landlines is 25% higher than RR for adults with cell only Landline RR for landline only is 25% higher than RR for adults with cells and Cell RR for cell only adults is 25% higher than RR for adults with cell and landline

Results: 

Results

Results: 

Results

An Application: 

An Application TX Behavioral Risk Factor Surveillance System (BRFSS), Nov and Dec 2006 1300+ interviews with a selected adult in the household Asked about cell phone usage TX Cell phone survey, Nov 2006 through Feb 2007 298 interviews with cell phone respondents Asked about landlines and adults in household Methods Selection probabilities—TX has various regional strata based on exchanges. We had ho way to identify the stratum that each cell phone respondent belonged. So, we go forward with the relative shares approach Used design weights for calculating shares

Shares: 

Shares Phone groups: About 58% of respondents in the cell phone survey reported that they lived in a household with a landline, 42% cell only About 73% of respondents in the landline survey reported using a cell phone As expected, usage shares vary across age groups We calculated the usage ratios for 6 age groups: 18-24, 25-34, 35-44, 45-54, 55-64, 65+ We have at least 30 cases per age cell in each of the samples Need to investigate collapsing for some smaller cells

Combining samples: 

Combining samples Combined two overlapping samples using weighted average of effective sample sizes Finally, we poststratified the combined sample based on age, sex, race, and Hispanic origin and ratio adjusted to match population controls in TX.

Prognosticators?: 

Prognosticators? There is anecdotal evidence that wireless telephone service is beginning to substitute on the margins for traditional (wireline) telephone service; and it may do so increasingly as technology improves, competition and subscribership increase, and prices fall. --The Council of Economic Advisers, February 8, 1999 Progress Report: Growth and Competition in U.S. Telecommunications 1993-1998,

Conclusions: 

Conclusions The frameworks for both weighting methods work with the right information The information to determine group membership will be self reported Both of these methods are applicable at any level of geography and neither requires any external information about the size of the phone groups. Differential nonresponse by phone status will disrupt the balance and could bias results

Conclusions: 

Conclusions Under the assumption of equal response likelihood within each sample (Scenarios 2+3), the share ratios method performs well. When response differs across groups within each sample (Scenarios 4+5), both methods are off the mark. The ratios can also be calculated within weighting cells to mitigate risk of violating this assumption. We need to understand nonresponse patterns so that we can develop weighting methodologies to counter any negative effects.

Conclusions: 

Conclusions In applying the methods, the relative shares method was fairly easy to implement in TX BRFSS, a sample with many geographic based strata. We couldn’t move forward with the selection probabilities since we did not have a way to assign cell respondents to the appropriate stratum. Although we don’t have a benchmark specific to TX, the 29% cell only seems slightly high. Maybe the within sample response assumption is violated. Are cell only users more likely to participate, or Are they more likely to have their phone on and therefore be more accessible?

Contact Information: 

Contact Information Randal ZuWallack Randal.S.ZuWallack@orcmacro.com