Research Design and Analysis 710; Fall 2005 Qualitative Research for Health Services Researchers: Research Design and Analysis 710; Fall 2005 Qualitative Research for Health Services Researchers Jane Forman, ScD, MHS Ann Arbor VA HSR&D


Objectives: Objectives Describe uses of qualitative methods in health services research. Compare the defining features of quantitative and qualitative methods. Strategic disarmament (or, address common questions about qualitative research). Qualitative study design: matching methods to research questions. Sampling. Data collection techniques. Data analysis process.


How Do We Use Qualitative Methods in Health Services Research?: How Do We Use Qualitative Methods in Health Services Research? Identify variables important to the phenomenon under study. Capture and understand views, motivations, experiences of participants. Provide a comprehensive description; understand processes, mechanisms. Provide more in-depth information about possible underlying mechanisms for the relationships observed in our quantitative analyses. Develop surveys -- generate survey items; test item validity


Identify variables important to the phenomenon you’re studying.: Identify variables important to the phenomenon you’re studying. Identify and examine factors that act as barriers and facilitators. Example – TRIP. “Identify and examine factors that facilitate and impede the adoption and implementation of evidence-based infection prevention practices in VA medical centers.” Understand the ways in which factors interact in context (e.g., the context of patients’ lives, in an ICU) Example – ABATE. Provide us with the insight and understanding essential for developing and testing interventions to ensure the successful translation of research into practice Example – TRIP. “Develop, evaluate and disseminate portable, deployable, and flexible interventions to facilitate adoption and implementation of proven infection prevention practices in VA hospitals.”


Capture and understand views, motivations, experiences of participants.: Capture and understand views, motivations, experiences of participants. Open ended questions enable participants to choose what is most relevant for the discussion and describe what it means to them, thus providing a window onto their experiences Examples – WYTFU, ABATE, TRIP


Provide a comprehensive description; understand processes, mechanisms: Provide a comprehensive description; understand processes, mechanisms Qualitative study Example – WYTFU Mixed methods study: Provide more in-depth information about possible underlying mechanisms for the relationships observed in our quantitative analyses. Example – ABATE


Survey Development: Survey Development Generate survey items Example – Collaborative Palliative Care Test item validity


One definition of qualitative research: One definition of qualitative research “An inquiry process of understanding…that explore[s] a social or human problem. The researcher builds a complex, holistic picture, analyzes words, reports detailed views of informants, and conducts the study in a natural setting.” Creswell, 1998, p.15 “The researcher also recognizes and reflects on their impact on data collection and the writing of the research.” Creswell, 2005, personal communication


Breadth vs. Depth: Breadth vs. Depth Quantitative Qualitative


Linear vs. Iterative Research Process(Crabtree and Miller): Linear vs. Iterative Research Process (Crabtree and Miller)


Strategic Disarmament: Strategic Disarmament Is Qualitative Research… Rigorous? Biased? Done with too small a sample size to mean anything? Valid? Just the researcher’s interpretation? Any other worries?


Rigor: Rigor Study design Sampling Data collection e.g., translation of research questions to interview guide, use of a priori theory to guide the questions, extensive data collection from multiple sources Data analysis e.g., code development and other systematic procedures, using multiple coders. Example from HSR&D


Bias: Bias Point of view is up front instead of hidden. Claim: There is no such thing as a neutral stance. Self-disclosure. Reflexivity is important Researchers should constantly take stock of their actions and their role in the research process.


Sample Size: Sample Size Intent is different in quantitative and qualitative research. Quantitative: statistical inference. Qualitative: understanding. Health services researchers tend to over-sample when they use qualitative methods.


Validity: Validity Quantitative and qualitative researchers view validity differently. Internal Validity Quantitative: Are you measuring what you are intending to measure? Qualitative: Are your findings accurate? Procedures: member checking, triangulation, negative cases, peer audit, external audit. External Validity Quantitative: Generalizability through statistical inference. Qualitative: Transferability through understanding of relationship between context and findings.


Interpretation: Interpretation Are qualitative findings just the researcher’s interpretation? Procedures: member-checking, rigor of analysis. Strategic study design. Reflexivity. Transparency: documentation of the research process. Answer: Both qualitative and quantitative research are interpretive. Qualitative research is more interpretive than quantitative research.


Qualitative Research Questions: Qualitative Research Questions Research question: a question that the research is designed to address, vs. questions an interviewer asks an interviewee. Pose as a general question. There’s one thing you’re trying to understand. Don’t think in terms of variables that you’re trying to relate to each other. Do think in terms of identifying factors and understanding how they work. Questions can evolve.


Relating vs. Identifying Variables/ Factors: Relating vs. Identifying Variables/ Factors VARIABLES/ FACTORS (e.g., barriers and facilitators) OUTCOMES QUALITATIVE: Exploring, Identifying, Understanding QUANTITATIVE: Relating


Research Strategy and Design: Research Strategy and Design Research design is not an entire advance blueprint; we make decisions throughout the research process But, a detailed design from which to start is very important and useful. You need a rigorous plan. Think widely and creatively, then pare down given best fit with the essence of your inquiry, and constraints on it Talk through methods, techniques, and data source issues with others


Matching Methods to Research Questions: Matching Methods to Research Questions How and why particular methods, techniques, and data sources might yield data which will help you to answer your questions (vs. I know how to do interviews so I’ll do them) (Mason p.27) What data sources and methods of data generation are potentially available or appropriate? What can these methods and sources feasibly tell me? How or on what basis do I think they could do this? Which of my research questions could they help me to address? Which elements of the background (literature, theory, research) do they relate to?


Matching Methods to Research Questions What constitutes knowledge or evidence relevant to my research questions?How do I go about generating such knowledge and evidence?: Matching Methods to Research Questions What constitutes knowledge or evidence relevant to my research questions? How do I go about generating such knowledge and evidence? (Mason 2002, Ch.2)


Sampling: Representational vs. Purposeful: Sampling: Representational vs. Purposeful Representational Goal: Enable generalizations from study samples to populations. The sample displays variables (e.g., age, gender) in similar proportions and patterns to the total population about which you wish to make generalizations. Statistical conventions are used to calculate the probability that patterns observed in the sample will exist in the wider population.


Sampling: Representational vs. Purposeful: Sampling: Representational vs. Purposeful Purposeful Goal: To understand a phenomenon, not to represent a population. The selection of information-rich cases for intensive study. Commonly used in qualitative research. Selected types of purposeful sampling: Maximum variation Stratified Typical case Deviant case.


Sample Size: Sample Size Sample should be large enough to make meaningful comparisons in relation to your research questions. Factors that affect sample size: Number of comparison groups (more comparisons  larger sample size). Detail, complexity and depth (more detailed, complex, and in-depth  smaller sample size) Develop explanations to account for similarities and differences in particular contexts. Seeing nothing new in newly sampled units involves recognizing what is there and what can be made out of the data already collected, and then deciding whether it is sufficient to create an intended product. There’s a tendency to over-sample in the health sciences. Underestimation of how much you get out of the data.


Sampling Decisions: Sampling Decisions Sampling is part of the detailed design that you develop before you begin your study. You make sampling decisions during data collection based on what you find in your data.


Qualitative Data Generation: Qualitative Data Generation Interviews Observation Documents and records “Personal experience materials” Correspondence Diaries Narratives Audio-visual materials


An interview: An interview Hamlet: Do you see yonder cloud that’s almost in the shape of a camel? Polonius: By th’ mass, and ‘tis like a camel, indeed. Hamlet: Methinks it is like a weasel. Polonius: It is back’d like a weasel. Hamlet: Or like a whale? Polonius: Very like a whale. William Shakespeare, Hamlet, act 3, scene 2


Qualitative Interview Characteristics: Qualitative Interview Characteristics Usually open-ended; can include both open and closed-ended questions Degree of structure: Un-, semi-, and highly-structured Researcher vs. participant directed Individual vs. group Face-to-face vs. telephone Dense data Data = text. ½ hour interview = 15 pages singled spaced.


When to use individual interviews: When to use individual interviews When individual’s experience and unique interpretation of it is of interest When a topic may be too sensitive to discuss in a group When respondents are too dissimilar to be meaningfully grouped


Focus groups: Focus groups Facilitated group discussions Focused, guided by a set of topics using open-ended questions Nurture different perceptions/points of view within the group. Participants generally homogenous on particular characteristics of relevance to the topic Depend on group interaction Participants are influencing and being influenced by other participants. Number of people: small enough for everyone to have a chance to talk, large enough to provide diversity of opinions Usually 5-7 people.


When to Use Focus Groups: When to Use Focus Groups When group interaction will help address your research question. E.g., When you want to bring out diverse points of view and contrast them in real time. E.g., When you want to observe interaction itself. When breadth of data generated is more important than depth. NOT when topic is sensitive


Qualitative Interviews Require Planning and Skill-building: Qualitative Interviews Require Planning and Skill-building Preparation for the interview Interview guide (see handout) Simultaneous management of intellectual and social dynamics Establishing rapport Listening skills “think[ing] on your feet…in ways consistent with your research questions” (Mason, p.67) Interview Guide example – see handout.


Good interviewing techniques: Good interviewing techniques Careful listening Waiting Not leading the interviewee Being supportive


Not so good interviewing techniques: Not so good interviewing techniques Asking too many questions for the time or type of participant The more questions you have, the more structured the interview has to be. Asking too many questions at once Asking closed-ended questions Asking leading questions Speaking rapidly, nervously, impatiently Interrupting


Observation: Observation Always used in ethnography, but observation does not equal ethnography Use when the data generated will be helpful in addressing your research questions Objects: e.g., daily routines, interactions, conversations, non-verbal behavior, active construction of texts, physical space Can supplement peoples’ accounts in interviews Inevitably selective – use same techniques to decide what to focus on as you use in creating interview guides Challenges Skills, training Going from the general to the specific Observation protocol example – see handout


Document Sources: Document Sources Chart reviews Patient information Policy documents Operational Procedures


“Qualitative analysis transforms data into findings. No formula exists for that transformation. Guidance, yes. But no recipe.”-- M.Q. PattonHere’s some guidance.: “Qualitative analysis transforms data into findings. No formula exists for that transformation. Guidance, yes. But no recipe.” -- M.Q. Patton Here’s some guidance.


What is Data Analysis?: What is Data Analysis? Goal of Analysis: Facilitate the interpretation of your data. Make sense of your data. Data Analysis: All techniques aimed at breaking data up; reconfiguring data; (dis)playing (with) data To enable researcher to see something new in those data To move from looking at data to looking for something specific in the data Learn by doing.


Basic Elements of Data Analysis: Basic Elements of Data Analysis Immersion: Getting a sense of the whole Reading Reduction: Develop a consistent approach to the data Coding Synthesis Abstracting to themes Building an argument


Preliminary Coding: Preliminary Coding Preliminary or Open coding: the first step in identifying patterns and themes. Highlighting text you think is relevant and interesting and commenting in the margins Identify key phrases and terms. Emic = categories that the people interviewed have created to make sense of their world. Inductive.


Example – Preliminary Coding: Example – Preliminary Coding CONNECT Study: Goal was to design instrument to measure differences in patients’ and physicians’ views of the illness experience. (Bokhour, Parker, and Haidet) Qualitative interview phase: to test and identify main dimensions of patients’ views of the illness experience RQ: What are the main dimensions of patients’ views of the illness experience?


Coding -- Where Do I Begin?: Coding -- Where Do I Begin? Model: Cause, Meaning of Symptoms, Treatment and Perceived Efficacy, Perception of the Seriousness of their Illness, Control Over the Condition “Theory-derived sensitizing concepts” Etic = labels imposed by the researcher. Deductive. Preliminary or Open coding: the first step in identifying patterns and themes. Highlighting text you think is relevant and interesting and commenting in the margins Identify key phrases and terms. Emic = categories that the people interviewed have created to make sense of their world. Inductive.


Codes: Codes Codes are tools to work with. They are key to developing a consistent approach to the data. Code definitions should be clear and codes should be mutually exclusive. This doesn’t mean, though, that a chunk of data can only have one code. Create a codebook: Work back and forth between the categories to refine the meaning and accuracy of the categories and the placement of the data in the categories.


Codes -- continued: Codes -- continued The set of categories should comprise a whole picture. REMEMBER: Always keep in mind that you need to produce slices of data that will help you address your research questions. Process of moving back and forth from data to research questions. Codebook example -- See handout


Next Steps: Next Steps Code data set. Reorganize data and (in some cases) code further. Continue to memo. Synthesis/Interpretation Data Display Conclusion Drawing and Verification Writing


Data Display Example: Data Display Example


Pitfalls: Pitfalls Don’t attach labels to lines of data before you have a sense of the whole and sense of analytic direction Premature analytic closure and a commitment to some a priori view Cookbook application of techniques


References: References John Creswell, Qualitative Inquiry and Research Design, Sage, 1998 Michael Q. Patton, Qualitative Research & Evaluation Methods, 3rd ed., Sage, 2002 Jennifer Mason, Qualitative Researching, Sage, 2002 Margarete Sandelowski, series of articles on qualitative research in Research in Nursing and Health. Interviewing: Robert S. Weiss, Learning from Strangers, The Free Press, 1994 Focus Groups: David L. Morgan, Focus Groups as Qualitative Research, Sage, 1997