Clinical research-getting started Slides

Clinical Research: Getting Started:

Clinical Research: Getting Started Edward A. Panacek, MD, MPH UC Davis Medical Center Department of Emergency Medicine (Syllabus) SAEM, May16, 2007, Chicago

Lecture Objectives:

Lecture Objectives Oriented to novice researchers ! An understanding of the research process Knowledge of the mechanics of research Research questions, hypotheses, analysis How to get started…..

What is research?:

What is research? Think about this for a moment

What is research? Common answers::

What is research? Common answers: An answer to a question The development of new knowledge The organized collection and analysis of observations or data A way to see your name in print in the journals A vehicle for tenure at the university

What is research?:

What is research? The NIH definition: “A systematic investigation designed to develop or contribute to generalizable knowledge”

What is research?:

What is research? In the purest, simplest sense, it is…. “A search for truth” truth in the universe

Truth in the universe:

Truth in the universe Can this ever really be known? Can we study the entire universe? Can our measurements be absolutely accurate? Can our data handling be error free?

Truth….or not truth…:

Truth….or not truth… Research: A process of drawing inferences

Why do research?:

Why do research? A desire to know something Intellectual curiosity To contribute to the knowledge base of the specialty To change the world ! Researchers can have an influence much greater than clinicians or educators

How is research performed?:

How is research performed? By using the The scientific method! And what is that?

The scientific method is the basic anatomy of research:

The scientific method is the basic anatomy of research

The relationship of the scientific method to the “truth”:

The relationship of the scientific method to the “truth” External validity Internal validity

Research versus the truth:

Research versus the truth Each step in the performance of research and the drawing of inferences can suffer from error There are two main types of error: Systematic error = AKA: bias Random error = occasional mistakes = Sloppiness, etc.

Types of error: The mirror image analogy:

Types of error: The mirror image analogy Consider observing or measuring something by looking at its’ reflection in a mirror You can’t see it directly Bends or distortions in the mirror (like a carnival mirror) are the equivalent of systematic error . They are always there Dirt on the mirror is like random error . Polishing the mirror (data cleanliness) can minimize the effects of random error

Goals of properly done research:

Goals of properly done research Minimize the effects of random error. Attention to detail 2. Eliminate sources of systematic error. Know sources of bias, to recognize them Eliminate or control them in the study design Both are designed to bring the investigator closer to the real truth

The research process: Starts with a research question:

The research process: Starts with a research question

Sources of research questions:

Sources of research questions Journal clubs Literature searches Presentations at meetings “Limitations” or “future directions” discussions in articles Daily clinical experiences in the ED

Projects always start with the all important research question:

Projects always start with the all important research question Take a moment. Consider a potential research question for your next project

All research starts with the question to be addressed:

All research starts with the question to be addressed Great question potential for a great project Flawed question Confused or incomplete project Impractical question Waste of time performing the project No question ? No answers !

Where in a manuscript do you find the author’s research question?:

Where in a manuscript do you find the author’s research question? In the last paragraph of the introduction

Pitfalls in research questions: I:

Pitfalls in research questions: I Problem: Too many questions Can’t focus on one specific question Desire to answer several questions with one project Solution Group into categories Prioritize in order of importance or feasibility Start with the most important one

Pitfalls in research questions II:

Pitfalls in research questions II Problem: Too broad a question Can’t focus on details Results in no testable hypothesis Solution Divide and conquer Break down into sub-questions Then group and prioritize

Research projects and questions:

Research projects and questions Each project must have one primary research question Answering this is the priority of the study It is the target which drives all other decisions in the study Can sometimes add in one or two sub-questions, but be careful !

Projects without questions:

Projects without questions Occasionally, some purely observational studies do not seem to have a focused research question e.g. simple descriptive summary of all the county EMS runs for the year However, even such studies should have some reason for doing them, some purpose, some question

A properly formatted research question: The four PICO elements:

A properly formatted research question: The four PICO elements P patients I intervention (independent variable) C comparison O outcome (dependent variable) S subjects E exposure C comparison O outcome

Using PICO to develop a question:

Using PICO to develop a question Includes each of the following: Intended patients/subjects to study The independent variable of interest Intervention, exposure, pt characteristic The item of primary interest of the study Comparison group or procedure Outcome measure Combines them together as a question

Question example: What is the HIV infection rate in IV drug abusers?:

Question example: What is the HIV infection rate in IV drug abusers? Issues to address: Define rate (prevalence, incidence?) Define drug abuser Define study population sample source Define HIV infection (PCR or Western blot) Final result: What is the rate of HIV zero positivity by PCR test in a participating sample of SFGH methadone clinic patients?

PICO in practice:

PICO in practice Essentially all proposed research questions should include each of the PICO elements

Research question comments:

Research question comments It has been proposed that if your entire question can be stated in one breath, it is too short That is extreme. It need not include your entire inclusion and exclusion criteria The research question is analogous to the pt chief complaint. It is not the entire history and exam.

Refining the research question: Making it a “Finer” question:

Refining the research question: Making it a “Finer” question F feasible I interesting N novel E ethical R relevant

Considering the FINER elements of a question:

Considering the FINER elements of a question Feasible = can it be done with the resources that are available? Interesting = do you have passion for it? Novel = is it a new research idea? Ethical = does it comply with research ethics and codes of conduct? Relevant = does it pass the “so what” test?

Applying the FINER criteria:

Applying the FINER criteria Carefully review your proposed research question Research in progress meetings can be helpful for this Consider how it meets each of the FINER criteria……or not Then decide: Do you still want to proceed with your proposed project? Is it feasible, etc.?

The FINER review:

The FINER review This is where most good research questions die an early death (or should have died)

Other Research Project Terms:

Other Research Project Terms Aims Purpose Objectives Questions Hypotheses All projects must have a research question. A study hypothesis should also be considered. Other requirements are determined by grant application instructions, etc.

Study goals:

Study goals Some grant applications request specific terms be used Objectives , aims or purpose generally define the general or overall goals of the project Sometimes written as the general focus and scope of the project Can involve multiple study questions

Grant proposal example*:

Grant proposal example* The purpose of this study is to assess tissue oxygen delivery and utilization in trauma patients with shock The objective of this research is to develop appropriate endpoints for resuscitation in ED patients with trauma The study aims are to determine the effects of resuscitative efforts on measures of oxidative stress, e.g. glutathione and ketones * From prior Chuck Cairns, MD, EMF grant application

In grant applications, study questions are often listed among the goals:

In grant applications, study questions are often listed among the goals “The goal of this project is to answer the following questions:” In trauma pts, does aggressive resuscitation improve measurements of glutathione and ketones? In trauma pts, are improvements in glutathione and ketones associated with less morbidity and mortality?

Hypotheses:

Hypotheses For each project, with it’s proposed question you should also establish your hypotheses: Null hypothesis (H o ) Alternative hypothesis (H a ) Study hypothesis

Hypotheses:

Hypotheses What are hypotheses and why have them?

Hypotheses:

Hypotheses A hypothesis is the investigator’s educated “best guess” of the answer to a question. A prediction of the real “truth”

What is the null hypothesis?:

What is the null hypothesis? This concept is remarkably poorly understood, but serves as the basis for classic statistical testing

What is the null hypothesis?:

What is the null hypothesis? That there is no difference between the study groups ! (data set A) - (data set B) = (0) = empty set = null set

The alternative hypothesis:

The alternative hypothesis That there is a difference between the two study groups (Data A) - (Data B) = (some #)  0 = not null However, it does not define the direction of the difference Concept of two-tailed vs. one-tailed statistical tests

What is the study hypothesis?:

What is the study hypothesis? The investigator’s theorized answer to the research question It usually is the alternative hypothesis but with a specific direction of difference It could be the null hypothesis What are examples of this?

When Ho is the study hypothesis:

When H o is the study hypothesis Pharma wants to show that their new 4 th generation cephalosporin is as good as other existing drugs in that category When an investigator tries to show that a cheaper or safer therapy is as good as the expensive/toxic therapy

Why have a null hypothesis?:

Why have a null hypothesis? Is it possible to test a question? H o is boring and usually not our real interest, so do we really need it? Why not just test the alternative hypothesis (H a ) ? Concepts of Karl Popper

Karl Popper:

Karl Popper An intellectual leader of the scientific revolution Believed science should be based on the hypothetico-deducto process Propose a hypothesis and test it If evidence can be found to contradict the hypothesis, then it is fully rejected However, evidence in support is only true for the specific conditions of that experiment

Accepting and rejecting hypotheses:

Accepting and rejecting hypotheses When there are only two possible hypotheses and there is evidence that one is not true, by default, the other hypothesis is accepted as being true If reject H o , means H a is accepted

Hypothesis testing question::

Hypothesis testing question: Your project is finished and in your submitted abstract, you’ve written your results as: “ treatment A was 20% better than Tx B in terms of outcome Y (p=0.03). Therefore, treatment A should be the preferred Tx for this condition” Your student research assistant asks you what “p” means, what does .03 mean, and why are you so confident in your conclusion? What, exactly, would you say to the student ?

What you should say::

What you should say: “p” = probability 0.03 means “if H o were true, the probability of obtaining results with this, or a more extreme distribution difference, by chance alone is only .03 (3%) “ “Since that is a very small probability, I am accepting it as sufficient evidence that the H o is not true and therefore I am accepting the H a instead (i.e. that there is a real treatment difference).”

Let’s make sure we understand this principle:

Let’s make sure we understand this principle Another study example

A study finds that a computerized reminder system in the ED is associated with a 10% increase in wound check f/u visits (p=0.03):

A study finds that a computerized reminder system in the ED is associated with a 10% increase in wound check f/u visits (p=0.03) Which of these interpretations is correct? “With this observed 10% difference between groups, there is only a 3% probability that the intervention had no effect”. “Given the assumption that the intervention had no effect, there is only a 3% probability of finding the observed 10% difference by chance alone”.

Statistical analyses:

Statistical analyses Statistical tests do not care about: Truth in the universe Your research question Your hypothesis Stats tests are only a tool to quantitate the effects of chance on data distributions Investigators, not stats tests, accept or reject hypothesis and draw conclusions

Interpreting study results:

Interpreting study results Your data is your data Your results are your results Sometimes the results are self-evident More often, statistical tests are needed to quantitate the effects of random chance on the data Do not confuse clinical significance with statistical significance

The next steps in a research project: Fleshing out the study protocol:

The next steps in a research project: Fleshing out the study protocol Defining all variables Identifying the sample population Inclusion/exclusion criteria Stipulating study interventions/measurements Sample size calculations Analysis plan Selecting an overall study design

Study designs:

Study designs Those that “prove” cause & effect RCTs Those that argue cause & effect Cohort studies Those that show associations Case-control Cross-sectional

Those topics are complex and are the subject of other lectures or courses:

Those topics are complex and are the subject of other lectures or courses EMBRS (Emergency Medicine Basic Research Skills) Course 12 day workshop sponsored by EMF/ACEP Given once a year in Dallas Spends an entire day on: Research study designs Developing study protocols Other courses and lectures Textbooks

Getting Started: Summary:

Getting Started: Summary Understand the basic anatomy of research Every project must start with a question Do not rush through this step Properly format the question using PICO Apply the FINER criteria to refine Be realistic. Do not proceed if not feasible Generate your study hypotheses Understand hypothesis testing

This was only “getting started”:

This was only “getting started” Well done studies are a lot of work Good luck with your next projects !

The End:

The End Thank you to the SAEM for this opportunity to participate in the meeting (The syllabus for this lecture has been sent to SAEM and should be on the web site soon)