Bias in clinical Research

Bias in clinical research:

Bias in clinical research Lecture By Dr . Amit Bhatt

Slide 2:

If an experiment was conducted to find out the distribution of sizes of fish in a lake, a net could be used to catch a representative sample of fish. If net had a mesh size of 1 cm then no fish with sizes less than 1cm would be found. This is a result of the method of selection (selection bias) - from the experiment, there is no way of knowing whether there are any fish smaller than 1 cm.

Slide 3:

Introduction The quality of a clinical study depends on internal and external factors. Studies have internal validity when, random error apart, reported differences between exposed and unexposed individuals can be attributed only to the exposure under investigation. Internal validity can be affected by two types of error: random error and systematic error. - Random error depends on chance and can be minimized by increasing the sample size (1). For example if we measure serum creatinine three times in the same individual by using the same instrument and the same laboratory reagents we can obtain slightly different values as a result of clinical fluctuations in the measurement. Systematic errors depend on any flaw that systematically leads to an overestimation or underestimation of the measurement and this type of error is independent of sample size (1). Systematic error occurs if the device we use, for example a sphygmomanometer for measuring blood pressure (BP), systematically overestimates or underestimates the “true” BP of the subject being evaluated. Systematic error is also referred to as bias . The list of biases is a long one (2). We describe three types of selection biases (incidence-prevalent bias, loss-to–follow-up bias and publication bias) and a series of information biases [i.e. misclassification bias (recall bias, interviewer bias, observer bias and regression dilution bias) and lead time bias]. 1) Rothman KJ . Random error and the role of statistics. In Epidemiology. An Introduction. Oxford University Press. New York. USA; pp. 113-142, 2002. 2) Sackett DL . Bias in analytic research. J Chronic Dis 1979; 32: 51-63.

Slide 4:

SELECTION BIAS (OR BERKSONIAN BIAS) - This type of error occurs when a systematic error in the enrollment of individuals in a study – cases or controls in case-control study, or exposed or unexposed individuals in a prospective cohort study – determines a biased association between exposure and outcome. - Here we consider three types of selection bias: the incidence-prevalence bias, the loss-to-follow-up bias and the publication bias.

Slide 5:

Incidence - prevalence bias An important selection bias is the incidence-prevalence bias (or Neyman bias or survival bias) , i.e. a bias that occurs when we try to estimate the risk of a disease on the basis of data collected at a given time point in a series of survivors rather than on data gathered during a certain time period in a group of incident cases ( Example 1 ) or when the sample of cases offers a distorted frequency of the exposure ( Example 2 ). This type of bias is of particular relevance in cross-sectional studies. SELECTION BIAS (OR BERKSONIAN BIAS)

Slide 6:

Incidence - prevalence bias (Example 1) We consider a cohort of 20 individuals with a follow-up from t0 - t2. During this follow-up, 4 individuals developed a moderate disease and 4 individuals got a severe form of the same disease. Therefore, the true risk ratio between the risk of severe disease and that of moderate disease is 1. However, people with severe disease have a higher risk to die, and before the time “t1”, 2 individuals died. If at time “t1” we perform a cross-sectional analysis (survey) we get a prevalence ratio of severe versus moderate disease of 0.5 (in fact at time “t1” we have only one case with severe disease and two cases with moderate disease), a figure that does not reflect the true risk ratio that is 1. Therefore, whenever possible we should estimate the occurrence of a disease in terms of incidence, i.e. by counting all cases that occur in a sample in a given time interval rather than in a given time point.

Slide 7:

Incidence - prevalence bias (Example 2a) As further example of incidence-prevalence bias we can consider the following one. In a case-control study, Ibanez et al. ( 3 ) assessed the association between chronic use of Aspirin and occurrence of end stage renal disease (ESRD). Disease status With ESRD (cases) Without ESRD (controls) Users of Aspirin 81 94 Nonusers of Aspirin 439 888 Total 520 982 In this study, the odds ratio (OR) of the use of Aspirin is: OR =(81/439)/(94/888)= 1.74 An odds-ratio of 1.74 means that the odds of exposure to Aspirin were 74% higher in individuals with ESRD than in those without this complication. In this study, the selection process of cases and controls as well as the assessment of exposure status were performed in an appropriate manner, thus we can assume that this OR reflects the situation in the reference population, i.e. that we can look at this OR as at the “true” OR.

Slide 8:

Incidence - prevalence bias (Example 2b) We now consider a hypothetical situation where the selection of cases from the target population, but not that of controls, is biased because affected by previous knowledge by the investigator of the exposure status. In other words, the investigator, influenced by his knowledge of the exposure status, tends to gather cases mainly among individuals known to be Aspirin users. Disease status With ESRD (cases) Without ESRD (controls) Users of Aspirin 130 94 Nonusers of Aspirin 390 888 Total 520 982 The OR in this situation is: (130/390)/(94/888)= 3.15 Bias in the selection of cases produces a considerable alteration in the estimate of the OR (3.15 vs 1.74). To avoid this problem, the selection process of cases and controls from the target population should be rigorously the same and should be independent from the exposure status (i.e. the investigator should be blinded to the exposure status). This type of bias, which we herein describe in relationship with case-control studies, can occur with any type of study design.

Slide 9:

Loss-to–follow-up bias This bias occurs in prospective cohort studies when individuals lost to follow-up do not have the same probability of having the clinical outcome of interest in comparison with individuals who remain under observation. In the recent literature, a potential source of loss-to-follow-up bias can be found in the CHOIR study (a randomized clinical trial testing the effect of anemia correction in patients with chronic kidney disease not receiving dialysis) (4). The CHOIR protocol terminated from the follow-up all individuals who achieved ESRD, thus potentially generating a bias. In fact, although the relative impact of the intervention was unaffected by this bias (the proportion of excluded ESRD patients was similar between the two CHOIR study arms: 18% vs 15%), this was not true for the cumulative incidence of events in the two groups that was influenced by censoring ESRD patients. We consider a hypothetical prospective study aimed at determining the incidence rate of renal insufficiency in hypertensive and normotensive individuals. It is well known that in this type of study the follow-up duration must be extended to several years. As a consequence, normotensives who did not develop any disease after several years of observation may be less stimulated to continue the study while hypertensives, that most likely develop comorbid conditions, can be more motivated to continue the study participation. Thus, loss to follow-up bias may distort the true risk of renal insufficiency in hypertensive versus normotensive individuals. SELECTION BIAS (OR BERKSONIAN BIAS)

Slide 10:

Publication bias This bias is generally considered as a selection bias. Indeed, the acceptance of the validity of published findings as applied to a given reference population is conditional not only on the fact that each published study is unbiased but also that published studies constitute an unbiased sample of all studies performed. When these assumptions are not met, a literature review based on either meta-analytic or conventional narrative approaches will give a distorted view of the exposure-outcome association of interest. Publication bias is generated in the selection process of the information that is eventually published. Several factors influence publication, the most important being study size and design, quality, funding and prestige ( 5 ). Kasiske and al. collated 13 studies on lipid lowering and renal outcomes in patients with chronic kidney disease ( 6 ). In this systematic review, only 2 studies out of 11 (i.e. 18%) reported a negative effect of treatment on albuminuria or proteinuria. A possible explanation for this phenomenon is that journal editors might have been biased in accepting studies reporting a negative effect of treatment or, alternatively, that authors of negative studies might have not submitted them because of the low probability for these studies to be accepted. SELECTION BIAS (OR BERKSONIAN BIAS)

Slide 11:

Publication bias The simplest and most commonly used method to detect publication bias is the funnel plot (7-8). A funnel plot is a graph where the effect of a given treatment of each trials is plotted against some measure of its size, such as the precision, the standard error or the overall sample size. These plots are referred to as funnel plots because they should be shaped like a funnel if no publication bias is present. This shape is expected because the estimate of the effect of a treatment has a larger variability in smaller studies. Since smaller and negative studies are less likely to be published, trials in the bottom left hand corner of the graph are often omitted, creating a degree of asymmetry in the funnel. SELECTION BIAS (OR BERKSONIAN BIAS)

Slide 12:

Publication bias In the figure we report a hypothetical example of publication bias. In the left part of figure the shape of the plot suggests that smaller studies indicating a worse outcome with the treatment being tested, might have been not reported (just one study where treatment produced worse outcome). In the right part of figure there are smaller studies showing either beneficial or no beneficial effects of the treatment. In this case the plot is appropriately funnel-shaped, making publication bias less likely. SELECTION BIAS (OR BERKSONIAN BIAS)

I want to talk about . . .:

I want to talk about . . . What is publication bias? Why does it matter? What is the evidence for it? What can be done about it? How has the BMJ responded?

There are many types of bias:

There are many types of bias Selection bias : biased allocation to comparison groups Performance bias : unequal provision of care except treatment being evaluated Detection bias : biased assessment of outcome Attrition bias : biased occurrence and handling of deviations from protocol and loss to follow up . . . and on and on (From Egger et al BMJ 2001;323:42-46 (7 July)

What is publication bias (1)?:

What is publication bias (1)? A definition: “Publication bias refers to the greater likelihood that studies with positive results will be published” JAMA 2002;287:2825-2828

What is publication bias (2)?:

What is publication bias (2)? An alternative definition: Publication bias is the selective or multiple publication or suppression of trial results so that the scientific record is distorted

Why does it matter?:

Why does it matter? Distorts the scientific record Hides the “truth” Influences doctors’ decision making Misleads policy makers Causes harm to patients Costly for the health service A form of scientific and research misconduct

Who is to blame?:

Who is to blame? Wicked researchers? Very wicked sponsors? Editors: the wickedest of all? (and let’s not forget reviewers)

What is the evidence for it (1)?:

What is the evidence for it (1)? Stern and Simes BMJ 1997;315:640-645 Question : To what extent is publication influenced by study outcome? Studies submitted to an Australian ethics committee over 10 years Examined protocols Questionnaire to authors (70% response)

Stern and Simes: results:

Stern and Simes: results All studies (n=520) Clinical trials (n=130) Positive>negative 2.32 (1.47 to 3.66) 3.13 (1.76 to 5.58) Time to publication 4.8 vs 8.0 yrs 4.7 vs 8.0 yrs

Stern and Simes: conclusions:

Stern and Simes: conclusions Positive trials are more likely to be submitted for publication Positive trials are more likely to be published Positive trials are more likely to be published quickly Implications for systematic reviews Important to register all trials

What is the evidence for it (2)?:

What is the evidence for it (2)? Lexchin and Bero BMJ 2003;326:1167-70 Question : Does drug industry sponsorship influence research quality and outcome? Meta-meta-analysis Industry research less likely to be published (more likely in symposium proceedings) No difference in methodological quality More likely to have a positive finding (OR 4.05 95% CI 2.98 to 5.51)

Melander et al: conclusion:

Melander et al: conclusion “Any attempt to recommend a specific selective serotonin reuptake inhibitor from the publicly available data ONLY is likely to be based on biased evidence.”

What is the evidence for it (4)?:

What is the evidence for it (4)? Olson et al JAMA 2002;287:2825-2828 Question : Is there publication bias in editorial decision making? 3 years, 745 manuscripts Positive vs negative OR 1.30 (0.87 to 1.86) Small effect of editorial decision making, much less than researchers not submitting negative studies Will this be true for journals less grand than JAMA?

What can be done about it (1)?:

What can be done about it (1)? Better conduct and reporting of RCTs (CONSORT) Better conduct and reporting of systematic reviews (QUORUM) “Publication” of unpublished trials Enlightened sponsors (a code of good practice Wager et al 2003 http://www.gpp-guidelines.org ) Better editorial policies Vigilant editors and reviewers Responsible authors

What can be done about it (2)?:

What can be done about it (2)? Publication of original protocols and deviations from protocol Declaration of competing (financial) interests by authors, reviewers, and editors Declaration of sponsorship/funding Registering all clinical trials

Conclusions:

Conclusions Publication bias is an important problem that impacts on patient care There is much evidence to support its existence There are many players There are many ways to reduce its effect, examples of good practice Ultimately there is a big responsibility on sponsors of trials, authors, and editors

Slide 28:

INFORMATION BIAS Information bias occurs during data collection. There are two (main) types of information bias: misclassification bias and lead time bias. Misclassification bias originates when the process to detect the exposure status (exposure identification bias) and/or the outcome (outcome identification bias) is imperfect, i.e. exposed/diseased individuals are classified as non-exposed/non diseased and vice-versa. A potential source of misclassification can be the use of an inaccurate instrumentation, such as using only one size blood pressure cuff to take measurements on both lean and obese adults. As an effect of the inappropriate size of cuff we can classify a normotensive individual as hypertensive and vice-versa. Misclassification can be nondifferential or differential. In the example below we refer to misclassification of exposure. Misclassification

Slide 29:

INFORMATION BIAS Non differential misclassification occurs when the misclassification of exposure is independent of disease status, i.e. it is the same in diseased individuals (cases) and non-diseased individuals (controls). Consider a hypothetical case-control study investigating the association between hypertension and renal dysfunction. With renal dysfunction (cases) With normal renal function (controls) Hypertensives 40 10 Normotensives 40 50 OR= (40/40)/(10/50)=5.00 The OR indicates that the odds of exposure to hypertension are 5.00 times higher in cases than in controls and, in the absence of misclassification, we consider this odds ratio as the true OR. Nondifferential misclassification

Slide 30:

INFORMATION BIAS Now, we can hypothesize a situation in which the misclassification of the exposure is equally frequent in cases and controls, i.e. in 30% of hypertensives with renal dysfunction (n=12) and in 30% of hypertensives with normal renal function (n=3). Since the misclassification of the exposure is identical in cases and controls it is said “nondifferential”. This new situation is illustrated below : With renal dysfunction (cases) With normal renal function (controls) Hypertensives 40-12= 28 10-3= 7 Normotensives 40+12= 52 50+3= 53 OR: (28/52)/(7/53)=4.10 This OR is lower than the “true” OR (4.10 instead of 5.00). In fact, nondifferential misclassification of exposure always leads to an underestimation of the strength of the association between the exposure and the disease, i.e. the RR and the OR tend toward 1.

Slide 31:

Differential misclassification occurs when the misclassification of exposure differs between cases and controls. For example, we consider that there is a 30% exposure misclassification only in controls: INFORMATION BIAS With renal dysfunction (cases) With normal renal function (controls) Hypertensives 40 10-3= 7 Normotensives 40 50+3= 53 Odds ratio: (40/40)/(7/53)=7.58 Here, differential misclassification leads to an overestimation of the strength of the association between exposure and disease. By contrast, if misclassification occurs in cases but not in controls the resulting OR leads to an underestimation of the strength of the same association. In summary, differential misclassification may either increase or decrease the strength of reported associations depending on the direction of the misclassification. Differential misclassification can also occur in cohort studies if exposed and unexposed individuals are misclassified. Differential misclassification

Slide 32:

INFORMATION BIAS The most common biases producing misclassification are recall bias, interviewer bias, observer bias and regression dilution bias. Recall bias Recall bias results from imprecise memory of past exposure and it is of particular concern in case-control studies. For example, a patient with glomerulonephritis may have gone through all his exposures in his head once the diagnosis was made to try to understand “Why me?”. An individual without illness might not have that memory due to limited reason to stretch to recall exposures. Methods used to prevent recall bias include: a) verification of responses from study individuals by using hospital records or other reliable sources of clinical information; b) use of diseased controls. Patients (cases) are much more prone to remember relevant exposures than healthy controls. Therefore, a control group composed by individuals affected by a disease different from that of cases can be used to introduce a similar bias also in the odds of exposure of controls; c) use of objective markers of exposure. For example in a hypothetical case-control study investigating the association between exposure to hydrocarbons and the risk of glomerulonephritis we can estimate exposure to hydrocarbon by measuring metabolites of this compound in biological samples.

Slide 33:

INFORMATION BIAS Interviewer bias This is the tendency of the interviewer to obtain answers that support preconceived notions. Interviewer bias may happen as a consequence of trying to “clarify” questions when such clarifications are not part of the study protocol or when certain words are more emphasized during the interview of cases but not of controls (or vice-versa). For example an interviewer testing the hypothesis that hydrocarbon exposure may cause glomerulonephritis may adopt a more inquisitive attitude in cases than in controls thus making the ascertainment of exposure more likely in diseased than in non diseased individuals. To avoid interviewer bias we should carefully standardize the interview and use an identical approach in cases and controls. Furthermore, the interviewer should be blinded to the case-control status. Observer bias Knowledge of exposure status by the outcome assessor may influence the assessment process and therefore produce biased results. An obvious example is the assignment of a kidney biopsy specimen to a diagnosis of hypertensive nephropathy where the pathologist knows that the patient has a long history of hypertension. Possible remedies to observer bias are: 1) blinding the outcome assessor to exposure status; 2) when applicable, labelling the outcome as “possible”, “probable” and “definite”. Observer bias should be suspected if results emerge in the “possible” category only; 3) involving multiple outcome assessors.

Slide 34:

INFORMATION BIAS Regression dilution bias It is well known that a variable that shows an extreme value on its first measurement will tend to be less extreme on subsequent assessments (‘regression to the mean’ phenomenon). The regression dilution bias is related to regression to the mean and originates in longitudinal studies investigating the association between baseline measurements of a continuous variable and the risk of a given outcome .

Slide 35:

As a consequence of the “regression to the mean”, individuals in the top category of blood pressure level frequently show higher systolic pressure at the first visit than at the second, while individuals in the bottom category have a systolic pressure at the first visit that is somewhat lower than that at the second visit: in other words many values converge ( regress ) toward the mean ( see figure ). Thus, if patients would be classified as hypertensive and normotensive on the basis of blood pressure measurement at the first visit we had have a misclassification of exposure (normotensives erroneously classified as hypertensives and vice-versa). A classical example of this bias is the association between systolic pressure and the risk of myocardial infarction. During the baseline assessment of systolic pressure we have two sources of variations: variations in the measurement of arterial pressure among individuals and deviations between the baseline and the usual arterial pressure level within each individual. Regression to the mean refers to the noise introduced by within individual variation. It is the tendency for high values of continuous variables (like blood pressure) measured at a given time point to decrease and for low values to increase, when repeated measurements are performed. Regression dilution bias (Example 1)

Slide 36:

As further example we can consider a sample of individuals where everyone usually has a creatinine of 0.9 mg/dL. However, at this particular moment of blood sampling some values were higher due to a steak from last night’s dinner or volume depletion, whereas others were lower due to pregnancy or steroid use for muscle building. If we were to take only those individuals with creatinine > 1 mg/dL and follow them, they will all fall back to their true baseline values of 0.9 mg/dL eventually and we would erroneously claim that their kidney function improved. The regression dilution bias always produces an underestimation of the true effect. To avoid the regression dilution bias the classification of patients should be based on more than one blood pressure measurement. Regression dilution bias (Example 2)

Slide 37:

INFORMATION BIAS Lead time bias Another important information bias which may occur in prospective studies aimed at evaluating the efficacy of screening is the lead time bias. This is the added time of illness attributable to the fact that we apply to selected patients different criteria for the diagnosis of the disease (i.e. diagnosis made in the latency period or in early phase versus diagnosis made at an advanced stage).

Slide 38:

INFORMATION BIAS Lead time bias (Example 1) A subject gets diseased after a given time and the disease is diagnosed because of symptoms. The disease is a serious one and the patient dies quite soon after diagnosis. The dead person was young and this causes sorrow and concern in the community where he/she lived. Experience with this first case prompts a screening program to detect the disease at an early, pre-clinical stage. In the second subject disease is diagnosed during the screening program at an asymptomatic stage. The second patient receives some form of treatment but the disease progresses until death. The survival from diagnosis in this case is much longer than in the first case and we may be tempted to conclude that early diagnosis was useful. In fact this is so because we applied different criteria for the detection of the disease, i.e. we diagnosed it at a symptomatic phase in the first patient and at a pre-clinical, asymptomatic phase in the second patient. It is therefore possible that had we diagnosed the disease at a pre-clinical stage in both patients the survival might have been identical. Thus, if we are going to assess the usefulness of a screening program we should take into proper account the “lead time”.

Slide 39:

INFORMATION BIAS Lead time bias Lead time bias may be a problem in studies examining the effect of early referral on the outcome of chronic kidney disease (CKD) ( 9 ). If we do not appropriately account for the phase of disease when CKD is diagnosed we may overestimate the beneficial effect of early referral. To avoid the lead time bias we can calculate the risk (or rate) of mortality in all screened and control individuals rather than the cumulative probability of survival from diagnosis in cases. 9) Korevaar JC, Jansen MA, Dekker FW, et al. Netherlands Cooperative Study on the Adequacy of Dialysis Study Group. When to initiate dialyses: effect of proposed US guidelines on survival. Lancet 2001; 358:1046–1050.

Slide 40:

SUMMARY Bias appears a pervasive problem in clinical research. Bias can be prevented at two levels: 1) by choosing the appropriate study design for addressing the study hypothesis and 2) by carefully establishing the procedures of data handling and the definitions of exposures and outcomes. Suspecting bias is an important aspect of critical appraisal and this attitude is precious to properly evaluate clinical and epidemiological studies.

Best Of luck :

Best Of luck Thank U all