Lecture 1 Intro Systematic Reviews

Views:
 
Category: Entertainment
     
 

Presentation Description

No description available.

Comments

Presentation Transcript

Introduction to Systematic Reviews: 

Introduction to Systematic Reviews Madhukar Pai, MD Systematic Reviews Group Division of Epidemiology University of California, Berkeley madhupai@uclink.berkeley.edu

Case Study 1: “Egg on their faces: the story of human albumin solution”*: 

Case Study 1: “Egg on their faces: the story of human albumin solution”* Human albumin solution, a blood product, has been used in the treatment of blood loss and burns since the attack on Pearl Harbour over half a century ago. In the UK alone, an estimated 100,000 patients are treated with human albumin solution each year, at a cost to the NHS of close to 12 million. In 1996, the global albumin market was worth £900,000. But is human albumin administration beneficial? *1. Roberts I, et al. Egg on their faces. The story of human albumin solution. Eval Health Prof. 2002;25(1):130-8. 2. Cochrane Injuries Group Albumin Reviewers. Human albumin administration in critically ill patients: systematic review of randomised controlled trials. BMJ 1998;317:235-40.

“Egg on their faces: the story of human albumin solution”: 

“Egg on their faces: the story of human albumin solution” To answer this question a systematic review of controlled trials comparing albumin with crystalloid was conducted by the Cochrane Injuries Group. 30 RCTs including 1419 randomised patients identified. A meta-analysis showed that the risk of death among those treated with albumin was higher than in the comparison groups. The pooled risk ratio was 1.68 (95% CI 1.26, 2.23) The data suggested that for every seventeen critically ill patients treated with albumin there is one extra death. Roberts I, et al. Egg on their faces. The story of human albumin solution. Eval Health Prof. 2002;25(1):130-8.

“Egg on their faces: the story of human albumin solution”: 

“Egg on their faces: the story of human albumin solution” The results were widely reported in the media and stimulated an immediate response from the regulatory agencies, the industry and the medical profession. The industry launched a “Albumin Support Programme” to resuscitate the ailing US $ 1.5 billion global albumin market. The objective was to disseminate evidence supporting albumin: (1) the preparation of literature reviews supporting the use of albumin to be sent to leading regulatory authorities (2) preparation and dissemination of a Cochrane critique dossier (3) the establishment of a medical advisory panel to write articles supporting the use of albumin. The industry set aside US $2.2 million for the program. Roberts I, et al. Egg on their faces. The story of human albumin solution. Eval Health Prof. 2002;25(1):130-8.

“Egg on their faces: the story of human albumin solution”: 

“Egg on their faces: the story of human albumin solution” “Despite vigorous attempts by the plasma products industry to limit the impact of the systematic review on albumin sales, the use of albumin declined steeply. Throughout the UK albumin sales fell by 40%. The decline in albumin use occurred despite vigorous criticism of the review in the letters pages of the BMJ. The decline in albumin sales is a clear indication that doctors took into account the evidence presented in the systematic review and that many doctors changed their practice in response.” Roberts I, et al. Egg on their faces. The story of human albumin solution. Eval Health Prof. 2002;25(1):130-8.

“Egg on their faces: the story of human albumin solution”: 

“Egg on their faces: the story of human albumin solution” Roberts I, et al. Egg on their faces. The story of human albumin solution. Eval Health Prof. 2002;25(1):130-8.

Case study 2: “Same trials, different takes”: 

Case study 2: “Same trials, different takes” Mammography for breast cancer is an established screening method Is screening with mammography justifiable? Gotzsche & Olsen [Nordic Cochrane Centre] conducted a systematic review in 2000 and updated it in 2001. They identified 8 large RCTs on this topic, with over 182,000 women randomized Gotzsche & Olsen. Is screening for breast cancer with mammography justifiable? Lancet 2000;355:129 – 34. Olsen & Gotzsche. Cochrane review on screening for breast cancer with mammography. Lancet 2001;358:1340-42.

“Same trials, different takes”: 

“Same trials, different takes” The authors found that no trial data were of high quality Two were of medium quality, and the rest were poor quality or flawed. When the results of the two medium quality trials were combined, the risk ratio was 1.00 (95% CI 0.96, 1.05) They concluded that “screening for breast cancer with mammography is unjustified” and “any hope or claim that screening mammography with more modern technologies than applied in these trials will reduce mortality without causing too much harm will have to be tested in large, well-conducted randomised trials…” Gotzsche & Olsen. Is screening for breast cancer with mammography justifiable? Lancet 2000;355:129 – 34. Olsen & Gotzsche. Cochrane review on screening for breast cancer with mammography. Lancet 2001;358:1340-42.

“Same trials, different takes”: 

“Same trials, different takes” Olsen & Gotzsche. Cochrane review on screening for breast cancer with mammography. Lancet 2001;358:1340-42.

“Same trials, different takes”: 

“Same trials, different takes” The US Preventive Services Task Force reviewed the same set of trials: “Recently, a 2001 Cochrane Collaboration review of the same trials concluded that six of the eight trials were "flawed" or of "poor quality" and that the pooled results from the remaining two better trials did not support a benefit from mammography. Although the USPSTF was concerned about many (but not all) of the flaws identified in this review, it did not consider the presence of flaws sufficient reason in itself for rejecting trial results. The meta-analysis performed for the USPSTF on the most current published data found that the pooled effect size of the combined trials was sizable and statistically significant: the summary relative risk (RR) of breast cancer death among women randomized to screening in seven trials that included women older than 50 was 0.77 (95 percent CI, 0.67-0.89). The USPSTF recommends screening mammography, with or without clinical breast examination, every 1-2 years for women aged 40 and older.” http://www.ahcpr.gov/clinic/3rduspstf/breast cancer/

Case study 3: “Is passive smoking harmful?”: 

Case study 3: “Is passive smoking harmful?” A topic of great debate and controversy for many years First few epidemiologic studies were published in 1918 Vigorously attacked by the tobacco industry Too small an association Potential bias Potential confounding Lack of biological proof Evidence accumulated over the next 2 decades It was not until about 10 years ago when several official bodies/agencies concluded that passive smoking is a cause of lung cancer The tobacco industry continues to dispute this claim!! Hackshaw AK et al. BMJ 1997;315:980-88. Hackshaw AK. Stat Meth Med Res 1998;7:119-136.

“Is passive smoking harmful?”: 

“Is passive smoking harmful?” Hackshaw et al. conducted a very comprehensive systematic review in 1997: They identified 37 published studies that reported risk of lung cancer among lifelong non-smoking women according to the husband’s smoking status Their meta-analysis revealed that the overall risk of lung cancer among lifelong non-smoking women was 1.24 times higher when their husbands smoked, as compared to those women whose husbands did not smoke. Hackshaw AK et al. BMJ 1997;315:980-88. Hackshaw AK. Stat Meth Med Res 1998;7:119-136.

“Is passive smoking harmful?”: 

“Is passive smoking harmful?” Hackshaw AK et al. BMJ 1997;315:980-88. Hackshaw AK. Stat Meth Med Res 1998;7:119-136.

“Is passive smoking harmful?”: 

“Is passive smoking harmful?” Hackshaw AK et al. BMJ 1997;315:980-88. Hackshaw AK. Stat Meth Med Res 1998;7:119-136.

“Is passive smoking harmful?”: 

“Is passive smoking harmful?” Hackshaw AK et al. BMJ 1997;315:980-88. Hackshaw AK. Stat Meth Med Res 1998;7:119-136.

What is evidence-based medicine?: 

What is evidence-based medicine? The practice of EBM is the integration of individual clinical expertise with the best available external clinical evidence from systematic research and patient’s values and expectations

What is evidence-based public health?: 

What is evidence-based public health? Evidence-based public health is “the development, implementation, and evaluation of effective programs and policies in public health through application of principles of scientific reasoning including systematic uses of data and program planning models.” --Brownson, Ross C., Gurney, James G., and Land, Garland H. Evidence-Based Decision Making in Public Health. Journal of Public Health Management Practice. 1999; 5(5): 86-97.

The importance of research synthesis: 

The importance of research synthesis We need evidence for both clinical practice and for public health decision making Where does evidence come from? An good review is a state-of-the-art synthesis of current evidence on a given research question Given the explosion of medical literature, and the fact that time is always scarce, review articles play a big role in decision-making To keep up to date in Internal Medicine, need to read 17 articles a day, 365 days a year!

The importance of research synthesis: 

The importance of research synthesis Given that most clinicians and public health professionals do not have the time to track down all the original articles, critically read them, and obtain the evidence they need for their questions, Systematic reviews and clinical practice guidelines may be their best source of evidence Several “pre-digested” sources of evidence are currently available The EBM movement is heavily dependent on these pre-appraised evidence sources

The importance of research synthesis: 

The importance of research synthesis Karl Pearson is probably the first medical researcher to use formal techniques to combine data from different studies (1904): He synthesized data from several studies on efficacy of typhoid vaccination His rationale for pooling data: “Many of the groups… are far too small to allow of any definite opinion being formed at all, having regard to the size of the probable error involved.” Egger et al. Systematic reviews in health care. London: BMJ Publications, 2001.

Prof Archibald Cochrane, CBE (1909 - 1988): 

Prof Archibald Cochrane, CBE (1909 - 1988) The Cochrane Collaboration is named in honour of Archie Cochrane, a British researcher. In 1979 he wrote, "It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomized controlled trials” Source: http://www.cochrane.org/cochrane/archieco.htm

The Cochrane Collaboration: 

The Cochrane Collaboration Archie Cochrane’s challenge led to the establishment during the 1980s of an international collaboration to develop the Oxford Database of Perinatal Trials. His encouragement, and the endorsement of his views by others, led to the opening of the first Cochrane centre (in Oxford, UK) in 1992 and the founding of The Cochrane Collaboration in 1993. Source: http://www.cochrane.org/cochrane/archieco.htm

Systematic reviews/meta-analyses indexed in PubMed – 10 years: 

Systematic reviews/meta-analyses indexed in PubMed – 10 years Search: meta-analysis(MeSH) OR meta-analysis(tw) OR systematic review(tw)

Are these the same or different?: 

Are these the same or different? Traditional, narrative review Systematic review Overview Meta-analysis Pooled analysis

Types of review articles: 

Types of review articles All reviews (also called overviews) Systematic reviews Meta-analyses Individual patient data meta-analyses (pooled analyses)

In practice, not all meta-analyses are conducted as part of systematic reviews: 

In practice, not all meta-analyses are conducted as part of systematic reviews All reviews (also called overviews) Systematic reviews Meta-analyses Individual patient data meta-analyses (pooled analyses)

Some definitions: 

Some definitions Traditional, narrative reviews, usually written by experts in the field, are qualitative, narrative summaries of evidence on a given topic. Typically, they involve informal and subjective methods to collect and interpret information. “A systematic review is a review in which there is a comprehensive search for relevant studies on a specific topic, and those identified as then appraised and synthesized according to a predetermined and explicit method.”* *Klassen et al. Guides for reading and interpreting systematic reviews. Arch Pediatr Adolesc Med 1998;152:700-704.

Some definitions: 

Some definitions “A meta-analysis is the statistical combination of at least 2 studies to produce a single estimate of the effect of the healthcare intervention under consideration.”* Individual patient data meta-analyses (pooled analyses) involves obtaining raw data on all patients from each of the trials directly and then re-analyzing them. *Klassen et al. Guides for reading and interpreting systematic reviews. Arch Pediatr Adolesc Med 1998;152:700-704.

Elements of a Systematic Review: 

Elements of a Systematic Review Formulate the review question & write a protocol Search for and include primary studies Assess study quality Extract data Analyze data Interpret results & write a report

All systematic reviews are not meta-analyses!: 

All systematic reviews are not meta-analyses! “…it is always appropriate and desirable to systematically review a body of data, but it may sometimes be inappropriate, or even misleading, to statistically pool results from separate studies. Indeed, it is our impression that reviewers often find it hard to resist the temptation of combining studies even when such meta-analysis is questionable or clearly inappropriate.” Egger et al. Systematic reviews in health care. London: BMJ books, 2001:5.

Are all reviews equal?: 

Are all reviews equal? In 1987, Cynthia Mulrow published an interesting article entitled “The Medical Review Article: State of the Science.” She examined 50 review articles published in 4 major general medical journals [Annals of Internal Med; Archives of Internal Med; JAMA; New Engl J Med] Findings: 80% addressed a focused review question 2% described the method of locating evidence 2% used explicit criteria for selecting studies for inclusion 2% assessed the quality of the primary studies 6% performed a quantitative analysis Mulrow C. The medical review article: state of the science. Annals Int Med 1987;106:485-88.

Are all reviews equal?: 

Are all reviews equal? In 1999, Cynthia Mulrow’s survey was repeated. This time 158 reviews published in 6 major general medical journals [Annals of Internal Med; JAMA; New Engl J Med; BMJ; Am J Med; J of Int Med] Findings: 34% addressed a focused review question 28% described the method of locating evidence 14% used explicit criteria for selecting studies for inclusion 9% assessed the quality of the primary studies 21% performed a quantitative analysis McAlister et al. The medical review article revisited: has the science improved? Annals Int Med 1999;131:947-51

Why aren’t traditional, narrative reviews good enough?: 

Why aren’t traditional, narrative reviews good enough? Can be subjective, prone to bias and error Literature search may be patchy and inadequate Selective citation of literature No description of the methods used by the review Usually uses vote-counting: can be misleading Usually not quantitative: can’t pick up small effects Readers can’t judge the quality of the review Readers can’t replicate or verify the review Hard to separate research evidence from anecdotal experiences Narrative reviews may disagree with each other

How are systematic reviews better?: 

How are systematic reviews better? You don’t have to be an “expert” to do one! More objective, less prone to bias and error Literature search is comprehensive, exhaustive and repeatable Clear description of the methods used Explicit criteria for choosing studies Includes assessment and discussion of quality of primary studies Quantitative synthesis avoid vote counting Can pick up small effects by pooling data Readers can replicate or verify the review

When can meta-analyses mislead?: 

When can meta-analyses mislead? When a meta-analysis is done outside of a systematic review When poor quality studies are included or when quality issues are ignored When inadequate attention is given to heterogeneity Indiscriminate data aggregation can lead to inaccurate conclusions When reporting biases are a problem Publication bias Time lag bias Duplicate publication bias Language bias Outcome reporting bias Egger M et al. Uses and abuses of meta-analysis. Clinical Medicine 2001;1:478-84.

Meta-analyses vs. large clinical trials: 

Meta-analyses vs. large clinical trials

How to read a systematic review?: 

How to read a systematic review? Research on quality of systematic reviews has shown that Not all SRs are truly systematic The quality of SRs are highly variable Cochrane reviews, on average, may be more rigorous and better reported than journal reviews However, recent studies show that even Cochrane reviews have methodological problems Since the quality of systematic reviews cannot be taken for granted, the reader has the responsibility of critically appraising them

Instruments for appraising the quality of a systematic review: 

Instruments for appraising the quality of a systematic review Oxman & Guyatt index of the scientific quality of research overviews (1991) Users’ Guides to the Medical Literature checklist on how to use review articles (2002)

What to look for in a systematic review?: 

What to look for in a systematic review? A clearly defined, explicit question Comprehensive and systematic search for studies Explicit, reproducible strategy for screening and including studies (inclusion/exclusion criteria) Assessment of quality of primary studies Explicit, reproducible data extraction Appropriate analysis and reporting of results Exploration of heterogeneity, publication bias, etc. Discussion should consider limitations and strength of evidence Interpretation supported by data Implications for patient care and future research

authorStream Live Help