Health Research Evaluation; HIA; etc..

Category: Education

Presentation Description

No description available.


Presentation Transcript


Health programmes evaluation : definition, components ; need for evaluation; HIA : definition; purposes; types; methodology; evidence-based ( tight and broad focus); data collection; stages of HIA ; DALYs : definition; formula; Errors in experimental designs Situations that distort the effectiveness of interventions By : Dr. Sam A. Thamby Faculty of Pharmacy; AIMST University ( Malaysia )


Health care programmes comprise health-related services ( informational and/or interventional ). Evaluation : the critical and objective assessment of the degree to which the health-related services ( informational or interventional) fulfill stated goals . The achievement must be compared with pre-determined standards of expectations (the aims and goals stated before commencing the health programme). Basic components of health care programme evaluation: Structure : framework of the service, equipments used, etc; Process : how the service is organized, methodology (‘field work’) Outcome : the observed result (compared with the pre-determined target result);


Why must health promotion programmes/ interventions be evaluated ? To assess whether the programme or intervention was successful or not ; WHO (1995) – health promotion is an investment , and evaluation is conducted to assess the costs and benefits of this investment . Evaluators assess one or more of the following: Contribution to knowledge base of health promotion; Insights that will result in more effective health promotion practice ; Relative costs and benefits (financial, and health-wise); Levels of stakeholders ’ (the public, communities, funders, politicians, health promotion practitioners, policy makers, etc.) satisfaction ; Evidence to influence policy makers to: upgrade the health policy; focus on research in health promotion (financial, upgrading the facilities, etc.);


HEALTH IMPACT ASSESSMENT (HIA) Health promotion programmes (HPP) require an appraisal or assessment of the health impact . HIA applies to programmes (mainly interventional) ; healthy public policies ; HIA is also used to identify the errors in the experimental designs of the programme or intervention . HIA : ‘A combination of procedures, methods and tools by which a programme or intervention or policy , may be judged as to it’s potential effects on the health of a population , and the distribution of those effects within a population .’ (European Centre of Health Policies,1999) ‘distribution of those effects within a population’ refers to equitable benefits of the programme or intervention


Purposes of HIA : Knowledge : To have improved knowledge about the potential impact of a programme / intervention /policy; Policy : To inform decision makers (law makers) and the ‘affected’ people (people affected by the policy); To facilitate adjustment of the proposed policy (with regard to a particular health promotion programme) in order to mitigate the negative effects and maximize the positive effects . HIA can also lead to informed policy making and provides opportunities to ‘adapt decisions’ to avoid potential harm.


THREE TYPES OF HIA HIA Prospective Predicts the effect(s) of a HPP / programme before it is implemented Retrospective Identifies the consequences of a HPP / programme after it’s implementation Concurrent Aims to identify consequences as policies or programmes are being implemented (especially when negative consequences are anticipated)


METHODOLOGY of HIA Evaluate the objectives (general and specific) of the programme / intervention / policy; Identify the ‘options’ ( different ways to conduct the health care programme; different ways to implement the policies); Identify and measure the costs and benefits associated with each option (all types of costs) Identify and assess the uncertainties of each option , errors in experimental designs ; Assess the ‘balance’ between each option (benefits : limitations) Present the results Monitoring protocol to further assess the consequences (any latent effects); * Experimental designs (like RCTs) are a key feature in steps c, d and e.


EVIDENCE-BASED HIAs HIA should be based on strong evidences to justify the results . ‘ Tight focus ’: Based on epidemiological data , QoL issues (death, disabilities ) ; pharmacological data (dose-response relationships) etc.; Outcomes are usually expressed in terms of death and disability . ‘ Broad focus ’: More wide-ranging; includes informed opinions and local knowledge ; Relevance : A good HIA combines both these evidences, and analyses the relevance to… Situational validation – will the HPP/programme objectives be relevant to the ‘health problem’ in that particular country???? Societal vindication – will the HPP/programme have value for society??? Social choice – will the HPP/programme benefit each individual???


DATA (INFO.) COLLECTION METHODS USED IN HIAs Informant (samples/cases/respondents) interviews Focus group discussions Equity audits , secondary analysis of existing data , Documentary sources (for retrospective type of HIA) Study tools or instruments (survey forms or questionnaires) Community profiling Experts’ opinions


STAGES OF HIA Screening : to establish whether a particular HPP, policy is relevant to health (individual, community, country); involves the use of checklists and study tools ; Scoping : to identify the relevant health issues and public concerns that are under the scope of that HPP, policy ; Appraisal : to identify the potential impacts of the HPP/policy on the health and well-being of the public , with evidence based from the earlier steps (1 & 2); can include health promoting measures ; Reporting : communicating with the ‘stakeholder s’ about the expected impacts on health , how the existing HPP/policy can be modified to have a more positive impact on the public ; Monitoring : the compliance with the recommendations following steps 1-4;


Disability-Adjusted Life Years; A measure of overall disease burden ; 1 DALY = 1 YEAR OF HEALTHY LIFE LOST The concept was developed by the WHO . Becoming increasingly common in the field of public health and HIA ( Health Impact Assessment ) . Purpose : It was designed to quantify the impact of ‘premature’ (early) death and disability on a population by combining them into a single, comparable measure . Mortality and morbidity are combined into a single, common metric (a common measure). DALYs


Earlier, health liabilities were expressed using one measure: expected or average number of Years of Life Lost (YLL) . This measure does not take the impact of disability into account , which can be expressed by: Years Lived with Disability (YLD) . DALY = YLL + YLD The DALY relies on time as the most appropriate measure of the effects of chronic illness, ( time lost due to premature death and time spent disabled by disease) . One DALY, is equal to one year of healthy life lost . Japanese life expectancy statistics are used as the standard for measuring premature death, as the Japanese have the longest life expectancies (on average) .



Types of errors:

Types of errors HIA assesses the ‘errors’ in experimental designs of HPP/interventions/policy. Type-1 error : an erroneous conclusion that a programme or policy has achieved significant change, when in reality, it has failed to do so (the differences between the experimental and control groups are due to some other extraneous factors rather than the intervention); A good experimental design must exclude (rule out) all the other extraneous factors . Type-2 error : an erroneous conclusion that that programme or policy has failed to have a significant impact, when in reality it may actually have had an impac t; E.g., a health promotion programme (intervention) may actually have had an impact on the target group, but the study instruments might have been ‘insufficiently sensitive’ to differentiate (discriminate) between the experimental and control group ;


Type-3 error : asserting that a programme or policy has failed to achieve successful results , when the reality is that the experimental design was poorly planned and executed , so that a true result could never have been observed (poor methodological design – sample selection, study instruments, results analyses); Type-4 error : Conducting an evaluation of a programme that no one cares about and is irrelevant to ‘decision makers’ ; Evaluation just for the sake of evaluation is central to this error . Type-5 error : An intervention is shown to produce a genuine statistically significant effect , but the change is so slight to have any practical significance when considering the entire population (e.g., a new costly drug only mildly controls BP than the existing medications, however the HPP shows a statistically significant impact).


Failure to detect ‘change’ due to an intervention can also occur because of… ‘ Mixed effect’ on populations (inequitable distribution) : The positive effect of the programme (interventional) in one section of the population is diluted by the zero effect in the rest of the population ; ‘ Negative reactance ’ effect;


Green and Lewis (1986) listed the following situations that can cause errors and distort the effectiveness of programmes (interventions): Delay of impact emergence (Sleeper effect) : The intervention actually has an effect (impact), which emerges a long time after the evaluation (latent manifestation of the impact); Decay of impact (Backsliding effect) : The intervention produces an effect, which ‘decays’ rapidly ; Without continuing the evaluation, if the intervention is claimed as successful, the error produced is that the intervention produced an effect which though beneficial, is not long-lasting .


‘ Borrowing from the future ’: The intervention triggers changes (effects) that would have occurred normally , the intervention might have just hastened the occurrence of the effect . ‘Secular’ trends : (changes over a long period of time - months, years) Positive secular trend : overestimating an intervention’s effect ; e.g.,: natural immunity improving resistance against a disease rather than the intervention; (similar to Type-I error). Negative secular trend : discounting / overlooking / rejecting a potential influence (effect); Contrast effects : premature termination of the intervention can cause a ‘ backlash ’, resulting in slowing or reversal of the outcomes , that would have not occurred had the intervention continued ;



authorStream Live Help