How to criticise quantitative research?Lecture I – Introduction and Methods: How to criticise quantitative research? Lecture I – Introduction and Methods Chee-Wee Tan Queen Margaret University, Edinburgh


Slide2: How to criticise research? Am I qualified to criticise research?


All papers are not created equal: All papers are not created equal Don’t take quality of papers for granted. There are as many poorly written papers as there are good ones. And of course, no one is perfect! Criticise papers with a grain of salt, preferably with an entire shaker!


Aims: Aims Know basic structure of papers Systematic reflection of a paper Appreciate some aspects to look out for criticism in each section of a paper


Definition of criticise: Definition of criticise Criticize: to consider the merits and demerits of and judge accordingly (Merriam-Websters Online Dictionary, http://www.m-w.com)


Types of papers: Types of papers Theoretical (formal or qualitative) Experimental Clinical studies Focus of this lecture Quantitative (could be laboratory or clinical) Empirical Methodological Systematic reviews Qualitative (not covered here)


Anatomy of a research paper: Anatomy of a research paper Introduction Hypothesis & Aims Methods Results Discussion Conclusion Paper structure used as critique approach


Section: Introduction: Section: Introduction Background May or may not assume reader’s prior knowledge of topic Important for setting tone of paper Essential info: Relevant background of research question Reason prompting conduct of research


Section: Introduction: Section: Introduction Common errors: Irrelevant information Background not adequate Weak justification


Section: Hypothesis & Aims: Section: Hypothesis & Aims Not usually explicitly stated But should be clear from introduction Common error: The aim is not stuck to in the study Magical appearance of extra aims


A word on assumptions: A word on assumptions All of us hold assumptions Research isn’t exempt Assumptions may be theory, discipline or method related. Questions to ask: What assumptions are explicitly or implicitly held by the researchers? Are the assumptions appropriate?


Section: Methods – Research design: Section: Methods – Research design What sort of research design? Does it do what it says on the tin? Simplest designs comparing differences or finding relationships? Does the context of research allow researcher to make causal inferences?


Section: Methods - Recruitment: Section: Methods - Recruitment How were participants recruited? Any problems with this recruitment approach? Is it ethical? 2 goals of recruitment strategy Comparability of intervention and control groups Confidence in generalisability Avoid selection bias


Section: Methods - Groups: Section: Methods - Groups Method of allocation to groups. Random? Any other procedures randomised? Blinding of researchers and/or participants Is intervention reasonably administered, or similar to practical situations? Validity and reliability of the procedures Procedures standardised?


Section: Methods - Analysis: Section: Methods - Analysis How was sample size determined? Was anticipated power stated explicitly? Were types of analysis and statistical tests planned a priori?


Conclusion: Conclusion Introduction Background and rationale Hypothesis and aims Clearly justified? Assumptions Appropriate? Methods Participants Procedures Planned analysis