How to criticise quantitative research?Lecture I – Introduction and Methods: How to criticise quantitative research? Lecture I – Introduction and Methods Chee-Wee Tan
Queen Margaret University, Edinburgh Slide2: How to criticise research?
Am I qualified to criticise research?
All papers are not created equal: All papers are not created equal Don’t take quality of papers for granted.
There are as many poorly written papers as there are good ones.
And of course, no one is perfect!
Criticise papers with a grain of salt, preferably with an entire shaker!
Aims: Aims Know basic structure of papers
Systematic reflection of a paper
Appreciate some aspects to look out for criticism in each section of a paper Definition of criticise: Definition of criticise Criticize:
to consider the merits and demerits of and judge accordingly
(Merriam-Websters Online Dictionary, http://www.m-w.com) Types of papers: Types of papers Theoretical (formal or qualitative)
Clinical studies Focus of this lecture
Quantitative (could be laboratory or clinical)
Qualitative (not covered here) Anatomy of a research paper: Anatomy of a research paper Introduction
Hypothesis & Aims
Paper structure used as critique approach Section: Introduction: Section: Introduction Background
May or may not assume reader’s prior knowledge of topic
Important for setting tone of paper
Relevant background of research question
Reason prompting conduct of research Section: Introduction: Section: Introduction Common errors:
Background not adequate
Weak justification Section: Hypothesis & Aims: Section: Hypothesis & Aims Not usually explicitly stated
But should be clear from introduction
The aim is not stuck to in the study
Magical appearance of extra aims A word on assumptions: A word on assumptions All of us hold assumptions
Research isn’t exempt
Assumptions may be theory, discipline or method related.
Questions to ask:
What assumptions are explicitly or implicitly held by the researchers?
Are the assumptions appropriate? Section: Methods – Research design: Section: Methods – Research design What sort of research design?
Does it do what it says on the tin?
comparing differences or finding relationships?
Does the context of research allow researcher to make causal inferences? Section: Methods - Recruitment: Section: Methods - Recruitment How were participants recruited?
Any problems with this recruitment approach?
Is it ethical?
2 goals of recruitment strategy
Comparability of intervention and control groups
Confidence in generalisability
Avoid selection bias Section: Methods - Groups: Section: Methods - Groups Method of allocation to groups. Random?
Any other procedures randomised?
Blinding of researchers and/or participants
Is intervention reasonably administered, or similar to practical situations?
Validity and reliability of the procedures
Procedures standardised? Section: Methods - Analysis: Section: Methods - Analysis How was sample size determined?
Was anticipated power stated explicitly?
Were types of analysis and statistical tests planned a priori?
Conclusion: Conclusion Introduction
Background and rationale
Hypothesis and aims