It is getting more and more difficult to keep up with newly published articles even just in one medical subspecialty. With so much to read, from background articles for research papers to articles for journal club to manuscript assignments as a peer reviewer, it is important to be efficient and attack every scientific article or manuscript strategically.
Clinical research is my focus area, but the following updated tips for interpreting a journal article may apply to other areas of research as well.
Background: Do the authors summarize previously published studies leading up to the present study? What don’t we already know about this topic?
- Do the authors do a good job justifying the reason for the study? This should not be lengthy if there is clearly a need for the study.
- Do the authors present a hypothesis? What is it?
- What is the primary aim/objective of the study? Do the authors specific secondary aims/objectives?
Study Design: Do the authors explicitly state the design used in the present study? If so, what is it?
Retrospective (“case-control study”): Starts with the outcome then looks back in time for exposure to risk factors or interventions
- Can calculate odds ratios to estimate relative risk.
- Cannot calculate risk/incidence (not prospective).
Cross-sectional (“prevalence study”): Takes a snapshot of risk factors and outcome of interest at one point in time or over a specific period of time
- Can calculate prevalence.
- Cannot calculate risk/incidence (not longitudinal).
Prospective: Gold standard for clinical research–may be observational or interventional/experimental. Check if the study is prospectively registered (e.g., clinicaltrials.gov) because most journals expect this.
Observational (“cohort study”)
- May or may not have a designated control group (can start with defined group and risk factors are discovered over time such as the Framingham Study).
- Can calculate incidence and relative risk for certain risk factors.
- Identify causal associations.
Interventional/Experimental (“clinical trial”)
- What is the intervention or experiment?
- Is there blinding? If so, who is blinded: single, double, or triple (statistician blinded)?
- Are the groups randomized? How is this performed?
- Is there a sample size estimate and what is it based on (alpha and beta error, population mean and SD, expected effect size)? This should be centered around the primary outcome.
- What are the study groups? Are the groups independent or related?
- Is there a control group such as a placebo (for efficacy studies) or active comparator (standard of care)?
Measurements: How are the outcome variables operationalized? Check the validity, precision, and accuracy of the measurement tools (e.g., survey or measurement scale).
- Validity: Has the tool been used before? Is it reliable? Does the tool make sense (face validity)? Is the tool designed to measure the outcome of interest (construct validity)?
- Precision: Does the tool hit the target?
- Accuracy: Are the results reproducible?
Analysis: What statistical tests are used and are they appropriate? How do the authors define statistical significance (p-value or confidence intervals)? How are the results presented in the paper and are they clear?
- Categorical variables with independent groups: 1 outcome and 2 groups = Chi square test (exact tests are used when n<5 in any field); multiple outcomes or multiple groups = Kruskal Wallis (with one-way ANOVA and post-hoc multiple comparisons test (e.g., Tukey-Kramer).
- Continuous variables with independent groups: 1 outcome and 2 groups = Student’s t test (if normal distribution) or Mann-Whitney U test (if distribution not normal); multiple outcomes or multiple groups = ANOVA with post-hoc multiple comparisons testing; multiple outcomes and multiple groups = linear regression.
- Continuous variables with related groups: paired t test or repeated-measures ANOVA depending on the number of outcomes and groups.
- Are the results statistically significant? Clinically significant?
- Do the results make sense?
Conclusions: I personally tend to skip the discussion section of the paper at first and come up with my own conclusions based on the study results; then I read what the authors have to say later.
- Did the authors succeed in proving what they set out to prove?
- Read the discussion section. Do you agree with the authors’ conclusions?
- What are possible future studies based on the results of the present study and how would you design the next study?