When I hear clinical research articles presented for Journal Club, the presentations are sometimes very dry and remind me of book reports–just regurgitating statements made by the authors of the article. In reality, Journal Club should offer a “deep dive” into study design and scientific methodology.
In the following outline, I suggest a format for evaluating clinical research articles layer by layer. This can be used by the presenter as well as the discussion moderator to promote a more interactive Journal Club. Coincidentally, the same format can also be used by journal reviewers and editors when reviewing submitted manuscripts (in other words–this is how I review manuscripts).
Background: Provide a brief synopsis of preliminary studies cited in the introduction leading up to the present study.
- Is the present study justified? (Should not be lengthy if there is clearly a need for the study)
- Do the authors present a hypothesis or hypotheses?
- What is the primary aim/objective of the study? Secondary aims/objectives?
Study Design: Do the authors explicitly state the design used in the present study? If so, what is it?
Retrospective (aka “case-control”): Starts with the outcome then looks back in time for exposure to risk factors or intervention
- Can calculate odds ratios to estimate relative risk.
- Cannot calculate risk/incidence (not prospective).
Cross-sectional (aka “prevalence study”): Takes a snapshot of risk factors and outcome of interest at one point in time or over a specific period of time
- Can calculate prevalence.
- Cannot calculate risk/incidence (not longitudinal).
Prospective (aka “cohort”): Gold standard for clinical research–may be observational or interventional/experimental (Is it prospectively registered? Check clinicaltrials.gov)
- Observational (cohort study)
- May or may not have a designated control group (can start with defined group and risk factors are discovered over time such as the Framingham Study).
- Can calculate incidence and relative risk for certain risk factors.
- Identify causal associations.
- Interventional/Experimental
- What is the intervention or experiment?
- Is there blinding? If so, who is blinded: single, double, or triple (statistician blinded)?
- Are the groups randomized? How is this performed?
- Is there a sample size estimate and what is it based on (alpha and beta error, population mean and SD, expected effect size)?
- What are the study groups? Are the groups independent or related?
- Is there a control group–placebo (for efficacy studies) or active comparator (standard of care)?
Measurements: How are the outcome variables operationalized? Check the validity, precision, and accuracy of the measurement tools (e.g., survey or measurement scale).
- Validity: Has the tool been used before? Is it reliable? Does the tool make sense (face validity)? Is the tool designed to measure the outcome of interest (construct validity)?
- Precision: Does the tool hit the target?
- Accuracy: Are the results reproducible?
Analysis: What statistical tests are used and are they appropriate? How do the authors determine statistical significance (p-value or confidence intervals)? How are the results presented in the paper and are they clear?
- Categorical variables with independent groups: 1 outcome and 2 groups = Chi square test (exact tests are used when n<5 in any field); multiple outcomes or multiple groups = Kruskal Wallis (with one-way ANOVA and post-hoc multiple comparisons test (e.g., Tukey-Kramer).
- Continuous variables with independent groups: 1 outcome and 2 groups = Student’s t test (if normal distribution) or Mann-Whitney U test (if distribution not normal); multiple outcomes or multiple groups = ANOVA with post-hoc multiple comparisons testing; multiple outcomes and multiple groups = linear regression.
- Continuous variables with related groups: paired t test or repeated-measures ANOVA depending on the number of outcomes and groups.
- Are the results statistically significant? Clinically significant?
- Do the results make sense?
Conclusions: Skip the discussion section of the paper at first and come up with your own conclusions based on the study results; then read what the authors have to say.
- Did the authors succeed in proving what they set out to prove?
- Read the discussion section. Do you agree with the authors’ conclusions?
- What are possible future studies based on the results of the present study and how would you design the next study?