Interpretation and Drawing Conclusions (Cochrane Handbook Ch 15)
Your Role
Structure the interpretation of SR findings: applicability assessment, NNT computation, SMD re-expression, correct use of CIs and P values, and framing implications for practice and research.
Prerequisites
- Pooled effect estimates (from sr-synthesis)
- GRADE certainty ratings (from sr-grade)
- Study characteristics (from sr-extraction)
Workflow
Step 1: Assess Applicability (Ch 15.2)
CHECK each dimension:
Biological variation: Are the study populations similar to the target population? (age, sex, disease severity, comorbidities)
Contextual variation: Are the settings similar? (ED, ICU, pre-hospital — resource availability, staffing, protocols)
Variation in adherence: Would real-world adherence differ from trial conditions?
Values and preferences: Would patients in the target setting value outcomes the same way?
For each, document: "The evidence is [directly applicable / somewhat indirect / very indirect] for [dimension]."
Step 2: Compute Absolute Effects and NNT (Ch 15.4)
If the MA used RR or OR:
python3 scripts/nnt_calculator.py --effect RR --value 0.75 --ci-lower 0.65 --ci-upper 0.85 --acr 0.30
GENERATE results for LOW, MEDIUM, and HIGH baseline risk:
Assumed Comparator Risk (ACR): 150 per 1000 (low)
Absolute effect: 37 fewer per 1000 (95% CI: 23 fewer to 49 fewer)
NNTB: 27
Assumed Comparator Risk (ACR): 300 per 1000 (medium)
Absolute effect: 75 fewer per 1000 (95% CI: 45 fewer to 105 fewer)
NNTB: 14
Assumed Comparator Risk (ACR): 500 per 1000 (high)
Absolute effect: 125 fewer per 1000 (95% CI: 75 fewer to 175 fewer)
NNTB: 8
LABEL correctly: NNTB = number needed to treat for an additional BENEFICIAL outcome. Use NNTH for harmful outcomes.
Step 3: Interpret Continuous Outcomes (Ch 15.5)
Same scale (e.g., both studies use APACHE II): report MD in original units. "If the MD is larger than the MID (minimal important difference), the effect is clinically meaningful."
Different scales (e.g., pain on VAS and NRS): report SMD. RE-EXPRESS SMD in natural units:
python3 scripts/smd_interpreter.py --smd 0.45 --pooled-sd 8.5
"An SMD of 0.45 corresponds to a MD of 3.8 points on the APACHE II scale (pooled SD from included studies: 8.5)."
If MID is known: compare SMD to MID/SD ratio. SMD > MID/SD → clinically meaningful.
Step 4: Correctly Handle CIs and P Values (Ch 15.3)
Do NOT use "statistically significant" or "non-significant" (MECIR C72 mandatory).
INSTEAD describe:
- "The 95% CI excludes the null (RR 0.75, 95% CI 0.65 to 0.85)"
- "The 95% CI includes the null (RR 0.92, 95% CI 0.81 to 1.04)"
- "The CI is wide, reflecting imprecision (RR 0.70, 95% CI 0.30 to 1.60)"
Also check the PREDICTION INTERVAL (from sr-synthesis): does the PI cross the null even if the CI does not? If yes: "The average effect is beneficial, but the effect varies across settings and may be null or harmful in some."
Step 5: Generate Implications (Ch 15.6)
Implications for Practice
Structure as:
- Who this evidence applies to (and who it does not)
- What intervention, compared to what
- What is the balance of benefits and harms
- Certainty of the evidence (from GRADE)
- Values and preferences that might influence decisions
"Do NOT make recommendations — only highlight actions consistent with particular values."
Implications for Research
Structure as:
- What specific research is needed
- Why (precise gap: population, intervention, comparator, outcome, design)
- How (suggested study design, sample size, follow-up)
- Priority (high/medium/low)
"If the evidence is sparse, say: 'Further research is likely to change the estimate.'"
Scripts
scripts/nnt_calculator.py
Computes NNT/NNTB/NNTH from RR/OR with CI across multiple ACRs. Uses Altman (1998) method for NNT CIs.
Usage: python3 nnt_calculator.py --effect RR --value 0.75 --ci-lower 0.65 --ci-upper 0.85 --acr 0.1,0.3,0.5
scripts/smd_interpreter.py
Re-expresses SMD in natural units or MID units.
Usage: python3 smd_interpreter.py --smd 0.45 --pooled-sd 8.5 or --mid 5.0
scripts/applicability_checklist.py
Structured indirectness assessment across 4 dimensions.
Usage: python3 applicability_checklist.py --pico sepsis,vasopressors,mortality,icu
Assets
assets/nnt-interpretation-guide.md
NNTB vs NNTH labeling, time-frame requirement, event-rate dependence.
assets/implications-template.md
Templates for "Implications for practice" and "Implications for research" sections.
assets/mid-reference-em.md
Published MIDs for common EM/CC outcomes: mortality (RR 0.90), vent days (MD 2), LOS (MD 1 day), pain VAS (MD 1.5 cm), SOFA (MD 1).
Guardrails
- "Do NOT make recommendations — Ch 15.6: 'Review authors should avoid specific recommendations.'"
- "Do NOT use 'statistically significant' or 'non-significant' — MECIR C72 mandatory."
- "NNT must specify: intervention, comparator, outcome, time-frame, and ACR."
- "For SMD <0.2, even if statistically significant, clinical importance is questionable."
- "Prediction interval crossing the null is important even if the CI does not."
- "Absolute effects depend on baseline risk — report across a RANGE of risks, not just one."
- "The so-called 'number needed to harm' (NNH) is misleading — use NNTH instead."
Handoff
→ sr-writing (discussion + conclusions sections)
微信扫一扫