# Early sepsis prediction using a hybrid LSTM-GAT model: a study on the PhysioNet 2019 dataset.

- **DOI:** [10.1136/bmjhci-2025-101762](https://doi.org/10.1136/bmjhci-2025-101762)
- **Rating:** Limited (2.8 out of 5)
- **Verdict:** The hybrid LSTM-GAT modestly improves on in-family baselines, but missing strong comparators prevents a credible state-of-the-art or deployment claim.
- **Framework version:** v5.9
- **Pipeline version:** v0.4
- **Evaluation date:** 2026-10-08
- **Evaluation page:** https://www.nabu.science/paper/10.1136%2Fbmjhci-2025-101762

## Key claims

1. The LSTM-GAT model achieved an area under the receiver operating characteristic curve (AUROC) of 0.853±0.005, an F1-score of 0.627±0.006 and a specificity of 0.872±0.007, outperforming baseline models.
   - Support: Tentative
   - Type: Primary finding
   - Basis: comparison is confined to in-family baselines, with ambiguity over repeated splits versus repeated seeds
   - Location: Abstract, Results
2. Despite being trained on fixed temporal windows, the model generalised well across multiple prediction horizons without retraining.
   - Support: Tentative
   - Type: Secondary
   - Basis: retrospective interval analysis reports only sensitivity and shows non-monotonic performance near onset
   - Location: Abstract, Results
3. Compared with the internal vital-sign-only AUROC of 0.730, the external AUROC was 0.70, with an AUPRC of 0.25.
   - Support: Supported
   - Type: Secondary
   - Basis: direct independent-cohort result, but restricted to the vital-sign model
   - Location: Results, External validation
4. The highest true positive rate in our model occurred 6–8 hours before onset, followed by a slight decline.
   - Support: Supported
   - Type: Secondary
   - Basis: reported retrospective interval analysis supports the direction, without broader horizon-specific performance measures
   - Location: Discussion, temporal analysis paragraph

## Strengths

- **Multiple complementary evaluation perspectives** The study reports discrimination, precision-recall, utility, alert-burden, temporal, threshold, and inference-time analyses rather than relying on AUROC alone.
  - Dimension: methodological rigour
  - Location: Methods, Model training and evaluation and Further analytical analyses; Results
- **Independent external cohort included** A vital-sign-only version is evaluated on MIMIC-III using fixed PhysioNet-derived thresholds and training-set standardisation, exposing a real cross-site performance reduction.
  - Dimension: methodological rigour
  - Location: Methods and Results, External validation
- **Relevant clinical and technical context** The introduction situates the model among SOFA and SIRS, PhysioNet challenge winners, recurrent and transformer models, graph methods, and clinically evaluated systems.
  - Dimension: positioning
  - Location: Introduction

## Limitations

- **Strong field baselines are missing** The same-task experiment compares only LSTM, GAT, temporal GAT, and the hybrid, although the introduction identifies XGBoost as a top PhysioNet approach. This prevents a persuasive state-of-the-art comparison.
  - Dimension: methodological rigour
  - Location: Introduction, PhysioNet challenge paragraph; Methods, Model architectures
- **Deployment language exceeds validation stage** The paper calls the model clinically deployable despite retrospective analysis, no operational workflow test, and external validation limited to vital signs with AUROC 0.70 and AUPRC 0.25.
  - Dimension: reporting
  - Location: How this study might affect research, practice or policy; Results, External validation
- **Comparability and reporting are inconsistent** Methods says the extended labels make prior-study results non-comparable, but Discussion claims the model outperformed all cited studies. The text also conflicts over AUPRC leadership and whether repeats were splits or seeds.
  - Dimension: positioning
  - Location: Methods, Data and Model training and evaluation; Discussion, paragraphs 1–3

## Reliability checks

Caveats (4 of 4 checks).

- **Results check:** Noted. One reported value does not reconcile, but the key claims do not rest on it.
  - Minor reporting notes (1):
    - Descriptive: "Given that 6 of the 10 top features were vital signs, we evaluated the model’s predictive power using only these inputs, excluding laboratory and demographic variables." (Results, feature-importance paragraph) vs "Feature importance analysis showed that six of the top 10 important predictors were vital signs (temperature (Temp), oxygen saturation (O2Sat), diastolic blood pressure (DBP), heart rate (HR) and systolic blood pressure (SBP)), along with two lab values (haematocrit (Hct) and haemoglobin (Hgb)) and three contextual variables: Unit1 (medical ICU), Unit2 (surgical ICU) and ICU length of stay (ICULOS)." (Discussion, feature-importance paragraph). Counting the enumerated predictors gives five vital signs, two laboratory variables and three contextual variables, totaling ten rather than supporting six vital signs. → Conclusion holds
- **Reporting and transparency:** Noted. The repetition procedure does not clearly identify whether the reported variability includes new data splits, and the section headed conflicts of interest contains no explicit disclosure.
  - Conflict of interest: not stated · Ethics approval: declared · Preregistration: not stated · Data availability: declared · Code availability: declared
- **Reference integrity:** Clean
  - 34 references in manuscript
  - 32 of 32 checkable references found in an index
  - 2 have no canonical index record — counted, but not index-checkable
  - 2 references confirmed by manual review
  - Citation details diverge for 1 reference:
    - #9 Yadgarov MY Landoni G Berikashvili LB et al Early detection of sepsis using machine learning algorithms: a systematic review and network meta-analysis Front Med (Lausanne) 2024 11 1491358 10.3389/fmed.2024.1491358 39478824 PMC11523135 (DOI resolves to: “Early detection of sepsis using machine learning algorithms: a systematic review and network meta-analysis” — Yadgarov, 2024 (Frontiers in Medicine). Differs from the DOI’s record on: container, authors) https://doi.org/10.3389/fmed.2024.1491358
  - References not verified automatically are resolved manually within 24 hours.

## Post-publication record

- **Status:** Clean
- **Summary:** No retraction or expression of concern found in Crossref (includes Retraction Watch).
- **Last checked:** 2026-10-17
