Loading evaluation data…
Paediatric cardiology presents challenges due to the rarity and complexity of conditions like congenital heart disease. Using retrospective electronic healthcare records from 1,522 Great Ormond Street Hospital cases, we benchmark machine learning models to predict length of stay and retrieve similar patient cases. BioClinical-BERT is used for embedding-based retrieval, while a Random Forest model achieves the best length of stay prediction accuracy (0.88 ± 0.02), outperforming clinicians. The Random Forest model shows mean precision and sensitivity of 0.77 ± 0.03 and 0.76 ± 0.04 during NHS silent deployment (1,052 admissions). K-means identifies three clinically distinct subgroups. Cosine similarity retrieval reveals diagnosis-driven top matches, while complications dominate broader sets. In a 25-case intensive care unit pilot, clinician-rated utility improves from 4.23 ± 2.42 to 4.41 ± 2.16 (scale 0-10). Our models surpass clinician performance in length of stay prediction and show promise for case retrieval, supporting data-driven decision-making in paediatric cardiology and beyond.
a Random Forest model achieves the best length of stay prediction accuracy (0.88 ± 0.02), outperforming clinicians.
Table 1 identifies a higher-accuracy model, while the small clinician comparison used BioClinical-BERT without a superiority test
The Random Forest model shows mean precision and sensitivity of 0.77 ± 0.03 and 0.76 ± 0.04 during NHS silent deployment (1,052 admissions).
prospective single-site monitoring supports the direction, with limited duration and a minor accuracy inconsistency
K-means identifies three clinically distinct subgroups.
three clusters were recovered, but clinical separation was modest and only one LoS contrast was significant
Cosine similarity retrieval reveals diagnosis-driven top matches, while complications dominate broader sets.
Table 3 contradicts the stated retrieval pattern and contains invalid uncertainty values for bounded metrics
In a 25-case intensive care unit pilot, clinician-rated utility improves from 4.23 ± 2.42 to 4.41 ± 2.16 (scale 0-10).
the uncontrolled pilot cannot establish improvement from a 0.18-point change with standard deviations above two points
Derived from the full evaluation — not a separate score.
Strengths
The Random Forest pipeline processed 1,052 admissions over six months without exposing predictions to clinicians. This supplies prospective performance evidence rarely available in retrospective EHR studies.
↳ Results, Silent Deployment Performance; Fig. 3
Decision Tree, Random Forest, Gradient Boosting, LSTM, RoBERTa, and BioClinical-BERT variants were evaluated, with patient-level stratification and upsampling restricted to training folds.
↳ Methods, Model Comparison and Rule-based Approaches; Results, Table 1
The study connects categorical LoS prediction to bed planning and tests similar-case presentation during ICU consultations, establishing a visible application pathway.
↳ Discussion, paragraph 2; Methods, Pilot Study in Clinical Care
Limitations
Table 3 contains impossible standard deviations for several bounded metrics, and its values do not support the claim that diagnosis drives top retrieval while complications dominate broader sets. This materially limits confidence in the retrieval conclusions.
↳ Results, Table 3; Discussion, patient-similarity paragraph
Training and test datasets were combined for encoding and standardisation, allowing held-out information to influence preprocessing. The prospective deployment partly tests real-world performance but does not repair the retrospective comparison design.
↳ Methods, Model Comparison and Rule-based Approaches
Table 1 reports BioClinical-BERT accuracy of 0.89 versus 0.88 for Random Forest, while Fig. 2 compares clinicians with BioClinical-BERT on only 50 cases and reports no significance test for superiority.
↳ Abstract; Results, Table 1; Fig. 2 caption
The six-month silent deployment provides the paper's strongest evidence and makes the contribution more consequential than a retrospective benchmark alone. Methodological confidence is reduced because training and test data were combined for encoding and standardisation, and the downstream retrieval evaluation lacks component ablations or a retrieval baseline. Table 3 also conflicts with the narrative account of retrieval and reports invalid uncertainty values for several bounded metrics. The manuscript is well structured and engages relevant deployment literature, but claims of clinician superiority, improved pilot utility, and broad transferability require substantial qualification.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Limited2.8
Confidence mediumThe integrated LoS, retrieval, silent-deployment, and pilot workflow constitutes a useful applied contribution in paediatric cardiology, although its modelling ingredients are established. The advance is limited by overstated comparative, clustering, retrieval, and utility claims.
“During the silent mode deployment, the model achieved a mean accuracy of 0.776.”
Patient-level stratification, multiple model families, training-fold-only upsampling, and prospective monitoring support the design. Methodological Rigour is constrained by joint train/test preprocessing, incomplete tuning detail, missing retrieval ablations, and unstable or inconsistent downstream evaluations.
“both training and test datasets were combined for consistent encoding. The encoded features were then standardised.”
The manuscript follows a recognisable progression from model development through deployment, retrieval, and pilot evaluation. Its abstract nevertheless misidentifies the best model, conflates the model used in the clinician comparison, and frames a small utility-score change as improvement.
“a Random Forest model achieves the best length of stay prediction accuracy (0.88 ± 0.02), outperforming clinicians.”
The Introduction and Discussion engage relevant patient-similarity, clinical-AI, and silent-deployment literature and acknowledge the single-centre and objective-function limitations. Claims of strong generalisability and transferability remain more expansive than the single-specialty evidence supports.
“the proposed methodology and model architecture are transferable and scalable to other hospital settings.”
Concerns4 of 4 checks
Several result tables and narrative claims are mutually inconsistent, including material problems in the retrieval validation and comparative-performance reporting. These issues could change the interpretation of the retrieval and clinician-superiority claims.
Ethics approval and secure handling of anonymised records are declared, with no text-grounded conduct concern identified. Unverified repository, supplementary, or disclosure details are assessability gaps rather than reliability findings.
Flags: 1 declared / 5 total
60 of 62 checkable references verified
77 references in manuscript 15 are books, websites or datasets — counted, but not index-checkable
No retraction notice found in Retraction Watch.
Sources: Retraction Watch ✓
Where this paper’s evidence sits on the path from initial observation to real-world use.
A six-month silent deployment covering 1,052 admissions and a 25-patient ICU pilot move the work beyond retrospective benchmarking. Readiness remains preliminary because predictions were not clinician-visible during deployment and the pilot had no controlled decision-impact evaluation.
“the model’s predictions were stored without clinician visibility.”
AI-generated, human-governed. Something look off? Contact us to request a review.