Assembling the evidence…
BACKGROUND: Models trained on accelerometer data have been proposed for detecting prodromal Parkinson's disease (PD). However, uncertainties in diagnosis timing in the UK Biobank (UKBB) may affect generalizability to other cohorts.
OBJECTIVES: The aim of the study was to evaluate the performance of previously published models for prodromal PD detection in other international cohorts.
METHODS: We applied the models to data from German and British cohorts of individuals with isolated or idiopathic rapid eye movement sleep behavior disorder and healthy controls. We compared hourly acceleration patterns and classification performance across cohorts.
RESULTS: The British cohort exhibited visually similar activity patterns to UK Biobank but weaker statistical differences and reduced model performance. The German cohort showed no significant group differences and lower performance. No pair of cohorts demonstrated statistical equivalence.
CONCLUSIONS: Models trained on UK Biobank data may capture early clinical disease rather than universal prodromal markers. Prospective validation in well-characterized cohorts is essential before clinical translation. © 2026 The Author(s). Movement Disorders published by Wiley Periodicals LLC on behalf of International Parkinson and Movement Disorder Society.
The British cohort exhibited visually similar activity patterns to UK Biobank but weaker statistical differences and reduced model performance. The German cohort showed no significant group differences and lower performance.
UK Biobank versus British AUROC difference is not significant (P = 0.066), and comparisons rest on five model replicates, not participants
No pair of cohorts demonstrated statistical equivalence.
equivalence margin of roughly 0.015 AUROC with five model replicates makes failure to show equivalence close to a foregone result
These findings suggest that delayed diagnosis recording in UKBB contributes to disease-stage misclassification and inflated performance estimates, limiting transferability.
diagnosis-timing mechanism is never tested and is fully confounded with cohort, device, preprocessing and phenotype differences
Models trained on UK Biobank data may capture early clinical disease rather than universal prodromal markers.
hedged reading of the differential transfer pattern, though alternative explanations are acknowledged and not ruled out
Derived from the full evaluation — not a separate score.
Strengths
The five published Lasso models are applied unchanged to two independent, prospectively characterised cohorts in two countries, which is the appropriate design for a generalisability question. Polysomnography-confirmed iRBD with longitudinal follow-up gives more precise disease staging than the EHR-linked labels the models were trained on.
↳ Methods, Study Cohorts; Methods, Model Application and Evaluation
The validation is motivated by published evidence that UK Biobank Parkinson's diagnosis dates are recorded several years late, which would make nominally prodromal participants already early-clinical. This gives the study a specific, pre-identified target rather than a generic replication aim.
↳ Introduction; Discussion, Diagnosis Timing
AUROCs are reported with 95% confidence intervals for every cohort, and both significance and equivalence testing are presented. The inability to compute wear-time bias-corrected features, affecting five of 81 variables and 6.7% of coefficient mass, is stated explicitly rather than left implicit.
↳ Methods, Preprocessing; Results, Discrimination Performance
Limitations
The Welch tests and TOST compare AUROCs across five model replicates applied to the same test cohort (df 4.878), so the p-values reflect variation between models rather than participant-level sampling uncertainty. No participant-level bootstrap or DeLong intervals are reported, and the roughly two dozen hourly tests carry no multiplicity correction.
↳ Methods, Model Application and Evaluation; Results, Discrimination Performance
The Conclusion states that delayed diagnosis recording in UK Biobank contributes to disease-stage misclassification and inflated performance, but the diagnosis lag is never quantified, stratified or manipulated. Cohort, device (AX3 versus AX6), preprocessing and phenotype (iRBD versus early clinical Parkinson's) differences are fully confounded with the proposed mechanism.
↳ Conclusion; Discussion, Diagnosis Timing; Methods, Preprocessing
The limitations section states that modest sample sizes limit statistical power, yet the Conclusion asserts the German cohort showed 'no discernible prodromal signal' and the Discussion describes 'no meaningful model performance', against a reported AUROC confidence interval of 0.56-0.62 that excludes chance. A failure to detect a difference under acknowledged low power does not support a null conclusion.
↳ Limitations and Future Work; Discussion, Cultural Factors; Conclusion
The TOST bound of δ = 0.5·SD corresponds to roughly 0.015 AUROC and is not justified. With five model replicates, equivalence is close to unattainable by construction, so the abstract's statement that no cohort pair demonstrated equivalence carries little information.
↳ Methods, Model Application and Evaluation; Results, Discrimination Performance; Abstract
The descriptive core of this study is credible in direction: five published Lasso models applied unchanged to two independent iRBD cohorts yield AUROCs of 0.62 and 0.59 against a reported 0.72 in UK Biobank. What drove the scores down is the inferential apparatus built on top of that result, where Welch tests and TOST compare five model replicates on the same test data (df 4.878) rather than participants, and the equivalence margin of roughly 0.015 AUROC makes the headline non-equivalence statement nearly automatic. Assessments converged closely on this methodological reading and on the device and preprocessing shift (AX3 to AX6, five missing features covering 6.7% of coefficient mass) going unprobed, but diverged on how firmly the paper states its case: the abstract hedges with 'may capture', while the Conclusion asserts that delayed diagnosis recording 'contributes to' inflated performance and that the German cohort showed 'no discernible prodromal signal' against a confidence interval of 0.56-0.62. The lower Positioning score reflects the stated power limitation being followed directly by null-style conclusions rather than tempering them. The Contribution score holds at the lower edge of the well-supported band because the validation step itself is genuinely needed and clearly targeted, even where the mechanistic explanation outruns the design.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Sound3.0
Confidence mediumThe study supplies a missing external-validation step for a published UK Biobank prodromal Parkinson's model, testing it against a documented concern about delayed diagnosis recording, and reports a clear performance drop from AUROC 0.72 to 0.62 and 0.59. The advance is real but narrow: two modest iRBD-only cohorts, and the proposed label-timing mechanism is never directly tested.
recent work indicates that PD diagnosis dates in UKBB are frequently delayed relative to clinical onset
Applying the pre-trained models unchanged to independent prospective cohorts is the right basic design, but the inferential statistics compare five model replicates rather than participants, the equivalence margin is roughly 0.015 AUROC, and device (AX3 vs AX6), preprocessing and five missing features are never assessed for directional bias. The Conclusion attributes the transfer failure to UK Biobank diagnosis-date delay, a mechanism the design cannot isolate from cohort, device and phenotype differences.
It could not compute wear-time bias-corrected features, resulting in five missing variables out of 81
The paper is short and logically ordered, moving from the diagnosis-timing concern through cohorts, methods and results to interpretation, and the abstract hedges appropriately. The Conclusion, however, is firmer than the abstract, asserting that delayed recording 'contributes to' inflated performance and that the German cohort showed 'no discernible prodromal signal' against a confidence interval excluding chance.
The German cohort showed no discernible prodromal signal.
The paper engages directly with the published UK Biobank model and the diagnosis-timing literature and states its limitations on sample size, diversity, missing features and the imperfection of iRBD as a prodromal proxy. Those limitations are not carried through: the acknowledged power constraint is followed by null-style conclusions, and the iRBD-proxy caveat does not soften the interpretation.
Sample sizes in the German and British cohorts were modest, limiting statistical power.
Caveats4 of 4 checks
Reported numbers reproduce arithmetically and the abstract matches the results, but the inferential statistics are computed across five model replicates rather than participants, the equivalence margin is extremely tight, and one standard-error/degrees-of-freedom pairing is internally inconsistent.
No conflicts of interest are declared and the data availability statement is a reasonable restricted-access arrangement for clinical cohort data, but no ethics or consent statement appears in the text despite human-participant accelerometry and polysomnography data.
Flags: 2 declared / 5 total
18 references in manuscript 16 of 16 checkable references found in an index 2 are books, websites or datasets — counted, but not index-checkable
No retraction notice found in Retraction Watch.
Sources: Retraction Watch ✓
Where this paper’s evidence sits on the path from initial observation to real-world use.
The work is a pilot-scale validation whose results constrain interpretation of existing models rather than delivering a tool or protocol. The negative and weak findings indicate substantial further work before any deployment.
AI-generated, human-governed. Something look off? Contact us to request a review.