Crunching the numbers. Responsibly.
evidence was obtained of an OA Citation advantage but not of an OA Publication Advantage
matched observational comparisons do not isolate OA effects, and publication-analysis counts conflict across Tables 2 and 3
the data in DOAJ in a field labelled “the first calendar year a journal provided online Open Access content” differed in about 40 per cent of cases from the year indicated by the publishers in a verification round
the reported verification round found 42% disagreement among 141 responding journals
Derived from the full evaluation — not a separate score.
Strengths
The citation-growth pattern is examined separately in DOAJ- and OAD-based flipping samples with limited overlap, reducing dependence on a single OA directory.
↳ Data, DOAJ-based and OAD-based approaches, pp. 5–6; Table 3, pp. 15–16
Each study journal is compared with a discipline-and-year control and a tailored control that additionally accounts for national orientation.
↳ Definition of control groups, pp. 8–9
The review includes positive, null, mixed, and critical evidence, while the limitations identify Scopus coverage, language and field biases, and residual hybrid-OA contamination.
↳ Earlier studies, pp. 3–4; Limitations, p. 20
Limitations
Table 2 reports 113 usable DOAJ study journals for short-term publication growth, but Table 3 reports 116 paired observations for the corresponding comparison. This directly weakens confidence in the publication-output null.
↳ Tables 2 and 3, pp. 15–16
The paper acknowledges that flipping may coincide with restyling, a language switch, scope changes, or editorial-board replacement, but these factors are absent from the comparative design.
↳ Suggestions for further research, pp. 20–21
Only about 15% of DOAJ journals are represented in Scopus, and the paper states that results based on Web of Science may not be similar.
↳ Limitations, p. 20
The two-source, two-control-group design meaningfully improves on simple before-and-after comparisons and consistently identifies faster citation-impact growth in the sampled flipping journals. However, the analysis cannot separate access-modality changes from contemporaneous changes in language, scope, styling, or editorial leadership identified in the future-research section. The publication-output conclusion is additionally weakened by the unreconciled difference between the usable DOAJ counts in Tables 2 and 3. The paper is therefore most informative as comparative observational evidence rather than as a causal or decision-ready estimate of the effect of flipping.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Sound3.5
Confidence mediumThe study meaningfully extends prior flipping research through two independently assembled journal sets, verified switch years for one set, two control definitions, and geographic decomposition. The advance remains incremental because it does not resolve the causal mechanism behind the observed citation pattern.
“two well-defined control group were created, enabling paired statistical tests”
The paired pre/post design, dual control groups, field-normalised impact measure, and explicit treatment of backlog timing support comparative inference. Selection into flipping, unmeasured contemporaneous journal changes, variable-specific exclusions, and the Table 2–3 count conflict introduce meaningful uncertainty.
“the change in access modality may at the same time be accompanied by other changes”
The paper follows a clear sequence from data construction and indicators to paired comparisons, limitations, and conclusions. Interpretation is impeded by the unreconciled analytic counts and by describing observational comparative evidence as “robust evidence” of an advantage.
“This outcome provides robust evidence”
The literature review covers positive, null, mixed, and methodologically critical findings, while the limitations identify Scopus coverage, language and field bias, and residual hybrid-OA contamination. Concurrent editorial changes are deferred to future research rather than fully integrated into the certainty of the central interpretation.
“The current study is purely statistical”
Concerns4 of 4 checks
An unreconciled analytic-count conflict affects the reported publication-output analysis. A likely mismatch between an in-text citation and the listed McKiernan reference is also present but is secondary to the table inconsistency.
No ethics approval was required for this bibliometric study, and no misconduct or suspicious availability claim is evident. An explicit conflict disclosure is absent despite SCImago affiliations and reliance on SCImago-derived bibliometric data.
Flags: 0 declared / 5 total
23 of 25 checkable references verified
36 references in manuscript 11 are books, websites or datasets — counted, but not index-checkable
No retraction notice found in Retraction Watch.
Sources: Retraction Watch ✓
Where this paper’s evidence sits on the path from initial observation to real-world use.
The analysis supplies real-world comparative benchmarks that can inform discussion of journal conversion. It does not support a prescriptive decision rule because contextual mechanisms and concurrent journal changes require qualitative follow-up.
“Suggestions are made for qualitative follow-up studies”
AI-generated, human-governed. Something look off? Contact us to request a review.