evidence was obtained of an OA Citation advantage but not of an OA Publication Advantage
matched observational comparisons do not isolate OA effects, and publication-analysis counts conflict across Tables 2 and 3
the data in DOAJ in a field labelled “the first calendar year a journal provided online Open Access content” differed in about 40 per cent of cases from the year indicated by the publishers in a verification round
the reported verification round found 42% disagreement among 141 responding journals
Derived from the full evaluation — not a separate score.
Strengths
The citation-growth pattern is examined separately in DOAJ- and OAD-based flipping samples with limited overlap, reducing dependence on a single OA directory.
↳ Data, DOAJ-based and OAD-based approaches, pp. 5–6; Table 3, pp. 15–16
Each study journal is compared with a discipline-and-year control and a tailored control that additionally accounts for national orientation.
↳ Definition of control groups, pp. 8–9
The review includes positive, null, mixed, and critical evidence, while the limitations identify Scopus coverage, language and field biases, and residual hybrid-OA contamination.
↳ Earlier studies, pp. 3–4; Limitations, p. 20
Limitations
Table 2 reports 113 usable DOAJ study journals for short-term publication growth, but Table 3 reports 116 paired observations for the corresponding comparison. This directly weakens confidence in the publication-output null.
↳ Tables 2 and 3, pp. 15–16
The paper acknowledges that flipping may coincide with restyling, a language switch, scope changes, or editorial-board replacement, but these factors are absent from the comparative design.
↳ Suggestions for further research, pp. 20–21
Only about 15% of DOAJ journals are represented in Scopus, and the paper states that results based on Web of Science may not be similar.
↳ Limitations, p. 20
The two-source, two-control-group design meaningfully improves on simple before-and-after comparisons and consistently identifies faster citation-impact growth in the sampled flipping journals. However, the analysis cannot separate access-modality changes from contemporaneous changes in language, scope, styling, or editorial leadership identified in the future-research section. The publication-output conclusion is additionally weakened by the unreconciled difference between the usable DOAJ counts in Tables 2 and 3. The paper is therefore most informative as comparative observational evidence rather than as a causal or decision-ready estimate of the effect of flipping.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Sound3.5
Confidence mediumThe study meaningfully extends prior flipping research through two independently assembled journal sets, verified switch years for one set, two control definitions, and geographic decomposition. The advance remains incremental because it does not resolve the causal mechanism behind the observed citation pattern.
“two well-defined control group were created, enabling paired statistical tests”
The paired pre/post design, dual control groups, field-normalised impact measure, and explicit treatment of backlog timing support comparative inference. Selection into flipping, unmeasured contemporaneous journal changes, variable-specific exclusions, and the Table 2–3 count conflict introduce meaningful uncertainty.
“the change in access modality may at the same time be accompanied by other changes”
The paper follows a clear sequence from data construction and indicators to paired comparisons, limitations, and conclusions. Interpretation is impeded by the unreconciled analytic counts and by describing observational comparative evidence as “robust evidence” of an advantage.
“This outcome provides robust evidence”
The literature review covers positive, null, mixed, and methodologically critical findings, while the limitations identify Scopus coverage, language and field bias, and residual hybrid-OA contamination. Concurrent editorial changes are deferred to future research rather than fully integrated into the certainty of the central interpretation.
“The current study is purely statistical”
Concerns4 of 4 checks
An unreconciled analytic-count conflict affects the reported publication-output analysis. A likely mismatch between an in-text citation and the listed McKiernan reference is also present but is secondary to the table inconsistency.
No ethics approval was required for this bibliometric study, and no misconduct or suspicious availability claim is evident. An explicit conflict disclosure is absent despite SCImago affiliations and reliance on SCImago-derived bibliometric data.
Flags: 0 declared / 5 total
23 of 23 checkable references verified
36 references in manuscript 13 have no canonical index record — counted, but not index-checkable 2 references confirmed by manual review
References not verified automatically are resolved manually within 24 hours.
No retraction or expression of concern found in Crossref (includes Retraction Watch).
Sources: Crossref (incl. Retraction Watch) ✓
Where this paper’s evidence sits on the path from initial observation to real-world use.
The analysis supplies real-world comparative benchmarks that can inform discussion of journal conversion. It does not support a prescriptive decision rule because contextual mechanisms and concurrent journal changes require qualitative follow-up.
“Suggestions are made for qualitative follow-up studies”
AI-generated, human-governed. Something look off? Contact us to request a review.
Keep reading
Other papers Nabu has evaluated blind, starting with the same field.
Baptiste Scancar, Jennifer Anne Byrne, David Causeur, et al. · 2026 · Media and communications
These findings suggest that paper mills inflate the citation metrics of supported publications and affected journals at scale.
TentativeChao Ma, Yiwei Li, Feng Guo, et al. · 2019 · Economics
papers published between October and December on average get as much as 18.5% fewer citations than those published in the other months in the year
TentativeMasashi Kawai, Koichi Ojiro, Makoto Asaoka, et al. · 2022 · Other social sciences
日本の機関リポジトリによるオープンアクセス論文の数は限られるものの，その被引用数と様々なタイプのオープンアクセス論文の被引用数の比較からは，日本の機関リポジトリによるオープンアクセスのメリットは確認できなかった．
Tentative