BACKGROUND: In the past few years there has been an ongoing debate as to whether the proliferation of open access (OA) publishing would damage the peer review system and put the quality of scientific journal publishing at risk. Our aim was to inform this debate by comparing the scientific impact of OA journals with subscription journals, controlling for journal age, the country of the publisher, discipline and (for OA publishers) their business model.
METHODS: The 2-year impact factors (the average number of citations to the articles in a journal) were used as a proxy for scientific impact. The Directory of Open Access Journals (DOAJ) was used to identify OA journals as well as their business model. Journal age and discipline were obtained from the Ulrich's periodicals directory. Comparisons were performed on the journal level as well as on the article level where the results were weighted by the number of articles published in a journal. A total of 610 OA journals were compared with 7,609 subscription journals using Web of Science citation data while an overlapping set of 1,327 OA journals were compared with 11,124 subscription journals using Scopus data.
RESULTS: Overall, average citation rates, both unweighted and weighted for the number of articles per journal, were about 30% higher for subscription journals. However, after controlling for discipline (medicine and health versus other), age of the journal (three time periods) and the location of the publisher (four largest publishing countries versus other countries) the differences largely disappeared in most subcategories except for journals that had been launched prior to 1996. OA journals that fund publishing with article processing charges (APCs) are on average cited more than other OA journals. In medicine and health, OA journals founded in the last 10 years are receiving about as many citations as subscription journals launched during the same period.
CONCLUSIONS: Our results indicate that OA journals indexed in Web of Science and/or Scopus are approaching the same scientific impact and quality as subscription journals, particularly in biomedicine and for journals funded by article processing charges.
after controlling for discipline, age of the journal and the location of the publisher the differences largely disappeared in most subcategories except for journals that had been launched prior to 1996
broad stratification supports convergence in many subgroups but does not provide full joint adjustment
Overall, average citation rates, both unweighted and weighted for the number of articles per journal, were about 30% higher for subscription journals.
direct descriptive comparison is consistent with the reported JCR and Scopus ratios
OA journals that fund publishing with article processing charges (APCs) are on average cited more than other OA journals.
direct within-OA descriptive comparison is reported across the revenue-model analysis
In medicine and health, OA journals founded in the last 10 years are receiving about as many citations as subscription journals launched during the same period.
the age-by-discipline comparison directly supports the bounded subgroup finding
Derived from the full evaluation — not a separate score.
Strengths
The analysis combines Ulrichsweb, JCR, SCImago, and DOAJ data, comparing 610 OA journals with 7,609 subscription journals in JCR and larger overlapping Scopus samples.
↳ Abstract, Methods; Methods, merged dataset paragraph
The Methods identify incomplete 2011 SCImago article and citation counts, explain the assumed effect on relative comparisons, and reserve detailed presentation for JCR results.
↳ Methods, SCImago data-problem paragraphs; Results, Table 1 discussion
The literature review discusses both reported OA citation advantages and Davis’s randomized trial finding no citation effect, rather than presenting the field as settled.
↳ Background, Earlier Studies
Limitations
Only journals included in Web of Science or Scopus could enter the citation analysis. The paper acknowledges this restriction but does not examine how selective indexing may alter OA-versus-subscription comparisons.
↳ Methods, database coverage limitation
Age, geography, and discipline are examined in broad subgroup combinations rather than a fully joint adjusted model. Their correlated influence therefore remains incompletely separated.
↳ Methods, citation-metric analysis; Results, Figures 1–2
The Discussion states that funding mechanism is irrelevant to journal quality, although older OA journals and some non-medical subgroups retain citation gaps. The final recommendation also extends beyond the indexed corpus.
↳ Results, Figure 2 discussion; Discussion, paragraphs 4–6; Conclusions
The large linked dataset and article-weighted comparisons provide a useful extension of earlier journal-level studies. Figures 1 and 2 support a qualified finding that citation differences narrow in several newer, geographic, and medical subgroups, while Table 1 documents the larger unadjusted gap. The score is moderated because index inclusion is selective and the broad stratifications do not jointly separate all correlated determinants of citation rates. The Discussion and Conclusions also generalize beyond the bounded indexed-journal evidence when treating funding model as irrelevant to quality and advising author choice.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Sound3.3
Confidence mediumThe study meaningfully extends earlier OA-versus-subscription comparisons through larger JCR and Scopus samples, article-weighted estimates, and stratification by age, geography, discipline, and APC model. The advance remains incremental because related comparisons predated it and the analysis uses broad subgrouping.
“We feel this adds a new and important dimension to the analysis as compared to earlier studies.”
The descriptive design uses multiple databases, journal- and article-weighted comparisons, and transparent handling of incomplete Scopus data. Restriction to indexed journals is acknowledged but its selection consequences are not traced, and the stratifications do not jointly adjust all identified factors.
“A limitation of this method is that journals not indexed in Web of Science or Scopus cannot be included”
The paper is coherently structured and the abstract preserves important age, discipline, and geography qualifiers. The Discussion and Conclusions broaden those bounded citation findings into claims about journal quality and author choice, creating a material language-evidence mismatch.
“Today the funding mechanism of a journal is irrelevant in considering its quality.”
The paper substantively engages mixed prior evidence, including a randomized study reporting no citation effect, and discloses its database and conflict-of-interest limitations. The index-selection limitation is not carried consistently into the broad concluding recommendation about OA journals.
“there is no reason for authors not to choose to publish in OA journals just because of the ‘OA’ label”
Caveats4 of 4 checks
The reported samples and headline comparisons are internally coherent, and the incomplete SCImago data are disclosed. Relative Scopus comparisons nevertheless depend on the untested assumption that incomplete counting affected OA and subscription journals similarly.
Directly relevant OA publishing affiliations are disclosed, with no indication that they were concealed. No human-participant ethics declaration was required for this bibliometric database study.
Flags: 1 declared / 5 total
12 of 12 checkable references verified
20 references in manuscript 8 have no canonical index record — counted, but not index-checkable 4 references confirmed by manual review
References not verified automatically are resolved manually within 24 hours.
No retraction or expression of concern found in Crossref (includes Retraction Watch).
Sources: Crossref (incl. Retraction Watch) ✓
Where this paper’s evidence sits on the path from initial observation to real-world use.
The analysis uses citation records from journals operating in real publishing conditions, so its findings can inform policy discussion. It does not provide an operational assessment tool, implementation protocol, or journal-level decision rule.
AI-generated, human-governed. Something look off? Contact us to request a review.
Keep reading
Other papers Nabu has evaluated blind, starting with the same field.
Baptiste Scancar, Jennifer Anne Byrne, David Causeur, et al. · 2026 · Media and communications
These findings suggest that paper mills inflate the citation metrics of supported publications and affected journals at scale.
TentativeChao Ma, Yiwei Li, Feng Guo, et al. · 2019 · Economics
papers published between October and December on average get as much as 18.5% fewer citations than those published in the other months in the year
TentativeMasashi Kawai, Koichi Ojiro, Makoto Asaoka, et al. · 2022 · Other social sciences
日本の機関リポジトリによるオープンアクセス論文の数は限られるものの，その被引用数と様々なタイプのオープンアクセス論文の被引用数の比較からは，日本の機関リポジトリによるオープンアクセスのメリットは確認できなかった．
Tentative