PURPOSE: To examine whether National Institutes of Health (NIH) funded articles that were archived in PubMed Central (PMC) after the release of the 2008 NIH Public Access Policy show greater scholarly impact than comparable articles not archived in PMC.
METHODS: A list of journals across several subject areas was developed from which to collect article citation data. Citation information and cited reference counts of the articles published in 2006 and 2009 from 122 journals were obtained from the Scopus database. The articles were separated into categories of NIH funded, non-NIH funded and whether they were deposited in PubMed Central. An analysis of citation data across a five-year timespan was performed on this set of articles.
RESULTS: A total of 45,716 articles were examined, including 7,960 with NIH-funding. An analysis of the number of times these articles were cited found that NIH-funded 2006 articles in PMC were not cited significantly more than NIH-funded non-PMC articles. However, 2009 NIH funded articles in PMC were cited 26% more than 2009 NIH funded articles not in PMC, 5 years after publication. This result is highly significant even after controlling for journal (as a proxy of article quality and topic).
CONCLUSION: Our analysis suggests that factors occurring between 2006 and 2009 produced a subsequent boost in scholarly impact of PubMed Central. The 2008 Public Access Policy is likely to be one such factor, but others may have contributed as well (e.g., growing size and visibility of PMC, increasing availability of full-text linkouts from PubMed, and indexing of PMC articles by Google Scholar).
2009 NIH funded articles in PMC were cited 26% more than 2009 NIH funded articles not in PMC, 5 years after publication.
reported means imply 21.9%, while residual article-level differences limit interpretation
NIH-funded 2006 articles in PMC were not cited significantly more than NIH-funded non-PMC articles.
the reported null tests support the comparison within the selected journal sample
This result is highly significant even after controlling for journal (as a proxy of article quality and topic).
journal-blocked analysis supports the association but not control of article-level quality
Our analysis suggests that factors occurring between 2006 and 2009 produced a subsequent boost in scholarly impact of PubMed Central. The 2008 Public Access Policy is likely to be one such factor, but others may have contributed as well.
the before-after pattern cannot distinguish policy effects from concurrent discovery changes
This study illustrates a positive impact of the 2008 NIH Public Access Policy.
causal policy impact is not identified by the observational comparison
Derived from the full evaluation — not a separate score.
Strengths
The 2006 and 2009 cohorts bracket implementation of the NIH policy, while within-journal analysis reduces differences in journal visibility, topic, and review standards.
↳ Literature Review; Methods; Results, Analysis of 2006 and 2009 Articles
The paper reports no significant PMC citation difference for 2006 and a significant association for 2009, rather than presenting only the policy-consistent result.
↳ Results, Analysis of 2006 Articles; Results, Analysis of 2009 Articles
After finding that PMC papers had more authors, the study reports a multiple regression holding author count constant and retains a significant PMC association.
↳ Limitations, paragraph 2
Limitations
The Abstract and Results call the difference 26%, but means of 29.14 and 23.91 imply approximately 21.9%. This affects the paper's headline quantitative claim, though not the direction of the association.
↳ Abstract, Results; Results, Analysis of 2009 Articles
The Conclusion attributes a positive impact to the 2008 policy even though the observational comparison cannot exclude residual article differences or contemporaneous changes in PMC and Google Scholar visibility.
↳ Discussion; Limitations; Conclusion
The sample is drawn from selected high-impact and core journals, excludes open-access titles, and does not support discipline-specific estimates.
↳ Methods, journal selection and exclusions; Limitations, final paragraph
The 2006/2009 comparison, within-journal blocking, and author-count regression make the descriptive PMC association informative and place the paper above a simple uncontrolled citation comparison. The Limitations section nevertheless acknowledges undefined intrinsic differences between deposited and non-deposited articles, leaving the central policy attribution unresolved. The final Conclusion therefore carries more causal certainty than the Abstract and Discussion, which identify Google Scholar indexing, PMC visibility, and other contemporaneous changes. The restricted journal frame and absence of discipline-specific analysis also limit application beyond the sampled biomedical titles.
Nabu’s assessment, alongside the field’s view.
Are you an author of this paper?
Sound3.0
Confidence highThe pre/post-policy comparison across 122 journals provides a meaningful incremental contribution to a mixed open-access citation literature. Its advance is constrained by the inaccurate headline percentage and inability to isolate the policy from other changes.
The findings of these studies, however, have been mixed.
Within-journal blocking and the author-count regression address identifiable confounders, but undefined differences between deposited and non-deposited articles remain. The final causal policy attribution therefore exceeds what the observational design identifies.
there is likely to be some intrinsic difference(s) between 2009 NIH funded articles deposited vs. not deposited in PMC
The paper presents its research question, methods, null 2006 result, and positive 2009 association in a readily traceable sequence. Reporting is reduced because the Conclusion drops the qualifications and alternative explanations developed in the Abstract and Discussion.
The 2008 Public Access Policy is likely to be one such factor, but others may have contributed as well
The Literature Review fairly presents mixed findings and engages Craig et al.'s concerns about timing and selection bias. However, the acknowledged residual group differences are not carried into the final assertion of a positive policy impact.
This study illustrates a positive impact of the 2008 NIH Public Access Policy.
Lower confidence on Contribution, Positioning — domain match limited.
Caveats4 of 4 checks
The qualitative pattern is internally stable, but the paper contains a numerical discrepancy in its headline percentage and a smaller effect-size inconsistency. Its final causal interpretation is also stronger than the qualified account elsewhere in the paper.
The study uses bibliometric data and does not present a human-subject ethics issue or a suspicious research-conduct claim. Missing conflict, data, code, and preregistration statements do not establish misconduct for this study type and publication era.
Flags: 0 declared / 5 total
14 of 14 checkable references verified
28 references in manuscript 14 are books, websites or datasets — counted, but not index-checkable
References not verified automatically are resolved manually within 24 hours.
No retraction or expression of concern found in Crossref (includes Retraction Watch).
Sources: Crossref (incl. Retraction Watch) ✓
Where this paper’s evidence sits on the path from initial observation to real-world use.
The analysis uses citation outcomes observed under an implemented public-access mandate, giving it more practical relevance than a laboratory or hypothetical study. Residual confounding prevents it from serving as a validated causal policy evaluation.
AI-generated, human-governed. Something look off? Contact us to request a review.
Keep reading
Other papers Nabu has evaluated blind, starting with the same field.
Nicholas Fraser, Fakhri Momeni, Philipp Mayr, et al. · 2020 · Other social sciences
bioRxiv-deposited journal articles had sizably higher citation and altmetric counts compared to nondeposited articles.
SupportedMcGargill, Liu, Kuhns, et al. · 2026 · Other social sciences
Reviewers effectively evaluated Quality and Impact as separate dimensions, with Quality scores being more consistent across reviewers than Impact scores.
TentativeNoémie Aubert Bonn, Andrew J. Stewart, Lukas Hughes-Noehrer · 2026 · Other social sciences
Lack of time and resources was the most frequently reported barrier.
Supported