nabu.science
MethodologyAboutContact

 

  • Snapshot
  • Key Claims
  • Strengths & Limitations
  • Commentary
  • From the Authors
  • Author Note / Field Feedback
  • Dimension breakdown
  • Impact Profile
Free · About 5 minutes
Report #NBU-59CE0B3B·Framework v5.9·Pipeline v0.4·Evaluated 29 Sept 2026·OECD: 1.2 Computer and information sciences·Subfield: Information Systems and Management·Basis: Full text·Type: Cross-sectional study

Free online availability substantially increases a paper's impact

Steve LawrenceSteve Lawrence·2001·Nature·10.1038/35079151 ↗

Primary claim

Free online availability substantially increases a paper’s impact

Tentative · causal wording is not identified by the cross-sectional comparison
In brief

A foundational large-scale association between online availability and citations whose causal framing exceeds what its cross-sectional design can establish.

ReliabilityCaveats3 of 4 checks completeRead the entry ↓
Unlikely to hold — the claim's weakest requirement is that the effect exists at all. Based on how the study was designed and conducted, not on statistics extracted from the paper. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.
Abstract

The volume of scientific literature far exceeds the ability of scientists to identify and use all relevant information. The ability to locate relevant research quickly will dramatically improve communication and scientific progress. Although availability varies greatly by discipline, more than a million research articles are now freely available on the web.

Key Claims

Primary finding · Tentative

Free online availability substantially increases a paper’s impact

Unlikely to hold — the claim's weakest requirement is that the effect exists at all. Based on how the study was designed and conducted, not on statistics extracted from the paper. Experimental. Every point below is traceable to a quoted passage, but this forecast has not yet been validated against known replication outcomes.

causal wording is not identified by the cross-sectional comparison

Title
Quote verified
Secondary · Well-supported

The mean number of citations to offline articles is 2.74, and the mean number of citations to online articles is 7.03, an increase of 157%.

directly reported descriptive means from the analysed article sample

Full text, headline results paragraph
Quote verified
Secondary · Supported

Averaging the percentage increase across 1,494 venues containing at least five offline and five online articles results in an average of 336% (median 158%) more citations to online computer-science articles

within-venue comparison supports the direction, though its percentage convention and confounding limit interpretation

Full text, paragraph beginning “We analysed differences within publication venues”
Quote verified
Secondary · Supported

Restricting our analysis to the top 20 publication venues by average citation rate gives an increase of 286% (median 284%) in the citation rate for online articles.

reported top-venue comparison tests venue quality but not article-level selection

Full text, paragraph beginning “If we assume”
Quote verified

Derived from the full evaluation — not a separate score.

Strengths · Limitations

Strengths

#contribution

Large early bibliometric evidence base

The analysis covers 119,924 conference articles and reports citation differences between online and offline papers, providing a substantial descriptive evidence base for the e-access debate.

↳ Full text, paragraph beginning “We analysed 119,924”

#methodological rigour

Within-venue comparison tests one alternative

Comparing online and offline papers within 1,494 publication venues partially reduces broad venue-level quality differences, with a further restriction to the top 20 venues.

↳ Full text, paragraphs beginning “We analysed differences within publication venues” and “If we assume”

#application pathway

Clear relevance to access decisions

The final paragraph identifies authors and publishers as actors who can make research easier to access, giving the findings a visible scholarly-communications pathway.

↳ Full text, final paragraph

Limitations

#methodological rigour

Causal framing exceeds observational evidence

The title states that free availability increases impact, but the reported analysis establishes correlation and makes causal interpretation conditional on an untested quality assumption.

↳ Title; full text, paragraph beginning “If we assume”

#methodological rigour

Selection and timing remain unresolved

The analysis does not separate accessibility effects from author self-selection, article-level prominence, or the possibility that citation accumulation preceded online detection.

↳ Full text, methods and within-venue analysis paragraphs

#application pathway

Evidence is narrowly field-specific

The sample is limited to computer-science and related conference articles, whose publication and indexing environment may not represent disciplines centered on journal literature.

↳ Full text, paragraph beginning “We analysed 119,924”

Does this match your read? Add your assessment ↓

Nabu Commentary

The large sample and within-venue comparison make the paper a meaningful early descriptive contribution to the e-access debate. The analysis nevertheless remains cross-sectional: online status, citation accumulation, author self-selection, and article-level prominence are not separated. That limitation is decisive because the title and discussion attribute higher citation rates to easier availability rather than retaining associational language. The presentation is concise and the numerical comparisons are easy to locate, but the single-field setting and unresolved causal identification constrain translation and transferability.

Evaluate another paper

Free · About 5 minutes · Research preview

Author Note / Field Feedback

Nabu’s assessment, alongside the field’s view.

Are you an author of this paper?

Dimension breakdown

Evidence Score

Out of 5
012345
2.7±0.1

Contribution
3.6

The analysis adds a large early empirical comparison of citation rates across 119,924 computer-science conference articles, including within-venue and top-venue summaries. Its contribution is meaningful as descriptive evidence but does not resolve whether access itself causes higher citation counts.

Full text, paragraphs beginning “We analysed 119,924” and “We analysed differences within publication venues”

“The mean number of citations to offline articles is 2.74, and the mean number of citations to online articles is 7.03”

Methodological Rigour
2.0

Within-venue comparisons partially address venue-level differences, but author self-selection, article-level quality, and exposure-outcome ordering remain uncontrolled. The title and discussion then attribute the citation difference to easier availability, activating stacked serious methodological ceilings.

Title; full text, paragraphs beginning “We analysed differences within publication venues” and “If we assume”

“the analysis by venue suggests that online articles are more highly cited because of their easier availability”

Reporting
3.2

The letter moves concisely from data sources to pooled, venue-level, and top-venue results, and it states its similar-quality assumption explicitly. Precision is reduced because the title converts the observed association into an unconditional causal conclusion.

Title; full text, paragraph beginning “If we assume”

“Free online availability substantially increases a paper’s impact”

Positioning
2.8

The study is situated in the contemporary e-access debate and tests one stated quality-comparability assumption through a top-20-venue restriction. It does not discuss self-selection into online posting or reverse causation, alternatives decisive for the causal interpretation.

Full text, paragraphs beginning “If we assume” and “Free online availability facilitates”

“This assumption is likely to be more valid for top-tier conferences”

Lower confidence on Contribution, Positioning — domain match limited.

Reliability

Caveats3 of 4 checks


Internal CoherenceNoted

The descriptive citation figures are internally coherent, but the unconditional causal framing exceeds the observational analysis and its stated assumptions. The venue-level percentage convention is disclosed, although its implications for the mean are not examined.

Research ConductClean

The bibliometric study did not require human-participant ethics review, and the supplied text presents no location-referenced conduct concern. Named databases and acknowledged coverage limitations provide basic source transparency.

Flags: 0 declared / 5 total

Reference IntegrityNot checked

Reference integrity check could not run: no references section detected in extracted text.

0 references in manuscript

Post-Publication RecordClean

No retraction or expression of concern found in Crossref (includes Retraction Watch).

Last checked: 17 Oct 2026

Sources: Crossref (incl. Retraction Watch) ✓

Impact Profile

Impact areaPolicy and practice · SDG 9
Type of advanceEmpirical
Route to useNamed
AdvanceRestates
CitationsFWCI 87.9
Policy · patentNot yet traced
ReachedNot reachedNot assessable or unverified
ReadinessValidated
Foundational
Controlled
Piloted
Validated
Implementable
FoundationalImplementable

The correlational claim is supported by analysis of real-world citation data on 119,924 articles across 1,494 venues and multiple publication years.

From the paper
“To maximize impact, minimize redundancy and speed scientific progress, authors and publishers should aim to make research easy to access.”

The paper names the actors, authors and publishers, who should act on the findings to make research freely accessible, but not a specific decision setting.

Pathway

Authors, journal publishers and learned societies would change access practices, making research articles freely available online to increase dissemination and citation.

Barrier

The evidence is correlational, so unmeasured quality or self-selection differences between online and offline articles could confound the claimed causal effect.

Evaluated 13 Nov 2026 · GBD 2023 · glm_v1_3_3 · impact_profile_v1_3_3

Report an issue →

AI-generated, human-governed. Something look off? Contact us to request a review.

Keep reading

Related evaluations

Other papers Nabu has evaluated blind, starting with the same field.

  1. Scientific production in the era of large language models

    Keigo Kusumegi, Xinyu Yang, Paul Ginsparg, et al. · 2025 · Media and communications (incl. library and information sciences)

    Primary claim

    scientists adopting LLMs to draft manuscripts demonstrate a large increase in paper production, ranging from 23.7-89.3% depending on scientific field and author background

    Tentative
    Evidence Score
    2.8±0.2
    Reliability
    Caveats
  2. The relationship between bioRxiv preprints, citations and altmetrics

    Nicholas Fraser, Fakhri Momeni, Philipp Mayr, et al. · 2020 · Other social sciences

    Primary claim

    bioRxiv-deposited journal articles had sizably higher citation and altmetric counts compared to nondeposited articles.

    Supported
    Evidence Score
    3.8±0.1
    Reliability
    Caveats
  3. Life After Benchmark Saturation: A Case Study of CORE-Bench

    Nadgir, Kapoor, Liu, et al. · 2026 · Computer and information sciences

    Primary claim

    we find that despite accuracy saturation, CORE-Bench v1.1 remains useful for measuring efficiency, reliability, model performance, and scaffold performance

    Supported
    Evidence Score
    3.8±0.1
    Reliability
    Concerns

How the rubric works →