# Who is using AI to code? Global diffusion and impact of generative AI

- **DOI:** [10.1126/science.adz9311](https://doi.org/10.1126/science.adz9311)
- **Rating:** Sound (3.7 out of 5)
- **Verdict:** A strong behavioral measurement study of AI coding diffusion, constrained by observational productivity identification in open-source Python.
- **Framework version:** v5.0
- **Pipeline version:** v0.4
- **Evaluation date:** 2026-07-18
- **Evaluation page:** https://www.nabu.science/paper/10.1126%2Fscience.adz9311

## Key claims

1. Today, AI writes an estimated 29% of Python functions in the US
   - Support: Supported
   - Type: Primary finding
   - Basis: large-scale detector estimate with misclassification correction, limited by detector generalization and GitHub-Python scope
   - Location: Abstract; Results, Figure 2A; SI S1.3
2. quarterly output, measured in online code contributions, has increased by 3.6% because of this
   - Support: Tentative
   - Type: Secondary
   - Basis: fixed-effects association supports direction, but causal wording exceeds the observational design
   - Location: Abstract; Results, Figure 3C; Table S7; SI S1.5
3. AI seems to benefit experienced, senior-level developers
   - Support: Supported
   - Type: Secondary
   - Basis: interaction estimates consistently favor experienced users across commit and library outcomes
   - Location: Abstract; Results, Figure 3D; SI S4.3; Table S9
4. early-career developers showed no significant benefits from AI adoption
   - Support: Supported
   - Type: Secondary
   - Basis: subgroup estimates are non-significant, though null effects depend on power and model specification
   - Location: Abstract; Results, Figure 3D; SI S4.3; Table S9
5. genAI helps programmers expand their capabilities to new domains of software development
   - Support: Supported
   - Type: Secondary
   - Basis: library-combination proxies and robustness checks support domain expansion, but the construct is indirect
   - Location: Results, Figure 3C; SI S4.1; Table S7

## Strengths

- **Behavioral measurement at unusual scale** The study applies a custom AI-code detector to 5 million functions from 31 million Python commits, moving beyond self-reported AI use or tool-access proxies. This gives the paper a distinctive empirical basis for tracking diffusion over time and across countries.
  - Dimension: contribution
  - Location: Figure 1; Results, Detecting AI generated code; Figure 2
- **Extensive validation and robustness checks** The classifier is evaluated on held-out data, WildChat interactions, newer LLM-generated code, and retraining scenarios, while the productivity regressions are accompanied by placebo tests and measurement-error analysis. These checks directly address several plausible threats to the adoption and productivity estimates.
  - Dimension: methodological rigour
  - Location: SI S2.1–S2.2; SI S4.2–S4.5; Tables S7–S9; Figure S6
- **Policy-relevant heterogeneity in benefits** The paper finds that productivity and exploration gains are concentrated among experienced developers, while less experienced developers show no statistically significant productivity benefit. This result gives the work practical relevance for workforce development and inequality debates.
  - Dimension: application pathway
  - Location: Results, Figure 3D; SI S4.3; Discussion

## Limitations

- **Observational design under causal wording** The productivity estimates come from user and quarter fixed-effects regressions with lagged AI-use measures, not from randomized or quasi-random assignment. The article acknowledges this in the SI, but phrases such as 'because of this' and 'substantially increases output' overstate the causal identification.
  - Dimension: methodological rigour
  - Location: Abstract; Results, Figure 3C; Discussion; SI S1.5
- **Detector performance remains context-dependent** The detector performs strongly on held-out synthetic data, but true-positive probabilities are lower on WildChat code and weaker for some newer models before retraining. Because adoption and productivity estimates depend on this detector, residual measurement uncertainty remains material.
  - Dimension: methodological rigour
  - Location: SI S2.1–S2.2; Figure S4; Figure S5; Table S4
- **External validity is materially bounded** The study focuses on open-source Python contributions on GitHub and covers six major countries, leaving uncertainty about other languages, proprietary enterprise codebases, lower-income countries, and China-specific platform substitution. The authors discuss these limits, but they constrain direct generalization.
  - Dimension: application pathway
  - Location: Discussion, Limitations; Figure 2B
- **Replication link is not verifiable** The paper states that code and data are available, but the supplied article text provides only 'this link' rather than a working repository or persistent identifier. This prevents independent verification of the replication package from the provided document.
  - Dimension: methodological rigour
  - Location: Data and materials availability; SI Materials and Methods, Data

## Reliability checks

Caveats (4 of 4 checks).

- **Internal coherence:** Noted. Core numerical claims are broadly consistent, including the 29% adoption estimate and 3.6% productivity calculation, but the paper does not reconcile different developer-count figures in one place. The 28.6% versus 29% distinction is explicitly explained in SI S1.5.
- **Research conduct:** Noted. The article states that code and data are available, but the supplied text gives only 'this link' rather than a verifiable repository. Public GitHub data, self-reported location, and name-based gender inference are described, with limitations discussed, but no ethics or COI declaration is evident in the supplied text.
- **Reference integrity:** Clean. All 49 cited references resolved to real publications via PubMed, Crossref, or OpenAlex.
  - References not verified automatically are resolved manually within 24 hours.

## Post-publication record

- **Status:** Clean
- **Summary:** No retraction or expression of concern found in Crossref (includes Retraction Watch).
- **Last checked:** 2026-10-17
