AI adoption
Different studies show why AI activity and net productivity need to be measured separately. In METR’s early-2025 randomised trial, 16 experienced open-source developers completed 246 real tasks. Access to AI made them 19% slower, although they believed afterwards that it had made them about 20% faster. This is a finding about those tools, developers and tasks.
Anthropic’s June 2026 analysis of around 400,000 Claude Code sessions found that sessions rated novice reached its strictest verified-success measure 15% of the time, compared with 28–33% for intermediate-to-expert sessions. Expertise and success were inferred from session records; these figures do not directly measure real-world productivity or a person’s ability to check work.
Our perspective
Our reading is that faster drafting or coding does not remove the need to evaluate the result. Before reducing production capacity, examine the effort required for review, correction and approval. Measure the complete task, and ensure someone has enough subject knowledge to assess the output. These studies do not all test the same mechanism, so they should inform questions about your workflow rather than support a universal productivity claim.
METR’s sample is small and reflects early-2025 tools and conditions. Anthropic studies its own product and uses transcript-based estimates of expertise and success. The studies do not demonstrate that only experts can verify AI work well, or that all time saved is transferred to checking.
Sources
METR — Measuring the Impact of Early-2025 AI on Experienced Open-Source Developer Productivity, July 2025 · Anthropic — Agentic coding and persistent returns to expertise, June 2026.
Explore further
Guide
AI assessment: understand the work before choosing technology
Le Signal Spentia
Three dated, sourced facts, an article and a podcast. Delivered in French every Monday.