Does better work always mean better workers?
A Google RCT found senior lawyers gained real judgment from 3 months of AI use while juniors showed zero average skill gain.
Google Research ran a three-month randomized controlled trial with 133 patent lawyers across eleven firms, giving most early access to an AI patent-drafting tool while a control group waited. AI access reliably improved drafting quality for everyone, but when the tool was removed for an unassisted redlining task, only senior lawyers (7+ years) showed a durable judgment gain; junior lawyers showed no average improvement, with their scores instead splitting into more very-low and more good outcomes.
Why it matters
This is a rare, well-designed field experiment (not a lab study) isolating AI's effect on skill-building versus immediate output, showing that routine AI use can boost junior output today while leaving their unassisted judgment no better than before. That has direct implications for how firms structure apprenticeship-style work in any expertise-intensive profession, including software engineering, where junior staff increasingly lean on AI for tasks that used to build foundational skill.
What is verified
Google Research ran a three-month randomized controlled trial with 133 lawyers at eleven IP law firms, giving two-thirds early access to a then-unreleased Google Labs AI patent writing assistant (now part of Gemini Notebook) while the rest served as a control group. [1]
At 10 days, AI access raised drafting quality scores by 0.34 SD (about 10 percentile points); at 90 days this rose to 0.38 SD (about 11 percentile points), driven by fewer poor scores and more good scores, not more excellent ones. [1]
In a 90-day unassisted redlining task (no AI allowed), treated lawyers outperformed controls by 0.32 SD overall, but this was driven entirely by senior lawyers (7+ years experience), who outperformed controls by 0.45 SD (about 13 percentile points); junior lawyers showed no average improvement and their scores instead bifurcated into more very-low and more good scores. [1]
Qualitative analysis found junior lawyers in both treatment and control groups worked sequentially, over-focused on low-stakes copy-editing and surface synonym swaps, and often diagnosed flaws without executing fixes, a pattern the three months of AI use did not remediate. [1]
Senior lawyers who used the AI tool described treating its output as a 'logic auditor' that forced them to articulate the reasoning behind structural edits, sharpening their expertise. [1]
What is not
- The study covers only three months versus years normally needed to build durable expertise, so long-term skill trajectories for juniors remain unknown.
- Sample is limited to patent lawyers at firms with business ties to Google, which may limit generalizability to other professions or to software engineering specifically.
- No data is given on what mitigations (if any) could help juniors convert AI-assisted repetition into durable judgment.
- The underlying NBER paper is referenced but not included in evidence, so methodological details beyond this summary are not independently verifiable here.
The case for
The experiment used a credible design: sustained three-month real-world AI usage, a genuine unassisted evaluation task, and blinded expert grading across five quality dimensions, which the authors argue is rarely achieved together. The consistent senior-gain/junior-stagnation pattern across both the aggregate and qualitative editing-behavior analysis strengthens the core finding.
The case against
The sample is a narrow slice of patent lawyers at firms with pre-existing business relationships with Google, observed for only three months against a career-length skill-building process, so it's unclear how far this generalizes to other professions, including software engineering, or to longer time horizons as AI tools keep improving.
What to do with this
Treat this as a credible signal rather than a settled fact: it's strong enough to warrant auditing how junior staff use AI tools and building in unassisted skill checkpoints, but not strong enough to justify restricting junior AI access outright given the narrow sample and short time window.
Sources
- 1Google Research Blogresearch.google · Primary source
Original source published 7 October 2026 at 22:19.
An AI-assisted briefing by Coding Partners. We link the evidence behind the reporting so you can explore the original work and assess the claims. How CP Signal works.