News · 2026-10-08

Google’s lawyer trial separates better AI-assisted work from better unaided judgment

A randomized trial of Google’s patent-drafting assistant improved lawyers’ AI-assisted drafts, but its later tool-free advantage was concentrated among experienced attorneys. Junior lawyers showed no average improvement on that unaided assessment, despite better assisted work. Google Research’s October 7 account highlights a practical distinction: a better deliverable with an assistant present is different evidence from stronger judgment after it is removed.

Key facts

• The trial randomized 133 attorneys; 91 completed the final unaided assessment.
• The intervention lasted three months across 11 participating patent-law firms.
• Senior attorneys’ unaided advantage was about 0.45 standard deviations; juniors had no mean gain.
• Primary sources are Google Research’s account and the full working paper.

Google Research frames the question as “Does better work always mean better workers?” The distinction is easy to miss in a workplace demonstration. A tool can correct a draft, suggest a strong opening or reduce errors without teaching its user how to recognize those problems independently. Conversely, a professional can learn from a suggestion even when the finished artifact does not look dramatically different.

David Autor, Tanya Rodchenko, Josh Martin, Zanna Iscenko, Scott Strand, David Pearl and Melissa Ferere studied a then-unreleased Google assistant called InFlow. Roughly two thirds of randomized participants received access for three months; controls received basic AI instruction without the generative writing tool during the intervention. Random assignment was stratified by firm and experience. Eleven firms participated out of 24 recruited firms with regular, non-exclusive Google relationships.

The assisted tasks asked lawyers to draft patent sections from invention materials at about ten days and again at ninety days. The later assignment was harder. Time was capped, and the materials came from a reserved benchmarking dataset. Independent patent attorneys, blinded to treatment assignment, scored the results on five dimensions: enforceability, technical accuracy, strategic ambiguity, completeness and alignment, and clarity. These are professional criteria rather than a model’s preference for polished prose.

With the tool available, treatment lawyers scored about a third of a standard deviation above controls on the first assessment and somewhat more on the second. The improvement appeared across rubric dimensions and reduced low-scoring work. Junior lawyers had larger estimated assisted gains, although the difference between junior and senior treatment effects was not statistically distinguishable from zero. The result supports an assisted quality benefit without proving that one experience group benefited more.

The later unaided task changed the question. All participants were asked to redline an AI-generated patent draft without AI, identifying and repairing defects. Senior lawyers with at least seven years of experience showed the clearer treatment-control advantage. Junior lawyers had no average gain, and their scores spread toward both stronger and weaker outcomes. That distribution is more informative than saying AI uniformly harmed or helped juniors.

The mechanism remains a hypothesis. In four interviews, senior participants described using the assistant as a “logic auditor,” pressure-testing arguments against their existing knowledge. Junior participants valued a faster start and moving sooner into review. Those accounts suggest that expertise can change how a tool’s output becomes feedback. Four selected interviews do not establish the causal explanation, and the experiment did not manipulate practice time to test whether fewer repetitions caused the pattern.

An analogy is a navigation system used by an experienced driver and a beginner. Both may reach the destination more reliably with directions. The experienced driver may compare the route with a known street network and learn a useful shortcut. The beginner may arrive without acquiring the same map. This analogy explains a possibility, not the study’s proven mechanism. The outcome measured here was patent redlining, rather than memory for streets or a general learning test.

Several limitations narrow the conclusion. Firms rejected a baseline unaided judgment test, so the researchers compared endline groups rather than measuring each person’s change from a starting score. The final assessment had 91 participants, not all 133 randomized attorneys. Participants were told to avoid AI, but the researchers could not directly monitor that rule. Three months is short relative to professional expertise formation, and there was no later assessment of persistence.

The paper is a working paper and discloses Google support for experiment costs and company affiliations among coauthors. Workplace productivity measures were exploratory, available for only 68 people and too imprecise to establish ordinary on-the-job efficiency gains. The results are stronger evidence about controlled task quality than about billable output, client value or litigation outcomes.

The implication is a measurement choice for employers and educators. Assess assisted performance and unaided capability separately, rather than treating one as a proxy for the other. The new lesson on randomized trials and treatment effects explains what random assignment can identify, while automation bias describes a separate risk of accepting suggestions too readily. The honest caveat is that this one tool, population and short horizon cannot settle whether AI improves or erodes professional learning generally.

Primary source, verified: read the paper →

Ground Truth. (2026, October 8). Google’s lawyer trial separates better AI-assisted work from better unaided judgment. Ground Truth. https://groundtruth.day/news/google-patent-lawyers-ai-output-versus-learning.html

@misc{groundtruth:google-patent-lawyers-ai-output-versus-learning,
  title  = {Google’s lawyer trial separates better AI-assisted work from better unaided judgment},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {oct},
  url    = {https://groundtruth.day/news/google-patent-lawyers-ai-output-versus-learning.html}
}

Topics: work · human-ai-collaboration · education · evaluation · causal-inference