In a three-month experiment with 133 U.S. patent lawyers, access to an AI writing assistant improved the quality of their patent drafts, with the largest gains among junior lawyers. But on a later task completed without AI, senior lawyers who had been using the tool outperformed senior lawyers who had not, while junior lawyers showed no average advantage. The findings were released as a National Bureau of Economic Research working paper.
Artificial intelligence (AI) is increasingly used in professional fields such as law, medicine, engineering, and consulting, where specialized knowledge and expert judgment are essential. Traditionally, professionals have developed their expertise through practical experience, performing tasks under the supervision of more experienced colleagues. AI tools can improve productivity by helping people complete tasks faster, produce higher-quality work, and perform activities that would otherwise require greater expertise.
Research suggests that these benefits are often particularly pronounced among less experienced workers, reducing performance differences between beginners and experts. However, improved performance with AI assistance does not necessarily mean that users are developing the skills needed to perform the same tasks independently.
Some studies have found that people who rely heavily on AI develop weaker problem-solving, programming, or diagnostic skills than those who work without such assistance. Other studies have demonstrated that AI can support learning and improve people’s ability to perform tasks independently after AI assistance is removed. An important factor appears to be how AI is used, as people who actively evaluate, interpret, and verify AI-generated outputs tend to learn more than those who simply accept its answers.
Study author David Autor of the Massachusetts Institute of Technology and colleagues at Google set out to examine how sustained AI use shapes the quality of lawyers’ work, their productivity, and their expert judgment. They conducted a preregistered randomized controlled trial, tracking practicing lawyers for three months and comparing those given AI assistance with those who worked without it.
The researchers began with 24 intellectual property law firms that do regular, nonexclusive business with Google, and 11 agreed to take part. Two groups of lawyers from one firm participated two months apart and were analyzed separately, giving the study 12 firm groups ranging from 3 to 49 recruited lawyers each. Participants were paid their usual hourly rate for the time they spent on study tasks.
Within each firm, the researchers randomly assigned about two-thirds of participants to receive AI assistance and the remaining third to a control group without it. Of the 156 lawyers put forward by the firms, 133 completed onboarding and were randomized: 90 to the AI group and 43 to the control group. Some dropped out over the course of the study, and 91 completed all of the tasks.
Lawyers in the AI group received access to an unreleased writing tool developed by Google Labs and designed to assist with professional writing. The tool drew on user-provided context, style guidance, and writing goals, and its main features included modifying text based on prompts, giving feedback from multiple simulated editorial perspectives, and checking documents for internal inconsistencies. Lawyers in the control group used standard legal tools but were not given access to generative AI writing tools.
Ten days after the AI group gained access to the tool, all participants completed a patent drafting task. Working from a main claim describing an invention, along with figures and notes, they wrote five to seven dependent claims, which build on the main claim, and a detailed description of the invention. They were asked to spend no more than two hours on the task and to keep the write-up within 1,000 words.
At 90 days, participants completed a second, more challenging drafting task and a redlining task, in which they reviewed and marked up an existing AI-generated patent draft. Participants were instructed not to use AI on the redlining task, which the researchers chose because it draws heavily on expert judgment. Blinded patent attorneys at an independent law firm scored all of the work on enforceability, technical accuracy, strategic ambiguity (using flexible claim language to maximize the scope of protection), completeness, and clarity.
Lawyers with AI access produced better drafts at both 10 days and 90 days, outperforming the control group by about a third of a standard deviation, a moderate improvement. The gains came mainly from fewer weak drafts rather than more excellent ones. The gains appeared larger for junior lawyers (those with fewer than seven years of experience) than for senior lawyers, though the difference between the two groups was not statistically significant. AI users also finished the first drafting task about 10 minutes faster, on average, than the control group, whose members took about 110 minutes.
On the redlining task, which both groups completed without AI, lawyers who had spent three months using the tool again outperformed the control group by a moderate margin. But this advantage came entirely from senior lawyers. Junior lawyers who had used AI performed no better on average than junior lawyers in the control group.
Junior lawyers’ scores did shift, however. Those in the AI group had far fewer mediocre scores than control juniors, but more poor scores and more good ones, with no increase in top scores. The researchers suggest that AI assistance may have served as a springboard for some juniors and a cushion for others, though the study could not determine why.
In other words, the lawyers who benefited most from AI while using it showed no average gain in their ability to work without it. Junior lawyers also believed the tool was improving their work, a belief that held for their AI-assisted drafts but not for their unassisted redlining.
The researchers noted that junior lawyers using AI worked faster, while senior lawyers drafted better but barely faster. This pattern is consistent with the idea that the time and effort spent on a task matter for learning, though the study did not test this directly.
“Foundational expertise may be a prerequisite for extracting durable skill from AI-assisted practice,” the study authors concluded.
The study contributes to the scientific understanding of how AI assistance affects professional expertise. However, it focused only on patent lawyers, and the effects in other fields and on other types of professional tasks might differ. The sample was also small, especially the junior group, and was drawn from firms that do ongoing patent work for a sophisticated client, so it may not be representative of professional services generally.
The researchers were unable to give participants a skills test before the study began, so they relied on random assignment to infer that differences at the end reflect skill acquisition. They also could not directly verify that participants avoided AI on the redlining task. An audit flagged 15 of 91 submissions for possible AI use; excluding them left the senior lawyers’ advantage in place, though somewhat weaker, while the overall group difference was no longer statistically significant.
Additionally, three months is brief relative to the years it takes to build patent expertise. The paper has not yet been peer-reviewed. The study was conducted and funded by Google, which developed the AI tool, and six of the seven authors work at Google.
The paper, “Does AI Assistance Enhance or Erode Expertise? Evidence from a Three-Month Field Experiment in Patent Drafting,” was authored by David Autor, Tanya Rodchenko, Josh Martin, Zanna Iscenko, Scott Strand, David Pearl, and Melissa Ferere.