Artificial Intelligence1 min read

Harvard researchers find AI coding agents shifted the bottleneck to code review

By · Published by Everything Blog

In short

Academic analysis indicates that while automated generation expanded initial code creation by roughly 30 percent, final finished output failed to demonstrate measurable improvement. Instead, the review phase lengthened by 49 percent as human professionals struggled to verify the expanded volume of material. The comprehensive investigation evaluated 725,938 employees across 718 enterprises by linking Jellyfish engineering logs with Revelio Labs records. The findings emphasize that organizations deploying artificial intelligence must balance drafting speed with human oversight constraints.

Key points

  • Generating materials faster with artificial intelligence does not translate into higher f…: Generating materials faster with artificial intelligence does not translate into higher finished output if human checkpoints face severe review backlogs.
  • Professional sectors such as legal services risk duplicating these engineering bottleneck…: Professional sectors such as legal services risk duplicating these engineering bottlenecks through generative workflows that place heavy verification demands on human practitioners.
  • Organizations evaluating new workplace automation must look beyond initial drafting speed…: Organizations evaluating new workplace automation must look beyond initial drafting speeds to rigorously measure revision rates, reviewer hours, and approved deliverables.

Sources checked by our analysis pipeline: 1

Editor’s Note: Engineers at firms that adopted AI coding agents added about 30 percent more code, yet finished work did not measurably rise and getting code through review took 49 percent longer, according to a working paper by Harvard Ph.D. candidates Fiona Chen and James Stratton covering 718 firms. The paper, a job-market paper with no indication of completed peer review, measures what happens at the human checkpoint once drafting speeds up. Harvey’s motion-to-dismiss workflow and Cooley’s redaction agent, still in development with Google Cloud, both keep a lawyer at a comparable checkpoint, and neither announcement reports a measure of that review stage.

For eDiscovery, privacy and compliance professionals, the study suggests a measurement approach for legal AI buyers: consider approved work product, revision rates and reviewer time alongside drafting output. It also shows that AI authorship can leave metadata in collaboration systems that records and legal hold programs may need to reach. Watch for a revised version of the paper and for any legal vendor that publishes review-stage figures, keeping in mind that the study examined software, not legal…

Artificial Intelligence