Artificial Intelligence1 min read

The AI cost of JavaScript frameworks

By · Published by Everything Blog

In short

Software engineers frequently assert that specific web ecosystems enable automated tools to produce faster, cleaner code without offering supporting data. To test this assumption, a new benchmark requires the Opus 5.5 model to build a standardized Pokémon application across different frameworks in sandboxed environments. The initiative measures token consumption, lines of code, completion time, and Lighthouse scores while accounting for the stochastic behavior of language models. Evaluation relies on the accessibility tree rather than raw DOM elements to verify application functionality fairly across different tech stacks.

Key points

  • Developers currently lack empirical evidence to support claims that specific frameworks o…: Developers currently lack empirical evidence to support claims that specific frameworks optimize code generated by artificial intelligence.
  • The new testing suite tracks resource usage by measuring total token consumption, develop…: The new testing suite tracks resource usage by measuring total token consumption, development duration, and resulting Lighthouse scores.
  • Benchmark reliability must overcome testing challenges such as stochastic model behavior…: Benchmark reliability must overcome testing challenges such as stochastic model behavior and variance between independent Lighthouse audit runs.

Sources checked by our analysis pipeline: 1

One of my pet peeves with the web development community is that so many of the decisions we make are vibes-based. Frameworks are “blazing fast,” tools will “supercharge” your development, the right abstraction will benefit both you and your users (DX and UX all in one!). All of this breathless boasting is usually offered with zero evidence. Alex Russell has called this “frontend’s lost decade”, Baldur Bjarnason has called programming a pop culture, and I’ve tried to counter this trend in my blog with facts and benchmarks (at my best, anyway).

For a long time it seemed like our community’s allergy to evidence was driven by the desire for greater DX. In other words, devs just liked the way it felt to type JSX in React, and the arguments promoting React or related tools (“the virtual DOM is fast!”) were largely post-hoc rationalizations to justify those initial gut feelings. (Jonathan Haidt would likely agree with this assessment.)

But in the agentic era, when developers are no longer typing code by hand and are thus less constrained by DX, I still…

Artificial Intelligence