Artificial Intelligence1 min read

Vibecoding an Inference Chip

By · Published by Everything Blog

In short

The automated setup improved machine learning inference speeds from an initial baseline of 6 tokens per second to 213 tokens per second. A cloud agent operating inside Conductor leveraged Claude and OpenAI models, including Astra and Opus, to complete the optimization. The developer exercised minimal oversight, providing occasional machine learning suggestions while lacking a background in Verilog and holding little recent C++ experience. Over hundreds of experiments, the project burned through around $10,000 in tokens while generating millions of lines of code.

Key points

  • Automated coding agents can independently drive significant hardware-level optimizations,…: Automated coding agents can independently drive significant hardware-level optimizations, boosting model processing speeds from 6 up to 213 tokens per second.
  • Complex hardware programming can be achieved by developers lacking specific domain expert…: Complex hardware programming can be achieved by developers lacking specific domain expertise in specialized languages like Verilog.
  • Strategic guidance was minimal, consisting largely of high-level machine learning advice…: Strategic guidance was minimal, consisting largely of high-level machine learning advice and one summarized prompt from DeepGrove's founders at the 10 tokens per second mark.

Sources checked by our analysis pipeline: 1

One of the more entertaining things about programming recently is that coding agents let me work in entirely new domains by giving me access to new programming languages and paradigms. Lately, that has meant hacking on hardware, now that writing low-level software is much easier. So, in this blog post, I’m going to try to recount the experience of vibe coding an FPGA, going through a series of “oh shit” moments several times, and giving myself a new appreciation for what’s possible if you have a large token budget (thanks YC) and interesting problems to point the models at.

I overheard some founders recently talk about how FPGAs work. I’ll admit I was relatively naive about them. I’d never really experimented with them in my startup or in college, so I knew little about them before. The basic idea I was familiar with: you can program a circuit into a reprogrammable chip. So rather than having a general purpose computing architecture (like a CPU) you can have a reprogrammable one, i.e. the configuration loaded onto the chip determines how its…

Artificial Intelligence