Artificial Intelligence1 min read

Provable Inference

By · Published by Everything Blog

In short

The new system, named FlashProve, uses zkSNARKs optimized with assistance from Claude to deliver verification speeds orders of magnitude faster than earlier open-source alternatives. When benchmarked on an H100 GPU running a 400-billion-parameter mixture-of-experts model, it generated proofs at roughly 110 tokens per second. While producing a 19-megabyte proof for 1,024 tokens takes 70 to 95 times the standard inference time, the mathematical guarantees remain auditable. To facilitate adoption and scrutiny, the creators released five core artifacts, including executable verifiers, under the open-source Apache 2.0 license.

Key points

  • Cryptographic verification via zkSNARKs prevents intermediaries from secretly swapping mo…: Cryptographic verification via zkSNARKs prevents intermediaries from secretly swapping models or altering outputs without exposing proprietary model weights.
  • Researchers utilized the formal verification language Lean to author top-level mathematic…: Researchers utilized the formal verification language Lean to author top-level mathematical statements that validate the security claims.
  • The collaborative team released five distinct artifacts, including a C++ verifier and pro…: The collaborative team released five distinct artifacts, including a C++ verifier and protocol specifications, under the Apache 2.0 license.

Sources checked by our analysis pipeline: 1

Roy Rinberg (Anthropic), Alex Allain (Anthropic); Alessandro Chiesa (EPFL, StarkWare), Miranda Christ (MIT, RESI), Rahul Dalal (Anthropic STEM Fellows Program, Succinct), Giacomo Fenzi (EPFL), Ben Mayer (Anthropic Fellows Program, MIT), Guy Weissenberg (EPFL)

Executive summary

When you talk to a friend in person, you have many ways to know it is really them: you can see them, hear them, shake their hand. When you talk to an AI model, you have almost none. A user querying a model today cannot check that the responses came from the specific model that a developer claims is responding, as opposed to one secretly substituted in by a routing service for cost reasons, or even by an attacker who has tampered with the token stream. Judging from the model’s output alone is unreliable, even if it produces high-quality responses. Cryptographic proofs of inference solve this problem: they let a model provider commit to a model and then prove that each response was produced by that model. Such proofs could enable applications ranging from consumer products to internal security, and could contribute to a toolkit…

Artificial Intelligence