The experiment ran five small models—including Llama 3.2 3B and Mistral 7B—on an Apple M3 Pro with 36 GB of RAM to propose answers and cite sources. Claude Sonnet 5.5 was deployed exclusively to evaluate whether retrieved passages backed the proposals, raising 1,000-question accuracy from 77.1% to 85.0%. To eliminate errors that broke previously correct responses, researchers revised the protocol to protect verified leading answers. On disputed questions with five alternative models, this adjusted methodology lifted accuracy from 73.1% to 85.9%.
Key points
Verifying cited reference passages yields superior accuracy over relying strictly on stan…: Verifying cited reference passages yields superior accuracy over relying strictly on standard majority voting among models.
Safeguarding an initially supported leading answer successfully reduced the rate of newly…: Safeguarding an initially supported leading answer successfully reduced the rate of newly introduced errors to 0.75%.
External frontier models can act effectively as specialized verification engines for loca…: External frontier models can act effectively as specialized verification engines for locally hosted lightweight artificial intelligence systems.
Sources checked by our analysis pipeline: 1
Verify, Then Vote
We gave five small local models the same questions and checked their sources before picking an answer. Our first paper reports what got better, a safety test that failed and how we can rerun decisions without asking the models again.
When five models disagree
We wanted to see how far a few small models could get running on one machine. We gave five of them the same question, let them look things up and compared their answers. They often disagreed. The obvious fix is a vote, but models can share the same mistake and a vote can’t catch that.
So we tried something else: check the sources each model cited, then use a fixed rule to pick the answer. We called the paper Verify, Then Vote. We kept answering, checking and choosing as separate steps and saved the results of each, so we could go back and reuse them.
Five local models and one checker
We used Llama 3.2 3B, Mistral 7B, Gemma 2 2B, Phi-3 mini and Qwen 2.5 3B, running through Ollama on an Apple…
This release expands the platform's automated capabilities by enabling artificial intelligence to operate continuously across diverse workflows. By linking directly into Computer, the software provides a persistent environment capable of tackling complex, simultaneous operations. The deployment signals a shift toward persistent digital agents capable of managing diverse responsibilities in parallel.
Network forensic analysts often struggle to feed massive capture files into large language models due to restrictive token context limits. Researchers addressed this constraint by condensing verbose traffic logs into a streamlined symbolic language called PacketGlyphs, stripping out heavy payload data while retaining critical flow patterns. This significant footprint reduction lets existing AI models review entire cellular capture sessions at once with high forensic accuracy.
Proprietary laboratories have captured nearly every budget tier on Village’s LLM Atlas, pushing alternative open-weight architectures down to an unprecedented 3% market share. Rapid release cycles from major AI providers allow them to leverage massive inference infrastructure and distill affordable tiers directly from frontier flagships. This dynamic leaves open-source alternatives largely confined to the cheapest edge deployments while struggling to maintain mid-tier competitiveness.