Technology2 min read

Five Qwen3.8-27B models outperform Fable 5 Using GVS5H Harness

By · Published by Everything Blog

In short

### Briefing Five Qwen3.8-27B Models Outperform Fable 5 Using GVS5H Harness:** Key Findings:**** Five Qwen3.8-27B models significantly outperform Fable 5 across eleven open and closed-weight models on the 100 latest LiveCodeBench problems. - The method, ledger-based zero-shot self-orchestration, achieves gains of up to 25.6 percentage points on pinned backends and offers two routes to frontier-level accuracy. - Inference-time organization can approach frontier coding accuracy at a fraction of the cost,…

Key points

  • ### Briefing **Five Qwen3.8-27B Models Outperform Fable 5 Using GVS5H Harness:** - **Ke…: ### Briefing **Five Qwen3.8-27B Models Outperform Fable 5 Using GVS5H Harness:** - **Key Findings:**** - Five Qwen3.8-27B models significantly outperform Fable 5 across eleven open and closed-weight models on the 100 latest LiveCodeBenc
  • - The method, ledger-based zero-shot self-orchestration, achieves gains of up to 25.6 per…: - The method, ledger-based zero-shot self-orchestration, achieves gains of up to 25.6 percentage points on pinned backends and offers two routes to frontier-level accuracy.
  • - Inference-time organization can approach frontier coding accuracy at a fraction of the…: - Inference-time organization can approach frontier coding accuracy at a fraction of the cost, or slightly exceed it on self-hostable weights.

Sources checked by our analysis pipeline: 1

Opus 5 ran on an earlier version of the scaffold, in a single pass served through OpenRouter rather than five on a pinned backend.

Abstract. Frontier coding performance is typically bought with larger proprietary models at high cost. We introduce ledger-based zero-shot self-orchestration, a training-free method in which fresh instances of one model decompose problems and coordinate through a shared filesystem holding a plan, notes and current solution. Across eleven open and closed-weight models on the 100 latest hard LiveCodeBench problems, the method yields gains of up to 25.6 percentage points on pinned backends and offers two routes to frontier-level accuracy. Orchestrated Qwen3.8 Flash Next reaches 93.0% pass@1 against Fable 5's 90.4% at 9% of the cost, and locally served, open-weight Qwen3.8-27B rises from 66.8% to 92.4%. Gains are not universal: some models are unchanged or worse. Transcript analysis attributes the gain to decomposition and persistent context. Inference-time organization can approach frontier coding accuracy at a fraction of the cost, or slightly exceed it on self-hostable weights.

Needs uv and an API key for the model you want to test.

cd codebase/v2-current

export OPENAI_API_KEY=...

LCB_RELEASE=release_v6 \

ESCALATION_CLOUD_MAX_TOKENS=128000 \

ESCALATION_CLOUD_TIMEOUT=7200 \

MULTIAGENT_MODEL=openai:gpt-5.6-terra \

uv run --no-project --python 3.12 --with 'datasets<4' --with numpy --with anthropic \

python escalation/run_bench.py --engine multiagent --only lcb --lcb 100 --parallel 8

--engine multiagent

runs the manager;--engine single

is the one-call baseline.- Other models:

anthropic:<model>

,dashscope:<model>

,openrouter:<model>

, each with its own*_API_KEY

. - The pass@1 score prints at the end. Results are written to

runs/results.json

, workspaces toruns/ws/

.

Code is under the MIT License. The paper, figures and run data are under CC BY 4.0. The LiveCodeBench fork, the benchmark problem statements and the LaTeX template files keep their own licenses. See NOTICE.md for which license covers which path.

Original source: github.com

Technology

More in Technology