Artificial Intelligence1 min read

Ggml v1.0.0

By · Published by Everything Blog

In short

The release streamlines the core scheduler by eliminating unneeded input uploads and limiting prefetching to data from preceding execution splits. Efficiency for complex architectures is further improved through a Mixture of Experts cache that offloads select misses to the CPU for asynchronous evaluation. In addition to broad computational enhancements, the interface prunes outdated functions and updates its network protocol to block non-compliant peers.

Key points

  • System developers must update integrated clients because interface cleanups and network c…: System developers must update integrated clients because interface cleanups and network changes purposefully break backward compatibility with legacy endpoints.
  • Memory bandwidth and pipeline efficiency are substantially improved through scheduler adj…: Memory bandwidth and pipeline efficiency are substantially improved through scheduler adjustments that avoid uploading unread inputs.
  • Hardware backends including Metal, Vulkan, SYCL, and OpenCL now offer specialized acceler…: Hardware backends including Metal, Vulkan, SYCL, and OpenCL now offer specialized acceleration pathways for complex model architectures.

Sources checked by our analysis pipeline: 1

Repository navigation

Overview

ggml 1.0.0 prunes the deprecated public API and the legacy DIAG_MASK

ops, adds configurable YaRN truncation and FP8 support, and brings broad performance work across backends: few-row quantized matmuls and MoE fusion on CUDA/Metal, a Vulkan prefill FlashAttention path, a device-side MoE scheduler on SYCL, decomposed Adreno FlashAttention in OpenCL, new WebGPU kernels, expanded Hexagon op coverage, and scheduler improvements that avoid uploading unused inputs.

API changes

- Pruned deprecated API and removed the legacy

DIAG_MASK_INF

/DIAG_MASK_ZERO

ops, bumping the RPC protocol to reject incompatible peers (llama/29801, llama/31946) - Added

ggml_rope_set_truncate

/ggml_rope_get_truncate

to control YaRN correction-bound rounding (llama/30206, llama/30802) - Implemented the backward pass for

TRI

(llama/30712) - Added

MUL_MAT

src1 precision tests and gating forggml_prec_set_acc

(llama/31896, llama/31851) - Bumped the RPC protocol to v9 with faster loading and graceful failure on unreachable endpoints or failed handshakes (llama/29413, llama/30415)

Core changes

- Scheduler skips uploading inputs that are not read and prefetches only inputs produced by earlier splits (llama/31419, llama/32100)

- MoE cache can compute a share of the cache misses on the CPU and copy the…

Artificial Intelligence

More in Artificial Intelligence