math/ggml 0.23.0 NetBSD

ggml v0.27.0 extends the backend scheduler API with user callbacks for split weight copying and compute-buffer reallocation, and bumps the RPC protocol to v8 to support tensor-parallel RPC servers. Backends gained substantial new fast paths: chunked gated-delta-net kernels on CUDA/HIP, KleidiAI SME2 k-quant matmuls and AVX2 Q2_0 dot products on CPU, L2-aware coopmat2 matmul scheduling and bf16 activations on Vulkan, reordered-layout MoE kernels (MXFP4, IQ3, Q5_K, grouped XMX GEMM) on SYCL, dp4a MoE GEMMs on OpenCL, tensor-API and wider-head-dim flash attention on Metal, and a DMA/HVX copy overhaul with new pooling and 3D im2col ops on Hexagon. Correctness work concentrated on non-contiguous and strided views, grid/workgroup size limits, out-of-bounds accesses, and well-defined mul_mat_id …

ggml v0.26.0 brings a new alloc_buffer_n/get_alloc_size_n buffer allocation API, sparse flash attention kernels on SYCL, Vulkan and Metal, and major lightning indexer improvements (halved score memory, tiling, MUSA support). The CPU backend gains BF16 ops and a tiled k-quant mul_mat, CUDA gains a model-driven W4A4 (NVFP4/MXFP4) mul_mat path plus MMVQ shared-expert fusion, and the Hexagon backend adds a sampler and more quant types. Model loading is faster with stricter GGUF size validation, Windows ARM64 MSVC builds are enabled, and WebGPU/OpenVINO/OpenCL/SYCL pick up numerous new kernels, ops and fixes.

0.25.3 A small maintenance release: the core graph size calculation (ggml_graph_nbytes) no longer triggers a UBSan "non-zero offset to null pointer" diagnostic, and the CI setup moves CUDA builds to the hf-jobs runner and adds a new AMD Vulkan build job. 0.25.2 A small point release focused on backend improvements: a new CUDA convolution kernel with an implicit-GEMM fast path, Vulkan fixes and tuning (misalignment handling in convolution shaders and cooperative-matrix support for Adreno GPUs), an optimized DP4A binary kernel for OpenCL Q6_K GEMM, and a precision guard in the Hexagon backend. 0.25.1 A hotfix release focused on the CUDA backend: sparse flash attention, which had been disabled due to a batch-dependent gate, is re-enabled …

Version 0.23.0 adds new operators and backend scheduling APIs for sparse attention, asynchronous execution, and allocation-dependency tracking. It includes broad correctness and performance work across CPU, GPU, and accelerator backends, plus an expanded RPC stack with Apple RDMA support.

This release adds tensor-split support to the multi-backend (meta) backend with improved split-state propagation, reworks the Metal kernels into per-op sources with parallel compilation, and fixes ggml_clamp to be a proper non-in-place op. It also brings new ops (POOL_1D, PAD_REFLECT_1D), Q2_K SYCL kernels, MoE bias fusion on OpenCL, and assorted fixes across the CUDA, Metal, SYCL, Vulkan, OpenCL and WebGPU backends.