Developer intelligence
Models, frameworks, libraries, and repositories with practical value for AI developers.
Your stack
Add your stack to personalize.
The latest llama.cpp release (b11345) brings support for q2_k and q3_k quantization types to Hexagon backend, enabling more efficient LLM inference on devices with Qualcomm Snapdragon NPUs.
The b11344 release of llama.cpp addresses two broken cases when running using CUDA on NVIDIA Volta GPUs. No new features or breaking changes are introduced.
Release b11342 for llama.cpp resolves a cache directory creation bug caused by symlinks with some buggy libstdc++ versions, improving reliability on affected Linux distributions.
llama.cpp's latest release reroutes Volta (sm70) CUDA kernels to use Turing's MMVQ parameter table. This improves K-quant batch-1 decode performance on Tesla V100 GPUs by 3.17% when running quantized models.
The b11331 release of llama.cpp improves CUDA support for NVFP4 quantization, adds BF16 compute support for quantized models on compatible hardware, and adjusts precision handling for better performance on supported GPUs.
The latest llama.cpp release (b11330) introduces MTP support for Qwen4Exp, streamlines state management, and cleans up code, benefitting developers using this open-source LLM runtime.
The llama.cpp b11324 release introduces a new memory optimization: it avoids creating a second full-size copy of each tensor by employing direct-IO for mmap, reducing RAM usage across supported platforms.
NVIDIA's open-source Do Inference Now (DIN) Deploy offers new C++ samples combining ONNX Runtime and TensorRT RTX for easier and faster local AI inference.
The latest llama.cpp release adds better documentation and simplified setup for AOCL-BLAS builds, expands platform binaries, and disables openEuler support.
The latest llama.cpp release fixes previous issues with qwen4exp model support and removes openEuler binaries, affecting users working with these environments.
The llama-bench tool in llama.cpp b11317 now correctly logs error-level messages due to a verbosity filter fix, with minor cleanup of dead code. This release is available for major operating systems and hardware platforms.
The llama.cpp library has released version b11304, which addresses Key-Value (KV) handling during training. This release enhances training reliability for developers working with llama-based models.
llama.cpp b11306 introduces enhanced testing for graph shape changes in test-llama-archs and disables openEuler builds for several configurations. Developers should review platform support and consider the updated graph shape validation mechanism.
