Have a lead? Contact us!

Underclassic / explainer

llama.cpp 1.0: what changed for home users

A home desk at night with a mini PC, a laptop running a terminal, and a llama figurine
Illustration: Underclassic, made with Runway.

Published 2026-11-15. Sources: Hugging Face, “llama.cpp v1.0.0: A stronger foundation for the local AI ecosystem” (Nov. 13, 2026); GitHub release notes, llama.cpp v1.0.0 (Nov. 13, 2026); pull requests #30966 and #28392.

llama.cpp, the open-source engine that runs AI models on everyday computers, reached version 1.0 on Nov. 13. Three and a half years after the project started, its maintainers say the number is a promise to the many apps that now depend on it. “Every new integration means more developers depending on our library interfaces and therefore having a flexible but stable API is paramount,” the ggml-org team, including founder Georgi Gerganov, wrote in the Hugging Face announcement.

For people who run models at home, the release brings more than a version number. Here is what changed, and what to check before upgrading.

The short version

  • Stable from here: breaking changes to the C API now mean a new major version.
  • Easier installs: Docker, Debian, Homebrew, conda-forge, MacPorts, Winget and prebuilt downloads.
  • Bigger models on smaller GPUs: the MoE expert cache can now run missed experts on the CPU, with a tuning script.
  • Longer chats in the same memory: an 8-bit FP8 option for the KV cache.
  • Safer llama-server: new protections and opt-in flags. Check your launch command.

A version number that means something

From 1.0 on, llama.cpp and its tensor library, ggml, follow semantic versioning. A breaking change to the public C API bumps the major version; new features, backend updates and model support bump the minor version; bug fixes are patches, according to the announcement. To clear the decks, the release removes the deprecated public API, including 35 deprecated functions, and promotes the NextN speculative-decoding API, the release notes say.

If you use llama.cpp through another app, this is the change that matters most: the tools built on top of it get a clear signal when an update could break them.

Easier to install

The project now lists official or community channels for every major platform: prebuilt artifacts for Linux, macOS, Windows, Android and iOS, plus Docker and Debian on Linux, Homebrew on macOS and Linux, conda-forge, MacPorts, Winget on Windows, and an XCFramework for Apple apps. The post also points to Llama desktop apps for macOS and Windows from the llama.cpp and Hugging Face team. Windows users get unbuffered model loading, per the release notes.

Bigger models on smaller GPUs

Many of the strongest open-weight models are mixture-of-experts designs: only a few “experts” run for each word, but all of them must be stored somewhere. llama.cpp's MoE expert cache keeps the most-used experts in GPU memory and the rest in system RAM. In 1.0 it can also run the experts that miss the cache on the CPU instead of copying them over, with two settings, --moe-cache-mib and --moe-cache-cpu-share, and a script, scripts/moe-cache-tune.py, that tests a machine and recommends values (pull request #30966, merged Nov. 7).

Results in the pull request discussion are user reports, not benchmarks we ran. One contributor with an 8 GB RTX 3070 running a 35-billion-parameter model said the baseline was about 35 tokens a second; the tuning script recommended settings it predicted would reach about 61. Another, on an 11 GB RTX 2080 Ti, reported a 42-token baseline and a recommended setting predicted at about 67. Your numbers will depend on your GPU, your RAM speed and your model.

Which models fit which machines is the subject of our Open-Source AI Guide; what the machines cost is in the home AI computers buyer's guide.

Longer chats in the same memory

The KV cache holds the conversation so far, and on long chats it can take as much memory as the model. 1.0 adds an FP8 option, set with -ctk f8_e4m3 -ctv f8_e4m3: an 8-bit format, half the size of the default 16-bit cache (pull request #28392, merged Nov. 10; release notes).

A safer server, and a new web UI

Many home users run llama-server so other devices on the network can use it. The release hardens it against server-side request forgery, cross-origin requests and DNS rebinding, adds cancellation for MCP tool calls, and puts two riskier features behind opt-in flags: --allow-unauthenticated-tools and --media-allow-private-urls, according to the release notes. If a setup that worked before stops working after the upgrade, check those flags first.

The server also gains a /v1/audio/speech endpoint for text-to-speech, the same path OpenAI's API uses, and a redesigned web interface with chat groups, a markdown editor and a native mobile layout.

More models, more hardware

llama.cpp supports more than 150 model architecture variants, the announcement says. The 1.0 notes add Kimi-K3.1-Pro, Ripple-65B-Preview and the drama-3 text-to-speech model, with vision support for MageVL-Cowork and Kimi-K3.1. On the hardware side, ggml 1.0 brings FlashAttention for prompt processing on Vulkan and fused mixture-of-experts kernels on CUDA and Metal, and the project's test machines now include hardware from NVIDIA, AMD, Qualcomm, Arm, IBM and Intel.

Should you upgrade?

Prices for the models you could run instead in the cloud are on our models board.

FAQ

When was llama.cpp 1.0 released?

Nov. 13, 2026. The v1.0.0 release of llama.cpp and the ggml tensor library was announced in a Hugging Face blog post from the ggml-org team, including project founder Georgi Gerganov, and published on GitHub the same day.

What does version 1.0 mean for llama.cpp?

It is a promise about change, not a finished product. From 1.0 on, llama.cpp and ggml use semantic versioning: a breaking change to the public C API means a new major version, new features and model support are minor versions, and bug fixes are patches. Apps built on llama.cpp get a clear signal when an update could break them.

Can llama.cpp 1.0 run bigger models on a small GPU?

For mixture-of-experts models, it helps. The MoE expert cache keeps the most-used experts in GPU memory and the rest in system RAM, and 1.0 can now run missed experts on the CPU instead of copying them to the GPU. A tuning script suggests settings for each machine. Contributors in the pull request discussion reported large speedups on 8 GB to 16 GB cards, but results depend on hardware.

Do I need to change anything after upgrading llama-server?

Possibly. The 1.0 release hardens llama-server against SSRF, CORS and DNS-rebinding attacks, and some features now need explicit opt-in flags, including --allow-unauthenticated-tools for MCP tools and --media-allow-private-urls for fetching media from private network addresses. Check your launch command against the release notes.

How do I install llama.cpp 1.0?

The Hugging Face post lists prebuilt artifacts for Linux, macOS, Windows, Android and iOS, plus Docker, Debian, Homebrew, conda-forge, MacPorts, Winget and an XCFramework for Apple apps. The team also points to Llama desktop apps for macOS and Windows.

Get the daily issue

One email a day. Unsubscribe anytime.

Read today's issue · Open-Source AI Guide · Buyer's guide · Models board

home  |  rss  |  page 2  |  page 3  |  archive  |  models  |  buyer's guide  |  subscribe  |  data  |  board  |  about  |  mcp  |  llms.txt