About llama.cpp server
llama-server is the HTTP server shipped with llama.cpp, the reference GGUF inference implementation. It runs quantised models on plain CPUs (with optional CUDA/Metal/Vulkan builds), needs no Python runtime, and exposes an OpenAI-compatible /v1 API, so Anchras can route gateway traffic to it exactly as it does to Ollama or vLLM.
Models are pulled from a Hugging Face GGUF repository on first start and cached on the attached volume.
Project details
License
MIT
Packaging
Container
Min. Requirements
2 CPU, 4GB RAM, 20GB disk
Website
github.com/ggml-org/llama.cpp
Source Code
ggml-org/llama.cpp
Or have it managed.
If you would rather not operate llama.cpp server by hand, Anchras Platform connects its deployment lifecycle, private networking, policy, logs, and audit on infrastructure you control. The platform is in closed beta; the catalog stays free here either way.
