Llama Cpp Plugin for Local GGUF Inference and Embeddings
This page covers the Llama Cpp plugin for running local GGUF text inference and generating embeddings using node-llama-cpp. It is intended for developers who need on-device AI without cloud dependencies.
Read this when
- You are installing, configuring, or auditing the llama-cpp plugin
Llama Cpp plugin
Run local GGUF text inference and generate embeddings using node-llama-cpp.
Distribution
- Package:
@openclaw/llama-cpp-provider - Installation method: npm; ClawHub
Surface
providers: llama-cpp; contracts: embeddingProviders
Default text model
When running interactive setup, OpenClaw presents Gemma 4 E4B IT Q4_K_M as a bundled download of about 5.0 GB. At least 16 GiB of total system RAM is required for this offer. Models already cached are still recognized on systems with less memory.
To use a different model, assign any custom GGUF to params.modelPath. The RAM restriction for bundled downloads does not apply to custom models. On machines that don't meet the requirement, you can run a smaller model via Ollama or LM Studio, or pick a cloud provider instead.