Wllama
FreeWebAssembly binding for llama.cpp - Enabling in-browser LLM inference
FreeFree tier
Inputs: text, image, audioOutputs: text
About Wllama
Wllama is a WebAssembly binding for llama.cpp that enables running large language models (LLMs) directly in the browser without any backend server or GPU. It leverages WebAssembly SIMD for efficient CPU inference and supports WebGPU for hardware acceleration. Key capabilities include an OpenAI-compatible API, multimodal input (images and audio), tool calling, model splitting for files over 2GB, automatic single/multi-thread switching, and inference in a Web Worker to avoid UI blocking. The project is open-source and distributed as an npm package (@wllama/wllama) for easy integration into JavaScript and TypeScript projects.
Key Features
OpenAI-compatible API with full TypeScript types
WebGPU support for hardware acceleration
Multimodal support: image and audio file input
Tool calling support
Runs inference directly in browser via WebAssembly SIMD, no backend or GPU needed
No runtime dependency (pure npm package)
Ability to split models into smaller files and load them in parallel
Auto switch between single-thread and multi-thread builds based on browser support
Inference runs inside a Web Worker, does not block UI render
Pros & Cons
Pros
- No backend server required – fully client-side
- Privacy: user data stays in the browser
- Works on any device with WebAssembly support, no GPU necessary
- Supports modern LLM capabilities: multimodal, tool calling, embeddings
- OpenAI-compatible API makes integration straightforward
- Open source and free to use
- Active development with latest llama.cpp features (WebGPU, multimodal)
Cons
- Multi-thread support requires specific Cross-Origin headers (COEP/COOP) to be set
- Maximum file size limited to 2GB due to ArrayBuffer constraints; larger models must be split
- WebGPU on Firefox (compat mode) may have significantly degraded performance
- Requires relatively modern browser with WebAssembly SIMD support for best performance
Best For
In-browser text completion and chatEmbedding generation and cosine similarity computationMultimodal vision tasks (image description, analysis)Tool calling (function calling) with LLMsPrivacy-preserving LLM applications (data never leaves the browser)Educational demos and prototypes requiring client-side inference
FAQ
How do I use Wllama in a React/TypeScript project?
Install the package with 'npm i @wllama/wllama', then import the Wllama class and create an instance with configuration paths. See the examples folder for a complete React context example.
What are the main limitations of Wllama?
Multi-thread support requires Cross-Origin-Embedder-Policy and Cross-Origin-Opener-Policy headers. Maximum model file size is 2GB (larger models must be split). WebGPU on Firefox may have degraded performance.
Does Wllama support multimodal inputs?
Yes, as of version 3, Wllama supports multimodal input including images and audio files for vision and audio tasks.
Can I use tool calling with Wllama?
Yes, Wllama supports tool calling (function calling) as a built-in feature, demonstrated in the examples/tools directory.