Swiss Army Llama

Free

Comprehensive set of tools for working with local LLMs for various tasks.

FreeFree tier

Inputs: text, audioOutputs: text

Type

Open Source

Visit Website GitHub (38 stars)

About Swiss Army Llama

Swiss Army Llama is a FastAPI service designed to streamline working with local large language models (LLMs). It provides REST endpoints for obtaining text embeddings and completions via llama_cpp, with built-in support for processing a variety of document types including PDFs (with OCR), Word files, and audio files (transcribed with Whisper). Embeddings are cached in SQLite to avoid redundant computation. The service optionally uses RAM disks to accelerate model loading. It offers semantic similarity computation using a Rust-based library with advanced measures (Spearman's rho, Kendall's tau, approximate distance correlation, Jensen-Shannon dependency measure, Hoeffding's D) and supports semantic search across cached embeddings with FAISS vector search. All endpoints are accessible through an intuitive Swagger UI, making integration into custom applications straightforward.

Key Features

REST API for text embeddings and completions using llama_cpp

Document processing: PDFs (with OCR), Word files, and other formats via textract

Audio transcription and embedding using Whisper

SQLite-based caching of embeddings to avoid recomputation

Optional RAM disk support for faster model loading

Advanced similarity measures: Spearman's rho, Kendall's tau, distance correlation, Jensen-Shannon dependency, Hoeffding's D

FAISS vector search for semantic search across cached embeddings

Convenient Swagger UI for exploring and testing endpoints

Pros & Cons

Pros

Operates entirely locally, ensuring data privacy and no cloud costs
Supports a wide range of file types including scanned PDFs and audio
Caches embeddings in SQLite for efficiency on repeated queries
Offers advanced statistical similarity measures beyond cosine similarity
Open-source and free to use with no usage limits
Includes a user-friendly Swagger UI for easy testing and integration

Cons

Requires local setup and management of LLM models and dependencies
Performance depends on local hardware (CPU/GPU) and may be slow on modest machines
Relies on llama_cpp which may not support the latest or all model architectures
No hosted cloud version; users must deploy and maintain their own instance

Best For

Building local semantic search applications for documentsAutomating embedding generation for large collections of text, PDFs, Word files, and audioPerforming text similarity and correlation analysisIntegrating local LLM capabilities into custom applications via REST APIRunning offline LLM tasks without relying on cloud services

FAQ

What file types does Swiss Army Llama support?

It supports PDFs (with OCR), Word documents, audio files (via Whisper transcription), and other common formats through the textract library.

Does the tool cache embeddings?

Yes, all computed embeddings are cached in SQLite. If the same input is processed again, the cached embedding is retrieved to save computation.

What similarity measures are available for semantic search?

Beyond cosine similarity (via FAISS), the tool offers advanced measures including Spearman's rho, Kendall's tau, approximate distance correlation, Jensen-Shannon dependency measure, and Hoeffding's D, which can be applied as a second pass on top results.

Is Swiss Army Llama free and open-source?

Yes, it is completely free and open-source, hosted on GitHub under the repository Dicklesworthstone/swiss_army_llama.