Byaldi
FreeQuick ColPali deployment
FreeFree tier
About Byaldi
Byaldi is a simple wrapper around the ColPali repository, designed to make late-interaction multi-modal models (such as ColPali and ColQwen2) easy to use with a familiar API modeled after RAGatouille. It allows users to load models, create indexes from documents, and query them in just a few lines of code. The library currently supports all models supported by the colpali-engine backend, including vidore/colqwen2-v1.0, and aims to support all ColVLM models. Future plans include HNSW indexing, pooling, and 2-bit quantization. Byaldi is open source and provided under a permissive license by AnswerDotAI.
Key Features
Simple wrapper around ColPali for multi-modal retrieval
Supports ColQwen2 and all models supported by colpali-engine
API modeled after RAGatouille for minimal code
Load models from pretrained checkpoints or existing indexes
Create indexes from documents (PDFs via pdf2image) and search them
Designed for GPU acceleration, with optional Flash Attention support
Planned support for VisRAG, HNSW indexing, pooling, and quantization
Open source and free to use
Pros & Cons
Pros
- Extremely easy to use with a few lines of code
- Supports state-of-the-art multi-modal models like ColPali and ColQwen2
- Open source with permissive license
- Active development with planned improvements (HNSW, quantization)
- Works with PDFs and other document formats via pdf2image
Cons
- Pre-release software with uncompressed indexes and potential quirks
- Requires GPU for reasonable performance (CPU/MPS not recommended for encoding)
- Requires external dependencies: poppler and optionally flash-attn
- Limited to models supported by colpali-engine backend
- Documentation and examples still maturing
Best For
Multi-modal document retrieval (images + text)Building RAG pipelines that can search PDFs and imagesQuick prototyping of late-interaction retrieval systemsAcademic research on vision-language models for retrieval
FAQ
What is Byaldi?
Byaldi is a mini sister project of RAGatouille, acting as a simple wrapper around the ColPali repository to make late-interaction multi-modal models easy to use with a familiar API.
Which models does Byaldi support?
Currently, Byaldi supports all models supported by the underlying colpali-engine, including vidore/colqwen2-v1.0. It aims to support all ColVLM models and eventually VisRAG.
What hardware do I need?
Byaldi uses multi-billion parameter models and a GPU is recommended for smooth operations. Encoding documents on CPU or MPS will suffer from poor performance.
Is Byaldi free?
Yes, Byaldi is open source and free to use. It is hosted on GitHub under the AnswerDotAI organization.