FAISS Vector Search: Fast Similarity Search at Billion Scale

FAISS (Facebook AI Similarity Search) is a library for efficient similarity search and clustering of dense vectors, capable of handling datasets with billions of vectors. It is the go-to tool when you need low-latency, high-throughput vector search without metadata filtering, and it supports both CPU and GPU acceleration. This guide covers installation, index types, GPU usage, and integration with LangChain and LlamaIndex.

What it does

FAISS provides a set of algorithms for searching through large collections of vectors (embeddings) to find the nearest neighbors of a query vector. It is designed for production-scale use, with C++ performance and Python bindings. You can choose between exact search (slow but 100% accurate) and several approximate methods that trade a small amount of accuracy for orders of magnitude speed improvement. The library also supports GPU acceleration, which can speed up search by 10-100x on large datasets.

Before you start

• Platforms: Linux and macOS are supported.
• Python: You need a working Python environment with pip.
• GPU (optional): If you plan to use GPU acceleration, you need a CUDA-capable GPU and the appropriate NVIDIA drivers.
• Installation: Install the CPU-only version with pip install faiss-cpu or the GPU version with pip install faiss-gpu. Do not install both.

Quick start

Installation

# CPU only
pip install faiss-cpu

# GPU support
pip install faiss-gpu

Basic usage

import faiss
import numpy as np

# Create sample data (1000 vectors, 128 dimensions)
d = 128
nb = 1000
vectors = np.random.random((nb, d)).astype('float32')

# Create index
index = faiss.IndexFlatL2(d)  # L2 distance
index.add(vectors)             # Add vectors

# Search
k = 5  # Find 5 nearest neighbors
query = np.random.random((1, d)).astype('float32')
distances, indices = index.search(query, k)

print(f"Nearest neighbors: {indices}")
print(f"Distances: {distances}")

This creates a flat (exact) index using L2 distance, adds 1000 random 128-dimensional vectors, and searches for the 5 nearest neighbors of a random query vector.

Index types

1. Flat (exact search)

# L2 (Euclidean) distance
index = faiss.IndexFlatL2(d)

# Inner product (cosine similarity if normalized)
index = faiss.IndexFlatIP(d)

# Slowest, most accurate

Flat indexes perform an exhaustive search. They are the slowest but guarantee 100% recall. Use them only for small datasets (under 10,000 vectors) or as a quantizer for other index types.

2. IVF (inverted file) - Fast approximate

# Create quantizer
quantizer = faiss.IndexFlatL2(d)

# IVF index with 100 clusters
nlist = 100
index = faiss.IndexIVFFlat(quantizer, d, nlist)

# Train on data
index.train(vectors)

# Add vectors
index.add(vectors)

# Search (nprobe = clusters to search)
index.nprobe = 10
distances, indices = index.search(query, k)

IVF partitions the vector space into clusters (using k-means) and only searches the most promising ones. The nprobe parameter controls how many clusters are searched during a query. Higher nprobe gives better accuracy but slower search. IVF is a good choice for datasets between 10,000 and 1 million vectors.

3. HNSW (Hierarchical NSW) - Best quality/speed

# HNSW index
M = 32  # Number of connections per layer
index = faiss.IndexHNSWFlat(d, M)

# No training needed
index.add(vectors)

# Search
distances, indices = index.search(query, k)

HNSW builds a multi-layer graph structure. It does not require a separate training step and offers excellent search quality (around 99% recall) with fast query times. The M parameter controls the number of connections per layer; higher values improve accuracy but increase memory usage and build time.

4. Product Quantization - Memory efficient

# PQ reduces memory by 16-32×
m = 8   # Number of subquantizers
nbits = 8
index = faiss.IndexPQ(d, m, nbits)

# Train and add
index.train(vectors)
index.add(vectors)

Product quantization compresses vectors by splitting them into subvectors and quantizing each subvector independently. This can reduce memory usage by 16-32x compared to storing raw vectors, at the cost of some accuracy (typically 90-95% recall). Use PQ when you need to fit a very large dataset in memory.

Save and load

# Save index
faiss.write_index(index, "large.index")

# Load index
index = faiss.read_index("large.index")

# Continue using
distances, indices = index.search(query, k)

Saving and loading is straightforward. Note that trained indices (like IVF and PQ) must be saved after training, as training is expensive and cannot be repeated from the saved file alone.

GPU acceleration

# Single GPU
res = faiss.StandardGpuResources()
index_cpu = faiss.IndexFlatL2(d)
index_gpu = faiss.index_cpu_to_gpu(res, 0, index_cpu)  # GPU 0

# Multi-GPU
index_gpu = faiss.index_cpu_to_all_gpus(index_cpu)

# 10-100× faster than CPU

GPU acceleration can dramatically speed up both index building and search. The StandardGpuResources object manages GPU memory. For multi-GPU setups, index_cpu_to_all_gpus distributes the index across all available GPUs.

LangChain integration

from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings

# Create FAISS vector store
vectorstore = FAISS.from_documents(docs, OpenAIEmbeddings())

# Save
vectorstore.save_local("faiss_index")

# Load
vectorstore = FAISS.load_local(
    "faiss_index",
    OpenAIEmbeddings(),
    allow_dangerous_deserialization=True
)

# Search
results = vectorstore.similarity_search("query", k=5)

LangChain's FAISS wrapper provides a convenient interface for using FAISS as a vector store in LLM applications. Note the allow_dangerous_deserialization=True flag: loading a FAISS index from an untrusted source can be a security risk, so only use this with indices you trust.

LlamaIndex integration

from llama_index.vector_stores.faiss import FaissVectorStore
import faiss

# Create FAISS index
d = 1536
faiss_index = faiss.IndexFlatL2(d)

vector_store = FaissVectorStore(faiss_index=faiss_index)

LlamaIndex's FaissVectorStore wraps a FAISS index and integrates it with the LlamaIndex ecosystem. The dimension d must match the embedding dimension of your documents.

Best practices

1. Choose right index type - Flat for <10K, IVF for 10K-1M, HNSW for quality
2. Normalize for cosine - Use IndexFlatIP with normalized vectors
3. Use GPU for large datasets - 10-100× faster
4. Save trained indices - Training is expensive
5. Tune nprobe/ef_search - Balance speed/accuracy
6. Monitor memory - PQ for large datasets
7. Batch queries - Better GPU utilization

Performance

Index Type	Build Time	Search Time	Memory	Accuracy
Flat	Fast	Slow	High	100%
IVF	Medium	Fast	Medium	95-99%
HNSW	Slow	Fastest	High	99%
PQ	Medium	Fast	Low	90-95%

When not to use it

FAISS is a pure vector search library. It does not support metadata filtering, transactions, or full database features. If you need to filter search results by metadata fields (e.g., "only documents from 2024"), consider Chroma, Pinecone, or Weaviate instead. For simpler use cases with smaller datasets, Annoy may be easier to set up.

Limits and gotchas

• FAISS indexes are not portable across different versions of the library. An index saved with one version may not load with another.
• The allow_dangerous_deserialization=True flag in LangChain is required because pickle deserialization can execute arbitrary code. Only load indices from trusted sources.
• GPU indices cannot be directly saved and loaded; you must convert them to CPU first.
• Training (for IVF and PQ) requires the full dataset to be in memory. For very large datasets, consider using the IndexIDMap or incremental training approaches.

Related resources

• GitHub: https://github.com/facebookresearch/faiss ⭐ 31,700+
• Wiki: https://github.com/facebookresearch/faiss/wiki
• License: MIT