All Documents
3,528 documents available
Chunking & Embeddings
Documents a chunk preview API and embedding pipeline for building a GPT chat knowledge base with FAISS indices.
Message Chunking for MCP stdio Transport
Explains macOS 64KB pipe write limit and provides a chunking layer for MCP stdio transport that splits and reassembles large messages transparently.
Text Chunking - Anton (ChromaDB)
Summarises a talk on text chunking for RAG, covering heuristic vs semantic approaches, evaluation metrics, and practical advice.
Check overall status
Guides an AI agent to analyze uncommitted git changes, split them into logical chunks, and create focused commits matching repository conventions.
Chunking
Proposes splitting Kimchi proof polynomials into chunks to raise the circuit size limit from 2^16 to 2^32 for Mina zkApps.
With Jina Embeddings v3
Explains late chunking, a technique that embeds full documents before splitting into chunks to preserve context, improving retrieval recall by 3-10%.
Step 7: Action Chunking
Replaces single-action prediction with a sequence of actions (chunk) to capture temporal structure, with configurable execution modes.
Chapter 18: The Art of Chunking
Explains why project files and conversation logs need different chunking strategies for vector retrieval, with code examples and parameter tables.
Semantic Chunking
Replaces fixed-token chunking with sentence embeddings and cosine similarity to detect natural topic boundaries for better retrieval.
Document_Processing_Chunking
Describes a pluggable document processing architecture with a base class, registry, and two concrete processors for text and complex formats.
Static Hosting Performance
Explores performance bottlenecks in GitHub zip retrieval and evaluates caching strategies for edge-hosted static sites.
ChonkyDB One-Shot Retrieval Target
Proposes a new ChonkyDB endpoint that returns final structured records in one request, eliminating separate hydration roundtrips for Twinr's low-latency retrieval.
notes_on_chunking
Distills chunking best practices from a RAG transcript, covering strategy types, evaluation metrics, and data inspection.
Marker Page Chunking Benchmark
Benchmarks Marker chunk sizes (10, 20, 30) for PDF parsing speed and GPU memory, recommending chunk size 10 for production.
Search and Retrieval
Documents a 6-stage RAG search pipeline with hybrid retrieval, reranking, Chain of Thought reasoning, and source attribution.
chunking
Explains Docling's native chunker classes that split documents into token-aware chunks with metadata for downstream AI pipelines.
time-chunking
Explains time chunking (task batching) to reduce context switching, with implementation steps, strategies, and profession-specific examples.
Property chunking
Defines a YAML schema and configuration pattern for splitting large property lists into smaller API requests and merging results back into complete records.
Better File Chunking
Opens a discussion on chunking strategies for IPFS, aiming to reduce algorithm proliferation and improve default chunking.
Chapter 3. Storage and Retrieval
Summarizes key storage engine concepts from a book chapter, covering hash indexes, LSM-trees, B-trees, OLTP/OLAP, and column-oriented storage.
Long Context Chunking
Splits long documents into overlapping chunks and averages their embeddings to avoid context limit errors.
Efficient Content Retrieval
Documents a 20x performance improvement over PostgreSQL's built-in ts_headline by precomputing TSVECTOR and word arrays and using setweight/ts_filter.
curl-content-retrieval
Fetches a specific URL with curl to verify web resource availability and content for subdomain takeover validation.
开放领域问答中的密集段落检索
Explains dense passage retrieval for open-domain QA and provides a code template using Simple Transformers to train a DPR model.