All Documents
3,528 documents available
Kerchunk
Explains how Kerchunk creates cloud-friendly access to netCDF/HDF5 data by extracting metadata and enabling parallel chunk reads.
Semantic Chunking
Defines a quote-aware chunking and indexing strategy for semantic search over mailing-list corpora, with block parsing rules and retrieval flow.
AST-Based Chunking
Explains AST-based code chunking using tree-sitter parsers, with language configs, metadata, and fallback behavior.
rag-vector-chunking
Explains embedding-based retrieval, cosine similarity, RAG evaluation metrics, chunking strategies, and retrieval optimization techniques.
🧠 Adaptive RAG Chunking System
Replaces naive fixed-size chunking with a size- and structure-aware strategy that produces fewer, more coherent chunks for RAG retrieval.
SaltyRTC Chunking
Defines a binary chunking algorithm to split large messages into smaller chunks, originally for WebRTC data channel size limits.
Chunking Fundamentals
Teaches chunking strategies for RAG pipelines, covering fixed-size, recursive, and semantic splitting with LangChain code examples.
ADR-002: AST-Pure Chunking
Replaces a broken chunker with AST-pure logic so every line belongs to exactly one semantically typed chunk and no orphans remain.
Chunking Strategies
Introduces five text chunking strategies with Python implementations and shows how to pair them with cosine similarity for retrieval.
Chunking
Proposes splitting Kimchi proof polynomials into chunks to raise the circuit size limit from 2^16 to 2^32 for Mina zkApps.
Text Chunking - Anton (ChromaDB)
Summarises a technical session on text chunking strategies for RAG, covering heuristic and semantic approaches, evaluation metrics, and practical advice.
Chapter 18: The Art of Chunking
Explains why project files and conversation logs need different chunking strategies for vector retrieval, with code examples and parameter tables.
ADR: Preserve Original Memories During Chunking
Decides to keep original memory content unchanged during chunking, storing summaries and chunks as separate linked memories.
Late Chunking Strategy
Explains late chunking for RAG: embed entire documents first, then split into context-aware chunks to preserve cross-references.
embedding and chunking
Compares late chunking against traditional chunking for RAG embedding, with Jina API code and similarity benchmarks.
Check overall status
Guides an AI agent to analyze uncommitted git changes, split them into logical chunks, and create focused commits matching repository conventions.
ADR-0006: Paragraph-Boundary Chunking
Splits documents into chunks at paragraph boundaries for embedding and retrieval, with configurable token limits and overlap.
Document_Processing_Chunking
Describes a pluggable document processing architecture with a base class, registry, and two concrete processors for text and complex formats.
File Chunking Configuration
Configures file chunking thresholds and timeout settings to prevent timeouts when reviewing large files with an AI code reviewer.
Semantic Chunking
Replaces fixed-token chunking with sentence embeddings and cosine similarity to detect natural topic boundaries for better retrieval.
With Jina Embeddings v3
Explains late chunking, a technique that embeds full documents before splitting into chunks to preserve context, improving retrieval recall by 3-10%.
Running Map Chunking on VPS
Walks through running a map chunking script on a VPS, including screen sessions, options, and troubleshooting.
Step 7: Action Chunking
Replaces single-action prediction with a sequence of actions (chunk) to capture temporal structure, with configurable execution modes.
Differences with torchtext.datasets.CoNLL2000Chunking
Compares PyTorch's CoNLL2000Chunking dataset API with MindSpore's CoNLL2000Dataset, highlighting parameter differences and MindSpore's lack of web download.