Process Optimization
CHUNKING.md · 8 documents
Splitters
Introduces splitters that divide large text into manageable chunks for AI processing, with configurable sizes and overlap.
Create a list that will hold your chunks
Walks through document chunking strategies for RAG pipelines, from basic character splitting to recursive and token-based methods, with code examples in LangChain.
🧠 Adaptive RAG Chunking System
Replaces naive fixed-size chunking with a size- and structure-aware strategy that produces fewer, more coherent chunks for RAG retrieval.
Per-Example Level Test Chunking
Proposes adding per-example granularity to a test chunking algorithm so large spec files can be split across parallel CI workers.
Speichereffiziente PDF-Verarbeitung – Strategiebericht
Documents a memory-leak fix for PDF chunking on mobile devices, reducing peak RAM from ~417 MB to ~172 MB.
chunking
Explains Docling's native chunker classes that split documents into token-aware chunks with metadata for downstream AI pipelines.
time-chunking
Explains time chunking (task batching) to reduce context switching, with implementation steps, strategies, and profession-specific examples.
chunking
Explains the chunking architecture in Next.js Turbopack, covering graph walk, module batching, merging, chunk splitting, and output generation.