Coding & Development
CHUNKING.md ยท 12 documents
type(docs) ==> List[Document], hence used split_documents
Demonstrates four LangChain text splitters, RecursiveCharacter, Character, HTMLHeader, and RecursiveJson, with code examples and explains when to use each.
Document Chunking/Splitting in Langroid
Documents four chunking strategies for Langroid's ParsingConfig and shows how to configure them in DocChatAgentConfig.
AST-Based Chunking
Explains AST-based code chunking using tree-sitter parsers, with language configs, metadata, and fallback behavior.
2025-09-19-HLS-Chunking
Explains how to implement HLS video chunking on Vercel using AWS MediaConvert and S3 presigned URLs to bypass serverless limitations.
libragen: First-Class AST-Aware Code Chunking Support
Plans integrating AST-aware code chunking into libragen, replacing recursive text splitting for supported languages.
Message Chunking for MCP stdio Transport
Explains macOS 64KB pipe write limit and provides a chunking layer for MCP stdio transport that splits and reassembles large messages transparently.
/tmp/example-workspace/WORKSPACE.bazel:13:13
Explains Bazel's WORKSPACE chunking behavior where dependency resolution depends on load statement boundaries.
Document_Processing_Chunking
Describes a pluggable document processing architecture with a base class, registry, and two concrete processors for text and complex formats.
Haskell ASTChunking Enhancement
Documents a multi-layered Haskell code chunking system with configurable AST-based and regex strategies, context propagation, and Rust-accelerated performance.
Differences with torchtext.datasets.CoNLL2000Chunking
Compares PyTorch's CoNLL2000Chunking dataset API with MindSpore's CoNLL2000Dataset, highlighting parameter differences and MindSpore's lack of web download.
TileMap Chunk Manager
Divides a large tilemap into fixed-size chunks, loading and rendering only those visible to reduce memory and draw calls by over 99%.
1. Clone repository
Introduces a multi-agent financial research platform with 7 agents, a LangGraph pipeline, and a dashboard.