Preprint2024
Post-training sparse attention with double sparsity
Unknown
Double Sparsity reduces KV cache access in LLMs via post-training sparse attention combining token and channel sparsity.
0Jan 1, 2024Attention Mechanisms
arXiv
AI Research
AI & ML Research Papers
A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
All Papers
80 papersPreprint2024
Seerattention: Learning intrinsic sparse attention in your llms
Unknown
Seerattention introduces a block-sparse attention kernel that achieves a 7.3× speedup at 90% sparsity on 128k sequences, enabling efficient long-context LLMs.
0Jan 1, 2024Attention Mechanisms
arXiv
Preprint2026
The sparse frontier: Sparse attention trade-offs in transformer llms
Unknown
This paper systematically evaluates six training-free sparse attention methods for transformer LLMs, proposing a taxonomy along four design axes and deriving actionable insights.
0Jan 1, 2026TransformersAttention Mechanisms
Preprint2025
Can LLMs Maintain Fundamental Abilities under KV Cache Compression?
Unknown
This paper introduces a new KV cache compression method and evaluates its impact on LLMs' fundamental abilities.
0Jan 1, 2025
arXiv
Preprint2026
Fastkv: Decoupling of context reduction and kv cache compression for prefill-decoding acceleration
Unknown
FastKV decouples context reduction from KV cache compression to accelerate both prefill and decoding phases in LLM inference.
0Jan 1, 2026
Preprint2024
Model tells you what to discard: Adaptive kv cache compression for llms
Unknown
This paper proposes adaptive KV cache compression policies for LLMs to reduce memory usage while maintaining generation quality.
0Jan 1, 2024
ConferenceInternational Conference on Machine Learning2023
Online speculative decoding
Xiaoxuan Liu, Lanxiang Hu, Peter Bailis, et al.
Introduces online speculative decoding to reduce LLM serving latency by adapting draft models on the fly.
117Jan 1, 2023
arXiv
Preprint2024
Llmc: Benchmarking large language model quantization with a versatile compression toolkit
Unknown
This paper introduces Llmc, a versatile toolkit for benchmarking LLM quantization methods, enabling standardized evaluation and comparison.
0Jan 1, 2024Large Language ModelsReasoning
PreprintInternational Conference on Learning Representations2407
Leanquant: Accurate and scalable large language model quantization with loss-error-aware grid
Tianyi Zhang, Anshumali Shrivastava
Leanquant introduces a loss-error-aware grid quantization method for LLMs that achieves accurate and scalable compression by minimizing quantization error with respect to the model's loss function.
26Jan 1, 2407Large Language ModelsQuantization
arXiv
Preprint2025
Ostquant: Refining large language model quantization with orthogonal and scaling transformations for better distribution fitting
Unknown
Ostquant improves LLM quantization by applying orthogonal and scaling transformations to better fit weight distributions, reducing accuracy loss.
0Jan 1, 2025Large Language ModelsScaling Laws
arXiv
Preprint2024
A survey on knowledge distillation of large language models
Unknown
This survey systematically reviews knowledge distillation techniques for transferring capabilities from large proprietary LLMs to smaller models.
0Jan 1, 2024Large Language ModelsKnowledge Distillation
arXiv
PreprintElectronics2024
A Review of Large Language Models: Fundamental Architectures, Key Technological Evolutions, Interdisciplinary Technologies Integration, Optimization and Compression Techniques, Applications, and Challenges
Songyue Han, Mingyu Wang, Jialong Zhang, et al.
A comprehensive survey of LLMs covering architectures, key technologies, interdisciplinary integrations, optimization, applications, and challenges.
62Dec 21, 2024Large Language ModelsReinforcement Learning