A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Unknown
RocketKV is a training-free two-stage KV cache compression method that accelerates long-context LLM inference by combining coarse-grain permanent eviction with fine-grain dynamic sparsity.
Unknown
The paper proposes a dynamic low-rank sparse adaptation method that unifies LLM sparsity and low-rank adaptation for efficient fine-tuning.
Atindra Jha, Margaret Li, Jure Leskovec, et al.
Mixture-of-Experts models overfit more to repeated training data than dense models, with degradation increasing with sparsity; masking-based regularization can mitigate but not eliminate the gap.
Unknown
This paper proposes that model sparsity can simplify machine unlearning, offering a novel viewpoint for more efficient unlearning processes.
Unknown
ThinKV is a thought-adaptive KV cache compression framework that exploits attention sparsity patterns to reduce memory usage in reasoning models.
Xiaoqiang Lin, Aritra Ghosh, K. H. Low, et al.
REFRAG exploits block-diagonal attention sparsity in RAG to skip unnecessary decoding computations, achieving 30.85x TTFT speedup and 16x context extension without accuracy loss.
Xiaoyuan Su, Taghi M. Khoshgoftaar
A comprehensive survey of collaborative filtering techniques, categorizing them into memory-based, model-based, and hybrid approaches while addressing key challenges like data sparsity and scalability.
Shuo Yang, Ying Sheng, Joseph E. Gonzalez, et al.
Double Sparsity reduces KV cache access in LLMs via post-training sparse attention combining token and channel sparsity.
Yizhao Gao, Zhichen Zeng, Dayou Du, et al.
Seerattention introduces a block-sparse attention kernel that achieves a 7.3× speedup at 90% sparsity on 128k sequences, enabling efficient long-context LLMs.