AI Research

AI & ML Research Papers

A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.

All Papers

2 papers

Preprint2024

Post-training sparse attention with double sparsity

Unknown

Double Sparsity reduces KV cache access in LLMs via post-training sparse attention combining token and channel sparsity.

0Jan 1, 2024Attention Mechanisms

arXiv

Preprint2024

Seerattention: Learning intrinsic sparse attention in your llms

Unknown

Seerattention introduces a block-sparse attention kernel that achieves a 7.3× speedup at 90% sparsity on 128k sequences, enabling efficient long-context LLMs.

0Jan 1, 2024Attention Mechanisms

arXiv