AI Research

AI & ML Research Papers

A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.

All Papers

2 papers

ConferenceInternational Conference on Machine Learning2024

Star Attention: Efficient LLM Inference over Long Sequences

Shantanu Acharya, Fei Jia, Boris Ginsburg

Star Attention enables efficient LLM inference over long sequences via a two-phase block-sparse approximation that reduces memory and time by up to 11x while preserving 97-100% accuracy.

34Nov 26, 2024Large Language ModelsTransformers

arXiv

Preprint2024

Seerattention: Learning intrinsic sparse attention in your llms

Yizhao Gao, Zhichen Zeng, Dayou Du, et al.

Seerattention introduces a block-sparse attention kernel that achieves a 7.3× speedup at 90% sparsity on 128k sequences, enabling efficient long-context LLMs.

0Jan 1, 2024Attention Mechanisms

arXiv