A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Romain Cosentino, Sarath Shekkizhar
This paper links LLM reasoning ability to the density of self-attention graphs, showing higher intrinsic dimension in MLP inputs implies greater expressive capacity.
Rachit Bansal, Aston Zhang, Rishabh Tiwari, et al.
Proposes test-time training via targeted gradient updates on context to overcome static self-attention limitations in long-context LLMs, outperforming inference-time scaling.
Unknown
DocFormer introduces an encoder-only transformer with a CNN backbone that fuses visual, textual, and spatial features via a novel multi-modal self-attention layer for document understanding.
Ashish Vaswani, Noam Shazeer, Niki Parmar, et al.
Introduced the Transformer architecture, replacing recurrence and convolutions with self-attention mechanisms, achieving state-of-the-art results on machine translation and becoming the foundation for all modern large language models.