A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
E. Jahani, Benjamin S Manning, Joe Zhang, et al.
This paper shows that user prompt adaptation accounts for roughly half of performance gains from model upgrades in structured tasks but plays a limited role in open-ended creative tasks.
Yuren Mao, Yuhang Ge, Yijiang Fan, et al.
A comprehensive survey categorizing LoRA variants for LLMs into downstream adaptation, cross-task generalization, efficiency, privacy, and applications.
Yizhao Gao, Shu-Yu Guo, Shijie Cao, et al.
SeerAttention-R adapts sparse attention for long-context reasoning, achieving strong AIME benchmark results with 4K token budgets and large sparse blocks.
Shaowen Wang, Linxi Yu, Jian Li
Proposes a novel initialization scheme for LoRA that uses gradient approximation to accelerate convergence.
Unknown
MtLoRA introduces task-agnostic and task-specific low-rank adaptation modules for efficient multi-task learning in vision.
Soufiane Hayou, Nikhil Ghosh, Bin Yu
This paper identifies and addresses suboptimal fine-tuning in LoRA for wide models by proposing Lora+, a method that improves parameter efficiency and performance.
Unknown
Introduces a subspace-inspired Low-Rank Adaptation (LoRA) method that is computationally efficient and easy to implement for large language models.
Unknown
DyLoRA introduces a dynamic, search-free low-rank adaptation method that trains LoRA blocks for a range of ranks to improve parameter-efficient tuning of pre-trained models.
Menglin Yang, Jialin Chen, Jinkai Tao, et al.
This paper provides a comprehensive review of Low-Rank Adaptation (LoRA) for foundation models, covering its methodology, emerging frontiers, and applications across multiple domains.
Unknown
Gora introduces gradient-driven adaptive low-rank adaptation that matches full fine-tuning performance.
Unknown
This paper analyzes the expressive power of Low-Rank Adaptation (LoRA) for fine-tuning pre-trained models.
Unknown
LoRA freezes pre-trained model weights and injects trainable low-rank matrices to enable efficient fine-tuning of large language models.