A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Unknown
This survey systematically reviews Low-Rank Adaptation (LoRA) methods and their variants for Large Language Models (LLMs), emphasizing their efficiency in adapting models without full fine-tuning.
Unknown
Introduces Correlated Low-Rank Adaptation (CoLoRA), a novel framework that explicitly models correlations among low-rank factors to improve parameter-efficient fine-tuning of convolutional networks.
Unknown
This paper introduces Lora-fa, a memory-efficient low-rank adaptation method that reduces the memory footprint of fine-tuning large language models by avoiding the need to store full-rank gradients.
Unknown
Reflora introduces RefLoRA, a refactored low-rank adaptation method that improves the efficiency and stability of fine-tuning large models.
Unknown
QA-LoRA introduces a quantization-aware low-rank adaptation method that balances parameter efficiency and quantization robustness for fine-tuning large language models.
Unknown
LoFT is a novel low-rank adaptation method that behaves like full fine-tuning by addressing the limitations of LoRA.
Şuayp Talha Kocabay, Talha Rüzgar Akkuş, Kamer Ali Yuksel
ARCHead compresses the LLM output head by 3.7-3.9x using a quantized low-rank core, group-wise INT4 residuals, and activation-derived correction, achieving near-lossless perplexity.
Unknown
A scalable algorithm for fast model editing that leverages the low-rank structure of fine-tuning to edit very large pre-trained language models.
Unknown
This paper introduces Mtlora, a low-rank adaptation approach for efficient multi-task learning, with novel task-agnostic and task-specific modules applied to vision patch-merging layers.
Unknown
LoRA freezes pretrained weights and injects low-rank trainable matrices into Transformer layers, drastically reducing parameters for fine-tuning.
Shaowen Wang, Linxi Yu, Jian Li
Proposes a novel initialization scheme for LoRA that uses gradient approximation to accelerate convergence.
Unknown
MtLoRA introduces task-agnostic and task-specific low-rank adaptation modules for efficient multi-task learning in vision.