Preprint2025
How feature learning can improve neural scaling laws
Unknown
Develops a solvable model showing how feature learning improves neural scaling laws beyond the kernel limit.
0Jan 1, 2025Scaling Laws
AI Research
AI & ML Research Papers
A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
All Papers
24 papersPreprint2210
A solvable model of neural scaling laws
Unknown
This paper proposes a solvable model that explains the empirical neural scaling laws observed in large language models.
0Jan 1, 2210Large Language ModelsScaling Laws
arXiv
Preprint2210
Broken neural scaling laws
Unknown
Proposes a smoothly broken power law functional form (Broken Neural Scaling Law) to accurately model and extrapolate scaling behaviors in deep learning.
0Jan 1, 2210Scaling Laws
arXiv
Preprint2022
Revisiting neural scaling laws in language and vision
Unknown
This paper revisits neural scaling laws for language and vision models, finding that scaling trends hold across modalities but with different exponents and saturation points.
0Jan 1, 2022Scaling LawsComputer Vision
Preprint2022
Beyond neural scaling laws: beating power law scaling via data pruning
Unknown
This paper demonstrates that data pruning can improve neural scaling laws, reducing the resource costs of deep learning.
0Jan 1, 2022Scaling Laws
Preprint2402
A dynamical model of neural scaling laws
Unknown
This paper introduces a dynamical model that reproduces key observations about neural scaling laws, explaining why performance scales with training time and model size.
0Jan 1, 2402Scaling Laws
arXiv
Preprint2024
Explaining neural scaling laws
Unknown
This paper explains neural scaling laws by deriving them from realistic data assumptions within learning theory.
0Jan 1, 2024Scaling Laws
Preprint2025
Ostquant: Refining large language model quantization with orthogonal and scaling transformations for better distribution fitting
Unknown
Ostquant improves LLM quantization by applying orthogonal and scaling transformations to better fit weight distributions, reducing accuracy loss.
0Jan 1, 2025Large Language ModelsScaling Laws
arXiv
Preprint2026
SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation
Junsong Chen, Jincheng Yu, Yitong Li, et al.
SANA-Video 2.0 introduces hybrid linear-softmax attention and attention residuals to achieve softmax-level video quality with linear-complexity scaling, enabling 720p generation on a single GPU.
0Jul 23, 2026TransformersAttention Mechanisms
arXiv
Preprint2023
Scaling down to scale up: A guide to parameter-efficient fine-tuning
Unknown
A systematic overview of parameter-efficient fine-tuning methods, covering over 50 papers from 2019 to 2024.
0Jan 1, 2023Fine TuningScaling Laws
arXiv
PreprintarXiv.org2024
A survey on mixture of experts
Weilin Cai, Juyong Jiang, Fan Wang, et al.
A comprehensive survey on mixture of experts (MoE) as an effective method for scaling model capacity with minimal computation overhead.
175Jan 1, 2024Scaling Laws
Preprint2025
A survey on mixture of experts in large language models
Unknown
This survey comprehensively reviews mixture of experts (MoE) methods for scaling large language models efficiently, covering architectures, training, and applications.
0Jan 1, 2025Large Language ModelsScaling Laws