Preprint2210
A solvable model of neural scaling laws
Unknown
This paper proposes a solvable model that explains the empirical neural scaling laws observed in large language models.
0Jan 1, 2210Large Language ModelsScaling Laws
arXiv
AI Research
AI & ML Research Papers
A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Large Language Models
83 papersPreprint2024
Minicache: Kv cache compression in depth dimension for large language models
Unknown
MiniCache introduces a novel KV cache compression method that operates along the depth dimension of large language models, achieving significant memory savings.
0Jan 1, 2024Large Language Models
Preprint2024
Unlocking efficiency in large language model inference: A comprehensive survey of speculative decoding
Unknown
A comprehensive survey of speculative decoding techniques for efficient large language model inference.
0Jan 1, 2024Large Language Models
Preprint2024
Intactkv: Improving large language model quantization by keeping pivot tokens intact
Unknown
Intactkv improves large language model quantization by preserving pivot tokens, achieving state-of-the-art results.
0Jan 1, 2024Large Language ModelsQuantization
arXiv
ConferenceInternational Conference on Machine Learning2025
GuidedQuant: Large language model quantization via exploiting end loss guidance
Jinuk Kim, Marwa El Halabi, Wonpyo Park, et al.
GuidedQuant improves post-training quantization of large language models by using end-loss guidance to better preserve model accuracy.
19Jan 1, 2025Large Language ModelsQuantization
arXiv
Preprint2024
Llmc: Benchmarking large language model quantization with a versatile compression toolkit
Unknown
This paper introduces Llmc, a versatile toolkit for benchmarking LLM quantization methods, enabling standardized evaluation and comparison.
0Jan 1, 2024Large Language ModelsReasoning
PreprintInternational Conference on Learning Representations2407
Leanquant: Accurate and scalable large language model quantization with loss-error-aware grid
Tianyi Zhang, Anshumali Shrivastava
Leanquant introduces a loss-error-aware grid quantization method for LLMs that achieves accurate and scalable compression by minimizing quantization error with respect to the model's loss function.
26Jan 1, 2407Large Language ModelsQuantization
arXiv
Preprint2025
Ostquant: Refining large language model quantization with orthogonal and scaling transformations for better distribution fitting
Unknown
Ostquant improves LLM quantization by applying orthogonal and scaling transformations to better fit weight distributions, reducing accuracy loss.
0Jan 1, 2025Large Language ModelsScaling Laws
arXiv
Preprint2024
A survey on knowledge distillation of large language models
Unknown
This survey systematically reviews knowledge distillation techniques for transferring capabilities from large proprietary LLMs to smaller models.
0Jan 1, 2024Large Language ModelsKnowledge Distillation
arXiv
PreprintElectronics2024
A Review of Large Language Models: Fundamental Architectures, Key Technological Evolutions, Interdisciplinary Technologies Integration, Optimization and Compression Techniques, Applications, and Challenges
Songyue Han, Mingyu Wang, Jialong Zhang, et al.
A comprehensive survey of LLMs covering architectures, key technologies, interdisciplinary integrations, optimization, applications, and challenges.
62Dec 21, 2024Large Language ModelsReinforcement Learning
Preprint2026
K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs
Hao Liang, Qihan Lin, Zhaoyang Han, et al.
Introduces K12-KGraph, a curriculum-aligned knowledge graph from Chinese textbooks, with benchmark and training data to improve LLMs' curriculum cognition.
0Jul 23, 2026Large Language ModelsFine Tuning
arXiv
Preprint2022
Lora: Low-rank adaptation of large language models.
Unknown
LoRA freezes pre-trained model weights and injects trainable low-rank matrices to enable efficient fine-tuning of large language models.
0Jan 1, 2022Large Language Models