Preprint2019
Relational knowledge distillation
Unknown
Relational knowledge distillation extends traditional distillation by transferring structural relationships between data points from teacher to student.
0Jan 1, 2019Knowledge Distillation
AI Research
AI & ML Research Papers
A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Knowledge Distillation
11 papersPreprint2024
A survey on knowledge distillation of large language models
Unknown
This survey systematically reviews knowledge distillation techniques for transferring capabilities from large proprietary LLMs to smaller models.
0Jan 1, 2024Large Language ModelsKnowledge Distillation
arXiv
Preprint2019
On the efficacy of knowledge distillation
Unknown
This paper investigates the factors influencing the efficacy of knowledge distillation, a technique for compressing large models into smaller ones.
0Jan 1, 2019Knowledge Distillation
Preprint2020
Knowledge distillation meets self-supervision
Unknown
This paper integrates knowledge distillation with self-supervised contrastive learning to improve student model performance without requiring labeled data.
0Jan 1, 2020Knowledge DistillationComputer Vision
Preprint2019
Similarity-preserving knowledge distillation
Unknown
Proposes similarity-preserving knowledge distillation that uses pairwise similarity matrices as supervisory signals to train student networks.
0Jan 1, 2019Knowledge DistillationNeural Networks
Preprint2025
A comprehensive survey on knowledge distillation
Unknown
A comprehensive survey of knowledge distillation methods, covering recent techniques and categorizing approaches in the field.
0Jan 1, 2025Knowledge Distillation
arXiv
Preprint2024
Logit standardization in knowledge distillation
Unknown
Proposes logit standardization as a pre-processing step to improve vanilla knowledge distillation performance.
0Jan 1, 2024Knowledge Distillation
Preprint2020
Improved knowledge distillation via teacher assistant
Unknown
This paper proposes Teacher Assistant Knowledge Distillation (TAKD) to improve knowledge transfer by introducing an intermediate assistant model between teacher and student.
0Jan 1, 2020Knowledge Distillation
Preprint2024
A survey on knowledge distillation: Recent advancements
Unknown
A comprehensive survey of knowledge distillation advancements, covering foundational techniques, relation-based methods, and novel approaches.
0Jan 1, 2024Knowledge Distillation
Preprint2021
Knowledge distillation: A survey
Unknown
This survey comprehensively reviews knowledge distillation techniques for model compression and acceleration.
0Jan 1, 2021Knowledge Distillation
PreprintZenodo (CERN European Organization for Nuclear Research)2025
Abstract Functional Language Logic: A Competitive Mixture of Experts Architecture for Paradox-Free Reasoning and Adaptive Intelligence
Torres H., Juan P.
Proposes a Competitive Mixture of Experts (CMoE) framework using Functional Language Logic to replace probabilistic token prediction with deterministic mathematical reasoning for paradox-free AI.
584Nov 28, 2025Large Language ModelsTransformers