A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Jack Lanchantin, Angelica Chen, Janice Lan, et al.
This paper compares offline, semi-online, and fully online RL finetuning methods for LLMs, finding online and semi-online DPO and GRPO variants perform similarly and outperform offline methods, with multi-tasking across verifiable and non-verifiable
He-Yen Hsieh, H. T. Kung
ReRound is a post-training quantization method that uses a conditional diffusion model to resolve midpoint ambiguity in round-to-nearest, improving low-bit LLM quantization without calibration data.
Unknown
This paper introduces Online DPO with a fast-slow chasing mechanism to improve LLM alignment by dynamically updating the reference model during online preference learning.
Unknown
This paper proposes Softmax-DPO, a novel adaptation of Direct Preference Optimization for recommendation systems, integrating softmax loss to enhance implicit feedback modeling.
Unknown
Diffusion-DPO aligns diffusion models to human preferences by directly optimizing the model on user preference data using Direct Preference Optimization.
Unknown
This paper introduces Token-level Direct Preference Optimization (TDPO), a method that extends DPO to optimize at the token level for improved alignment of LLMs with human preferences.
Shunyu Liu, Wenkai Fang, Zetian Hu, et al.
This survey systematically reviews Direct Preference Optimization (DPO), a streamlined alternative to RLHF for aligning LLMs with human preferences, covering its variants, applications, and future directions.
Unknown
This paper extends the analysis of length exploitation from classical RLHF to Direct Preference Optimization (DPO), studying how DPO can over-optimize for response length rather than quality.
Unknown
This paper introduces Direct Preference Optimization with an offset (DPO-offset), a modification to DPO that improves alignment by incorporating a margin term to better separate preferred and dispreferred responses.
Unknown
This paper introduces -DPO, a dynamic preference optimization method that improves DPO by adapting the reference model during training to better align LLMs with human preferences.
Unknown
Direct Preference Optimization (DPO) simplifies RLHF by directly optimizing language model policies from preference data, eliminating the need for a separate reward model.
Xian Sun, Wei Chow, Yingshuo Wang, et al.
Introduces MIST benchmark and SC2W metric for selective trust in LLMs, plus SCOPE method using balanced DPO to reduce context-induced errors without sacrificing accuracy.