A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
Unknown
This paper critically evaluates speculative decoding's real-world effectiveness, revealing that prior benchmarks overstate its speedups due to idealized conditions.
Unknown
This paper introduces a decoding-time debiasing method using process reward models to reduce social biases in LLM generation, from fill-in tasks to open-ended generation.
Unknown
This survey provides a comprehensive overview of techniques for making large language models more efficient, covering model compression, efficient decoding, and system-level optimizations.
Ruoyu Qin, Weiran He, Weixiao Huang, et al.
Seer is a context-learning RL system that reduces long-tail latency by 72-94% and improves rollout throughput by 2.04x through divided rollout, context-aware scheduling, and adaptive grouped speculative decoding.
Xiaoqiang Lin, Aritra Ghosh, K. H. Low, et al.
REFRAG exploits block-diagonal attention sparsity in RAG to skip unnecessary decoding computations, achieving 30.85x TTFT speedup and 16x context extension without accuracy loss.
Yung-Sung Chuang, Linlu Qiu, Cheng-Yu Hsieh, et al.
Proposes Lookback Lens, a simple hallucination detector using attention weight ratios, effective across tasks and models, and reduces hallucinations via classifier-guided decoding.
Siqi Kou, Lanxiang Hu, Zhe He, et al.
Consistency LLMs accelerate LLM inference by refining the model to consistently predict the fixed point from any state, achieving 2.4-3.4x speedup over autoregressive decoding.
Xun Liang, Hanyu Wang, Yezhaohui Wang, et al.
A systematic review of controllable text generation for LLMs, defining core concepts, categorizing tasks, and analyzing methods including model retraining, fine-tuning, and decoding-time intervention.
Unknown
Hyperfitting fine-tunes LLMs to near-zero loss on small data, sharpening predictions to improve greedy decoding for long text generation.
Unknown
FastKV decouples context reduction from KV cache compression to accelerate both prefill and decoding phases in LLM inference.
Unknown
Introduces speculative decoding, an algorithm for faster sampling from autoregressive models by computing multiple tokens in parallel without altering outputs.
Xiaoxuan Liu, Lanxiang Hu, Peter Bailis, et al.
Introduces online speculative decoding to reduce LLM serving latency by adapting draft models on the fly.