Preprint2025
Towards urban general intelligence: A review and outlook of urban foundation models
Unknown
A review of urban foundation models, covering pre-training methods and recent advances toward urban general intelligence.
0Jan 1, 2025
AI Research
AI & ML Research Papers
A comprehensive index of artificial intelligence and machine-learning research with AI-generated summaries, citation metrics, and direct links to papers and code.
All Papers
32 papersPreprint2026
Reusing Pre-Training Data at Test Time is a Compute Multiplier
Unknown
The paper proposes reusing pre-training data at test time as a compute multiplier to improve model performance.
0Jan 1, 2026Scaling Laws
PreprintarXiv.org2025
Rethinking Reflection in Pre-Training
Essential AI Darsh J Shah, Peter Rushton, Somanshu Singla, et al.
This paper shows that language models' self-reflection ability emerges during pre-training, not just reinforcement learning, by introducing deliberate errors into chains-of-thought.
45Apr 5, 2025Reinforcement LearningReasoning
arXiv
PreprintarXiv.org2025
Reinforcement Pre-Training
Qingxiu Dong, Li Dong, Yao Tang, et al.
Introduces Reinforcement Pre-Training (RPT), reframing next-token prediction as a reasoning task trained with RL and verifiable rewards, improving language modeling accuracy and providing a strong foundation for RL fine-tuning.
37Jun 9, 2025Large Language ModelsFine Tuning
arXiv
Preprint2025
Generalization can emerge in tabular foundation models from a single table
Unknown
This paper demonstrates that tabular foundation models can achieve generalization through in-context learning from a single table, without requiring pre-training on multiple tables.
0Nov 1, 2025In Context Learning
arXiv
Preprint2025
Table foundation models: on knowledge pre-training for tabular learning
Unknown
This paper explores knowledge pre-training for tabular foundation models, aiming to improve performance on small to moderate-sized tabular data.
0May 1, 2025
arXiv
Preprint2023
Towards foundation models for scientific machine learning: Characterizing scaling and transfer behavior
Unknown
This paper investigates how pre-training can be leveraged for scientific machine learning, focusing on transfer learning behavior and scaling characteristics.
0Jan 1, 2023Scaling Laws
PreprintACM Computing Surveys2023
Recent Advances in Natural Language Processing via Large Pre-trained Language Models: A Survey
Bonan Min, Hayley Ross, Elior Sulem, et al.
This survey reviews the shift to PLM-driven NLP, covering pre-training, fine-tuning, prompting, and text generation, and discusses limitations and future directions.
1.6kJun 27, 2023Fine TuningPrompting
arXiv
PreprintarXiv.org2025
Pre-Training, Mid-Training, and RL Interplay
Charlie Zhang, Graham Neubig, Xiang Yue
A controlled experimental framework isolates causal contributions of pre-training, mid-training, and RL to reasoning LM generalization.
58Dec 8, 2025Reinforcement LearningReasoning
arXiv
Preprint2026
UltraViT: Latency-Optimized On-device Vision Encoder for Large Vision-Language Models
Ioannis Maniadis Metaxas, Adrian Bulat, Alberto Baldrati, et al.
UltraViT is a latency-optimized vision encoder for LVLMs using a pyramidal architecture with heterogeneous spatial mixers and a two-stage generative pre-training strategy, achieving 1.7x speedup on-device.
0Jul 25, 2026MultimodalAlignment
arXiv
PreprintarXiv.org2025
LLMs Can Get “Brain Rot”!
Shuo Xing, Junyuan Hong, Yifan Wang, et al.
Shows continual pre-training on junk web text causes lasting cognitive decline in LLMs, with dose-response effects and partial healing.
4Oct 15, 2025Large Language ModelsChain Of Thought
arXiv
PreprintConference on Empirical Methods in Natural Language Processing2024
Fine-tuning and Hallucinations
Zorik Gekhman, G. Yona, Roee Aharoni, et al.
This paper shows that supervised fine-tuning introducing new factual knowledge to LLMs is learned slowly and linearly increases hallucination, supporting that factual knowledge is acquired during pre-training.
298May 9, 2024Large Language ModelsFine Tuning
arXiv