Research
LittleLearner Models Trained Only on K-5 Curriculum Show Skills Are Elicited, Not Acquired
Researchers released LittleLearner, a family of language models trained from scratch on a strictly filtered K-5 elementary school curriculum, to answer whether capabilities beyond training data can be elicited or acquired through scaling, post-training, and in-context learning. The answer is largely no: scaling, post-training, and in-context learning amplify what the curriculum taught, but none meaningfully improve out-of-scope performance. The pretraining filter sets the effective capability ceiling, providing a controlled sandbox for studying knowledge acquisition and RL.
Aug 165 minNeura News