Scaling Agents via Continual Pre-training

Free

Scaling agentic capabilities via continual pre-training.

FreeFree tier

Type

Open Source

Visit Website GitHub (40 stars)

About Scaling Agents via Continual Pre-training

This research paper introduces Agentic Continual Pre-training (Agentic CPT), a novel training pipeline to build powerful agentic foundation models for autonomous tool use and multi-step reasoning. The authors identify that existing post-training approaches on general-purpose models underperform in agentic tasks due to optimization tensions between learning diverse behaviors and aligning to expert demonstrations. Agentic CPT resolves this by incorporating continual pre-training into the deep research agent pipeline. Based on this method, they develop AgentFounder-30B, which achieves state-of-the-art results on 10 benchmarks, including 39.9% on BrowseComp-en, 43.3% on BrowseComp-zh, and 31.5% Pass@1 on HLE, while retaining strong tool-use ability.

Key Features

Agentic Continual Pre-training (Agentic CPT) pipeline

Deep research agent model (AgentFounder-30B)

State-of-the-art performance on 10 benchmarks

Strong autonomous tool-use ability

Multi-step reasoning for complex problem-solving

Targets optimization tensions in post-training

Pros & Cons

Pros

State-of-the-art results on BrowseComp and HLE benchmarks
First to propose Agentic CPT for agentic foundation models
Strong tool-use ability retained
Open-source research paper with likely available code

Cons

Evaluated only on 10 benchmarks; generalizability may vary
Requires substantial computational resources (30B parameters)
As a research paper, not a ready-to-deploy production tool

Best For

Autonomous tool useMulti-step reasoning for complex problem-solvingDeep research tasksBuilding agentic foundation models

FAQ

What is AgentFounder?

AgentFounder is a deep research agent model developed using Agentic Continual Pre-training, achieving state-of-the-art performance on agentic benchmarks.

What does Agentic CPT address?

It addresses the optimization tensions that arise when post-training general-purpose models for agentic tasks, by incorporating continual pre-training to build robust agentic foundation models.

Is the code or model available?

The paper is open-access on arXiv; while the abstract does not explicitly link to code, the tool is categorized as open source, suggesting the model and code may be released.