Stream of Search (SoS): Learning to Search in Language
FreeLearning to search in language via unified search representations
About Stream of Search (SoS): Learning to Search in Language
Stream of Search (SoS) is a method proposed in a research paper that trains language models to perform search by representing the entire search process as a flattened string. It introduces a unified language for search that encompasses various symbolic search strategies. The approach is demonstrated on the game Countdown, where the model learns to combine input numbers with arithmetic operations to reach a target number. SoS involves pretraining a transformer-based language model from scratch on search trajectories generated by heuristic solvers. This pretraining increases search accuracy by 25% compared to models trained only on optimal trajectories. The model is further fine-tuned with policy improvement methods (APA and STaR), enabling it to solve 36% of previously unsolved problems, including those unsolvable by the heuristic solvers. The results indicate that language models can learn to solve problems via search, self-improve to use different search strategies, and potentially discover new ones.
Key Features
Pros & Cons
- Demonstrates that language models can learn effective search strategies from heuristic solvers
- Outperforms models trained only on optimal trajectories by 25% in search accuracy
- Fine-tuned models solve problems that were unsolvable by any heuristic solver
- Enables self-improvement to flexibly adopt different search strategies
- Requires pretraining from scratch on large datasets of search trajectories
- Demonstrated only on the Countdown game, so generalizability to other domains is unproven
- Computational cost of generating and learning from search streams may be high
- Not a ready-to-use tool; primarily a research paper with experimental results