Enfold: Folding World Model Imagination into Predictive Representations for Ultra-Efficient Embodied Control

Analysis

Why This Paper Matters

World generative models have been used primarily for what they produce—rendered futures, video-conditioned actions, or latent contexts—but their computation is expensive. Enfold challenges this paradigm by arguing that the most reusable asset is the internal computation that constructs a future, not the future itself. By transferring this computation into a representation inferred from the present, Enfold enables ultra-efficient embodied control, reducing action latency by up to 10.1x while maintaining strong performance. This is a significant shift because it suggests that the generative process can be amortized into a feedforward encoder, making real-time control feasible on resource-constrained robots.

The paper's approach is novel in that it uses the intermediate states of a generator as supervision for a current-only encoder, effectively distilling the generative computation into a representation that can be predicted from the present. This is distinct from prior work that either uses the generator's output or requires running the generator at inference. The finding that the learned representation can condition future generation and be read by task heads without task gradients reshaping the encoder is crucial, as it preserves the generative structure and prevents the encoder from overfitting to task-specific noise.

Technical Contributions

• Multi-level state distillation: Enfold exposes the intermediate states of a world generator as it processes the observed future and uses them to supervise a current-only encoder, transferring the generative computation into a representation.
• Gradient isolation: Task gradients are prevented from reshaping the encoder, ensuring that the representation retains the generative structure rather than being distorted by task-specific objectives.
• Latency reduction: By eliminating the need to execute the generator at deployment, Enfold achieves 3.7x lower action latency than Fast-WAM, and Enfold-Flash reaches 10.1x, enabling real-time control.
• Representation analysis: The paper provides evidence that the learned representation suppresses nuisance variation and captures long-horizon changes, and that actions adapt to human interventions, indicating that the representation is not a fixed trajectory replay.

Results

Enfold was evaluated on LIBERO, RoboTwin2.0, and real-robot tasks. It achieved strong control performance while reducing action latency by 3.7x relative to Fast-WAM, and Enfold-Flash reached 10.1x speedup. The representation analyses showed that it suppresses nuisance variation and preferentially captures changes that emerge over longer horizons. Additionally, when the current scene was altered by human intervention, both the generated continuation and the executed actions adapted, which is inconsistent with fixed trajectory replay. These results demonstrate that Enfold can maintain high performance while being significantly more efficient.

Significance

This work has broad implications for embodied AI and robotics. By showing that a world generator's internal computation can be enfolded into the present, it opens the door to using powerful generative models in real-time systems without the computational overhead. This could enable more responsive and adaptive robots that can react to changes in their environment. Furthermore, the idea of distilling generative computation into representations could be applied beyond control, potentially benefiting other areas such as planning and simulation. The paper's findings challenge the conventional use of generative models and suggest a new direction for efficient model-based reasoning.