The landscape of artificial intelligence is undergoing a profound transformation as we move from static content generation to dynamic, interactive simulation. At the forefront of this revolution stands PAN (General World Model), a groundbreaking model from MBZUAI’s Institute of Foundation Models that fundamentally redefines what’s possible in AI-driven video generation. Unlike conventional text-to-video models that produce single clips and then stop, PAN maintains an internal world state that persists and evolves over time. This innovative approach allows the model to function as a general world model designed to simulate and predict future states of the world as video, conditioned on history and natural language actions, enabling interactive simulations over long horizons. By bridging the gap between one-off generation and continuous simulation, PAN represents a significant leap forward in creating AI systems that can understand and interact with dynamic environments in meaningful ways.

• Architecture of PAN: Unpacking the GLP Framework
• Innovative Techniques: Causal Swin DPM and Sliding Window Diffusion
• Training and Data Construction: Building a Robust Model
• Performance Evaluation: Benchmarks and Results
• Expert Opinion
• The Future of Interactive Simulations

Architecture of PAN: Unpacking the GLP Framework

At the heart of PAN’s capabilities lies the Generative Latent Prediction (GLP) framework, a sophisticated architecture that fundamentally separates the prediction of world dynamics from visual rendering. This separation allows the model to maintain and update an internal state of the world, enabling it to function as a true interactive simulator rather than just a video generator. The GLP architecture operates through three distinct components: a vision encoder that maps input frames into latent representations, an autoregressive backbone that predicts future world states, and a video decoder that renders these states into observable video segments. For PAN specifically, this framework employs Qwen2.5-VL-7B-Instruct as the backbone for processing visual and language inputs, enabling the model to understand and predict world states with remarkable accuracy. Meanwhile, the visual rendering is handled by Wan2.1-T2V-14B, a powerful diffusion transformer adapted for high-fidelity video generation from latent representations. This architectural division is crucial because it allows PAN to focus computational resources where they’re most needed – maintaining consistent world dynamics across multiple steps while ensuring realistic visual output. By separating what happens in the simulated environment from how it appears visually, PAN achieves unprecedented stability in long-horizon simulations where traditional video models typically fail due to error accumulation and temporal drift.

Innovative Techniques: Causal Swin DPM and Sliding Window Diffusion

At the heart of PAN’s ability to generate coherent, long-form video sequences lies its innovative Causal Swin DPM (Diffusion Process Model), a mechanism that uses chunk-wise causal attention in a sliding temporal window to ensure smooth transitions between video frames and reduce error accumulation over time. This architectural breakthrough directly addresses one of the most persistent challenges in sequential video generation: the rapid degradation and temporal drift that occurs when naively chaining single-shot video models by conditioning only on the last frame. The Causal Swin DPM mechanism introduces a sliding window, chunk-wise causal denoising process that conditions on partially noised past chunks, which stabilizes long horizon video rollouts and reduces temporal drift compared to naive last frame conditioning. During operation, the decoder maintains a sliding temporal window containing two chunks of video frames at different noise levels. As denoising progresses, one chunk transitions from high noise to clean frames before exiting the window, while a new noisy chunk enters at the opposite end. The critical innovation is the chunk-wise causal attention mechanism, which ensures that later chunks can only attend to earlier ones but never to unseen future actions or frames. This architectural constraint maintains temporal causality while enabling smooth transitions between consecutive chunks. Additionally, PAN introduces controlled noise injection into conditioning frames rather than using perfectly sharp reference images. This deliberate noise addition suppresses incidental pixel details that don’t contribute to understanding scene dynamics while encouraging the model to focus on stable structural elements like object relationships and spatial layouts. The combination of these techniques – sliding window processing with causal attention and strategic noise injection – enables PAN to maintain remarkable consistency across extended simulation horizons, effectively preventing the error accumulation that typically plagues sequential video generation systems.

Training and Data Construction: Building a Robust Model

The training of PAN follows a meticulously designed two-stage process that transforms it from a collection of sophisticated components into a cohesive world model. In the first stage, the research team adapts the powerful Wan2.1-T2V-14B decoder into the novel Causal Swin DPM architecture. This foundational training was conducted on a massive compute cluster of 960 NVIDIA H200 GPUs with a flow matching objective and a hybrid sharded data parallel scheme [3]. The second stage involves integrating this adapted decoder with the frozen Qwen2.5-VL-7B-Instruct backbone under the Generative Latent Prediction (GLP) objective. Here, while the vision-language model remains static, the system learns query embeddings and fine-tunes the decoder to ensure that predicted latent states and their corresponding video reconstructions remain consistent across long simulation horizons.

The quality of any AI model is fundamentally tied to its training data, and PAN is no exception. The model’s training corpus was constructed from widely used, publicly accessible video sources that span everyday activities, human-object interactions, natural environments, and multi-agent scenarios. To ensure coherence and relevance, long-form videos were segmented into meaningful clips using shot boundary detection. A rigorous filtering pipeline was then applied to remove content that would hinder learning, such as static or overly dynamic clips, low-aesthetic-quality footage, videos with heavy text overlays, and screen recordings. This process utilized rule-based metrics, pre-trained detectors, and a custom VLM filter to guarantee only high-quality data proceeded to the final step: dense temporal recaptioning. This final curation step produces descriptions that emphasize motion and causal events, providing PAN with the rich context needed to understand and simulate action-conditioned dynamics rather than just isolated visual scenes.

Performance Evaluation: Benchmarks and Results

The true measure of any world model lies in its empirical performance, and PAN’s evaluation across multiple rigorous benchmarks demonstrates its significant leap forward in interactive simulation capabilities. Researchers assessed the model along three critical axes: action simulation fidelity metrics, long horizon forecasting, and simulative reasoning and planning, comparing it against both open-source competitors like WAN 2.1/2.2 and Cosmos 1/2, as well as leading commercial systems including KLING, MiniMax Hailuo, and Gen 3. For action simulation fidelity metrics – which evaluates how accurately the model executes language-specified actions while maintaining background stability – PAN reaches 70.3% accuracy on agent simulation and 47% on environment simulation, for an overall score of 58.6% [1]. This represents the highest fidelity among open-source models and surpasses most commercial baselines. In long horizon forecasting, PAN’s innovative architecture delivers exceptional temporal stability. The model scores 53.6% on Transition Smoothness – quantified through optical flow acceleration to measure motion continuity across action boundaries – and achieves 64.1% on Simulation Consistency, which monitors degradation over extended sequences [2]. These metrics demonstrate PAN’s ability to maintain coherent world states over extended interactions, a crucial capability for practical applications. Perhaps most impressively, PAN surpasses commercial systems such as KLING, MiniMax Hailuo, and Gen 3 on these comprehensive benchmarks [4], achieving state-of-the-art open-source results while remaining competitive with proprietary alternatives. In simulative reasoning tasks where PAN functions as an internal simulator within an agent loop, it achieves 56.1% accuracy in step-wise simulation – the best performance among open-source world models – validating its practical utility for planning and decision-making applications.

Expert Opinion

The development of PAN represents a significant milestone in the evolution of generative AI systems, moving beyond static content creation toward dynamic, interactive world simulation. At NeuroTechnus, we view this as part of a broader trend where AI models are increasingly required to maintain persistent internal states and reason about causal relationships over extended time horizons. The Generative Latent Prediction (GLP) architecture at PAN’s core demonstrates a crucial design principle that separates world dynamics from visual rendering – a concept we’ve seen gaining traction across multiple AI domains. This architectural approach allows the model to focus on understanding fundamental causal relationships rather than just surface-level visual patterns. What makes PAN particularly noteworthy is its ability to function as what we call a ‘world model’ – a system that maintains an internal representation of its environment and can simulate how that environment evolves in response to actions. This capability represents a fundamental shift from traditional video generation models that produce isolated clips without maintaining any persistent understanding of the world they’re depicting. The integration of Qwen2.5-VL-7B-Instruct for latent dynamics prediction with Wan2.1-T2V-14B for video diffusion creates a powerful synergy between high-level reasoning and detailed visual synthesis. From our perspective at NeuroTechnus, PAN’s most impressive achievement lies in its long-horizon stability – maintaining coherent simulations across multiple action steps without the rapid degradation that typically plagues sequential generation systems. The Causal Swin DPM mechanism with its sliding window approach and chunk-wise causal attention provides an elegant solution to the temporal consistency problem that has challenged previous attempts at interactive simulation. As we continue to develop our own generative AI systems, we recognize that these principles of persistent state maintenance, action-conditioned simulation, and long-term coherence will become increasingly central to creating truly intelligent systems capable of meaningful interaction with complex environments.

The Future of Interactive Simulations

The development of PAN represents a significant milestone in the evolution of interactive simulations, demonstrating how advanced world models can bridge the gap between generative video and persistent digital environments. By implementing the Generative Latent Prediction (GLP) architecture – which separates world dynamics from visual rendering – PAN achieves unprecedented action simulation fidelity metrics and long-horizon stability. This capability positions it as a transformative tool for robotics, autonomous systems, and AI agent training, where precise simulation of complex scenarios is paramount. Looking forward, three distinct scenarios emerge for PAN’s trajectory. In a positive outcome, PAN accelerates advancements in robotics and autonomous systems by enabling precise long-horizon simulations, fostering innovation in open-source AI ecosystems. A neutral scenario sees PAN gaining traction in specialized applications but facing challenges in scaling due to computational demands, with mixed adoption across academic and commercial sectors. Conversely, a negative path involves environmental and ethical concerns over energy use and synthetic reality risks leading to regulatory hurdles, slowing broader deployment and creating public distrust in AI simulations. Ultimately, PAN’s success will depend not just on technical performance but on responsible development practices that address these broader societal implications.

Frequently Asked Questions

What is PAN and what problem does it solve in AI?

PAN is a groundbreaking general world model developed by MBZUAI’s Institute of Foundation Models that enables dynamic, interactive video generation and long-horizon simulations. It solves the problem of static content generation by maintaining an internal world state that persists and evolves over time, allowing accurate prediction and simulation of future scenarios based on history and language actions.

How does the GLP architecture contribute to PAN’s capabilities?

The Generative Latent Prediction (GLP) framework at the core of PAN separates the prediction of world dynamics from visual rendering. It consists of a vision encoder mapping inputs to latent representations, an autoregressive backbone predicting future states, and a video decoder rendering these states, with Qwen2.5-VL-7B-Instruct handling dynamics and Wan2.1-T2V-14B managing visual output for stability in long simulations.

What innovative techniques does PAN use for handling long-horizon simulations?

PAN utilizes the Causal Swin DPM with a sliding temporal window and chunk-wise causal attention to ensure smooth transitions between video frames and reduce error accumulation. It also incorporates controlled noise injection into conditioning frames to suppress incidental details and focus on stable structural elements, enabling coherent simulations over extended horizons.

What performance metrics were reported for PAN in benchmarks?

PAN achieved 70.3% accuracy in action simulation fidelity, 53.6% on transition smoothness, and 64.1% on simulation consistency. These results demonstrate its superior performance compared to open-source and commercial baselines, highlighting its ability to maintain temporal stability and coherence in interactive simulations.