By 2026, the field of artificial intelligence (AI) is experiencing a profound paradigm shift. After large language models have pushed the boundaries of text processing in the digital realm, the industry consensus has clearly shifted toward the next frontier: the physical world.

AI is no longer content with mere conversations and content generation within screens; it is now striving to "extend into the physical realm." Jensen Huang, CEO of NVIDIA, refers to this trend as "physical AI." In his view, world models are the linchpin for realizing physical AI—they enable AI to comprehend the laws governing the physical world, thereby facilitating autonomous control of robots, self-driving cars, and other devices.

(Image source: Zheshang Securities)

Despite the immense potential and utility of world models, significant hurdles and challenges remain before they can be truly deployed on a large scale.

01. Conceptual Confusion: When Everyone Talks About the Same Thing, They Mean Different Things

The first major challenge confronting world models lies in their very definition. The term "world model" has become one of the hottest and most ambiguous in the AI field since 2025. A model capable of generating videos of flames, a language model that can conjure up playable games from scratch, and a physics engine that accurately simulates combustion processes are all now dubbed "world models." Wang Zhongyuan, President of the Beijing Academy of Artificial Intelligence (BAAI), bluntly stated that many video generation models, 3D reconstruction tools, and multimodal large models are all claiming this label, yet the industry has yet to reach a consensus on the definition, technical approaches, or evaluation criteria for world models.

(Image source: Zheshang Securities)

In an essay published in June of this year, Li Feifei attempted to establish an analytical framework for this chaotic landscape. She categorized existing world models into three types: renderers, which focus solely on visual realism, generating beautiful pixels and videos without ensuring physical or geometric accuracy; simulators, which pursue structural precision, outputting geometric data, material parameters, and collision grids rather than visuals; and planners, which bridge perception and action, enabling agents to predict changes in the world before taking action. However, this categorization itself underscores the problem—if even defining "what is a world model" requires an entire essay, it indicates that the field is far from achieving technical convergence.

Wang Zhongyuan approached the issue from another perspective. He divided current technical approaches into four categories: language-centric world models (such as VLA), pixel-centric world models (such as video generation), 3D structure-centric world models (such as 3D reconstruction), and visual representation-centric world models (such as the JEPA series). Each claims to be a world model, yet none is close to being a foundational model capable of truly understanding, predicting, and interacting with the real physical world. These four approaches are all advancing in their respective directions, but there is no clear hierarchy of superiority or a universally acknowledged "correct path."

02. The Data Dilemma: Where Does the Fuel for the Physical World Come From?

If conceptual confusion is a problem of "not knowing which way to go," then data scarcity is a problem of "wanting to go but having no path." Training a model that can comprehend the physical world requires entirely different data from that used to train large language models. Large language models can scrape nearly unlimited text from the internet—web pages, books, papers, forum posts—text data is virtually abundant and cost-effective. But what about data from the physical world? Humans intuitively understand that a cup falling off a table will shatter, but for AI to learn this causal relationship, it requires multimodal interaction data with precise geometric, physical annotations, and action labels. Such data is several orders of magnitude scarcer than internet videos.

Even more problematic is that even if data is available, it may not be the right kind. Wang Zhongyuan admitted that multimodal interaction data from the real physical world is extremely scarce, and different technical approaches have varying data requirements. Take embodied intelligence as an example: robots can complete specific tasks on assembly lines but lack generalization and versatility—the reason lies in the lack of a universal understanding of worldly commonsense and physical laws. Currently, successful applications of world models are still limited to specific domains such as autonomous driving or electronic games, and the scale and diversity of data in these fields are far from sufficient to support a general-purpose world model.

Synthetic data was once seen as a potential solution. Generating large amounts of virtual data using physical simulation engines and game engines is far cheaper than collecting real data. However, this approach has its own pitfalls. While various physical simulation tools can simulate the world, because human knowledge of real physics, engine rules, and algorithms are still incomplete, simulations can never achieve 100% realism. There is always a gap between the laws of motion of objects in simulated environments and those in the real world—this is what the industry often refers to as the "simulation-to-reality gap." AI-generated geometries may look fine but harbor hidden defects such as overlapping faces or incorrect dimensions, which become absurd once fed into a physics engine for calculation. Training models with defective data can only result in models learning a defective world.

03. The Architectural Quandary: Video Generation, 3D Reconstruction, or Latent Space Prediction?

Beyond data and concepts lies an even deeper issue: even with the right data and a clear definition, we still don't know what architecture to use to build this world model. This is not just a technical choice but a fundamental divergence in assumptions across the field.

Currently, representative technical approaches include Google's Genie3-based world simulator route, which creates an interactive video environment akin to a video game that evolves in real-time based on user input. For example, with a single command to "make it rain," the entire world responds dynamically. Its advantage lies in the bidirectional interaction between visuals and users, supporting long-term, coherent exploration. However, at its core, it is still based on video generation logic and does not truly grasp the underlying physical causality.

Another example is Li Feifei's World Labs team's spatial route, centered on 3D structures. Their Marble model can generate persistent, downloadable 3D environments, allowing users to generate an exportable 3D world with just a single prompt. However, critics point out that Marble seems more like a 3D rendering pipeline than a robot's brain. It captures what the surface looks like but does not incorporate the physical laws governing why the world behaves as it does. For humans, seeing a ball placed on a slope is enough to know it will roll down; but for a robot to make the same judgment, it needs information such as mass, friction, and velocity.

Then there is the cognitive route, represented by Yann LeCun's JEPA architecture. Its core idea is to predict the next representation rather than the next data point. The model does not need to waste computational power generating pixels but can focus solely on capturing world states useful for AI decision-making. This approach is theoretically closer to the concept of "mental models" in cognitive science—the brain does not store every pixel of the world but an abstract internal representation for reasoning and prediction. However, this route is far from mature, with a vast engineering gap between abstract representations and practical actions.

04. How Far Is the World Model from Large-Scale Application?

How far is the world model from the real world? There is no simple numerical answer to this question. From a conceptual perspective, it is separated from the real world by a "fog of definition"—when everyone uses the same term to mean different things, consensus remains elusive. From a data perspective, it is separated by a "sea of data"—the scarcity of interaction data from the real physical world causes even the most optimistic researchers anxiety. From an architectural perspective, it is separated by a "wall of paradigms"—video generation, 3D reconstruction, and latent space prediction each have their theoretical support and fatal flaws.

The Beijing Academy of Artificial Intelligence (BAAI) predicts that the next three to five years will at least be a stage of continuous evolution and iteration for world models. This prediction is both optimistic and cautious—optimistic in its belief that the direction is correct, cautious in its acknowledgment that the timeframe will not be short. Wang Zhongyuan compares world models to deep learning around 2012—at that time, data silos were severe, routes were undetermined, benchmarks were still in conflict, and the ChatGPT moment had yet to arrive.

However, the challenges faced by world models may be even more profound than those faced by deep learning back then. Deep learning deals with pattern recognition—finding statistical regularities in data. World models must handle causal reasoning—understanding why objects move as they do and why events unfold as they do. The gap between these two capabilities may not be bridged by a few years of technical iteration but by a fundamental shift in cognitive paradigms.

From a long-term perspective, we should not cling to the label of "world model" or to any specific technical approach. What truly matters is whether we can enable AI to move from "looking like the world" to "understanding the world," from pixel-level simulation to causal-level reasoning.

- End -