Creating Virtual Realms, Opening Digital Gateways: ByteDance Leads the Charge in World Models

Without any prior promotion, a Xiaohongshu post went viral in tech circles. Blogger Ju AI shared his visit to Film & Thunder, where Tim showcased ByteDance’s latest world model: crafting immersive worlds and interacting with them. By using classic films as templates, users can freely control camera movements, open portals to step into dreamlike environments, enable robots to master tasks in virtual worlds (including autonomous driving), and allow professionals like doctors to engage in simulated learning via VR glasses.

The post did not disclose the model’s name, nor did ByteDance issue an official announcement. However, a month earlier, insiders revealed that ByteDance was developing a real-time spatial video generation model based on its video generation framework Seedance, under the direct supervision of Zhang Yiming. He coordinated multiple business units within the company, allocating AI resources and computing power, with a potential launch as early as October.

Now October has arrived, and the demonstration preceded any formal announcement. While some netizens were awestruck, remarking, “This is the true metaverse. Vision Pro seems to be making a comeback,” others dismissed it as “just a video generation model with a fancy name.”

The term “world model” has surged in popularity within tech circles this year, though its precise definition remains elusive. Google hails it as a milestone toward Artificial General Intelligence (AGI), NVIDIA views it as the cornerstone of embodied AI, and ByteDance has made it a top priority in its annual strategy. After careful consideration, I aimed to answer three questions: Why is Zhang Yiming personally involved? What are world models truly competing on? And beyond Tim’s demonstration, what is ByteDance’s ultimate ambition?

01 Zhang Yiming Takes the Helm on a Strategic Project

Those familiar with Zhang Yiming know he rarely makes public appearances in recent years, let alone champions a specific product. Yet, according to media reports, he personally oversaw cross-departmental collaboration and resource allocation for this model, mobilizing teams from model R&D, cloud infrastructure, Pico hardware, and content divisions, with computing power given top priority. Internally, this project is known as the “world model.”

According to 36Kr, the world model ranks first among ByteDance’s four key AI priorities for 2026, with an exceptionally high data budget. ByteDance is treating it not as a pilot project but as its foremost priority. In early September, ByteDance secured a $29.6 billion syndicated loan (approximately RMB 213.6 billion), which the market interpreted as a strategic boost for AI infrastructure. With funding, talent, and the founder’s direct involvement, the project’s significance speaks for itself.

Why is Zhang Yiming personally engaged? It evokes memories of Mark Zuckerberg’s metaverse push five years ago. Reality Labs invested hundreds of billions, yet Horizon Worlds failed to gain mainstream traction. Meta’s approach began with hardware and 3D worlds, attempting to build an empty city from the top down. ByteDance’s strategy differs: it starts with video generation, where Seedance creates the initial visuals, and the world model evolves the environment based on human interactions. One approach grows from content, the other from hardware—fundamentally different starting points.

Another key consideration: According to LatePost, Zhang Yiming stated at a Seed team meeting in August that ByteDance would not take shortcuts through model distillation but would instead build AGI capabilities from a more foundational level, even if it meant temporary setbacks. He praised Seedance’s unique strengths and identified programming as a current focal point.

Together, these statements send a clear message: ByteDance is not scrambling to catch up in language models but aims to lead in the emerging arena of world models.

02 Three Distinct Approaches in the World Model Race

World models have never followed a single trajectory. After reviewing recent developments, I’ve identified several key approaches.

OpenAI pursues physical simulation. In February 2024, Sora’s technical report, titled “Video Generation Models as World Simulators,” positioned video generation models as world simulators. OpenAI aims to enable AI to deduce physical laws like a deity, calculating the trajectory of every grain of sand. This path is resource-intensive and slow but holds immense potential. Sora still struggles to accurately simulate basic physical processes like glass shattering, underscoring its complexity.

Google DeepMind emphasizes real-time interaction. Genie 3, released in August 2025, is officially described as the first real-time, interactive, and realistic world model, generating explorable 3D environments from text descriptions at 20-24 fps, anchored to the real world via Google Maps street views. It already allows users to navigate these generated worlds.

NVIDIA lays the groundwork for embodied AI. Cosmos 3, unveiled in May this year, is the world’s first fully open multimodal model, integrating visual reasoning, world generation, and action prediction into a unified system. It specializes in generating training data for robots and autonomous driving, compressing training evaluation cycles from months to days.

ByteDance takes a fourth, more accessible path. Its model, built on Seedance, generates 20 fps videos on demand, with end-to-end latency around 50 milliseconds and cloud-based rendering streamed to terminals. Applications include live streaming, interactive short dramas, and gaming, with plans to adapt to Pico headsets.

Notably, 50 milliseconds approaches the threshold of human perception, and 20 fps indicates that cloud rendering and streaming have reached practical engineering standards. This path’s value lies in its focus on real-time interaction.

ByteDance’s confidence in prioritizing world models stems from its robust content ecosystem. QuestMobile data shows that Hongguo Free Short Dramas had 356 million monthly active users in May this year, with users spending an average of 125 minutes daily in February. Douyin serves as a natural testing ground, CapCut as a creator tool, Volcano Engine provides computing power, and Pico offers terminals. The recently released Seedance 2.5 can already generate 30-second videos in a single pass and accept up to 50 multimodal reference materials. These capabilities form the bedrock for world models.

The domestic landscape is equally competitive. Alibaba developed Qwen-AgentWorld and Qwen-RobotWorld, Tencent’s HY-World 2.0 emphasizes editable 3D worlds, NIO, XPeng, and Li Auto focus on driving world models, while Kuaishou’s Kling and Tencent’s Hunyuan are also iterating. According to sources, ByteDance’s internal testing showed a roughly 10% gap with global leaders earlier this year, with a goal to release at least one model on par with Genie 3 by year-end. No absolute winner exists yet, but ByteDance holds the longest content leverage.

In fact, competition in world models has clearly differentiated. According to KPMG’s assessment at WAIC 2026, world model financing in the first half of 2026 nearly matched the total for 2025, with the industry’s focus shifting from feasibility to scalable application and commercial value creation.

03 Beyond Tim’s Demonstration: ByteDance’s Broader Vision

In Tim’s demonstration, the most groundbreaking aspects were not world-building or portals but the final two points: enabling robots to learn operations in virtual worlds, including autonomous driving, and allowing professionals like doctors to simulate learning with VR glasses. These moves shift world models from entertainment to productivity.

Subtle clues reveal ByteDance’s earlier strategic布局 (which translates to “layout” or “strategy”) in this area.

As early as October 2025, ByteDance’s Seed team released Seed3D 1.0. At the time, world simulators were deemed crucial for embodied AI, providing complex scenario simulations and high-quality synthetic data for robot training. The generated 3D models could seamlessly import into simulation engines like Isaac Sim.

Moreover, NVIDIA has already blazed this trail. Cosmos 3 is generating action data for companies like Agile Robots and Li Auto, allowing robots and autonomous vehicles to “fail” 100,000 times in simulation before real-world deployment. While ByteDance hasn’t publicly positioned itself against NVIDIA, Seed3D’s role suggests clarity: the value of world models lies not in generating visually stunning videos but in creating interactive, trainable worlds.

This also explains Pico’s role. After the VR industry cooled, Pico was seen as marginalized, but ByteDance didn’t abandon it. The PICO Space Pro, originally slated for a September release, was delayed to Q4 for software experience upgrades. By moving rendering to the cloud, world models reduce the need for local computing power in headsets, lowering hardware barriers and shifting competition to AI models, computing infrastructure, and content distribution. Pico transforms from a hardware competitor to a component of the content ecosystem—a role transition only achievable through world models.

Returning to the Xiaohongshu post: without official endorsement or ByteDance-confirmed details, its emergence signals that world models, by October, are ready for demonstration, discussion, and dissemination. Zhang Yiming’s strategy—using video generation for real-time interaction, leveraging Douyin and Hongguo for content, relying on Volcano Engine for computing power, and connecting with Pico terminals—also opens doors to robotics and vocational training.

Returning to the original topic, the true outcome of the world model race may take three to five years to unfold, but the direction is clear. Now, it’s up to time to validate.

This article is original to Xinmou. For reprint authorization or business cooperation, please contact us.

— END —