Functional classification of world models

Functional classification of world models

Observing models, moving models, acting models

The world isn't made up of words alone.

Language models excel at handling concepts. They can read texts, reason, explain, and grasp abstract relationships. However, the world we live in is made up of space, time, objects, forces, light, and friction, prior to language.

Therefore, the next major challenge for AI is not "describing the world," but "understanding what happens within the world." This is where the so-called world model becomes important.

However, the term "world model" currently encompasses a very wide range of things. AI that generates videos, AI that determines robot behavior, and systems that perform physical simulations are all sometimes referred to by the same name. What is needed, therefore, is a perspective that classifies world models not by name, but by function.

The primary function is "rendering".

This is the ability to transform the world into a form that humans can see. For example, this includes models that generate images from text or models that depict landscapes by changing the viewpoint. What is important for a renderer is visual naturalness and beauty. However, being visually correct is not the same as being structurally correct. Even if a building in an image looks beautiful, it does not necessarily mean that its interior space or physical integrity is maintained.

The second function is "simulation."

This is the ability to deal with the structure and physical changes of the world. It calculates how the world changes, including the position, shape, mass, collisions, deformations, fluids, and friction of objects. While a renderer outputs "what it looks like," a simulator deals with "what is happening." This capability is essential in fields such as architecture, robot training, autonomous driving, games, and digital twins in manufacturing.

The third function is "planning."

This is the ability to decide what to do next based on observation and objectives. A robot picking up a cup, a car driving through an intersection, a humanoid agent tidying a room—these actions require not just recognition, but the ability to choose actions while predicting the consequences.

I believe we should add a fourth function here: "proofreading" or "verification."

A world model, no matter how sophisticated, is not reality itself. Even if the generated 3D space looks natural, it may have dimensional inaccuracies or include physically impossible structures. Therefore, a world model needs a mechanism to measure "how well its output matches reality." This is a crucial condition for AI to be trusted in the real world.

I believe we should add a fourth function here: "proofreading" or "verification."

From this perspective, the development of the world model is not merely an evolution of image generation. What is ultimately needed is a model that integrates seeing, moving, acting, and error correction.

Language models have given machines the ability to speak about the world. The world model empowers machines to think, experiment, fail, correct, and act within the context of the world. In that sense, the core of the world model lies not in "perfectly depicting the world," but in "dealing with possibilities that exist within the constraints of the world."