Waymo has introduced the Waymo World Model, a frontier generative model that it hopes will enable self-driving vehicles to navigate more types of roads and environments.
Robotaxis today are constrained to drive in only certain areas they are trained to navigate. Giving autonomous vehicles free rein to drive in all types of streets and terrain is not yet feasible since it is extremely difficult to train them to anticipate all possible scenarios.
Enter world models. They are systems that simulate reality and predict what will happen next. For example, if one sees a ball rolling down a street, you anticipate seeing a child running after it. That is a world model running in a human brain.
Currently, self-driving systems rely mainly on large volumes of real-world driving data and traditional simulation. That means rare and unusual events – such as debris flying off a truck – can be difficult to model comprehensively.
Waymo says its World Model is designed to strengthen anticipation by generating entirely new, hyper-realistic 3D environments, allowing engineers to test how vehicles respond to scenarios that may be too rare or dangerous to capture at scale in the real world.
Waymo said the system can model extreme weather events, wrong-way drivers and unusual obstacles such as animals or oversized debris. Engineers can modify simulations using driving inputs, scene layouts or natural language prompts, enabling ‘what if’ testing of alternate driving decisions.
The company said its self-driving system has logged nearly 200 million fully autonomous miles on public roads. Behind the scenes, the Waymo Driver has navigated billions of miles in virtual environments. The new world model is aimed at making those simulations more realistic and more controllable.
The Waymo World Model was built on Google DeepMind’s Genie 3, a general-purpose model trained on an “extremely large and diverse set of videos” of the real world, according to the company. Through post-training, that world knowledge was transferred to Waymo’s model. As a result, the world model can generate “virtually any scene,” from day-to-day driving to rare events, according to the company.