Genie 4 builds worlds you can walk through, and now they have sound

Google DeepMind announced Genie 4 on 27 October. It is a “world model”: instead of making a fixed video, it generates a place you can move through in real time, using a keyboard to walk and look around, while the model works out what you should see and hear next.

What is new

Sound. DeepMind says Genie 4 is its first real-time world model to generate audio natively. Each world has its own ambient sound, from wind in trees to waves on a beach, plus music and, most ambitiously, characters who talk and react to you as you go.

Live steering. You can change a simulation with ordinary language while it runs, for example changing the weather, moving the camera or adding a person or object, and Gemini adapts the request to fit the scene.

Realism. DeepMind says Genie 4 handles physics such as aerodynamics and collisions better, and that its biggest leap is in how people move and respond.

Behind the scenes, Genie 4 uses other Google models. Gemini Omni grounds what the world looks like, and Gemini 3.8 Flash acts as a guide: it watches the scene, breaks goals into steps and checks each step worked, so events follow on from each other.

Why DeepMind is building it

DeepMind says simulating the world is a crucial part of building more general AI. One near-term use is training: robots and software agents can practise in endless, varied, safe simulations before they act in the real world, and DeepMind says it has been combining Genie 4 with its Gemini Robotics work. Companies are also testing it for quickly generating scenarios.

Who can use it

For now, only trusted testers. DeepMind plans to open it next to Google AI Ultra subscribers through Project Genie and to early-access partners in the Gemini API and Google AI Studio, with a waitlist open. DeepMind says the model went through internal safety evaluations, and that all video and audio it generates will carry SynthID, Google’s invisible watermark for AI-made content. That matters: worlds this realistic, with people who speak, could also be used to make convincing fake footage.