What is the 5/3 tiles? Cameras?
The model generates camera and Lidar data. As if it was a Waymo car that drove through the simulated scenario with its cameras running. This synthetic training data can then be used to train the driving models.
The Waymo World Model
211–220 of 699 posts
Re: The Waymo World Model
#212This is cool, but they are still not going about it the right way. Its much easier to build everything into the compressed latent space of physical objects and how they move, and operate from there. Everyone jumped on the end-2-end bandwagon, which then locks you into the input to your driving model being vision, which means that you have to have things like genie to generate vision data, which is wasteful.
The article is about using the world model to generate simulations, not for controlling the vehicle.
Re: The Waymo World Model
#213Suddenly all this focus on world models by Deep mind starts to make sense. I've never really thought of Waymo as a robot in the same way as e.g. a Boston Dynamics humanoid, but of course it is a robot of sorts. Google/Alphabet are so vertically integrated for AI when you think about it. Compare what they're doing - their own power generation , their own silicon, their own data centers, search Gmail YouTube Gemini wor…
But somehow google fails to execute. Gemini is useless for programming and I don’t think even bother to use it as chat app. Claude code + gpt 5.2 xhigh for coding and gpt as chat app are really the only ones that are worth it(price and time wise)
Gemini works well enough in Search and in Meet. And it's baked into the products so it's dead simple to use.
I don't think Google is targeting developers with their AI, they are targeting their product's users.
Re: The Waymo World Model
#2141. Still hard not to think that this is a huge waste of time as opposed to something that's a little more like a public transport train-ish thing, i.e. integrate with established infrastructure. 2. No seriously, is the filipino driver thing confirmed? It really feels like they're trying to bury that.
My view on Waymo and autonomous taxis in general is they will eventually make public transit obsolete. Once there is a robotaxi available to pick up and drop off every passenger directly from a to b, the whole system could be made to be super efficient. It will take time to get there though. But eventually I think we will get there. Human drivers will be banned, the roads will be exclusively used by autonomous vehicl…
If everyone in NYC tried to commute in a single-occupancy vehicle, there would be gridlock -- AVs or no.
Re: The Waymo World Model
#215It’s impressive to see simulation training for floods, tornadoes, and wildfires. But it’s also kind of baffling that a city full of Waymos all seemed to fail simultaneously in San Francisco when the power went out on Dec 22. A power outage feels like a baseline scenario—orders of magnitude more common than the disasters in this demo. If the system can’t degrade gracefully when traffic lights go dark, what exactly is…
All this simulation buys a single vehicle that drives better. That failure was a fleet-wide event (overloading the remote assistance humans).
That is, both are true: this high-fidelity simulation is valuable and it won't catch all failure modes. Or in other words, it's still on Waymo for failing during the power outage, but it's not uniquely on Waymo's simulation team.
Re: The Waymo World Model
#2161. Still hard not to think that this is a huge waste of time as opposed to something that's a little more like a public transport train-ish thing, i.e. integrate with established infrastructure. 2. No seriously, is the filipino driver thing confirmed? It really feels like they're trying to bury that.
My view on Waymo and autonomous taxis in general is they will eventually make public transit obsolete. Once there is a robotaxi available to pick up and drop off every passenger directly from a to b, the whole system could be made to be super efficient. It will take time to get there though. But eventually I think we will get there. Human drivers will be banned, the roads will be exclusively used by autonomous vehicl…
> Human drivers will be banned, the roads will be exclusively used by autonomous vehicles
I basically agree with your premise that public transit as it exists today will be rendered obsolete, but I think this point here is where your prediction hits a wall. I would be stunned if we agreed to eliminate human drivers from the road in my lifetime, or the lifetime of anyone alive today. Waymo is amazing, but still just at the beginning of the long tail.
Re: The Waymo World Model
#217> The Waymo World Model can convert those kinds of videos, or any taken with a regular camera, into a multimodal simulation—showing how the Waymo Driver would see that exact scene. Subtle brag that Waymo could drive in camera-only mode if they chose to. They've stated as much previously, but that doesn't seem widely known.
The purpose of lidar is to prove error correction when you need it most in terms of camera accuracy loss. Humans do this, just in the sense of depth perception with both eyes.
Human depth perception uses stereo out to only about 2 or 3 meters, after which the distance between your eyes is not a useful baseline. Beyond 3m we use context clues and depth from motion when available.
Re: The Waymo World Model
#218What is the 5/3 tiles? Cameras?
The model generates camera and Lidar data. As if it was a Waymo car that drove through the simulated scenario with its cameras running. This synthetic training data can then be used to train the driving models.
Wonder how it'll do. The trees change shape (presumably the Lidar patterns do too). I get the premise/why but it seems odd to me (armchair) to use fake data. Real trees don't change shape (in real time) although it can be windy.
It probably doesn't matter though, "this general blob over there"
Re: The Waymo World Model
#219Earlier quoted context omitted.
Tesla built something like this for FSD training, they presented many years ago. I never understood why they did productize it. It would have made a brilliant Maps alternative, which country automatically update from Tesla cars on the road. Could live update with speed cameras and road conditions. Like many things they've fallen behind
No Lidar anymore on the 2026 Volvo models ES60 and EX60. See for example: https://www.jalopnik.com/2032555/volvo-ends-luminar-lidar-20...
I love Volvo, am considering buying one in a couple weeks actually, but they're doing nothing interesting in terms of ADAS, as far as I can tell. It seems like they're limited to adaptive cruise control and lane keeping, both of which have been solved problems for more than a decade.
It sounds like they removed Lidar due to supplier issues and availability, not because they're trying to build self-driving cars and have determined they don't need it anymore.
Re: The Waymo World Model
#220Could these world models be used to build some sort of endless GranTurismo type street racing game?