Live data from Hacker News

The Waymo World Model

waymo.com

271–280 of 699 posts

Re: The Waymo World Model

#271

Earlier quoted context omitted.

So if they’re already “fusioning” all these things, why would LIDAR be any different?

Tesla went nothing-but-nets (making fusion easy) and Chinese LIDAR became cheap around 2023, but monocular depth estimation was spectacularly good by 2021. By the time unit cost and integration effort came down, LIDAR had very little to offer a vision stack that no longer struggled to perceive the 3D world around it. Also, integration effort went down but it never disappeared. Meanwhile, opportunity cost skyrocketed…

Monocular depth estimation can be fooled by adversarial images, or just scenes outside of its distribution. It's a validation nightmare and a joke for high reliability.

Re: The Waymo World Model

#272

Earlier quoted context omitted.

Google has been doing more R&D and internal deployment of AI and less trying to sell it as a product. IMHO that difference in focus makes a huge difference. I used to think their early work on self-driving cars was primarily to support Street View in thier maps.

It has always felt to me that the LLM chatbots were a surprise to Google, not LLMs, or machine learning in general.

Not true at all. I interacted with Meena[1] while I was there, and the publication was almost three years before the release of ChatGPT. It was an unsettling experience, felt very science fiction.

[1]: https://research.google/blog/towards-a-conversational-agent-...

Re: The Waymo World Model

#273
post #259

Dumb question - Why would Waymo disclose this much information to public and competitors?

Maybe to distract from the story that they use remote drivers after one of their cars hit a kid? [1] [1] https://people.com/waymo-exec-reveals-company-uses-operators... edit: fixed kill -> hit

It’s awful a child was hit, but they only suffered minor injuries [1]. Nowhere in your linked article does it say they were killed.

[1] https://people.com/waymo-car-hits-child-walking-to-school-du...

Re: The Waymo World Model

#274
post #265

All this work is impressive, but I'd rather have better trains

Me too but given our extensive car brain culture, Waymo is an amazing step to getting less drivers & cars off the road, and to further cement future generations not ever needing to drive or own cars

Re: The Waymo World Model

#275
post #163

1. Still hard not to think that this is a huge waste of time as opposed to something that's a little more like a public transport train-ish thing, i.e. integrate with established infrastructure. 2. No seriously, is the filipino driver thing confirmed? It really feels like they're trying to bury that.

My view on Waymo and autonomous taxis in general is they will eventually make public transit obsolete. Once there is a robotaxi available to pick up and drop off every passenger directly from a to b, the whole system could be made to be super efficient. It will take time to get there though. But eventually I think we will get there. Human drivers will be banned, the roads will be exclusively used by autonomous vehicl…

Automated taxis would still be stuck in traffic. Automation gets couple times in capacity, but the induced demand and extra cars looking for rides and parking will mean traffic.

Automation makes public transit better. There will be automated minibuses that are more flexible and frequent than today's buses. Automation also means that buses get a virtual bus lane. Taxis solve the last mile problem, by taking taxi to the station, riding train with thousands of people, and then taking more transit.

Also, we might discover the advantage of human powered transit. Ebikes are more efficient than cars and give health benefits. They will be much safer than automated cars. Could use the extra capacity for bike and bus lanes.

Re: The Waymo World Model

#276
post #186

Suddenly all this focus on world models by Deep mind starts to make sense. I've never really thought of Waymo as a robot in the same way as e.g. a Boston Dynamics humanoid, but of course it is a robot of sorts. Google/Alphabet are so vertically integrated for AI when you think about it. Compare what they're doing - their own power generation , their own silicon, their own data centers, search Gmail YouTube Gemini wor…

Which is why it's embarrassing how much worse Gemini is at searching the web for grounding information, and how incredibly bad gemini cli is.

Not my experience in either of those areas.

Re: The Waymo World Model

#277

Interesting, but it feels like it's going to cope very poorly with actually safety-critical situations. Having a world model that's trained on successful driving data feels like it's going to "launder" a lot of implicit assumptions that would cause a car to get into a crash in real life (e.g. there's probably no examples in the training data where the car is behind a stopped car, and the driver pulls over to another…

While there most likely is going to be some bias in the training of those kinds of models, we can also hope that transfer learning from other non-driving videos will at least help generate something close enough to the very real but unusual situations you are mentioning. We could imagine an LLM serving as some kind of fuzzer to create a large variety of prompts for the world model, which as we can see in the article seems pretty capable at generating fictive scenarios when asked to.

As always tho the devil lies in the details: is an LLM based generation pipeline good enough? What even is the definition of "good enough"? Even with good prompts will the world model output something sufficiently close to reality so that it can be used as a good virtual driving environment for further training / testing of autonomous cars? Or do the kind of limitations you mentioned still mean subtle but dangerous imprecisions will slip through and cause too poor data distribution to be a truly viable approach?

My personal feeling is that this we will land somewhere in between: I think approaches like this one will be very useful, but I also don't think the current state of AI models mean we can have something 100% reliable with this.

The question is: is 100% reliability a realistic goal? Human drivers are definitely not 100% reliable. If we come up with a solution 10x more reliable than the best human drivers, that maybe has some also some hard proof that it cannot have certain classes of catastrophic failure modes (probably with verified code based approaches that for instance guarantees that even if the NN output is invalid the car doesn't try to make moves out of a verifiably safe envelope) then I feel like the public and regulators would be much more inclined to authorize full autonomy.

Re: The Waymo World Model

#278

Earlier quoted context omitted.

It has always felt to me that the LLM chatbots were a surprise to Google, not LLMs, or machine learning in general.

Not true at all. I interacted with Meena[1] while I was there, and the publication was almost three years before the release of ChatGPT. It was an unsettling experience, felt very science fiction. [1]: https://research.google/blog/towards-a-conversational-agent-...

Well, I must say ChatGPT felt much more stable than Meena when I first tried it. But, as you said, it was a few years before ChatGPT was publicly announced :)

Re: The Waymo World Model

#279
post #265

All this work is impressive, but I'd rather have better trains

Isn't a vehicle that goes from anywhere to anywhere on your own schedule, safely, privately, cleanly, and without billions in subsidies better?

Re: The Waymo World Model

#280

IIUC, there's a confusion of meaning for "World Model", between Waymo/Deepmind's which is something that can create a consistent world (for use to train Waymo's Driver), vs Yann LeCun/Advanced Machine Intelligence (AMI) which is something that can understand a world.

I don't think there's a conflict. If you can predict the world you understand it.
Post reply on HN