Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

deepmind.google

Post reply on HN
Print view View original

11–20 of 115 posts

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#11

Post by jandrese »

tinfoilhatter wrote: post #9

[flagged]

It seems pretty clear to me that they're trying to develop AGI humanoid assistants/workers without the messy and expensive real world hardware. Basically approaching the problem from the other end than a company like Tesla that built a robot and are now trying to figure out how to make a computer drive it without needing constant hand holding.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#12

Post by woeirua »

Yet another blogpost that looks super impressive, until you get to the bottom and see the charts assessing held out task performance on ASKA and MineDojo and see that it's still a paltry 15% success rate. (Holy misleading chart batman!) Yes, it's a major improvement over SIMA 1, but we are still a long way from this being useful for most people.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#13

Post by eminence32 »

This is obviously just a research project, but I do wonder about the next steps:

* After exploring an learning about a virtual world, can anything at all be transferred to an agent operating in the real world? Or would an agent operating in the real world have to be trained exclusively or partially in the real world?

* These virtual worlds are obviously limited in a lot of important ways (for example, character locomotion in a game is absolutely nothing like how a multi-limbed robot moves). Does there eventually need to be more sophisticated virtual worlds that more closely mirror our real world?

* Google seems clearly interested in generalized agents and AGI, but I'm actually somewhat interested in AI agents in video games too. Many video games have companion NPCs that you can sort of give tasks to, but in almost all cases, the companion NPCs are nearly uncontrollable and very limited in what they can actually do.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#14

Post by Workaccount2 »

woeirua wrote: post #12

Yet another blogpost that looks super impressive, until you get to the bottom and see the charts assessing held out task performance on ASKA and MineDojo and see that it's still a paltry 15% success rate. (Holy misleading chart batman!) Yes, it's a major improvement over SIMA 1, but we are still a long way from this being useful for most people.

To be fair, it's 65% on all tasks (with a 75% human baseline) and 15% on unseen environments. They don't provide a human baseline for that, but I'd imagine it's much more than 15%.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#15

Post by singularity2001 »

oersted wrote: post #7

I get why they do it, they are a business. I just wish Google would get off their ivory tower and build in the open more like they used to (did they? maybe I'm misremembering...). They've acquired this bad habit of keeping all their scientific experiments closed by default and just publishing press releases. I wish it was open-source by default and closed just when there's a good reason. Don't get me wrong, I suppose…

Dreamer v3 was open, v4 coming soon?

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#16

Post by dang »

tinfoilhatter wrote: post #9

[flagged]

Could you please stop posting flamebait and breaking the site guidelines? You've unfortunately been doing it repeatedly, including this dreadful thread from a couple weeks ago: https://news.ycombinator.com/item?id=45781981. I realize the other person was doing it also, but you (Comments like what your account has been posting are not what this site is for, and destroy what it is for, so if you wouldn't mind reviewing https://news.ycombinator.com/newsguidelines.html and taking the intended spirit of the site more to heart, we'd be grateful.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#17

Post by nharada »

eminence32 wrote: post #13

This is obviously just a research project, but I do wonder about the next steps: * After exploring an learning about a virtual world, can anything at all be transferred to an agent operating in the real world? Or would an agent operating in the real world have to be trained exclusively or partially in the real world? * These virtual worlds are obviously limited in a lot of important ways (for example, character locom…

The end goal is to marry the lessons learned about HOW to learn in a virtual world with a high fidelity world model that's currently out of reach for this generation of AI. In a year or two once we have a world model that's realistic enough and fast enough, robots will be trained there and then (hopefully) generalize easily to the real world. This is groundwork trying to understand how to do that without having the models required to do it for real.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#18

Post by xnx »

JohnMakin wrote: post #10

Isn't most of this demo no man's sky? The voiceover doesn't make it clear that the world is not generated by SIMA.

It's hard to keep up with the many different models and pace of progress.

Genie 3 is Google's world generating model: https://deepmind.google/blog/genie-3-a-new-frontier-for-worl...

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#19

Post by lawlessone »

JohnMakin wrote: post #10

Isn't most of this demo no man's sky? The voiceover doesn't make it clear that the world is not generated by SIMA.

If it can get through those lengthy glitchy nms story mission tutorials quickly it's already a super intelligence.

as much as some AI annoys me. This would be great for making games more accessible.

Post reply on HN