Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

deepmind.google

Post reply on HN
Print view View original

101–110 of 115 posts

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#101

Post by ForHackernews »

dyauspitr wrote: post #81
woeirua wrote: post #24

Earlier quoted context omitted.

It really feels like we are determined to simulate every possible task in every possible environment instead of building true intelligence.

To be fair, humans are also pretty terrible at doing completely new things and until we’ve tried it a few times.

I don't think that's true. Humans are dramatically better than current AI systems at tackling novel problems and situations. Humans are capable of zero-shot learning by imagining how they might do something, we are able to apply general reasoning principles without previous examples.

https://arcprize.org/ is a whole category of problems that AI struggles with but humans are able to do.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#102

Post by efitz »

hansmayer wrote: post #100

Wow these folks really want to make the humans irrelevant, to the point of even killing the joy of mindlessly (or mindfully) playing a computer game.

Yeah, I was hoping that AI would make games more exciting, not play them for me.

On the other hand, it opens new worlds for farming…

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#103

Post by onlyrealcuzzo »

catgary wrote: post #37
golol wrote: post #6

The gap between high level and low level control of robots is closing. Right now thousands of hours of task specific training data is being collected and trained on to create models that can control robots to execute specific tasks in specific contexts. This essentially turns the operation of a robot into a kind of video game, where inputs are only needed a in low-dimensional abstract form, such as "empty the dishwas…

I work on a much easier problem (physics-based character animation) after spending a few years in motion planning, and I haven’t really seen anything to suggest that the problem is going to be solved any time soon by collecting more data.

Is Physics-based character animation an easier problem?

Almost any problem can be really hard depending on the amount of 9s.

Maybe there's more room for error in a lot of robotics applications than for your physics-based character animation?

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#104

Post by igleria »

hansmayer wrote: post #100

Wow these folks really want to make the humans irrelevant, to the point of even killing the joy of mindlessly (or mindfully) playing a computer game.

> make the humans irrelevant

Only the poor or undesirable ones. I have the (hopefully incorrect) feeling that universal basic income and similar proposals are red herrings (due to their impracticality) to distract the masses from the fact that once people with enough economic power don't need them to fulfill their wishes, they will be in no better position than cattle.

There might not be a universe where, for example, someone with ALS is able to benefit from a humanoid robot for their daily needs unless they already had enough money/resources before this incipient revolutionary technology is deployed en masse.

We are close to a point where even personal security or securing one's assets can be done with robots. So you would not even need to keep human private security happy.

Again, I really really hope this new technology benefits the average person. I'm not optimistic on that happening.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#105

Post by hansmayer »

efitz wrote: post #102
hansmayer wrote: post #100

Wow these folks really want to make the humans irrelevant, to the point of even killing the joy of mindlessly (or mindfully) playing a computer game.

Yeah, I was hoping that AI would make games more exciting, not play them for me. On the other hand, it opens new worlds for farming…

it feels like they've completely lost the plot.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#106

Post by in-silico »

hansmayer wrote: post #100

Wow these folks really want to make the humans irrelevant, to the point of even killing the joy of mindlessly (or mindfully) playing a computer game.

It is naive to think that their end-goal is to make models that play computer games. The reason they build models like this is because playing 3D computer games is the closest digital proxy for operating a robot or similar in the real world.

I also don't see how this "kills the joy" of playing computer games. You can still play games while this exists, nobody is going to stop you.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#107

Post by glial »

catgary wrote: post #37
golol wrote: post #6

The gap between high level and low level control of robots is closing. Right now thousands of hours of task specific training data is being collected and trained on to create models that can control robots to execute specific tasks in specific contexts. This essentially turns the operation of a robot into a kind of video game, where inputs are only needed a in low-dimensional abstract form, such as "empty the dishwas…

I work on a much easier problem (physics-based character animation) after spending a few years in motion planning, and I haven’t really seen anything to suggest that the problem is going to be solved any time soon by collecting more data.

https://danijar.com/project/dreamer4/

"We present Dreamer 4, a scalable agent that learns to solve control tasks by imagination training inside of a fast and accurate world model. ... By training inside of its world model, Dreamer 4 is the first agent to obtain diamonds in Minecraft purely from offline data, aligning it with applications such as robotics where online interaction is often impractical."

In other words, it learns by watching, e.g. by having more data of a certain type.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#108

Post by hansmayer »

in-silico wrote: post #106
hansmayer wrote: post #100

Wow these folks really want to make the humans irrelevant, to the point of even killing the joy of mindlessly (or mindfully) playing a computer game.

It is naive to think that their end-goal is to make models that play computer games. The reason they build models like this is because playing 3D computer games is the closest digital proxy for operating a robot or similar in the real world. I also don't see how this "kills the joy" of playing computer games. You can still play games while this exists, nobody is going to stop you.

Yeah, maybe read my first line. I said they are trying to make the humans irrelevant, and in the process also kill the joy of playing computer games. And nobody is stopping me from playing the computer games, just like I can write my e-mails without being bugged and begged to let the AI write it for me, because, WTF should I process my own thoughts and write my own e-mails, right?

I have no doubt that the oligpolists in charge of our tech landscape will at some point "infuse" this shit into the games or whatever the shitterm they use for coupling the text generators with perfectly fine and deterministic software in order to raise their DAU/MAU/WAU numbers and in the process make the outcomes of using the software less reliable, non-deterministic and absolutely frustrating.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#109

Post by qingcharles »

Workaccount2 wrote: post #45
vessenes wrote: post #40

OK, AI playing video games is cool. But you know what's really really cool? It looks like SIMA 2 is controlling the mouse and reading the screen at something approaching 30+fps. WANT. Computer use agents are so slow right now, this is really something. I wonder what the architecture is for this.

I desperately want an AI agent that can use my phone for me. Just something that takes instructs for each screen and execute it. "Open Chrome" "Go to xyz.com" "open hamburger menu" "Click login" etc. etc.

I have this in my bookmarks:

https://dafdef.com/aikey

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#110

Post by ksh09 »

vessenes wrote: post #40

OK, AI playing video games is cool. But you know what's really really cool? It looks like SIMA 2 is controlling the mouse and reading the screen at something approaching 30+fps. WANT. Computer use agents are so slow right now, this is really something. I wonder what the architecture is for this.

When, manus.ai came out I wondered the same, the "use computer" mode seemed really interesting to me, although I've seen JAYU [1] which implemented computer use with gemini. Moreover I saw somewhere I really don't remember navigating web browser through layouts akin vimium like experience.

Initially (my impression of) computer use was only opening chrome and doing things inside chrome, chrome-as-os experience, I think maybe cloudflare could do something better here, with their workers?

Per user instance seems really costly I really do wonder how did they architect it.

[1] https://ai.google.dev/competition/projects/jayu

Post reply on HN