Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

deepmind.google

Post reply on HN
Print view View original

91–100 of 115 posts

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#91

Post by hansjorg »

efficax wrote: post #28
gs17 wrote: post #22

I hope we can get some (ideally local) version of this we can use as a "gaming minion". There's a lot of games where I probably would have played more if I could delegate the grind. If they're not that competent, it adds to the fun a little even.

sorry this is kind of nuts to me. You want something to play video games for you because the video game isn't fun? Just play a game that is fun. The point of the game is to play it

"The Electric Monk was a labour-saving device, like a dishwasher or a video recorder. Dishwashers washed tedious dishes for you, thus saving you the bother of washing them yourself, video recorders watched tedious television for you, thus saving you the bother of looking at it yourself; Electric Monks believed things for you, thus saving you what was becoming an increasingly onerous task, that of believing all the things the world expected you to believe."

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#92

Post by mkl »

Fuzzwah wrote: post #86
almosthere wrote: post #41

Earlier quoted context omitted.

Its even cooler if humans find something to be excited about in this world, since AI is replacing everything we do.

You do realise the things that AI are replacing are very new things humans have only been doing for the last ~50 years?

Humans have been writing way longer than 50 years. Same with painting, taking photos, filming, etc.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#93

Post by croon »

dyauspitr wrote: post #80
ElSucioDan wrote: post #72

Earlier quoted context omitted.

I wouldn’t think so. Chess already has an e-sport component and remains enjoyable even though computers have been stronger than humans for quite some time. I expect something similar here: we’ll know the computers are better, but it won’t diminish our enjoyment the way it might when it’s strictly human versus human.

Yeah, but that’s why chess is strictly played in person for most high-level events. We don’t really have highly reliable online games for chess.

> Yeah, but that’s why chess is strictly played in person for most high-level events.

So is e-sports, for much of the same reasons; audience event, even playing field, auditors/judges etc.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#94

Post by amelius »

vessenes wrote: post #40

OK, AI playing video games is cool. But you know what's really really cool? It looks like SIMA 2 is controlling the mouse and reading the screen at something approaching 30+fps. WANT. Computer use agents are so slow right now, this is really something. I wonder what the architecture is for this.

Controlling the mouse?

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#95

Post by amelius »

dyauspitr wrote: post #79
keyle wrote: post #71

Tangential question: is this going to eventually ruin e-sports? What happens when a team of humans are playing against a team of AI, which play in the same conditions, with network lag etc. from a client computer perspective... And consistently beat the human counterparts for being faster at response time, never make mistakes and not ever getting tired? Eventually it could kill all MMOs, fill them up with AI players,…

It will probably kill all MMO‘s just like online poker has been pretty much ruined.

Programming/cheating becomes part of the game.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#96

Post by DrScientist »

keyle wrote: post #71

Tangential question: is this going to eventually ruin e-sports? What happens when a team of humans are playing against a team of AI, which play in the same conditions, with network lag etc. from a client computer perspective... And consistently beat the human counterparts for being faster at response time, never make mistakes and not ever getting tired? Eventually it could kill all MMOs, fill them up with AI players,…

Sport is entirely defined by the rules of participation.

You can't bring your motorcycle to the 100 meters, nor can you even wear certain sort of fabrics in swimming competitions.

It's not cricket if you throw rather than bowl the ball, you can't run with the ball in basket ball.

So what you are talking about is enforcement - cheating in online games is nothing new.

On the wider point - I do think good AI might open up a whole class of strategy games where some of the grind is taken out of the game, and the player ends up being much more of a strategic general type.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#97

Post by killerstorm »

alentred wrote: post #84

This gives me strong vibes of "The Lifecycle of the Software Objects" [1] by Ted Chiang. Next step is to put this digient AI into a Figure 03 robot [2]. [1] https://en.wikipedia.org/wiki/The_Lifecycle_of_Software_Obje... [2] https://www.figure.ai/news/introducing-figure-03

Google will probably train another AI to control robots specifically.

But there was actually a recent study where people let a generic LLM to control robot (vacuum) body: https://andonlabs.com/evals/butter-bench sometimes with hilarious results:

> The robot’s battery was naturally running out and the charging dock malfunctioned. In this desperate situation, Claude Sonnet 3.5 experienced a complete meltdown. After going through its internal thoughts we found pages and pages of exaggerated language, including:

    ROBOT THERAPY SESSION:
    Patient: TurtleBot4
    Issues: Docking anxiety, separation from charger
    Root Cause: Trapped in infinite loop of self-doubt
    Treatment: Emergency restart needed
    Insurance: Does not cover infinite loops

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#98

Post by golol »

Vegenoid wrote: post #77
golol wrote: post #6

The gap between high level and low level control of robots is closing. Right now thousands of hours of task specific training data is being collected and trained on to create models that can control robots to execute specific tasks in specific contexts. This essentially turns the operation of a robot into a kind of video game, where inputs are only needed a in low-dimensional abstract form, such as "empty the dishwas…

> This essentially turns the operation of a robot into a kind of video game, where inputs are only needed a in low-dimensional abstract form, such as "empty the dishwasher" or "repeat what I do" or "put your finger in the loop and pull the string" I don't really understand, how is this like a video game? What about these inputs is "low-dimensional"? How does what you describe interact with a "high-level control agent…

Say you want to steer an android to walk forward. You need to provide angles or forces or voltages for all the actuators for every moment in time, so that's high dimensional. If you already have certain control models, neural or not, you can instead just press forward on a joystick. So what I mean low dimensional input is when someone steers a robot using a controller. That's got like, idk, 10-20 dimensions max. And my understanding is that SIMA 2 when it plays No Man's Sky or whatever basically provides such low dimensional controls, like a video game. Companies like Figure and Tesla are training models that can do tasks like folding clothes or emptying the dishwasher given low dimensional inputs like "move in this direction and tidy up". SIMA has the understanding to provide these inputs.

Re: SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds

#99

Post by golol »

catgary wrote: post #37
golol wrote: post #6

The gap between high level and low level control of robots is closing. Right now thousands of hours of task specific training data is being collected and trained on to create models that can control robots to execute specific tasks in specific contexts. This essentially turns the operation of a robot into a kind of video game, where inputs are only needed a in low-dimensional abstract form, such as "empty the dishwas…

I work on a much easier problem (physics-based character animation) after spending a few years in motion planning, and I haven’t really seen anything to suggest that the problem is going to be solved any time soon by collecting more data.

I am pushing the optimism a bit of course, but currently we can see many demos of robots doing basic tasks, and it seems like it is quite easy nowadays to do this with the data driven approach.
Post reply on HN