Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini Robotics-ER 1.6

deepmind.google

Post reply on HN
Print view View original

71–80 of 91 posts

Re: Gemini Robotics-ER 1.6

#71

Post by nozzlegear »

tootie wrote: post #24
sho_hn wrote: post #2

It does all start to feel like we'd get fairly close to being able to convincingly emulate a lot of human or at least animal behavior on top of the existing generative stack, by using brain-like orchestration patterns ... if only inference was fast enough to do much more of it. The gauge-reading example here is great, but in reality of course having the system synthesize that Python script, run the CV tasks, come bac…

Is emulating human behavior really a valuable end goal though? Humans exist as the evolutionary endpoint of exhaustion hunting large pray and organic tool-making. We've built loads of industrial and residential automation tools in the last 100 years and none of them are humanoid. I'd imagine a household robot butler would be more like R2D2 with lots and lots of arms.

> Humans exist as the evolutionary endpoint

Just want to pedantically point out that we're not at our evolutionary endpoint yet. Humans are still evolving!

Re: Gemini Robotics-ER 1.6

#72

Post by nozzlegear »

skybrian wrote: post #8

Pointing a camera at a pressure gauge and recording a graph is something that I would have found useful and have thought about writing. Does software like that exist that’s available to consumers?

I wonder how the municipal employees would react to cameras suddenly appearing on the meters around my house.

Re: Gemini Robotics-ER 1.6

#73

Post by prossercj »

rcoveson wrote: post #66
vibe42 wrote: post #21

A parcel of land. A few robot legs and arms, big battery, off-the-shelf GPU. Solar panels. Prompt: "Take care of all this land within its limits and grow some veggies."

Jury is still out as to how well it works, but the traditional prompt is: "Be fruitful, and multiply"

Don't forget the extra instructions about not eating from the tree of the knowledge of good and evil.

Re: Gemini Robotics-ER 1.6

#74

Post by pstuart »

moonu wrote: post #5
sho_hn wrote: post #2

It does all start to feel like we'd get fairly close to being able to convincingly emulate a lot of human or at least animal behavior on top of the existing generative stack, by using brain-like orchestration patterns ... if only inference was fast enough to do much more of it. The gauge-reading example here is great, but in reality of course having the system synthesize that Python script, run the CV tasks, come bac…

Idk if you've seen this already but Taalas does this interesting thing where they embed the model directly onto the chip, this leads to super-fast speeds ( https://chatjimmy.ai ) but the model they're using is an old small Llama model so the quality is pretty bad. But they say that it can scale, so if that's really true that'd be pretty insane and unlock the inference you're talking about.

It's a fascinating proposition and no doubt they'll get bigger models in there, and likely be able to cluster multiple models for mega MOE. One thing that would really be great is if they could take the power requirements down -- the chip requires 2.5KW, which is modest in terms of what the big boys use but would be an issue on a battery powered robot.

Re: Gemini Robotics-ER 1.6

#75

Post by famouswaffles »

tootie wrote: post #67
hootz wrote: post #46

Earlier quoted context omitted.

It is when the world was made to interface with us. We can't use robots for everything if they aren't emulating us, because we would have to adapt everything for the non-humanlike robots.

We build our living spaces against the constraints of the human form, but that still doesn't imply the human form is optimal for anything. There's no reason a robot traveling over smooth surface should have legs instead of wheels or treads. There's no reason to have a head. Some kind of arm is a common design feature, but certainly no reason to have two. No reason to be symmetrical. A domestic robot may be constraine…

>We build our living spaces against the constraints of the human form, but that still doesn't imply the human form is optimal for anything.

We build just about everything we expect to interact with against the constraints of the human form, not just living spaces. And yes we because we built those spaces for the human body, the human body is by definition the optimal choice.

>There's no reason a robot traveling over smooth surface should have legs instead of wheels or treads.

There's a reason. The robot becomes useless for any surface that isn't smooth. What's it going to do about stairs ? You're not going to make a bespoke solution that generalizes for us better than 'feet that work'. Do you think it's better to built a million different complex robot bodies for every situation ? That defeats the purpose of being general purpose.

Re: Gemini Robotics-ER 1.6

#76

Post by Glemllksdf »

johntb86 wrote: post #60
Glemllksdf wrote: post #54

Earlier quoted context omitted.

Really unfortunate that I forgot what YT video i saw about this just 2 weeks ago. It was about Googles PaLM-E evolution and progress. It basically has two models one which controls the robot, the other is a llm and they are combined together in some attention layer.

Must have been https://www.youtube.com/watch?v=2mrGMMmrVNE from Welch Labs.

Yes :-)

Re: Gemini Robotics-ER 1.6

#77

Post by Glemllksdf »

shireboy wrote: post #62

Maybe dumb question: One of the use cases is instrument reading of analog instruments. My brain immediately goes to "this should have some sensor sending data, and not be analog". Is having a robot dog read analog sensors really a better fit in some cases?

Having a IoT system working flawlessly across all devices you own, would be great right?

Like your washing machine reporting its state, knowing if sun is out, running only when there is a lot of sun.

Your bsement heater sending out its stats.

And your industry machine doing the same thing.

Then you realize that we are talking about industry 4.0 for a decade now, everything IoT is either closed source or always costs extra and working together? hahahaahha...

I don't know why we can't have nice things it would be that easy :|

Re: Gemini Robotics-ER 1.6

#78

Post by moffkalast »

colinator wrote: post #56
moffkalast wrote: post #44

Earlier quoted context omitted.

SO-ARM101 I guess? Or more likely the Lekivi variant.

I was also just in the market for a small experiment robot. I got the hiwonder armpi-fpv. Avoid it, the actuators are pretty bad - they're very 'grindy', the robot jitters like crazy when it moves. Any such problems with the lekivi?

Hmm I've never used the hiwonder servos so I'm not sure how they compare with the feetech/waveshare STS type, but these have been surprisignly good overall. There is still considerable backlash which results in a cumulative 1-2cm of gripper translational error when accumulated along the arm, but the control is really stable. I don't think there's any jitter at all. They are a bit loud when moving at max speed, but there is also a STS3250 brushless variant that's stronger and really quiet. Expensive though.

I haven't tested the Lekivi specifically, but lots of SO-ARMs and a custom built lekivi-like robot. I think some people have had some issues with the rear omni wheel when moving forward but I haven't seen that myself.

Re: Gemini Robotics-ER 1.6

#79

Post by fennecfoxy »

pstuart wrote: post #74
moonu wrote: post #5

Earlier quoted context omitted.

Idk if you've seen this already but Taalas does this interesting thing where they embed the model directly onto the chip, this leads to super-fast speeds ( https://chatjimmy.ai ) but the model they're using is an old small Llama model so the quality is pretty bad. But they say that it can scale, so if that's really true that'd be pretty insane and unlock the inference you're talking about.

It's a fascinating proposition and no doubt they'll get bigger models in there, and likely be able to cluster multiple models for mega MOE. One thing that would really be great is if they could take the power requirements down -- the chip requires 2.5KW, which is modest in terms of what the big boys use but would be an issue on a battery powered robot.

"The chip" no, a whole rack/deployment they offer takes 2.5kW. Not just one chip. Squeezing 2.5kW thru 1 chip would be mental.

Re: Gemini Robotics-ER 1.6

#80

Post by fennecfoxy »

Glemllksdf wrote: post #54
colinator wrote: post #45

This seems perfect to hook up to my 'LLMs can control robots over MCP' system. The idea is that LLMs are great at writing code, so let's lean in to that. I'll give it a try! I just got a bigger robot, we'll see how it does... https://colinator.github.io/Ariel/post1.html

Really unfortunate that I forgot what YT video i saw about this just 2 weeks ago. It was about Googles PaLM-E evolution and progress. It basically has two models one which controls the robot, the other is a llm and they are combined together in some attention layer.

I would look into VLAs/VLMs if you haven't already.
Post reply on HN