Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Computer Use model

blog.google

Post reply on HN
Print view View original

41–50 of 339 posts

Re: Gemini 2.5 Computer Use model

#41

Post by layman51 »

jklinger410 wrote: post #28
password54321 wrote: post #15

doesn't seem like it makes sense to train AI around human user interfaces which aren't really efficient. It is like building a mechanical horse.

Why do you think we have fully self driving cars instead of just more simplistic beacon systems? Why doesn't McDonald's have a fully automated kitchen? New technology is slow due to risk aversion, it's very rare for people to just tear up what they already have to re-implement new technology from the ground up. We always have to shoe-horn new technology into old systems to prove it first. There are just so many facto…

About your self-driving car point, I feel like the approach I'm seeing is akin to designing a humanoid robot that uses its robotic feet to control the brake and accelerator pedals, and its hand to move the gear selector.

Re: Gemini 2.5 Computer Use model

#42

Post by bonoboTP »

CuriouslyC wrote: post #21

I feel like screenshots should be the last thing you reach for. There's a whole universe of data from accessibility subsystems.

The rendered visual layout is designed in a way to be spatially organized perceptually to make sense. It's a bit like PDFs. I imagine that the underlying hierarchy tree can be quite messy and spaghetti, so your best bet is to use it in the form that the devs intended and tested it for.

I think screenshots are a really good and robust idea. It bothers the more structured-minded people, but apps are often not built so well. They are built until the point that it looks fine and people are able to use it. I'm pretty sure people who rely on accessibility systems have lots of complaints about this.

Re: Gemini 2.5 Computer Use model

#44

Post by bonoboTP »

layman51 wrote: post #41
jklinger410 wrote: post #28

Earlier quoted context omitted.

Why do you think we have fully self driving cars instead of just more simplistic beacon systems? Why doesn't McDonald's have a fully automated kitchen? New technology is slow due to risk aversion, it's very rare for people to just tear up what they already have to re-implement new technology from the ground up. We always have to shoe-horn new technology into old systems to prove it first. There are just so many facto…

About your self-driving car point, I feel like the approach I'm seeing is akin to designing a humanoid robot that uses its robotic feet to control the brake and accelerator pedals, and its hand to move the gear selector.

Yeah, that would be pretty good honestly. It could immediately upgrade every car ever made to self driving and then it could also do your laundry without buying a new washing machine and everything else. It's just hard to do. But it will happen.

Re: Gemini 2.5 Computer Use model

#45

Post by bonoboTP »

odie5533 wrote: post #35
michaelt wrote: post #26

Earlier quoted context omitted.

In my country there's a multi-airline API for booking plane tickets, but the cheapest of economy carriers only accept bookings directly on their websites. If you want to make something that can book every airline? Better be able to navigate a website.

You can navigate a website without visually decoding the image of a website.

Except if its a messy div soup with various shitty absolute and relative pixel offsets where the only way to know what refers to what is by rendering it and using gestalt principles.

Re: Gemini 2.5 Computer Use model

#46

Post by bonoboTP »

CuriouslyC wrote: post #23
password54321 wrote: post #15

doesn't seem like it makes sense to train AI around human user interfaces which aren't really efficient. It is like building a mechanical horse.

We're training natural language models to reason by emulating reasoning in natural language, so it's very on brand.

It's on the brand of stuff that works. Expert systems and formal symbolic if-else, rules based reasoning was tried, it failed. Real life is messy and fat-tailed.

Re: Gemini 2.5 Computer Use model

#47

Post by ripbozo »

asadm wrote: post #14
siva7 wrote: post #11

Earlier quoted context omitted.

probably because its ip is coming from googles own subnet

isnt it coming from browserbase container?

Interestingly the IP I got when prompting `what is my IP` was `73.120.125.54` - which is a residential comcast IP.

Re: Gemini 2.5 Computer Use model

#48

Post by pixl97 »

odie5533 wrote: post #33
pixl97 wrote: post #19

Earlier quoted context omitted.

Right, let's make APIs for everything... [Looks around and sees people not making APIs for everything] Well that didn't work.

Every website and application is just layers of data. Playwright and similar tools have options for taking Snapshots that contain data like text, forms, buttons, etc that can be interacted with on a site. All the calls a website makes are just APIs. Even a native application is made up of WinForms that can be inspected.

Ah, so now you're turning LLMs into web browsers capable of parsing Javascript to figure out what a human might be looking at, let's see how many levels deep we can go.

Re: Gemini 2.5 Computer Use model

#50

Post by thomasm6m6 »

numpad0 wrote: post #43

How big are Gemini 2.5(Pro/Flash/Lite) models in parameter counts, in experts' guesstimation? Is it towards 50B, 500B, or bigger still? Even Flash feels smart enough for vibe coding tasks.

2.5 Flash Lite replaced 2.0 Flash Lite which replaced 1.5 Flash 8B, so one might suspect 2.5 Flash Lite is well under 50B
Post reply on HN