Live data from Hacker News

Gemini 2.5 Computer Use model

blog.google

31–40 of 339 posts

Re: Gemini 2.5 Computer Use model

#31
post #10
post #7

I've had good success with the Chrome devtools MCP ( https://github.com/ChromeDevTools/chrome-devtools-mcp ) for browser automation with Gemini CLI, so I'm guessing this model will work even better.

What sorts of automations were you able to get working with the Chrome dev tools MCP?

Not OP, but in my experience, Jest and Playwright are so much faster that it's not worth doing much with the MCP. It's a neat toy, but it's just too slow for an LLM to try to control a browser using MCP calls.

Re: Gemini 2.5 Computer Use model

#32
It is actually quite good at following instructions, but I tried clicking on job application links, and since they open in a new window, it couldn't find the new window. I suppose it might be an issue with BrowserBase, or just the way this demo was set up.

Re: Gemini 2.5 Computer Use model

#33
post #19

doesn't seem like it makes sense to train AI around human user interfaces which aren't really efficient. It is like building a mechanical horse.

Right, let's make APIs for everything... [Looks around and sees people not making APIs for everything] Well that didn't work.

Every website and application is just layers of data. Playwright and similar tools have options for taking Snapshots that contain data like text, forms, buttons, etc that can be interacted with on a site. All the calls a website makes are just APIs. Even a native application is made up of WinForms that can be inspected.

Re: Gemini 2.5 Computer Use model

#34
I sure hope this is better than pathetically useless. I assume it is to replace the extremely frustrating Gemini for Android. If I have a bluetooth headset and I try "play music on Spotify" it fails about half the time. Even with youtube music. I could not believe it was so bad so I just sat at my desk with the helmet on and tried it over and over. It seems to recognise the speech but simply fails to do anything. Brand new Pixel 10. The old speech recognition system was way dumber but it actually worked.

Re: Gemini 2.5 Computer Use model

#35

doesn't seem like it makes sense to train AI around human user interfaces which aren't really efficient. It is like building a mechanical horse.

In my country there's a multi-airline API for booking plane tickets, but the cheapest of economy carriers only accept bookings directly on their websites. If you want to make something that can book every airline? Better be able to navigate a website.

You can navigate a website without visually decoding the image of a website.

Re: Gemini 2.5 Computer Use model

#37

I feel like screenshots should be the last thing you reach for. There's a whole universe of data from accessibility subsystems.

and all sorts of situations where they don't work. When they do work it's great, but if they don't and you rely on them, you have nothing.

Re: Gemini 2.5 Computer Use model

#38
There are some absolutely atrocious UIs out there for many office workers, who spend hours clicking buttons opening popup after popup clicking repetitively on checkboxes etc. E.g. entering travel costs or somesuch in academia and elsewhere. You have no idea how annoying that type of work is, you pull out your hair. Why don't they make better UIs, you ask? If you ask, you have no idea how bad things are. Because they don't care, there is no communication, it seems fine, the software creators are hard to reach, the software is approved by people who never used it and decide based on gut feel, powerpoints and feature tickmarks. Even big name brands are horrible at this, like SAP.

If such AI tools allow to automate this soulcrushing drudgery, it will be great. I know that you can technically script things Selenium, AutoHotkey whatnot. But you can imagine that it's a nonstarter in a regular office. This kind of tool could make things like that much more efficient. And it's not like it will then obviate the jobs entirely (at least not right away). These offices often have immense backlogs and are understaffed as is.

Re: Gemini 2.5 Computer Use model

#39

Earlier quoted context omitted.

I know there was a memo telling Googlers they are expected to use AI at work and it’s expected for their performance to increase as a result.

The HBO's Silicon Valley ended way too soon. The plot pretty much writes itself.

Don't worry Maybe someone will create AI slop for this on Sora 2 or the likes (this was satire)

On a serious note, What the fuck is happening in the world.

Re: Gemini 2.5 Computer Use model

#40
post #34

I sure hope this is better than pathetically useless. I assume it is to replace the extremely frustrating Gemini for Android. If I have a bluetooth headset and I try "play music on Spotify" it fails about half the time. Even with youtube music. I could not believe it was so bad so I just sat at my desk with the helmet on and tried it over and over. It seems to recognise the speech but simply fails to do anything. Bra…

I was riding my motorcycle the other day, and asked my helmet to "call ." Gemini infuriatingly replied "I cannot directly make calls for you. Is there something else I can help you with?" This absolutely used to work.

Reminds me of an anecdote where Amazon invested howevermany personlives in building AI for Alexa, only to discover that alarms, music, and weather make up the large majority of things people actually use smart speakers for. They're making these things worse at their main jobs so they can sell the sizzle of AI to investors.

Post reply on HN