Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Computer Use model

blog.google

Post reply on HN
Print view View original

171–180 of 339 posts

Re: Gemini 2.5 Computer Use model

#171

Post by pbhjpbhj »

martinald wrote: post #63

Interesting, seems to use 'pure' vision and x/y coords for clicking stuff. Most other browser automation with LLMs I've seen uses the dom/accessibility tree which absolutely churns through context, but is much more 'accurate' at clicking stuff because it can use the exact text/elements in a selector. Unfortunately it really struggled in the demos for me. It took nearly 18 attempts to click the comment link on the HN…

18 attempts - emulating the human HN experience when using mobile. Well, assuming it hit other links it didn't intend to anyway. /jk

Re: Gemini 2.5 Computer Use model

#172

Post by CamperBob2 »

VirgilShelton wrote: post #141
y0eswddl wrote: post #136

Earlier quoted context omitted.

and soon/now triple as surveillance.

[flagged]

You have no idea if you have anything to hide or not. It's not your call, and never has been.

https://www.amazon.com/Three-Felonies-Day-Target-Innocent/dp...

Re: Gemini 2.5 Computer Use model

#173

Post by nkko »

atonse wrote: post #69
odie5533 wrote: post #31

Earlier quoted context omitted.

Not OP, but in my experience, Jest and Playwright are so much faster that it's not worth doing much with the MCP. It's a neat toy, but it's just too slow for an LLM to try to control a browser using MCP calls.

Yeah I think it would be better to just have the model write out playwright scripts than the way it's doing it right now (or at least first navigate manually and then based on that, write a playwright typescript script for future tests). Cuz right now it's way too slow... perform an action, then read the results, then wait for the next tool call, etc.

Exactly this. I’ve spent some time last week at a 50 something people web agency helping them setup QA process where agents explore the paths and based on those passes write automated scripts that humans verify and put into testing flow.

Re: Gemini 2.5 Computer Use model

#174

Post by fn-mote »

dekhn wrote: post #162
ncallaway wrote: post #130

Earlier quoted context omitted.

> we could have computer systems that can open chrome, start a video chat with somebody, go back and forth for a while to achieve a task, then hang up... with the person on the other end ever knowing they were dealing with a computer instead of a human. Doesn't that...seem bad? I mean, it would certainly be a monumental and impressive technical accomplishment. But it still seems...quite bad to me.

Good or bad? I don't know. It just seems inevitable.

The main reason you might not know if it is a human or not is that the human interactions are so bad (eg help desk call, internet provider, any utility, even the doctor’s office front line non-medical staff).

Re: Gemini 2.5 Computer Use model

#176

Post by fn-mote »

dhon_ wrote: post #142
simonw wrote: post #30

Earlier quoted context omitted.

Post edited: I was wrong about this. Gemini tried to solve the Google CAPTCHA but it was actually Browserbase that did the solve, notes here: https://simonwillison.net/2025/Oct/7/gemini-25-computer-use-...

I was concerned there might be sensitive info leaked in the browserbase video at 0:58 as it shows a string of characters in the browser history: nricy.jd t.fxrape oruy,ap. majro 3 groups of 8 characters, space separated followed by 5 for a total of 32 characters. Seemed like text from a password generator or maybe an API key? Maybe accidentally pasted into the URL bar at one point and preserved in browser history? I…

This is the kind of response that makes me feel like we are getting left behind by the LLM.

Very nice solve, ChatGPT.

Re: Gemini 2.5 Computer Use model

#177

Post by guelo »

Havoc wrote: post #170

At some point just having APIs for the web would just make sense. Rendering it and then throwing llms at interpreting it seems…suboptimal Impressive tech nonetheless

The programmable web is a dead 20 year old dream that was crushed by the evil tech monopolists, Facebook, Google, etc. This emerging llm based automation tech is a glimmer of hope that we will be able to regain our data and autonomy.

Re: Gemini 2.5 Computer Use model

#179

Post by SXX »

VirgilShelton wrote: post #141
y0eswddl wrote: post #136

Earlier quoted context omitted.

and soon/now triple as surveillance.

[flagged]

You have nothing to hide until you automatically marked for whatever and then judged also automatically by a buggy hallucinating AI overlord.

Might be because pattern on your face or T-shirt match something bad.

And this kind of stuff already happened in UK even before "AI craze". Hundreds of people been imprisoned because of faulty accounting system:

https://en.m.wikipedia.org/wiki/British_Post_Office_scandal

"Computer says you go to prison"!

Post reply on HN