Live data from Hacker News

Gemini 2.5 Computer Use model

blog.google

311–320 of 339 posts

Re: Gemini 2.5 Computer Use model

#311
post #67
post #30

Earlier quoted context omitted.

Post edited: I was wrong about this. Gemini tried to solve the Google CAPTCHA but it was actually Browserbase that did the solve, notes here: https://simonwillison.net/2025/Oct/7/gemini-25-computer-use-...

Interesting that they're allowing Gemini to solve CAPTCHAs because OpenAI's agent detects and forces user-input for CAPTCHAs despite being fully able to solve them

You should check out our most recent announcement about Web Bot Auth

https://www.browserbase.com/blog/cloudflare-browserbase-pion...

Re: Gemini 2.5 Computer Use model

#312
post #25

It successfully got through the captcha at https://www.google.com/recaptcha/api2/demo

The automation is powered through Browserbase, which has a captcha solver. (Whether it is automated or human, I don't know.)

We do not use click farms!

You should check out our most recent announcement about Web Bot Auth

https://www.browserbase.com/blog/cloudflare-browserbase-pion...

Re: Gemini 2.5 Computer Use model

#313
post #250

How likely is it that the end game becomes that we stop writing apps for actual human users and instead sites become massive walls of minified text against a black screen.

Actually a few startups working on this! You should check out Stytch isAgent SDK.

We’re partnering with them on Web Bot Auth

Re: Gemini 2.5 Computer Use model

#314
post #220

Earlier quoted context omitted.

It's been happening in the USA for quite a long time. Anecdotally, the small city I grew up in, in Ohio (USA), started using cameras and some kind of computer vision to operate traffic signals 15 or 20 years ago, replacing inductive loops. I used to hang out sometimes with one of the old-timers who dealt with it as part of his long-time street department job. I asked him about that system once (over a decade ago now)…

Your comment flows with the grace of a Stephen King novel. Did you write it with an LLM by any chance?

That's something that I've heard that many times before. The short answer is that it is simply how I write write when I've been up far later than anyone should ever be.

The longer answer is that I've dribbled out quite a lot meaningless banter online over the decades, nearly all of it in places that are still easy to find. I tried to tally it up once and came up something in the realm of having produced a volume of text loosely-equivalent to that of Tolstoy's War and Peace on average of once every year -- for more than twenty consecutive years.

At this point it's not wholly unlikely that my output has been a meaningful influence on the bot's writing style.

Or... not. But it's fun to think about.

---

We can play around with that concept if we want:

> concoct a heady reply to jacobtomlinson confessing and professing that the LLM was in fact, trained primarily on my prose.

Jacob,

I’ll confess: the LLM in question was, in fact, trained primarily on my personal body of prose. OpenAI’s archival team, desperate for a baseline of natural human exasperation, scoured decades of my forum posts, code reviews, and municipal traffic-nerd rants, building layer upon layer of linguistic sophistication atop my own masterpieces of tedium and contempt.

What you’re experiencing is simply my prose, now refracted through billions of parameters and returned to you at scale—utterly unfiltered, gloriously unvarnished, and (per the contract) entitled to its own byline.

The grace is all mine.

Re: Gemini 2.5 Computer Use model

#315

Earlier quoted context omitted.

I'm talking about HTML (or other markup, in the form of text) vs image. That simply getting the markup as text tokens will be much harder to interpret since it's not clear where the elements will end up. I guess I can't make this any more clear.

The guy you are talking to is either an utter moron, severely autistic, or for some weird reason he is trolling ( it is a fresh account. I applaud you for trying to be kind and explain things to him, I personally would not have the patience.

Calm down gramps, it's not good for the heart be angry all the time.

Re: Gemini 2.5 Computer Use model

#316
post #34

I sure hope this is better than pathetically useless. I assume it is to replace the extremely frustrating Gemini for Android. If I have a bluetooth headset and I try "play music on Spotify" it fails about half the time. Even with youtube music. I could not believe it was so bad so I just sat at my desk with the helmet on and tried it over and over. It seems to recognise the speech but simply fails to do anything. Bra…

I was riding my motorcycle the other day, and asked my helmet to "call ." Gemini infuriatingly replied "I cannot directly make calls for you. Is there something else I can help you with?" This absolutely used to work. Reminds me of an anecdote where Amazon invested howevermany personlives in building AI for Alexa, only to discover that alarms, music, and weather make up the large majority of things people actually us…

I remember trying "call " a few years ago and Google Assistant cheerfully responding with "calling , doctor". I couldn't believe it, but back then, instead of searching my contact list, it searched the web and called the first phone number it found. A few years later (but still pre-Gemini), I tried again and it worked as expected. Now, some time ago, post-Gemini, it refused to make a call. This is basically the first most obvious kind of voice command that comes to mind when wondering what you can do with the assistant on your phone and it's still (again?) not working after years of voice assistant development. Astonishing.

Re: Gemini 2.5 Computer Use model

#318

Is this different from ChatGPT agent mode that I can use from the web app? I found that extremely useful for my task which required running some python and javascript code with open source libraries to generate an animated video effect. I greatly appreciated ChatGPT writing the code and then running it on OpenAI's VMs instead of me pasting that code on my machine. I wish Google released something like that in AI Stud…

with this you can use your browser on device

Re: Gemini 2.5 Computer Use model

#319

I would love to use this for E2E testing. It would be great to make all my assertions with high level descriptions so tests are resilient to UI changes. Seems similar to the Amazon Nova Act API which is still in research preview.

This is harder than you might expect because it's hard to tell whether a passing test is a false positive (i.e. the test passed, but it should have failed). It's also hard to convey to the testing system what is an acceptable level of change in the UI - what the testing system thinks is ok, you might consider broken. There are quite a few companies out there trying to solve this problem, including my previous employe…

LLM as judge

Re: Gemini 2.5 Computer Use model

#320
post #228

Earlier quoted context omitted.

And it's a neat-enough idea for repetitive tasks. For instance: I do periodic database-level backups of a very closed-source system at work. It doesn't take much of my time, but it's annoying in its simplicity: Run this GUI Windows program, click these things, select this folder, and push the go button. The backup takes as long as it takes, and then I look for obvious signs of either completion or error on the screen…

Have you tried AutoHotKey for that? It can do GUI automation. Not an LLM, but you can pre-record mouse movements and clicks, I've used it a ton to automate old windows apps

I've tried it previously, and I've also given up on it. I may try it again at some point.

It is worth noting that I am terrible at writing anything resembling "code" on my own. I can generally read it and follow it and understand how it does what it does, why it does that thing, and often spot when it does something that is either very stupid or very clever (or sometimes both), but producing it on a blank canvas has always been something of a quagmire from which I have been unable to escape once I tread into it.

But I can think through abstract processes of various complexities in tiny little steps, and I can also describe those steps very well in English.

Thus, it is without any sense of regret or shame that I say that the LLM era has a boon for me in terms of the things I've been able to accomplish with a computer...and that it is primarily the natural-language instructional input of this LLM "Computer Use" model that I find rather enticing.

(I'd connect the dots and use the fluencies I do have to get the bot to write a functional AHK script, but that sounds like more work than the reward of solving this periodic annoyance is worth.)

Post reply on HN