How likely is it that the end game becomes that we stop writing apps for actual human users and instead sites become massive walls of minified text against a black screen.
Hopefully we get entirely off the internet.
Gemini 2.5 Computer Use model
251–260 of 339 posts
Re: Gemini 2.5 Computer Use model
#252I feel like screenshots should be the last thing you reach for. There's a whole universe of data from accessibility subsystems.
Re: Gemini 2.5 Computer Use model
#253The irony is that most of tech companies make their money by forcing users to wade through garbage. For example, if you could browse the internet and avoid ads, why wouldn't you? If you could choose what twitter content to see outside of their useless algorithms, why wouldn't you?
It's the same as saying if you could plunder and pillage why wouldn't you
Re: Gemini 2.5 Computer Use model
#254Many years ago I was sitting at a red light on a secondary road, where the primary cross road was idle. It seemed like you could solve this using a computer vision camera system that watched the primary road and when it was idle, would expedite the secondary road's green light. This was long before computer vision was mature enough to do anything like that and I found out that instead, there are magnetic systems that…
> But I guess the world is complicated enough and computers got fast enough that having an AI look at a computer screen and move/click a mouse makes sense. It's not that the world is particularly complicated here - it's just that computing is a dynamic and adversarial environment. End-user automation consuming structured data is a rare occurrence not because it's hard, but because it defeats pretty much every way peo…
And the race is not over yet, adversaries to automation will find way to block the last approach too, in the name of monetization
Re: Gemini 2.5 Computer Use model
#255Hey - I'm on the team that launched this. Please let me know if you have any questions!
I am on https://gemini.browserbase.com/ and just click the use case mentioned on the site "Go to Hacker News and find the most controversial post from today, then read the top 3 comments and summarize the debate."
It did not work, multiple times, just gets stuck after going to Hacker news.
Re: Gemini 2.5 Computer Use model
#256I would love to use this for E2E testing. It would be great to make all my assertions with high level descriptions so tests are resilient to UI changes. Seems similar to the Amazon Nova Act API which is still in research preview.
Use Playwright and some AI model to write Playwright script, running those scripts will be much faster.
Re: Gemini 2.5 Computer Use model
#257Earlier quoted context omitted.
The sound of a revving engine is often the first warning you have that someone is about to pass you and especially how they handle it is a good sign of how likely they are to attempt a close pass rather than overtake in the legal manner with the minimum distance.
That doesn't work for EVs. Situational awareness is important, don't rely on any one thing,
"don't rely on one thing, but also let's reduce the number of things" is rather mixed messaging.
Re: Gemini 2.5 Computer Use model
#258Is this different from ChatGPT agent mode that I can use from the web app? I found that extremely useful for my task which required running some python and javascript code with open source libraries to generate an animated video effect.
I greatly appreciated ChatGPT writing the code and then running it on OpenAI's VMs instead of me pasting that code on my machine.
I wish Google released something like that in AI Studio.
Re: Gemini 2.5 Computer Use model
#259It successfully got through the captcha at https://www.google.com/recaptcha/api2/demo
Post edited: I was wrong about this. Gemini tried to solve the Google CAPTCHA but it was actually Browserbase that did the solve, notes here: https://simonwillison.net/2025/Oct/7/gemini-25-computer-use-...
Any idea how Browserbase solves CAPTCHA? Wouldn't be surprised if it sends requests to some "click farm" in a low cost location where humans solve captchas all day :\
Re: Gemini 2.5 Computer Use model
#260I've always been interested in running LLM locally to automate browser tasks, but every time I've tried, I've found the browser API to be too complex. In contrast, writing scripts directly with Playwright or Puppeteer tends to be much more stable.