Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Computer Use model

blog.google

Post reply on HN
Print view View original

291–300 of 339 posts

Re: Gemini 2.5 Computer Use model

#291

Post by redman25 »

poopiokaka wrote: post #192
alach11 wrote: post #188

Earlier quoted context omitted.

Computer use is the most important AI benchmark to watch if you're trying to forecast labor-market impact. You're right, there are much more effective ways for ML/AI systems to accomplish tasks on the computer. But they all have to be hand-crafted for each task. Solving the general case is more scalable.

Not the current benchmarks, no. The demos in this post are so slow. Between writing the prompt, waiting a long time and checking the work I’d just rather do it myself.

They could literally run 24/7 overnight assuming they eventually become good enough to not need hand holding.

Re: Gemini 2.5 Computer Use model

#292

Post by informal007 »

xnx wrote: post #7

I've had good success with the Chrome devtools MCP ( https://github.com/ChromeDevTools/chrome-devtools-mcp ) for browser automation with Gemini CLI, so I'm guessing this model will work even better.

Computer use model comes from interactive demand with computer automatically, Chrome devtools MCP might be one of the core pushers.

Re: Gemini 2.5 Computer Use model

#294

Post by jklinger410 »

layman51 wrote: post #79
bonoboTP wrote: post #44

Earlier quoted context omitted.

Yeah, that would be pretty good honestly. It could immediately upgrade every car ever made to self driving and then it could also do your laundry without buying a new washing machine and everything else. It's just hard to do. But it will happen.

Yes, it sounds very cool and sci-fi, but having a humanoid control the car seems less safe than having the spinning cameras and other sensors that are missing from older cars or those that weren't specifically built to be self-driving. I suppose this is why even human drivers are assisted by automatic emergency braking. I am more leaning into the idea that an efficient self-driving car wouldn't even need to have a st…

> I am more leaning into the idea that an efficient self-driving car wouldn't even need to have a steering wheel, pedals

We always make our way back to trains

Re: Gemini 2.5 Computer Use model

#295

Post by jklinger410 »

layman51 wrote: post #41
jklinger410 wrote: post #28

Earlier quoted context omitted.

Why do you think we have fully self driving cars instead of just more simplistic beacon systems? Why doesn't McDonald's have a fully automated kitchen? New technology is slow due to risk aversion, it's very rare for people to just tear up what they already have to re-implement new technology from the ground up. We always have to shoe-horn new technology into old systems to prove it first. There are just so many facto…

About your self-driving car point, I feel like the approach I'm seeing is akin to designing a humanoid robot that uses its robotic feet to control the brake and accelerator pedals, and its hand to move the gear selector.

Open Pilot (https://comma.ai/openpilot) connects to your cars brain and sends acceleration, turning, etc signals to drive the car for you.

Both Open Pilot and Tesla FSD use regular cameras (ie. eyes) to try and understand the environment just as a human would. That is where my analogy is coming from.

I could say the same about using a humanoid robot to log on to your computer and open chrome. My point is also that we made no changes to the road network to enable FSD.

Re: Gemini 2.5 Computer Use model

#296

Post by thewebguyd »

ssl-3 wrote: post #228
panarky wrote: post #218

Earlier quoted context omitted.

It's not about being faster than you. It's about working independently while you do other things.

And it's a neat-enough idea for repetitive tasks. For instance: I do periodic database-level backups of a very closed-source system at work. It doesn't take much of my time, but it's annoying in its simplicity: Run this GUI Windows program, click these things, select this folder, and push the go button. The backup takes as long as it takes, and then I look for obvious signs of either completion or error on the screen…

Have you tried AutoHotKey for that? It can do GUI automation. Not an LLM, but you can pre-record mouse movements and clicks, I've used it a ton to automate old windows apps

Re: Gemini 2.5 Computer Use model

#297

Post by t_mann »

dhon_ wrote: post #142
simonw wrote: post #30

Earlier quoted context omitted.

Post edited: I was wrong about this. Gemini tried to solve the Google CAPTCHA but it was actually Browserbase that did the solve, notes here: https://simonwillison.net/2025/Oct/7/gemini-25-computer-use-...

I was concerned there might be sensitive info leaked in the browserbase video at 0:58 as it shows a string of characters in the browser history: nricy.jd t.fxrape oruy,ap. majro 3 groups of 8 characters, space separated followed by 5 for a total of 32 characters. Seemed like text from a password generator or maybe an API key? Maybe accidentally pasted into the URL bar at one point and preserved in browser history? I…

That's actually correct: https://awsm-tools.com/keyboard-layout?form%5Bfrom%5D=dvorak...

Impressive. This could legitimately have been tricky a puzzle on some Easter egg hunt, even for nerds.

Re: Gemini 2.5 Computer Use model

#298

Post by disqard »

omneity wrote: post #111
atonse wrote: post #69

Earlier quoted context omitted.

Yeah I think it would be better to just have the model write out playwright scripts than the way it's doing it right now (or at least first navigate manually and then based on that, write a playwright typescript script for future tests). Cuz right now it's way too slow... perform an action, then read the results, then wait for the next tool call, etc.

This is basically our approach with Herd[0]. We operate agents that develop, test and heal trails[1, 2], which are packaged browser automations that do not require browser use LLMs to run and therefore are much cheaper and reliable. Trail automations are then abstracted as a REST API and MCP[3] which can be used either as simple functions called from your code, or by your own agent, or any combination of such. You ca…

Looks useful! What would it take to add support for (totally random example :D) Harper's Magazine?

Re: Gemini 2.5 Computer Use model

#299

Post by omneity »

jarek83 wrote: post #274
omneity wrote: post #161

Earlier quoted context omitted.

Thanks! Let me know if you give it a shot and I’ll be happy to help you with anything.

You might want to change column title colors as they're not visible (I can see them when highlighting the text) https://herd.garden/docs/alternative-herd-vs-puppeteer/

Oh thanks! It was a bug in handling browser light mode. I just fixed it.

Re: Gemini 2.5 Computer Use model

#300

Post by qingcharles »

hugh-avherald wrote: post #248
mohsen1 wrote: post #124

> Solve today's Wordle Stucks with: > ...the task is just to "solve today's Wordle", and as a web browsing robot, I cannot actually see the colors of the letters after a guess to make subsequent guesses. I can enter a word, but I cannot interpret the feedback (green, yellow, gray letters) to solve the puzzle.

I found ChatGPT also struggled with colour detection when solving Wordle, despite my advice to use any tools. I had to tell it.

I tried to get GPT to play Wordle when Agent launched, but it was banned from the NYT and it had to play a knock-off for me instead.
Post reply on HN