Live data from Hacker News

Gemini 2.5 Computer Use model

blog.google

231–240 of 339 posts

Re: Gemini 2.5 Computer Use model

#231

Earlier quoted context omitted.

This is just like the comments suggesting we need sensors and signs specifically for self-driving cars for them to work. It'll never happen, so companies need to deal with the reality we have.

We can build tons of infrastructure for cars that didn't exist before but can't for other things anymore? Seems like society is just becoming lethargic.

No, it's just hilariously impractical if you bother to think about it for more than five seconds.

Re: Gemini 2.5 Computer Use model

#232
post #142

Earlier quoted context omitted.

I was concerned there might be sensitive info leaked in the browserbase video at 0:58 as it shows a string of characters in the browser history: nricy.jd t.fxrape oruy,ap. majro 3 groups of 8 characters, space separated followed by 5 for a total of 32 characters. Seemed like text from a password generator or maybe an API key? Maybe accidentally pasted into the URL bar at one point and preserved in browser history? I…

Is this as impressive as it initially seems though? A Bing search for the text shows up some Web results for Dvorak to QWERTY conversion, I think because the word ‘t.fxrape’ (keyboard) hits. So there’s a lot of good luck happening there.

Here's the chat session - you can expand the thought process and see that it tried a few things (hands misaligned with the keyboard for example) before testing the Dvorak keyboard layout idea.

https://chatgpt.com/share/68e5e68e-00c4-8011-b806-c936ac657a...

I also found it interesting that despite me suggesting it might be a password generator or API key, ChatGPT doesn't appear to have given that much consideration.

Re: Gemini 2.5 Computer Use model

#233
post #71

Many years ago I was sitting at a red light on a secondary road, where the primary cross road was idle. It seemed like you could solve this using a computer vision camera system that watched the primary road and when it was idle, would expedite the secondary road's green light. This was long before computer vision was mature enough to do anything like that and I found out that instead, there are magnetic systems that…

It’s funny that you used traffic signals as an example of overcomplicating a problem with AI because there turns out to be a YC funded startup making AI powered traffic lights: https://www.ycombinator.com/companies/roundabout-technologie...

And even funnier in that context: it’s called ‘roundabout technologies’.

Re: Gemini 2.5 Computer Use model

#235
post #168

> Solve today's Wordle Stucks with: > ...the task is just to "solve today's Wordle", and as a web browsing robot, I cannot actually see the colors of the letters after a guess to make subsequent guesses. I can enter a word, but I cannot interpret the feedback (green, yellow, gray letters) to solve the puzzle.

So I guess it’s browsing in grey scale?

I tested it and gemini seems not able to solve wordle in grayscale

https://g.co/gemini/share/234fb68bc9a4

Re: Gemini 2.5 Computer Use model

#236
post #168

> Solve today's Wordle Stucks with: > ...the task is just to "solve today's Wordle", and as a web browsing robot, I cannot actually see the colors of the letters after a guess to make subsequent guesses. I can enter a word, but I cannot interpret the feedback (green, yellow, gray letters) to solve the puzzle.

So I guess it’s browsing in grey scale?

It can definitely see color - I asked it to go to bing and search for the two most prominent colors in the bing background image and it did so just fine. It seems extremely lazy though; it prematurely reported as "completed" most of the tasks I gave it after the first or second step (navigating to the relevant website, usually).

Re: Gemini 2.5 Computer Use model

#237
post #89

I just have to say that I consider this an absolutely hilarious outcome. For many years, I focused on tech solutions that eliminated the need for a human to be in front of a computer doing tedious manual operations. For a wide range of activities, I proposed we focus on "turning everything in the world into database objects" so that computers could operate on them with minimal human effort. I spent significant effort…

> computer systems that can open chrome, start a video chat with somebody, go back and forth for a while to achieve a task, then hang up...

all the pieces are there, though I suspect the first to implement this will be scammers and spear phishers.

Re: Gemini 2.5 Computer Use model

#238
post #71

Many years ago I was sitting at a red light on a secondary road, where the primary cross road was idle. It seemed like you could solve this using a computer vision camera system that watched the primary road and when it was idle, would expedite the secondary road's green light. This was long before computer vision was mature enough to do anything like that and I found out that instead, there are magnetic systems that…

Ironically now that computer vision is commonplace, the cameras you talk about have become increasingly popular over the years because the magnetic systems do not do a very good job of detecting cyclists and the cameras double as a congestion monitoring tool for city staff.

Sadly, most signal controllers are still using firmware that is not trajectory aware, so rather than reporting the speed and distance of an oncoming vehicle, these vision systems just emulate a magnetic loop by flipping a 0 to a 1 to indicate mere presence rather than passing along the richer data that they have.

Re: Gemini 2.5 Computer Use model

#239
post #173
post #69

Earlier quoted context omitted.

Yeah I think it would be better to just have the model write out playwright scripts than the way it's doing it right now (or at least first navigate manually and then based on that, write a playwright typescript script for future tests). Cuz right now it's way too slow... perform an action, then read the results, then wait for the next tool call, etc.

Exactly this. I’ve spent some time last week at a 50 something people web agency helping them setup QA process where agents explore the paths and based on those passes write automated scripts that humans verify and put into testing flow.

That's nice. Do you have some tips/tricks based on your experience that you can share?

Re: Gemini 2.5 Computer Use model

#240

Really feels like computer use models may be vertical agent killers once they get good enough. Many knowledge work domains boil down to: use a web app, send an email. (e.g. recruiting, sales outreach)

Why do you need an agent use web app through UI? Can't agent be integrated into web app natively? IMO for verticals you mentioned the missing piece is for an agent to be able to make phone calls.
Post reply on HN