I wonder how it would behave in a scenario where it has to download some file from a shady website that has all those advertisement with fake "download"
Gemini 2.5 Computer Use model
241–250 of 339 posts
Re: Gemini 2.5 Computer Use model
#242Hey - I'm on the team that launched this. Please let me know if you have any questions!
Really cool stuff! Any interesting challenges the team ran into while developing it?
Re: Gemini 2.5 Computer Use model
#243Earlier quoted context omitted.
What sorts of automations were you able to get working with the Chrome dev tools MCP?
Not OP, but in my experience, Jest and Playwright are so much faster that it's not worth doing much with the MCP. It's a neat toy, but it's just too slow for an LLM to try to control a browser using MCP calls.
Actually the super power of having the LLM in the bowser may be that it vastly simplifies using LLMs to write Playwright scripts.
Case in point, last week I wrote a scraper for Rate Your Music, but found it frustrating. I'm not experienced with Playwright, so I used vscode with Claude to iterate in the project. Constantly diving into devtools, copying outter html, inspecting specific elements etc is a chore that this could get around, making for faster development of complex tests
Re: Gemini 2.5 Computer Use model
#244Earlier quoted context omitted.
About your self-driving car point, I feel like the approach I'm seeing is akin to designing a humanoid robot that uses its robotic feet to control the brake and accelerator pedals, and its hand to move the gear selector.
Yeah, that would be pretty good honestly. It could immediately upgrade every car ever made to self driving and then it could also do your laundry without buying a new washing machine and everything else. It's just hard to do. But it will happen.
By the time it happens you and me are probably under the ground.
Re: Gemini 2.5 Computer Use model
#245How ironic that words become more powerful than images.....
Re: Gemini 2.5 Computer Use model
#246Many years ago I was sitting at a red light on a secondary road, where the primary cross road was idle. It seemed like you could solve this using a computer vision camera system that watched the primary road and when it was idle, would expedite the secondary road's green light. This was long before computer vision was mature enough to do anything like that and I found out that instead, there are magnetic systems that…
In some European countries all of this is commonplace - check out the not just bikes video on the subject - https://youtu.be/knbVWXzL4-4?si=NLTMgHiVcgyPv6dc
Detects if you are coming to the intersection and with what speed, and if there is no traffic blocking you automatically cycles the red lights so you don’t have to stop at all.
Re: Gemini 2.5 Computer Use model
#247> Solve today's Wordle Stucks with: > ...the task is just to "solve today's Wordle", and as a web browsing robot, I cannot actually see the colors of the letters after a guess to make subsequent guesses. I can enter a word, but I cannot interpret the feedback (green, yellow, gray letters) to solve the puzzle.
It solved it in four twice for me.
Its like it sometimes just decides it can’t do that. Like a toddler.
Re: Gemini 2.5 Computer Use model
#248> Solve today's Wordle Stucks with: > ...the task is just to "solve today's Wordle", and as a web browsing robot, I cannot actually see the colors of the letters after a guess to make subsequent guesses. I can enter a word, but I cannot interpret the feedback (green, yellow, gray letters) to solve the puzzle.
I found ChatGPT also struggled with colour detection when solving Wordle, despite my advice to use any tools. I had to tell it.
Re: Gemini 2.5 Computer Use model
#249Earlier quoted context omitted.
So I guess it’s browsing in grey scale?
It can definitely see color - I asked it to go to bing and search for the two most prominent colors in the bing background image and it did so just fine. It seems extremely lazy though; it prematurely reported as "completed" most of the tasks I gave it after the first or second step (navigating to the relevant website, usually).
The models are mostly I believe capable of executing however as you rightly indicated 'lazy'. This 'laziness' I think is to conserve resource usage as much as possible as given the current state of AI market the infrastructure is being heavily subsidized for the user. This leads to perhaps the model being incentivized to produce an optimum result that satisfies the user by consuming the least amount of resources.
This is also why most 'vibe' coding projects fail as the model is always going to give this optimum ('lazy') result by default.
I have fun goading Gemini to break this ceiling when I work on my AI project - https://github.com/gingerhome/gingee
Re: Gemini 2.5 Computer Use model
#250How likely is it that the end game becomes that we stop writing apps for actual human users and instead sites become massive walls of minified text against a black screen.