Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gemini 2.5 Computer Use model

blog.google

Post reply on HN
Print view View original

261–270 of 339 posts

Re: Gemini 2.5 Computer Use model

#261

Post by ionwake »

bonoboTP wrote: post #139
measurablefunc wrote: post #123

Earlier quoted context omitted.

2D rendering is not necessary for processing information by neural networks. In fact, the image is flattened into 1D array & loses the topological structure almost entirely b/c the topology is not relevant to the arithmetic performed by the network.

I'm talking about HTML (or other markup, in the form of text) vs image. That simply getting the markup as text tokens will be much harder to interpret since it's not clear where the elements will end up. I guess I can't make this any more clear.

The guy you are talking to is either an utter moron, severely autistic, or for some weird reason he is trolling ( it is a fresh account. I applaud you for trying to be kind and explain things to him, I personally would not have the patience.

Re: Gemini 2.5 Computer Use model

#263

Post by fauigerzigerk »

skc wrote: post #250

How likely is it that the end game becomes that we stop writing apps for actual human users and instead sites become massive walls of minified text against a black screen.

If some functionality isn't used directly by humans why not expose it as an API?

If you're asking how likely it is that all human-computer interaction will take place via lengthy natural language conversations then my guess is no.

Visualising information and pointing at things is just too useful to replace it with what is essentially a smart command line interface.

Re: Gemini 2.5 Computer Use model

#264

Post by pseidemann »

emp17344 wrote: post #149
pseidemann wrote: post #121

Earlier quoted context omitted.

Funny thing is, most humans cannot properly control a computer. Intelligence seems to be impossible to define.

Is this a joke, or do you actually believe most people are incapable of using a computer?

I didn't write "incapable". Emphasis on "properly".

Re: Gemini 2.5 Computer Use model

#265

Post by jbarber »

zomgbbq wrote: post #148

I would love to use this for E2E testing. It would be great to make all my assertions with high level descriptions so tests are resilient to UI changes. Seems similar to the Amazon Nova Act API which is still in research preview.

This is harder than you might expect because it's hard to tell whether a passing test is a false positive (i.e. the test passed, but it should have failed).

It's also hard to convey to the testing system what is an acceptable level of change in the UI - what the testing system thinks is ok, you might consider broken.

There are quite a few companies out there trying to solve this problem, including my previous employer https://rainforestqa.com

Re: Gemini 2.5 Computer Use model

#266

Post by regularfry »

dekhn wrote: post #89

I just have to say that I consider this an absolutely hilarious outcome. For many years, I focused on tech solutions that eliminated the need for a human to be in front of a computer doing tedious manual operations. For a wide range of activities, I proposed we focus on "turning everything in the world into database objects" so that computers could operate on them with minimal human effort. I spent significant effort…

We will get to the point of degrading the computer output, having it intentionally make humanising mistakes, so that it's more believable.

Re: Gemini 2.5 Computer Use model

#267

Post by pseidemann »

fragmede wrote: post #198
emp17344 wrote: post #149

Earlier quoted context omitted.

Is this a joke, or do you actually believe most people are incapable of using a computer?

We should be very specific and careful with our words. pseidemann said "most humans cannot properly control a computer", which isn't the same as "most people are incapable of using a computer". I would agree with pseidemann. There's a level of understanding and care and focus that most people lack. That doesn't make those people less worthy of love and care and support, and computers are easier to use than ever. Most…

Well written but I disagree with the eugenics part. I think we can all achieve high quality of life with (very) good education and health care alone, and we have to. All other ways eventually turn into chaos, imho.

Re: Gemini 2.5 Computer Use model

#268

Post by pseidemann »

a_wild_dandan wrote: post #135
pseidemann wrote: post #121

Earlier quoted context omitted.

Funny thing is, most humans cannot properly control a computer. Intelligence seems to be impossible to define.

Intelligence is whatever an LLM can’t do yet. Fluid intelligence is the capacity to quickly move goal posts.

I'm not sure I understand your statement. Are you implying that once an LLM can do something, "it" is not intelligent anymore? ("it" being the model, the capability, or both?)

Re: Gemini 2.5 Computer Use model

#269

Post by Oras »

MiguelG719 wrote: post #57
Oras wrote: post #32

It is actually quite good at following instructions, but I tried clicking on job application links, and since they open in a new window, it couldn't find the new window. I suppose it might be an issue with BrowserBase, or just the way this demo was set up.

are you running into this issue on gemini.browserbase.com or the google/computer-use-preview github repo?

on gemini.browserbase.com

Re: Gemini 2.5 Computer Use model

#270

Post by falcor84 »

iLoveOncall wrote: post #13
xnx wrote: post #7

I've had good success with the Chrome devtools MCP ( https://github.com/ChromeDevTools/chrome-devtools-mcp ) for browser automation with Gemini CLI, so I'm guessing this model will work even better.

This has absolutely nothing in common with a model for computer use... This uses pre-defined tools provided in the MCP server by Google, nothing to do with a general model supposed to work for any software.

The general model is what runs in an agentic loop, deciding which of the MCP commands to use at each point to control the browser. From my experimentation, you can mix and match between the model and the tools available, even when the model was tuned to use a specific set of tools.
Post reply on HN