Live data from Hacker News

Computer use in Gemini 3.5 Flash

blog.google

71–80 of 175 posts

Re: Computer use in Gemini 3.5 Flash

#71
post #68
post #67

Earlier quoted context omitted.

That's better than the loop grok got stuck in trying to use git and push the work it did leading to a $15 api credit deduction.

Getting AI/ML to acknowledge "I don't know" is such a challenge.

Not true regarding ML, most ML methods support RMSE even if they are non parametric methods.

Re: Computer use in Gemini 3.5 Flash

#72

Today I asked Gemini to extract a table from an PDF appendix and create C++ data table with its contents. After 15 or so iterations with corrections and new mistakes, it eventually gave up. I was floored when it said “I’m sorry, I cannot do this simple task, I’ve exceeded my error threshold and cannot do this task for you. My LLM prediction engine invents data instead of doing a simple data copy/reformat”. Stunned to…

[dead]

Re: Computer use in Gemini 3.5 Flash

#73

Today I asked Gemini to extract a table from an PDF appendix and create C++ data table with its contents. After 15 or so iterations with corrections and new mistakes, it eventually gave up. I was floored when it said “I’m sorry, I cannot do this simple task, I’ve exceeded my error threshold and cannot do this task for you. My LLM prediction engine invents data instead of doing a simple data copy/reformat”. Stunned to…

You didn't say whether you were using the App but the App's performance seems to be severely throttled compared to API.

Re: Computer use in Gemini 3.5 Flash

#74

Earlier quoted context omitted.

Computer use is a great idea. It gets the job done when nothing else will. If you're a person trying to get their job done at a big company, but half your job is in 1-2 proprietary tools or is stuck behind an API you can't program against, computer use can allow you, a non-techie, to do your job more efficiently. I think it's an awesome way to circumvent gate keepers and the IT department to let people accomplish the…

How are folks using “computer use” to click things on intranet portals that are behind an SSO? Even this OP example shows visitors a url and enter this search term… that is port of useless. How can I automate things behind an SSO wall? Even if it means I manually authorize it once and watch it do things on its own..

I've never used Gemini computer use, but I assume it's the same:

Claude computer use takes control of your whole computer inputs (mouse and keyboard) plus screenshots. You just log in, tell Claude you're logged in, and let it get to work. It'll use the browser you're logged in with.

The chrome extension is a little better because it only takes control of its own chrome tabs (again: you just log in.)

Re: Computer use in Gemini 3.5 Flash

#75

Computer use is such a terrible idea. It's slow, insecure, error prone, expensive. I guess if you're trying to get people to tokenmaxx it may look like a valid strategy, but ain't no way this will be delightful to users. I think it's a symptom of just not understanding how LLMs should interface with the OS because we're still in their early days. Eventually there'll be an iPhone moment for the ergonomics of LLM usage…

[deleted]

Re: Computer use in Gemini 3.5 Flash

#76
post #20

Earlier quoted context omitted.

It's great for testing and QA automation for UIs. It's also possibly good for the vision impaired.

UI QA only works well if your model plausibly matches the average user behavior and/or real-world edge cases. These models are far from that, and they are much less random than you'd like them to be for fuzzing (mode collapse).

It doesn't need to be that kind of QA. Even just a basic "I want the AI to build the beginnings of a GUI app for me" will work much better if the AI can see the output of its work and iterate on it. Similar if you want the AI to fix a GUI bug—much better if you can show it the the bug and tell it how to test to see when it's gone.

Re: Computer use in Gemini 3.5 Flash

#78

Today I asked Gemini to extract a table from an PDF appendix and create C++ data table with its contents. After 15 or so iterations with corrections and new mistakes, it eventually gave up. I was floored when it said “I’m sorry, I cannot do this simple task, I’ve exceeded my error threshold and cannot do this task for you. My LLM prediction engine invents data instead of doing a simple data copy/reformat”. Stunned to…

You should just have it OCR a screenshot of the PDF that would probably work better

Re: Computer use in Gemini 3.5 Flash

#79
post #5

It's funny how in their own graph, https://storage.googleapis.com/gweb-uniblog-publish-prod/ima... Gemini 3.5 Flash is beat hands down by both Opus 4.8 and GPT 5.5, and yet the graph is drawn as if Gemini wins ... :-D

It's amazing how designers of charts trying to show their product is close to the leader always remember to start the axis at zero, and designers of charts trying to show how big their lead is always forget that

Re: Computer use in Gemini 3.5 Flash

#80

Today I asked Gemini to extract a table from an PDF appendix and create C++ data table with its contents. After 15 or so iterations with corrections and new mistakes, it eventually gave up. I was floored when it said “I’m sorry, I cannot do this simple task, I’ve exceeded my error threshold and cannot do this task for you. My LLM prediction engine invents data instead of doing a simple data copy/reformat”. Stunned to…

That's interesting because my experience has been almost the opposite. A few months ago I tested Gemini on converting screenshots of tables from PDF files into CSV. I tried it on several different tables and it got every one right. It consistently outperformed ChatGPT.
Post reply on HN