I wonder if it will be better at building TUI's. It has been absolutely abysmal at interacting with them and building them
Claude can build UI but it sucks at testing it and iterating on it. Fable showed some improvements in this regard but alas.
Computer use in Gemini 3.5 Flash
31–40 of 175 posts
Re: Computer use in Gemini 3.5 Flash
#32Earlier quoted context omitted.
The “correct”, elegant way for AI to interact with existing software would take decades and billions of dollars to build. Someone would have to do the hard work of building new APIs, solving decades of accessibility issues, etc. Or you can show an AI screenshots and ask it where to click.
I disagree if your application is networked. Most SaaS is built on RESTful APIs that can be converted trivially into interfaces / contracts for tool use.
So you can either wait for every application to do that, or at least make it possible for an LLM to do it… or you can make the LLM use a computer interface that works with every application by definition.
Re: Computer use in Gemini 3.5 Flash
#33Re: Computer use in Gemini 3.5 Flash
#34People using google’s models: am I holding it wrong or are the guardrails really overtuned? I had the dubious pleasure of testing gemini of late and I kept running into refusals. How do I transfer a sim number from one provider to another? No. What should I consider when making backups on ntfs less prone to data loss and more bitrot resistant? No. Evaluate this piece of code? No. I’m not sure if it’s cold feet from t…
I love antigravity. I’ve had zero issues with it.
Re: Computer use in Gemini 3.5 Flash
#35People using google’s models: am I holding it wrong or are the guardrails really overtuned? I had the dubious pleasure of testing gemini of late and I kept running into refusals. How do I transfer a sim number from one provider to another? No. What should I consider when making backups on ntfs less prone to data loss and more bitrot resistant? No. Evaluate this piece of code? No. I’m not sure if it’s cold feet from t…
I just asked gemini the question with sim number and it gives me full step by step guide.
Re: Computer use in Gemini 3.5 Flash
#36Computer use is such a terrible idea. It's slow, insecure, error prone, expensive. I guess if you're trying to get people to tokenmaxx it may look like a valid strategy, but ain't no way this will be delightful to users. I think it's a symptom of just not understanding how LLMs should interface with the OS because we're still in their early days. Eventually there'll be an iPhone moment for the ergonomics of LLM usage…
Computer use is a great idea. It gets the job done when nothing else will. If you're a person trying to get their job done at a big company, but half your job is in 1-2 proprietary tools or is stuck behind an API you can't program against, computer use can allow you, a non-techie, to do your job more efficiently. I think it's an awesome way to circumvent gate keepers and the IT department to let people accomplish the…
Yeah, it's not that computer use is the most theoretically optimal paradigm, but there's a reasonable case that given the constraints of modern software systems and how they're built, that it's the most realistically optimal paradigm.
Re: Computer use in Gemini 3.5 Flash
#37There's still no MCP support in the Gemini app, which is very useful to get various pieces of info as a user just via chatting. For example I recently wanted to get an Airbnb and wanted to filter by specific criteria including house image analysis and Gemini couldn't do it so I had to do it in Codex.
They only fixed stopping the model mid-generation losing the entire session pretty recently.
The Gemini apps suck.
Re: Computer use in Gemini 3.5 Flash
#38Computer use is such a terrible idea. It's slow, insecure, error prone, expensive. I guess if you're trying to get people to tokenmaxx it may look like a valid strategy, but ain't no way this will be delightful to users. I think it's a symptom of just not understanding how LLMs should interface with the OS because we're still in their early days. Eventually there'll be an iPhone moment for the ergonomics of LLM usage…
It's great for testing and QA automation for UIs. It's also possibly good for the vision impaired.
UI QA only works well if your model plausibly matches the average user behavior and/or real-world edge cases. These models are far from that, and they are much less random than you'd like them to be for fuzzing (mode collapse).
Re: Computer use in Gemini 3.5 Flash
#39People using google’s models: am I holding it wrong or are the guardrails really overtuned? I had the dubious pleasure of testing gemini of late and I kept running into refusals. How do I transfer a sim number from one provider to another? No. What should I consider when making backups on ntfs less prone to data loss and more bitrot resistant? No. Evaluate this piece of code? No. I’m not sure if it’s cold feet from t…
The context window size is also very small if you use Gemini in the app. It starts forget quite fast. In my opinion Gemini on app is useless additionally to the guardrails.
Re: Computer use in Gemini 3.5 Flash
#40There's still no MCP support in the Gemini app, which is very useful to get various pieces of info as a user just via chatting. For example I recently wanted to get an Airbnb and wanted to filter by specific criteria including house image analysis and Gemini couldn't do it so I had to do it in Codex.
I'm fairly convinced Claude's strongest point is the app. AI users aren't anywhere near as mature or smart as youtube/hn would have folks believe. The claude app is amazing for bridging that gap.
[flagged]