And structured APIs are about 1e9x more expensive than not invoking an LLM in the first place compared to using deterministic code to do something ... it's not like any of this is rational based on compute.
Computer Use is 45x more expensive than structured APIs
161–170 of 295 posts
Re: Computer Use is 45x more expensive than structured APIs
#162All the websites currently blocking Claude Code or other AI agents are fighting a losing battle. Computer-use is in the early stages, and the thing preventing mass-adoption seems to be the number of tokens it takes. Agents can fumble around trying 10 CLI commands that don't work before finding the right one and we barely notice. But other visual agents (browser use / computer use etc) end up eventually fumbling on to…
nobody can block actual LLM providers, they use spoofed requests to scan web for content, sometimes even using residential proxies.
Sure they can, proof of work seems to be effective. Anubis has become pretty popular
Re: Computer Use is 45x more expensive than structured APIs
#163Great guidance hidden in here for making it expensive for agents to navigate your website. Move elements on screen as the mouse moves, force natural mouse movement to make the UI work, change the button labels in the JS to be randomly named every visit, force scrolling to the bottom of the screen to check for hidden extra tasks... Hang on, that sounds like common corporate SaaS apps.
It's really weird, I'm seeing across the board that people who never believed in them before are suddenly all into good software eng practices (starting with writing a spec) because of AI.
It's kind of fascinating that we never were willing to do these things for humans but now that AI needs it ... we are all in. A bit depressing in the sense that I think mostly the reason we happy to do it for AI is that we perceive it will benefit us personally rather than some abstract future human.
Re: Computer Use is 45x more expensive than structured APIs
#164I’m missing the premise. For internal apps why would you ever reach for Computer Use vs just having your agent whip up a cli or MCP?
_of course_ computer use is worse. It is your last resort. Do not use it on state that lives in a DB that you own.
If anything I am impressed that it’s only 50x worse.
Re: Computer Use is 45x more expensive than structured APIs
#165I wouldn't call it a benchmark since it's just one sample. They do highlight a real problem, though. Computer use is immature right now and far behind language agents
Try playing fruit ninja via text and llm toolcalls though
Re: Computer Use is 45x more expensive than structured APIs
#166Earlier quoted context omitted.
I wish I could place a lot of money on the opposite side of this bet. I don't think many realize how could the cheap, alternative models are becoming. I prefer SOTA models for key work, but I can also spend 10X as many tokens on an open model hosted by a non-VC subsidized provider (who is selling at a profit) for tasks that can tolerate slightly less quality. The situation is only getting better as models improve and…
What open source model and what non-subsidized provider specifically?
GLM 4.7 Flash is 0.07/1m tokens in, 0.40/1m tokens out on AWS Bedrock us-east-1. That's less than 1/10 the price of Haiku 4.5
Bedrock isn't the cheapest either although I'm fairly sure they aren't being VC subsidized
There are definitely cheap tokens out there. The big gotcha is "for tasks that can tolerate slightly less quality"
Re: Computer Use is 45x more expensive than structured APIs
#167I'm always skeptical of the whole "computer use" concept. It's like hiring someone and inviting him to your house and telling him to go ahead, feel free to sleep on the bed, use the toilet, eat whatever is in the fridge, watch the TV, and oh here are the combinations for the safe... and that someone you hire is a monkey.
I feel like I am taking crazy pills. Are we really having an AI fart around with a mouse and clicking on things to accomplish stuff because we're not capable of making one kind of software query and command another piece of software? It kind of boggles my mind.
Re: Computer Use is 45x more expensive than structured APIs
#168Metadata and structure beats AI every time.
Re: Computer Use is 45x more expensive than structured APIs
#169Great guidance hidden in here for making it expensive for agents to navigate your website. Move elements on screen as the mouse moves, force natural mouse movement to make the UI work, change the button labels in the JS to be randomly named every visit, force scrolling to the bottom of the screen to check for hidden extra tasks... Hang on, that sounds like common corporate SaaS apps.
It's really weird, I'm seeing across the board that people who never believed in them before are suddenly all into good software eng practices (starting with writing a spec) because of AI. It's kind of fascinating that we never were willing to do these things for humans but now that AI needs it ... we are all in. A bit depressing in the sense that I think mostly the reason we happy to do it for AI is that we perceive…
It’s an interesting psychological phenomenon. It’s like the way I keep my house way tidier since I got a robot vacuum. Pick things up off the floor for aesthetics’ sake? Nah. Pick them up because the vacuum will attempt to eat them and might get sick? Of course!
Re: Computer Use is 45x more expensive than structured APIs
#170In an agentic world, the OS needs to be completely rethought. For example, every single app functionality should be exposable via an API while remaining human friendly. I think OpenAI designing their own phone is the next logical step. I hope they succeed which should bring major competition to Apple and Android.
The GUI was a mistake. Long live the shell!