Computer Use is 45x more expensive than structured APIs • Hacker News Board

Re: Computer Use is 45x more expensive than structured APIs

#111

Post by angry_octet » Tue, May 05, 2026, 8:33 PM UTC

Great guidance hidden in here for making it expensive for agents to navigate your website. Move elements on screen as the mouse moves, force natural mouse movement to make the UI work, change the button labels in the JS to be randomly named every visit, force scrolling to the bottom of the screen to check for hidden extra tasks...

Hang on, that sounds like common corporate SaaS apps.

Re: Computer Use is 45x more expensive than structured APIs

#112

Post by theabhinavdas » Tue, May 05, 2026, 8:38 PM UTC

For now.

Re: Computer Use is 45x more expensive than structured APIs

#113

Post by pjc50 » Tue, May 05, 2026, 8:44 PM UTC

ctoth wrote: post #72

merlindru wrote: post #44

I'm building something that fixes this exact problem[1]. The landing page doesn't advertise it yet, but essentially, I give agents a small set of tools to explore apps' surfaces, and then an API over common macOS functions, especially those related to accessibility. The agent explores the app, then writes a repeatable workflow for it. Then it can run that workflow through CLI: `invoke chrome pinTab` Why accessibility…

If agents is what it finally takes to get good a11y I'll take it. I'll bitch about it, but I'll take it.

Very real risk of this going in reverse: people building inaccessible websites to prevent AI use.

Re: Computer Use is 45x more expensive than structured APIs

#114

Post by izend » Tue, May 05, 2026, 8:46 PM UTC

merlindru wrote: post #44

I'm building something that fixes this exact problem[1]. The landing page doesn't advertise it yet, but essentially, I give agents a small set of tools to explore apps' surfaces, and then an API over common macOS functions, especially those related to accessibility. The agent explores the app, then writes a repeatable workflow for it. Then it can run that workflow through CLI: `invoke chrome pinTab` Why accessibility…

Does https://github.com/webmachinelearning/webmcp overlap ?

Re: Computer Use is 45x more expensive than structured APIs

#115

Post by deafpolygon » Tue, May 05, 2026, 8:50 PM UTC

This is missing the point that AI training probably costed boatloads more to achieve to get here.

Re: Computer Use is 45x more expensive than structured APIs

#116

Post by blurbleblurble » Tue, May 05, 2026, 8:51 PM UTC

pjc50 wrote: post #113

ctoth wrote: post #72

Earlier quoted context omitted.

If agents is what it finally takes to get good a11y I'll take it. I'll bitch about it, but I'll take it.

Very real risk of this going in reverse: people building inaccessible websites to prevent AI use.

"AI" is a made up hype thing. It's just computers and computer programs. For real!

Re: Computer Use is 45x more expensive than structured APIs

#117

Post by ipunchghosts » Tue, May 05, 2026, 8:53 PM UTC

I have a similar finding for a website I made that collates college town bar specials and live music. Using agents with vision models works but it's not as straightforward as one would initially think. U can check out the results here. https://www.nittanynights.com

Re: Computer Use is 45x more expensive than structured APIs

#118

Post by merlindru » Tue, May 05, 2026, 8:53 PM UTC

izend wrote: post #114

merlindru wrote: post #44

I'm building something that fixes this exact problem[1]. The landing page doesn't advertise it yet, but essentially, I give agents a small set of tools to explore apps' surfaces, and then an API over common macOS functions, especially those related to accessibility. The agent explores the app, then writes a repeatable workflow for it. Then it can run that workflow through CLI: `invoke chrome pinTab` Why accessibility…

Does https://github.com/webmachinelearning/webmcp overlap ?

Not really IMO, webmcp has devs change their apps. invoke just works with existing apps, especially ones that are accessible

invoke rather has overlap with Claude's and Codex' computer-use, except the steps are stored/scripted.

webmcp is bottom-up. computer-use & invoke are top-down

Re: Computer Use is 45x more expensive than structured APIs

#119

Post by caughtinthought » Tue, May 05, 2026, 8:57 PM UTC

Aurornis wrote: post #106

boringg wrote: post #103

Earlier quoted context omitted.

Tokens cheaper? I don't think that seems to be the case ... VC funded tokens were there to build user base and token price will go up as they eventually switch from growth to profitability.

I wish I could place a lot of money on the opposite side of this bet. I don't think many realize how could the cheap, alternative models are becoming. I prefer SOTA models for key work, but I can also spend 10X as many tokens on an open model hosted by a non-VC subsidized provider (who is selling at a profit) for tasks that can tolerate slightly less quality. The situation is only getting better as models improve and…

What open source model and what non-subsidized provider specifically?

Re: Computer Use is 45x more expensive than structured APIs

#120

Post by linkjuice4all » Tue, May 05, 2026, 9:03 PM UTC

ctoth wrote: post #72

merlindru wrote: post #44

I'm building something that fixes this exact problem[1]. The landing page doesn't advertise it yet, but essentially, I give agents a small set of tools to explore apps' surfaces, and then an API over common macOS functions, especially those related to accessibility. The agent explores the app, then writes a repeatable workflow for it. Then it can run that workflow through CLI: `invoke chrome pinTab` Why accessibility…

If agents is what it finally takes to get good a11y I'll take it. I'll bitch about it, but I'll take it.

I mean…I guess. But this is ridiculous - how many layers does our technology need to bash through to update two records on remote systems? I get that value is being added at some point - but just charge some micropayment for transactions. This is just too much.