Who would have thunk? You know what is a great LLM agent api? bash. vast corpus, text based, already traindd in the model.
Computer Use is 45x more expensive than structured APIs
211–220 of 295 posts
Re: Computer Use is 45x more expensive than structured APIs
#212In an agentic world, the OS needs to be completely rethought. For example, every single app functionality should be exposable via an API while remaining human friendly. I think OpenAI designing their own phone is the next logical step. I hope they succeed which should bring major competition to Apple and Android.
I still have to understand what my AI agents could do that I don't want to do myself. Buy stuff? No thanks, I want to see what I buy. I think that they are 99% a solution in search of a problem.
My family (unfortunately) uses InstaCart and probably 15% of items are a shitty "replacement" "not what I wanted". For time sensitive items, having the shitty replacement item NOW is better than having to wait for the "item I actually wanted", so we often just accept the inferior product. This is a dark pattern that I could see AI adopting -- it buys tons of cheap crap you didn't want, some of it was right, and you're left with a mess of returns to sort out, esp. if those returns require you to physically take some sort of action like physically returning the item to the store
Re: Computer Use is 45x more expensive than structured APIs
#213Great guidance hidden in here for making it expensive for agents to navigate your website. Move elements on screen as the mouse moves, force natural mouse movement to make the UI work, change the button labels in the JS to be randomly named every visit, force scrolling to the bottom of the screen to check for hidden extra tasks... Hang on, that sounds like common corporate SaaS apps.
I had one project where a desktop application deliberately hid the contents of all grid controls from Windows accessibility APIs, took measures to ensure checkbox and radio button selections made through accessibility APIs did not register, and all functions that allowed data to be exported were protected by CAPTCHAs.
Generative AI wasn't a thing at the time, but I had to resort to a combination of OCR, simulated user input, and print capture to drive the application and export data.
Had the developers been aware of the Windows DRM APIs that block screen capture, or the fact that text is easily recoverable from PostScript files with minimal formatting, I don't know what I would have done.
The irony is that the process this replaced involved giving cheap offshore labor full read-only remote access to all data in the system, which was by any measure a far more serious security risk than otherwise authorized employees using tools running locally with no network access provided by established, trustworthy vendors to automate their work.
Re: Computer Use is 45x more expensive than structured APIs
#214I want to just talk to the Mac and have it do things. I tried computer use and other alternatives, but the latency made it unusable.
I want to be able to control both Mac, apps and the browser. I also need it to figure out things by itself given a goal.
Claude Code with the --chrome flag is kind of good, but it's too slow. I wanted to try faster APIs, like the one hosted on Cerebras, but it's too expensive.
Any solution I might be missing?
Re: Computer Use is 45x more expensive than structured APIs
#215I did a simple computer use to search something, and used up 50% of my 5h plan limit from codex.
Re: Computer Use is 45x more expensive than structured APIs
#216Earlier quoted context omitted.
My friend at a faang was talking about the "massive overhauls to make everything ready for ai". I asked for an example. He said "basically just documenting the shit out of everything" I guess that just never occurred to anybody before.
Having the humans document the code seems backward (maybe that's not what they're doing, but "make everything ready for ai" sound manual). And hopefully there aren't that many scary surprises that humans need to manually document. One of the best parts of LLMs is that you can use them to bootstrap your documentation, or scan for outdated things, etc, far more quickly than ever before. Don't just throw a mountain at i…
> Having the humans document the code seems backward (maybe that's not what they're doing, but "make everything ready for ai" sound manual).
No, that's forward. Any documentation an AI can make, another AI can regenerate. If an LLM didn't write the code, it shouldn't document it either. You don't want to bake in slop to throw off the next LLM (or person).
Re: Computer Use is 45x more expensive than structured APIs
#217Earlier quoted context omitted.
They don’t need to be 100% effective they just need to make you afraid enough of being banned to not bother trying.
How do they know that the "you" accessing the site is the same "you" they previously banned? Face-scanning? Iris patterns?
You used your credit card to buy whatever service or product they sell.
Re: Computer Use is 45x more expensive than structured APIs
#218Great guidance hidden in here for making it expensive for agents to navigate your website. Move elements on screen as the mouse moves, force natural mouse movement to make the UI work, change the button labels in the JS to be randomly named every visit, force scrolling to the bottom of the screen to check for hidden extra tasks... Hang on, that sounds like common corporate SaaS apps.
So ASP WebForms was the technology we needed all along?
Re: Computer Use is 45x more expensive than structured APIs
#219In an agentic world, the OS needs to be completely rethought. For example, every single app functionality should be exposable via an API while remaining human friendly. I think OpenAI designing their own phone is the next logical step. I hope they succeed which should bring major competition to Apple and Android.
This will not happen. None of the existing apps people use daily on their phones have any incentive to support this. Social media wants the people to doomscroll, shopping apps and booking sites want to use their own dark patterns to make people believe they get a special discount if they buy _now_ and everything else just wants users to see the ads. Why on earth would they offer convenient hooks for AI chatbots?
Actually this more or less describes how accessibility APIs work.
Re: Computer Use is 45x more expensive than structured APIs
#220Earlier quoted context omitted.
because the social media sites that do will outcompete once people get personal AI coaches that tell them to use technology that is better for them.
How is an AI posting on your social media better for you?
People on LinkedIn who are trying to build their "personal brand" seem to favor it. In fact, that's basically all the platform is these days.