Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computer Use is 45x more expensive than structured APIs

reflex.dev

Post reply on HN
Print view View original

151–160 of 295 posts

Re: Computer Use is 45x more expensive than structured APIs

#151

Post by merlindru »

btown wrote: post #143
merlindru wrote: post #44

I'm building something that fixes this exact problem[1]. The landing page doesn't advertise it yet, but essentially, I give agents a small set of tools to explore apps' surfaces, and then an API over common macOS functions, especially those related to accessibility. The agent explores the app, then writes a repeatable workflow for it. Then it can run that workflow through CLI: `invoke chrome pinTab` Why accessibility…

If you're on macOS and interested in this space, I highly recommend you open up the system-provided Accessibility Inspector.app and play around with apps and browsers. See how the green cells might guide an LLM to only need to read/OCR specific parts of a screen, how much text is already natively available to the accessibility engine, and how this could lead to really effective hybrid systems - not just MCPs, but cod…

That's how I got into this thing in the first place, hah. Golden advice. It's incredibly cool to see what some apps offer. More of them have great accessibility support than you think (or at least than I thought!)

Re: Computer Use is 45x more expensive than structured APIs

#152

Post by faangguyindia »

nonameiguess wrote: post #63
embedding-shape wrote: post #25

Earlier quoted context omitted.

I think it's the third or forth time I see you bragging about HN how many apps you're able to develop with AI now. Care to link any of them, especially where we can see the actual code that you've produced here? Without being able to see actual results, I'm not sure what you want people to take away from your repeated comments.

Why even bother asking a guy with the statistical acumen to think he can make a reliable estimate of a monthly average from some span of time shorter than two months? He's probably just going to say it doesn't matter and unfortunately he's probably right. If you sound excited enough, you can convince other people and close deals, so who gives a shit if there's really a there there? We'll see how he's doing in another…

> statistical acumen to think he can make a reliable estimate of a monthly average from some span of time shorter than two months

Perhaps because those numbers are provided on the Playstore dashboard? You should question Google's acumen in providing those statistics to developers?

And people have been estimating ARR through projections for a long time.

I already have services running for a decade+ in a product which I posted here: https://news.ycombinator.com/threads?id=faangguyindia&next=4...

In the end, simplicity wins.

Re: Computer Use is 45x more expensive than structured APIs

#154

Post by faangguyindia »

embedding-shape wrote: post #64
faangguyindia wrote: post #34

Earlier quoted context omitted.

I only write here because people are spreading doomerism here with AI and I am excited about future. Well I am competing with geoip provider like maxmind. I developed custom traceroute and ping service to geolocate IPs with very high accuracy beating products like digital element, maxmind, ipinfo These companies have huge teams. But my 3 people company already beat them. Code doesn't matter much, it's not an opensour…

> Code doesn't matter much, it's not an opensource project. Heh, you're in for a rude awakening, sometime in the future :) But I won't spoil the surprise, you clearly have made up your mind about what to focus on. > My free app is http://macrocodex.app which I've developed along with a fitness coach. Crazy, this app you've run for ~1-2 months has 10K active users already, even though there is zero info about who runs…

>Heh, you're in for a rude awakening, sometime in the future :) But I won't spoil the surprise; you clearly have made up your mind about what to focus on.

Already running for a decade+ in production, recently talked about my stack here: https://news.ycombinator.com/threads?id=faangguyindia&next=4...

>Even though there is zero info about who runs it.

People in the community already know who runs it; most others don't care. You won't get 10K users without people getting results. It's a free app, so not like I am spending bucks to advertise it on social networks.

The app is completely free, doesn't upload data to any server (other than Sentrycrash reporting), doesn't ask for any email or phone number. When people get results, they share them with their friends. That's how it's growing.

>Says "Download on the App Store" on the landing page even though you then ask people to use the web app.

On iOS, we’ve a PWA app. I am well aware of it.

Re: Computer Use is 45x more expensive than structured APIs

#155

Post by drob518 »

notjustanymike wrote: post #135
angry_octet wrote: post #111

Great guidance hidden in here for making it expensive for agents to navigate your website. Move elements on screen as the mouse moves, force natural mouse movement to make the UI work, change the button labels in the JS to be randomly named every visit, force scrolling to the bottom of the screen to check for hidden extra tasks... Hang on, that sounds like common corporate SaaS apps.

Ah damn it, we invented Jira

Real LOL!

Re: Computer Use is 45x more expensive than structured APIs

#156

Post by drob518 »

btown wrote: post #143
merlindru wrote: post #44

I'm building something that fixes this exact problem[1]. The landing page doesn't advertise it yet, but essentially, I give agents a small set of tools to explore apps' surfaces, and then an API over common macOS functions, especially those related to accessibility. The agent explores the app, then writes a repeatable workflow for it. Then it can run that workflow through CLI: `invoke chrome pinTab` Why accessibility…

If you're on macOS and interested in this space, I highly recommend you open up the system-provided Accessibility Inspector.app and play around with apps and browsers. See how the green cells might guide an LLM to only need to read/OCR specific parts of a screen, how much text is already natively available to the accessibility engine, and how this could lead to really effective hybrid systems - not just MCPs, but cod…

Great idea.

Re: Computer Use is 45x more expensive than structured APIs

#157

Post by nijave »

Worf wrote: post #90

Is it possible to ask the vision agent to "map" the UI and expose it to another agent as a set of interfaces that resemble an API better? From what I understand the vision agent now should both know that "next page" shows more results and that they need to get more results in the first place. If one agent just explores the UI, maybe in a test environment, and outputs a somewhat-structured description of the various U…

That was my first thought as well. A lot of current web development relies heavily on code generation then has obsfuscation and compression slapped on top leading to complicated structures. Then on top of that, more code (client side/JavaScript) reconfigures everything again. You end up with fairly complicated html/css/JavaScript to wade through.

For better and worse, 5-10Mi isn't uncommon for a web app.

Instead of trying to go "bottom up" and, effectively, do what a browser engine is doing in reverse, it seems easier to go "top down" like a human does and go off the visual representation.

Re: Computer Use is 45x more expensive than structured APIs

#158

Post by faangguyindia »

Worf wrote: post #90

Is it possible to ask the vision agent to "map" the UI and expose it to another agent as a set of interfaces that resemble an API better? From what I understand the vision agent now should both know that "next page" shows more results and that they need to get more results in the first place. If one agent just explores the UI, maybe in a test environment, and outputs a somewhat-structured description of the various U…

>Is it possible to ask the vision agent to "map"

No most vision models focus on subset of an image at a time when using image -> text

image -> image uses whole image.

Re: Computer Use is 45x more expensive than structured APIs

#159

Post by faangguyindia »

rahulyc wrote: post #98

All the websites currently blocking Claude Code or other AI agents are fighting a losing battle. Computer-use is in the early stages, and the thing preventing mass-adoption seems to be the number of tokens it takes. Agents can fumble around trying 10 CLI commands that don't work before finding the right one and we barely notice. But other visual agents (browser use / computer use etc) end up eventually fumbling on to…

nobody can block actual LLM providers, they use spoofed requests to scan web for content, sometimes even using residential proxies.

Re: Computer Use is 45x more expensive than structured APIs

#160

Post by nijave »

zhxiaoliang wrote: post #145

I'm always skeptical of the whole "computer use" concept. It's like hiring someone and inviting him to your house and telling him to go ahead, feel free to sleep on the bed, use the toilet, eat whatever is in the fridge, watch the TV, and oh here are the combinations for the safe... and that someone you hire is a monkey.

In fairness, you're hoping the monkey does all the monkey tasks you'd rather not do yourself
Post reply on HN