Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computer Use is 45x more expensive than structured APIs

reflex.dev

Post reply on HN
Print view View original

81–90 of 295 posts

Re: Computer Use is 45x more expensive than structured APIs

#81

Post by overgard »

peyton wrote: post #60
overgard wrote: post #37

I've been thinking of things I'd want an agent for recently. The problem is, everything I think of is something that requires using quite a few different websites, saving a lot of data securely, and working with a lot of sensitive accounts (my email, etc.) The problem is, all the tasks are essentially: a) things agents probably just can't do, and b) things that absolutely cannot afford to be hallucinated or otherwise…

It’s great at 1. things you wouldn’t otherwise bother doing 2. things where it otherwise would get stuck iterating on hacky workarounds doomed to fail “Reverse engineer this app/site so we can do $common_task in one click”, “by the way, I’m logged in to $developer_portal, so try @Browser Use if you’re stuck”, etc. I just had Codex pull user flows out of a site I’m working on and organize them on a single page. It fou…

I guess the problem I see here is that if the use case is "things I otherwise wouldn't bother doing", that's fine, but it's pretty niche. I dunno, if you're talking about a human "Agent" (like say in sports or entertainment), they'd be a trusted person to handle business matters outside of your competency (contract negotiations, etc.). I don't see AI "agents" being at all like that, they're more like an intern you need to supervise constantly.

Re: Computer Use is 45x more expensive than structured APIs

#82

Post by swiftcoder »

aurareturn wrote: post #77
planb wrote: post #23

Earlier quoted context omitted.

This will not happen. None of the existing apps people use daily on their phones have any incentive to support this. Social media wants the people to doomscroll, shopping apps and booking sites want to use their own dark patterns to make people believe they get a special discount if they buy _now_ and everything else just wants users to see the ads. Why on earth would they offer convenient hooks for AI chatbots?

Why on earth would they offer convenient hooks for AI chatbots? Competition. If I ask my OS-level AI assistant to find a social media reel about a elephant dancing, the social media app that exposes a set of APIs for an AI agent might get used more. Watch how fast Meta adds this if a new hot shot social media app succeeds by designing for AI agents controlled by users.

Having used a chatbot to find a reel Meta was censoring from search in the past... I'm not sure how well the incentives align

Re: Computer Use is 45x more expensive than structured APIs

#83

Post by pmontra »

sbrother wrote: post #43
pmontra wrote: post #32

Earlier quoted context omitted.

I still have to understand what my AI agents could do that I don't want to do myself. Buy stuff? No thanks, I want to see what I buy. I think that they are 99% a solution in search of a problem.

Same. Well the biggest thing I don't want to do that they could help with is work. But in the cases where it can do that for me, there's no world where that benefit goes to me rather than my employer.

Well, that's the very nature of the employer / employee relationship. In my case I write software for my customers and I trade time for money. If I use an AI to write code two times faster my daily rate doesn't double. However I can keep my costumers.

That's only another step in the path I experienced since the 80s, when I had to type every single character because there was no auto complete, no command line history, very few libraries. I was very good at writing trees, hash tables, linked lists and so was everybody else. Nobody would hire me if I were that slow at writing code today.

Re: Computer Use is 45x more expensive than structured APIs

#84

Post by ai_fry_ur_brain »

k__ wrote: post #80
ai_fry_ur_brain wrote: post #73

Earlier quoted context omitted.

"Agentic engineering" were always just FADs to bring in more revenue for token providers. If I think an LLM is good for something I create well defined, very deterministic "middleware" for that purpose on top of Openrouter.

Agentic engineers can build well defined, very deterministic middleware on top of OpenRouter. Anthropic even says, that an agent based solution should only be your last resort and that most problems are well served with a one-shot. https://www.anthropic.com/engineering/building-effective-age...

Written 1.5 years ago. Anthropic would not advertise this stance today.

I'm much more agreeable with that type of LLM workflow. Running "agents" with monolithic "harness" for long time horizon tasks seems wasteful, unecessary but probably super appealing to lazy people.

Re: Computer Use is 45x more expensive than structured APIs

#85

Post by zozbot234 »

aurareturn wrote: post #8

In an agentic world, the OS needs to be completely rethought. For example, every single app functionality should be exposable via an API while remaining human friendly. I think OpenAI designing their own phone is the next logical step. I hope they succeed which should bring major competition to Apple and Android.

> In an agentic world, the OS needs to be completely rethought. For example, every single app functionality should be exposable via an API while remaining human friendly.

So, like a Unix system?

Re: Computer Use is 45x more expensive than structured APIs

#86

Post by ai_fry_ur_brain »

wahnfrieden wrote: post #79
ai_fry_ur_brain wrote: post #73

Earlier quoted context omitted.

"Agentic engineering" were always just FADs to bring in more revenue for token providers. If I think an LLM is good for something I create well defined, very deterministic "middleware" for that purpose on top of Openrouter.

It’s not a fad or without value.

Its very much valuable to lazy people who dont care about quality or doing hard things. I totally see the appeal for those people.

Re: Computer Use is 45x more expensive than structured APIs

#89

Post by orliesaurus »

Computer Use? Or Browser Use? IMHO big diff

The problem is that not everything from the 'past' can be accessed via APIs. It would be a fun time - remember Prism [1] - I would just run that and get all the API calls in a nice format and then replay them over and over to do things in succession.

In the new world, we have access to OpenAPI.json and whatnot, but in the world where things were built in the days pre-OpenAPI and pre-specs and best practices...I am not so sure! (and a lot of world lives then)

Alas, this works for a good chunk of things but not everything. Which is why the other technnology exists.

[1] https://stoplight.io/open-source/prism

Re: Computer Use is 45x more expensive than structured APIs

#90

Post by Worf »

Is it possible to ask the vision agent to "map" the UI and expose it to another agent as a set of interfaces that resemble an API better? From what I understand the vision agent now should both know that "next page" shows more results and that they need to get more results in the first place.

If one agent just explores the UI, maybe in a test environment, and outputs a somewhat-structured description of the various UI elements and their behavior, then another agent was given that description, would the other agent perform better that an agent that both explores the UI and tries to accomplish the given task at the same time?

With an example UI I made up, the description (API-like interface definition) could be something like:

  Get all reviews:

  To get all the reviews you need to go to each page and click "show full review" for every review summary in that page.

  Go to each page:

  Start at page 1 (the default when in the Reviews tab). Continue by clicking the "next" button until the "next" button is no longer available (as you've reached the last page).
So the second agent can skip some thinking about how to navigate because it already has that skill. The first agent can explore the UI on its own, once, without worrying about messing up if there's a test environment.

Or am I misunderstanding the article completely? Probably. But it's interesting nonetheless. Sorry if it makes no sense.

Post reply on HN