Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Computer Use is 45x more expensive than structured APIs

reflex.dev

Post reply on HN
Print view View original

291–295 of 295 posts

Re: Computer Use is 45x more expensive than structured APIs

#291

Post by phatskat »

misja111 wrote: post #257
titzer wrote: post #167

Earlier quoted context omitted.

I feel like I am taking crazy pills. Are we really having an AI fart around with a mouse and clicking on things to accomplish stuff because we're not capable of making one kind of software query and command another piece of software? It kind of boggles my mind.

There are several cases where this is not possible or desirable: - you are using AI as an E2E testing tool and it's a requirement to not just test the API but also the workings of the UI - you are using some third party tool and want to automate its usage

To the first point, why use AI for end-to-end testing at all? The main app I work on has accessible buttons, menus, etc, that can all be found deterministically, why waste the money having AI try to figure it out and possibly lie about the results?

Re: Computer Use is 45x more expensive than structured APIs

#292

Post by GarnetFloride »

zenoprax wrote: post #204
GarnetFloride wrote: post #175

Earlier quoted context omitted.

My manager just told me that after 12 years of trying to get one of the founders to understand the difference between dev docs and user docs, they tried getting Claude to do it and he finally got it that they are different. He'd been saying this whole time that customer could just read the dev docs. If they could they wouldn't need our software.

How firm is the boundary between a dev doc and a user doc in your opinion? I have found that the overlap can be quite large if the users are also technically proficient. Right now I'm trying to balance "how X works so you can use the app better" with "how X works so you can contribute or build your own plugin". DeepWiki really helps as a backstop for anything not already covered though it's not without its own caveat…

In general, I am talking about non-technical or technical in a different field.

In my case right now, our users are civil engineers, they just want to be able to use our software to model the environment. They really don't want to become an expert programmer on top of that.

They just want to be able to build their thing, like a bridge, so they can make money, and plug numbers into our software to do that.

It's like making a hammer, the documentation needed for forging a hammer out of steel will be radically different from using the hammer to build a house or doing ortho surgery.

Re: Computer Use is 45x more expensive than structured APIs

#294

Post by phatskat »

QuantumNomad_ wrote: post #171
fooker wrote: post #150

Earlier quoted context omitted.

Jira from first principles Almost sounds like an Orielly book

The O’Reilly animal for Jira is apparently some kind of duck or goose. Matthew B. Doar (2011). Practical JIRA Plugins . O’Reilly. https://www.oreilly.com/library/view/practical-jira-plugins/... In case anyone was wondering. Which they probably weren’t :p

According to O’Reilly’s Animal Menagerie [1] it’s a King Duck

[1] https://www.oreilly.com/animals.csp?x-search=duck&x-sort=old...

Re: Computer Use is 45x more expensive than structured APIs

#295

Post by merlindru »

willwade wrote: post #222
merlindru wrote: post #44

I'm building something that fixes this exact problem[1]. The landing page doesn't advertise it yet, but essentially, I give agents a small set of tools to explore apps' surfaces, and then an API over common macOS functions, especially those related to accessibility. The agent explores the app, then writes a repeatable workflow for it. Then it can run that workflow through CLI: `invoke chrome pinTab` Why accessibility…

Interesting! I started something - nowhere near as complete as that and quite different but again using accessibility UI elements. The BIG problem I've found is SOOOO much stuff does really poorly having these elements exposed. Here was my approach https://github.com/willwade/app-automate?tab=readme-ov-file#... - What I do here is build UI templates - either using UIAccess OR using a one pass using a vision model. No…

sorry i had missed this! this is extremely interesting. thank you for sharing. i thought about this approach and came to the same conclusions as the reddit thread you shared. then again, it probably works fine for 80% of cases...
Post reply on HN