Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What happens to SaaS in a world with computer-using agents?

docs.google.com

Post reply on HN
Print view View original

71–80 of 85 posts

Re: What happens to SaaS in a world with computer-using agents?

#71

Post by caspper69 »

mdaniel wrote: post #64
caspper69 wrote: post #23

Earlier quoted context omitted.

Ok, I will need to work on my reading comprehension skills. That being said, I thought the purpose of OCR was to take text from a non-digital source and make it digital. Why should we have to OCR something that exists already in a perfectly interchangeable digital format already?

> Why should we have to OCR something that exists already in a perfectly interchangeable digital format already? I'm with you in spirit, but in this specific context I think it's because the alternative would require the ~~LLM~~ Agent to be an HTML parser, or be bright enough to write themselves a Scrapy crawler. I suspect folks decided it's cheaper (by some metric) to just use the normal browser machinery to render…

Not everyone does their work in a web browser.

And even still, you don’t have to parse raw markup to grab properties from DOM elements. That could be handled by a browser plugin coupled with some some user guided training.

PDF is another beast entirely. I think there’s already a whole thread about that going on now. I’m going to zip my lips. I’m still waiting on Adobe to return my call from two years ago inquiring about the licensing costs of their parsing library for a small shop. Good thing I wasn’t relying on them to get that project done, and thank goodness for oss.

Re: What happens to SaaS in a world with computer-using agents?

#72

Post by Terr_ »

caspper69 wrote: post #70
Terr_ wrote: post #69

Earlier quoted context omitted.

> Like Homer Simpson's button pressing birdie toy? :smackshead: This comparison is especially apt, given that one of the main use-cases for LLMs is the same kind of... well, fraud : To give the illusion that you did the work of understanding or reviewing something, but actually just (smart-)phoning it in. In one Apple iPhone advertisement, the famous actor is asked by their agent what they think of a script. They did…

I think my quip about the toy flew over a lot of heads, so I appreciate that someone got it. The reality is that most applications and websites don’t expose enough context about the what of what you’re actually doing for AIs to be able to meaningfully infer from natural language the steps required to complete a given task. We humans are very good at filling in the blanks based on if we’re working in Photoshop or VS C…

> I am skeptical that models will be able to replicate a complex workflow when there’s very little in the way of labels and UI controls even visible.

Also, at least from the perspective of internal business software, a significant part of it is trying to get people to know what they're doing. There's a domain-model that's being taught at the same time, and it's institutionally-important that they are cognizant and aware of what they're agreeing to. Together this tends to lead to an arrangement of multiple screens, confirmation boxes, etc.

Many individuals instinctively dislike this, and it'll be their one of their first choices for "let my LLM assistant do it."

> I mean I can just see it now

Before these LLMs, I felt like Idiocracy had become politically prescient, but now it feels like I actually see a technology that could enable it.

Re: What happens to SaaS in a world with computer-using agents?

#73

Post by satvikpendem »

burnte wrote: post #42
Magmalgebra wrote: post #6

I think this post underestimates how the degree to which “what data is correct” is deeply contextual. My team created an identical hypothesis to this doc ~2 years ago and generated a proof of concept. It was pretty magic, we had fortune 500 execs asking for reports on internal metrics and they’d generate in a couple of minutes. First week we got rave reviews - followed by an immediate round of negative feedback as we…

> I think this post underestimates how the degree to which “what data is correct” is deeply contextual. I can't get anyone to listen to this point. I'm seeing plans going full steam ahead deploying AI when they don't even have a good definition of the PROBLEM much less how to train the AI to do things well and correctly. I was in a 90 minute meeting with some execs who were all high on ChatGPT Operators. He was sayin…

> I can't get anyone to listen to this point. I'm seeing plans going full steam ahead deploying AI when they don't even have a good definition of the PROBLEM much less how to train the AI to do things well and correctly.

First time? I did AI work years before the current generative AI boom and it was the same then too, managers wanted to stick AI into everything without even knowing what the hell they actually wanted in the end.

Re: What happens to SaaS in a world with computer-using agents?

#75

Post by caspper69 »

Terr_ wrote: post #72
caspper69 wrote: post #70

Earlier quoted context omitted.

I think my quip about the toy flew over a lot of heads, so I appreciate that someone got it. The reality is that most applications and websites don’t expose enough context about the what of what you’re actually doing for AIs to be able to meaningfully infer from natural language the steps required to complete a given task. We humans are very good at filling in the blanks based on if we’re working in Photoshop or VS C…

> I am skeptical that models will be able to replicate a complex workflow when there’s very little in the way of labels and UI controls even visible. Also, at least from the perspective of internal business software, a significant part of it is trying to get people to know what they're doing . There's a domain-model that's being taught at the same time, and it's institutionally-important that they are cognizant and a…

Brawndo is coming- it's got electrolytes and IT'S WHAT PLANTS CRAVE!

Life imitates art indeed.

I am sympathetic to wanting to automate complex workflows. Hell, I'm sympathetic to wanting to automate simple workflows. In fact, I bitch about the stupidity of the things I do at least once a week (no, you see, I take the numbers that show on this monitor, and I type them into a box on that monitor; why no cut & paste? faster to re-type the numbers; sigh).

But people provide context. Sure, an AI might tell you utility costs were up last quarter, but they won't know it was because of a water leak that went unnoticed and tripled the bill. Or it will tell you that wages were up, but not that it was because Bill from Operations had hernia surgery and we had to bring on a temp for 2 months. And it certainly won't tell you that Jim's back on the sauce, so we should probably begin putting out feelers for a new salesman.

So much of what business does is tracking metrics, yes, but the numbers never tell the whole story. There's always a backstory. Things that just can't be captured in raw data and hence can't be summarized by an AI. And AIs can't keep the ship sailing. Every small business has the guy/girl that does all the little things for everyone that absolutely holds the whole damn thing together. I'm not a BigCorp guy, but I imagine most departments are similar.

How about customer feedback? How can a model distill valuable (actionable) meaning from disparate communication mediums other than superficial high-level conclusions?

Expectations are just not realistic right now. There's going to be a lot of disappointment.

Re: What happens to SaaS in a world with computer-using agents?

#76

Post by lfxyz »

burnte wrote: post #42
Magmalgebra wrote: post #6

I think this post underestimates how the degree to which “what data is correct” is deeply contextual. My team created an identical hypothesis to this doc ~2 years ago and generated a proof of concept. It was pretty magic, we had fortune 500 execs asking for reports on internal metrics and they’d generate in a couple of minutes. First week we got rave reviews - followed by an immediate round of negative feedback as we…

> I think this post underestimates how the degree to which “what data is correct” is deeply contextual. I can't get anyone to listen to this point. I'm seeing plans going full steam ahead deploying AI when they don't even have a good definition of the PROBLEM much less how to train the AI to do things well and correctly. I was in a 90 minute meeting with some execs who were all high on ChatGPT Operators. He was sayin…

> People are either completely taken in by the marketing and believe like it's a religion, or they have solid, sensible concerns about reliability.

The other issue is that the first group are labelled as innovative go-getters, while the second group are labelled as negative crusty curmudgeons and this has an impact on the careers of both groups.

Re: What happens to SaaS in a world with computer-using agents?

#77

Post by joshstrange »

Magmalgebra wrote: post #6

I think this post underestimates how the degree to which “what data is correct” is deeply contextual. My team created an identical hypothesis to this doc ~2 years ago and generated a proof of concept. It was pretty magic, we had fortune 500 execs asking for reports on internal metrics and they’d generate in a couple of minutes. First week we got rave reviews - followed by an immediate round of negative feedback as we…

Yep, I had a similar experience around a year or so ago. Hooking an LLM up to my RDMBS was really cool for the first 1-2 questions but fell over almost immediately with questions that strayed much further than “how many rows are in this table”.

Sure, you can do some basic filtering (but it would fail here making bad assumptions) and any (correct) joins were a crap-shoot. I was including schema and sample rows from all my tables, I wrote 10’s of lines of instructions explaining the logic of the tables and that still didn’t begin to cover all the cases.

Prompt engineering tons of business logic is a horrible job. It hard to test and it feels so “squishy” and unreliable. Even with all of my rules, it would write queries that didn’t work and/or broke a rule/concept that I had laid out.

In my experience, you’re much better off using AI to help you write some queries that you add to the codebase (after tweaking/checking) then you are having AI come up with queries at run time.

Re: What happens to SaaS in a world with computer-using agents?

#78

Post by turnsout »

Something1234 wrote: post #7
turnsout wrote: post #3

We need a simple open-source protocol which includes authentication and ability for agents to make payments. Essentially what you want is the ability for an agent to take a core action (as the article mentions, like adding a record to a CRM). I fundamentally believe that human-oriented web apps are not the answer, and neither is REST. We need something purpose-built. The challenge is, it has to be SIMPLE enough for p…

This is the same dumb problem as always. Are you who you say you are and are you allowed to do such and such action? There’s existing solutions but everything is its own special snowflake. Oauth is a lie, sso sometimes works. But sso doesn’t provide a differentiation between my employee and their broken script.

Public key encryption solves this entirely

Re: What happens to SaaS in a world with computer-using agents?

#79

Post by reportgunner »

Bjorkbat wrote: post #32

This kind of reminds me of when there was a lot of hype around messenger apps and this idea that we'd just do everything through a chat interface / chat bot. It never panned out, arguably because the technology wasn't quite there yet (this was well before ChatGPT came out), but I thought the bigger problem was that people thought that a chat UI was the ultimate user interface. Just didn't feel right to me. For simple…

It was the same then as it is now. Chatbot providers had bots to sell, now autocomplete providers have autocomplete to sell. Marketing people just say what they get paid to say.
Post reply on HN