Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What happens to SaaS in a world with computer-using agents?

docs.google.com

Post reply on HN
Print view View original

31–40 of 85 posts

Re: What happens to SaaS in a world with computer-using agents?

#31

Post by lukev »

Magmalgebra wrote: post #6

I think this post underestimates how the degree to which “what data is correct” is deeply contextual. My team created an identical hypothesis to this doc ~2 years ago and generated a proof of concept. It was pretty magic, we had fortune 500 execs asking for reports on internal metrics and they’d generate in a couple of minutes. First week we got rave reviews - followed by an immediate round of negative feedback as we…

This is absolutely the problem. But there is a line of sight; namely, combining LLMs with existing semantic data technologies (e.g, RDF.)

This is why I'm building a federated query optimizer: we want to let the LLM reason and formulate queries at the ontological level, with query execution operating behind a layer of abstraction.

Re: What happens to SaaS in a world with computer-using agents?

#32

Post by Bjorkbat »

This kind of reminds me of when there was a lot of hype around messenger apps and this idea that we'd just do everything through a chat interface / chat bot.

It never panned out, arguably because the technology wasn't quite there yet (this was well before ChatGPT came out), but I thought the bigger problem was that people thought that a chat UI was the ultimate user interface. Just didn't feel right to me. For simple tasks, sure, but otherwise it felt like for "exploratory" tasks it made more sense to have a graphical user interface of some kind.

Same sentiments apply to the hype around agents. Even in a hypothetical world where agents work as well as any human I don't think an agent/chatbot UI is necessarily the ultimate user interface. If I'm asking an agent questions, it makes sense for it to show rather than tell in many contexts. Even in a world where agents capture much of the way we interact with computers, it might make more sense for them to show us using 3rd party SaaS apps.

Re: What happens to SaaS in a world with computer-using agents?

#33

Post by svieira »

caspper69 wrote: post #25
svilen_dobrev wrote: post #18

Earlier quoted context omitted.

check https://news.ycombinator.com/item?id=42974429 from few days ago.. the OP was re-advertising OAUth, but another idea might be, that new kind of interfaces are needed - application agentic interfaces - standing in middle between APP(Programming) (too detailed) and AHI(Human) screen/forms (too human targeted). IMO.

I propose the Open Agent Interface. We can call it OpenAI. I'll see myself out.

We could also call it the Open Agent Permissions Interface or OpenAPI for short.

Re: What happens to SaaS in a world with computer-using agents?

#34

Post by abakker »

lukev wrote: post #31
Magmalgebra wrote: post #6

I think this post underestimates how the degree to which “what data is correct” is deeply contextual. My team created an identical hypothesis to this doc ~2 years ago and generated a proof of concept. It was pretty magic, we had fortune 500 execs asking for reports on internal metrics and they’d generate in a couple of minutes. First week we got rave reviews - followed by an immediate round of negative feedback as we…

This is absolutely the problem. But there is a line of sight; namely, combining LLMs with existing semantic data technologies (e.g, RDF.) This is why I'm building a federated query optimizer: we want to let the LLM reason and formulate queries at the ontological level, with query execution operating behind a layer of abstraction.

Line of sight to a problem solving architecture, while cool, is nowhere near line of sight on upgrading the existing crappy data that is critically intertwined with literal thousands of apps in a typical enterprise.

Re: What happens to SaaS in a world with computer-using agents?

#35

Post by Magmalgebra »

lukev wrote: post #31
Magmalgebra wrote: post #6

I think this post underestimates how the degree to which “what data is correct” is deeply contextual. My team created an identical hypothesis to this doc ~2 years ago and generated a proof of concept. It was pretty magic, we had fortune 500 execs asking for reports on internal metrics and they’d generate in a couple of minutes. First week we got rave reviews - followed by an immediate round of negative feedback as we…

This is absolutely the problem. But there is a line of sight; namely, combining LLMs with existing semantic data technologies (e.g, RDF.) This is why I'm building a federated query optimizer: we want to let the LLM reason and formulate queries at the ontological level, with query execution operating behind a layer of abstraction.

Unfortunately this doesn't address the problem I'm describing.

My team had these ontologies available to the LLM and provided it in the context window. The queries were ontologically sensible at a surface level, but still wrong.

The problem is that your ontology is rapidly changing in non-obvious and hard to document ways e.g. "this report is only valid if it was generated on a tuesday or thursday after 1pm because that's when the ETL runs, at any other time the data will be incorrect"

Re: What happens to SaaS in a world with computer-using agents?

#36

Post by Magmalgebra »

llm_trw wrote: post #16
Magmalgebra wrote: post #6

I think this post underestimates how the degree to which “what data is correct” is deeply contextual. My team created an identical hypothesis to this doc ~2 years ago and generated a proof of concept. It was pretty magic, we had fortune 500 execs asking for reports on internal metrics and they’d generate in a couple of minutes. First week we got rave reviews - followed by an immediate round of negative feedback as we…

>Digging into use cases you’d fine that for a particular question you needed to not just get all the rows from a column, you needed to do some obscure JOIN ON operation. This fact was only known by 2 data scientists in charge of writing the report. >I still work on AI powered products and I don’t see even a little line of sight on this problem. Everyone’s data is immensely messy and likely to remain so. I've worked i…

Descriptions of tables is insufficient (we had it) - you also need descriptions of the systems writing to the tables.

My favorite example was a report that was only accurate if generated on a Tuesday or Thursday due to when the ETL pipeline ran. A small config change on the opposite side of a code base completely altered the semantics of the data!

Re: What happens to SaaS in a world with computer-using agents?

#37

Post by bushido »

It's an intriguing take, but as others have pointed out, the truth will be somewhere in the middle. I don't believe that AI will replace the entire SaaS interface. And I also don't think it will need as many services and APIs of yester-years.

This writeup seems to be authored by a senior designer at Salesforce and I can see the motivation from the their perspective. Their challenges are different than what a new SaaS product will encounter.

Like all the incumbents of their time they are a core-ish database that depended on a plethora of point solutions from vendors and partners to fill in the gaps their product left in constructing workflows. If they don't take an approach like being discussed here – or in the linked OpenAI/Softbank video – they will risk alienating their vendors/partners or worse see them becoming competitors in their own right.

Disclaimer – I'm biased too, I'm building one of the upstarts that aims to compete with Salesforce.

Re: What happens to SaaS in a world with computer-using agents?

#40

Post by dkkergoog »

sbmthakur wrote: post #30

I wonder how we will train Customer Support to tackle issues faced by LLMs. LLMs can already do basic Customer Support. But stuff like understanding bugs and deciding if they should escalate things to engineers feels like a hard thing for an LLM.

[dead]
Post reply on HN