41–50 of 85 posts
I wonder how we will train Customer Support to tackle issues faced by LLMs. LLMs can already do basic Customer Support. But stuff like understanding bugs and deciding if they should escalate things to engineers feels like a hard thing for an LLM.
Especially since most attempts will have a "under no circumstances should you voluntarily involve a human" in the prompt.
I think this post underestimates how the degree to which “what data is correct” is deeply contextual. My team created an identical hypothesis to this doc ~2 years ago and generated a proof of concept. It was pretty magic, we had fortune 500 execs asking for reports on internal metrics and they’d generate in a couple of minutes. First week we got rave reviews - followed by an immediate round of negative feedback as we…
I can't get anyone to listen to this point. I'm seeing plans going full steam ahead deploying AI when they don't even have a good definition of the PROBLEM much less how to train the AI to do things well and correctly. I was in a 90 minute meeting with some execs who were all high on ChatGPT Operators. He was saying we could replace 80 people at this company RIGHT NOW with this tool. I asked the presenter to type in one simple request to the AI, the entire demo went wildly off the rails from then on and the presenter wasn't even remotely bothered by that. People are either completely taken in by the marketing and believe like it's a religion, or they have solid, sensible concerns about reliability. But the number of people in category 2 is a smaller number than the true believers.
I learned about the idea of Generative UI from a Sharp Talk podcast, and it's stuck with me ever since. Many SaaS (especially the complex ones, which are the also the most important ones) have a tonne of UI often imposing a huge amount of non-work work onto users - all the clicking you have to do as part of entering or retrieving data, especially if the UI flow doesn't fit exactly what you're trying to do at that mom…
This is a good read that is a great starting point for thinking about this. It essentially takes the extreme position - SaaS no longer needs a UI, because the LLM is the UI. In reality, as always, I suspect the truth will be somewhere in between. SaaS products that succeed will be those that have a good UI _and_ and good API that LLMs can use. An LLM is not always the best interface, particularly for data access. For…
As an industry, we have been through a textual user interface already: terminals, and we moved away from that.
And voice UIs are not new either: we have had voice assistant for quite some time now, and they didn't see the success Apple, Google or Amazon were expecting (recently it came out that most of echo use cases were about setting timers).
Great doc. I wonder when we’ll be getting an OS that dedicates itself to Agents.
I’ll bet you a bag of peanuts that some SaaS company names their next AI Product “AgentOS” 2 bags of peanuts if the actual product isn’t an OS and barely passes as AI
I think this post underestimates how the degree to which “what data is correct” is deeply contextual. My team created an identical hypothesis to this doc ~2 years ago and generated a proof of concept. It was pretty magic, we had fortune 500 execs asking for reports on internal metrics and they’d generate in a couple of minutes. First week we got rave reviews - followed by an immediate round of negative feedback as we…
This is absolutely the problem. But there is a line of sight; namely, combining LLMs with existing semantic data technologies (e.g, RDF.) This is why I'm building a federated query optimizer: we want to let the LLM reason and formulate queries at the ontological level, with query execution operating behind a layer of abstraction.
You Will.
You can have Agents run behaviors async by attaching triggers to them, for example when you get a specific email or something gets updated in a CRM. You can also give the agent access to basically any third-party action you can think of.
Like others in this thread have pointed out, there's a nice middle-ground here between an LLM-only interface and some nice UI around it, as well as ways to introduce determinism where it makes sense.
The product is still in its early days and we're iterating rapidly, but feel free to check it out and give us some feedback. There's a decent free plan.
Earlier quoted context omitted.
>Digging into use cases you’d fine that for a particular question you needed to not just get all the rows from a column, you needed to do some obscure JOIN ON operation. This fact was only known by 2 data scientists in charge of writing the report. >I still work on AI powered products and I don’t see even a little line of sight on this problem. Everyone’s data is immensely messy and likely to remain so. I've worked i…
Descriptions of tables is insufficient (we had it) - you also need descriptions of the systems writing to the tables. My favorite example was a report that was only accurate if generated on a Tuesday or Thursday due to when the ETL pipeline ran. A small config change on the opposite side of a code base completely altered the semantics of the data!