Ask HN: Examples of agentic LLM systems in production?

#1

Post by SebaSeba » Mon, Dec 16, 2024, 2:40 PM UTC

Now that everybody and their mother are fuzzing in social media about LLM agents and agentic LLM systems (or something), are there actual examples of live applications that are based on an agentic LLM process flow?

I'd be curious to know and see such examples in order to derive some inspiration from them.

Re: Ask HN: Examples of agentic LLM systems in production?

#2

Post by klntsky » Mon, Dec 16, 2024, 3:23 PM UTC

Support bots. Scrapers. Personal assistants. Search startups like perplexity. Scammer bots. Bots that spread political agenda. "AI" memecoins.

Re: Ask HN: Examples of agentic LLM systems in production?

#3

Post by bronco21016 » Mon, Dec 16, 2024, 3:24 PM UTC

With all the agencies and the YouTube demos of n8n and Make.com they should be everywhere.

I look at my workplace and I see places where they might fit in but if the reliability isn’t 99.5% they won’t be trusted and I think that’s a problem.

I made a toy in n8n that collects transactions in YNAB via API and matches them to Amazon orders in GMail. It then uses GPT-4o with vision to categorize the product pictures according to my budget’s categories but I have to add the order link to the transaction memo and add a flag for human review because it’s only 80% or so. It has sped up the workflow for sure but nowhere near good enough to set it and forget it.

Re: Ask HN: Examples of agentic LLM systems in production?

#4

Post by SebaSeba » Mon, Dec 16, 2024, 3:26 PM UTC

klntsky wrote: post #2

Support bots. Scrapers. Personal assistants. Search startups like perplexity. Scammer bots. Bots that spread political agenda. "AI" memecoins.

When does a LLM customer support bot that is based for example on RAG architecture, become an LLM agent?

Re: Ask HN: Examples of agentic LLM systems in production?

#5

Post by austinkhale » Mon, Dec 16, 2024, 3:26 PM UTC

I know of many, many LLM systems in production system, since that's what I've been helping companies build since the start of the year. Mostly it's pretty rote automation work but the cost savings are incredible.

Agentic workflows are a much higher bar that are just barely starting to work. I can't speak to their efficacy but here's a few of the ones that are sort of starter-level agents that I've started seeing some companies adopt:

- https://www.intercom.com/fin

- https://www.rox.com/

- https://devin.ai/

- https://bolt.new/

- https://v0.dev/

Re: Ask HN: Examples of agentic LLM systems in production?

#6

Post by DataDaemon » Mon, Dec 16, 2024, 3:27 PM UTC

Agentic workflow is great only for demos without real business cases. Each agent can hallucinate, which will pass this hallucination to another agent. In the end, you have just garbage. But... It's better to be silent, we still need to inflate this bubble.

Re: Ask HN: Examples of agentic LLM systems in production?

#7

Post by beardedwizard » Mon, Dec 16, 2024, 3:28 PM UTC

SebaSeba wrote: post #4

klntsky wrote: post #2

Support bots. Scrapers. Personal assistants. Search startups like perplexity. Scammer bots. Bots that spread political agenda. "AI" memecoins.

When does a LLM customer support bot that is based for example on RAG architecture, become an LLM agent?

As soon as we admit to ourselves that agent is just another word for context isolation among coordinated llm tasks.

Will agents still matter once models do a better job paying complete attention to large contexts?

Re: Ask HN: Examples of agentic LLM systems in production?

#8

Post by klntsky » Mon, Dec 16, 2024, 3:28 PM UTC

SebaSeba wrote: post #4

klntsky wrote: post #2

Support bots. Scrapers. Personal assistants. Search startups like perplexity. Scammer bots. Bots that spread political agenda. "AI" memecoins.

When does a LLM customer support bot that is based for example on RAG architecture, become an LLM agent?

The definition of agent is blurry. I prefer to avoid that term because it does not mean anything in particular. These are implemented as chat completion API calls + parsing + interpretation.

Re: Ask HN: Examples of agentic LLM systems in production?

#9

Post by shmatt » Mon, Dec 16, 2024, 3:31 PM UTC

You'd probably have to define agents first. What large / mega caps call agents is LLM + RAG + API Calls to read data and trigger jobs. And there are plenty of those online

Re: Ask HN: Examples of agentic LLM systems in production?

#10

Post by SebaSeba » Mon, Dec 16, 2024, 3:31 PM UTC

bronco21016 wrote: post #3

With all the agencies and the YouTube demos of n8n and Make.com they should be everywhere. I look at my workplace and I see places where they might fit in but if the reliability isn’t 99.5% they won’t be trusted and I think that’s a problem. I made a toy in n8n that collects transactions in YNAB via API and matches them to Amazon orders in GMail. It then uses GPT-4o with vision to categorize the product pictures acco…

Interesting! To me 80% hitrate sounds actually pretty good and awesome if it actually improves productivity, though understandably not something that could be left on it's own devices.

I had no idea about Make.com or n8n, they seem interesting. Thanks for the tip! Will check them out.