Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Examples of agentic LLM systems in production?

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 80 posts

Re: Ask HN: Examples of agentic LLM systems in production?

#71

Post by BrandiATMuhkuh »

bradarner wrote: post #41
isoprophlex wrote: post #29

An anecote that helps you maybe: I do contracting work, we're building a text-to-sql automated business analyst. It's quite well-rounded: it tries to recover from errors, allows automatic creation of appropriate visualisations, has a generic "faq" component to help the user understand how to use the tool. The tool is available to some 10.000 b2b users. It's just a bunch of prompts conditionally slapped together in a…

The hype of Agentic AI is to LLMs what an MBA is to business. Overcomplicating something with language that is pretty common sense. I've implement countless LLM based "agentic" workflows over the past year. They are simple. It is a series of prompts that maintain state with a targeted output. The common association with "a floating R2D2" is not helpful. They are not magic. The core elements I'm seeing so far are: the…

Do you have some advice on how to build the structure on how to move from one prompt to the next?

Are you using a separate state manager + function calling so the LLM knows where it is?

Re: Ask HN: Examples of agentic LLM systems in production?

#72

Post by anon373839 »

Setting aside buzzwords, how are people currently dealing with the problem of LLM errors propagating/accumulating through a pipeline? All of these model call feeding into model calls feeding into model calls results in a pretty low probability that the overall task stays on a happy path. And adding even more calls to guardrail the steps adds compounding latency.

Re: Ask HN: Examples of agentic LLM systems in production?

#73

Post by arresin »

simonw wrote: post #35
isoprophlex wrote: post #29

An anecote that helps you maybe: I do contracting work, we're building a text-to-sql automated business analyst. It's quite well-rounded: it tries to recover from errors, allows automatic creation of appropriate visualisations, has a generic "faq" component to help the user understand how to use the tool. The tool is available to some 10.000 b2b users. It's just a bunch of prompts conditionally slapped together in a…

Love that. Reminds me of a time I was asked to build a "machine learning algorithm" driven recommendation system... and eventually I realized that delivering a recommendation system based on one big BM25 search query was fine, and the people asking for it to use "machine learning" didn't actually understand or care about the difference.

It's nice to combine the two but the ranking takes tuning.

Re: Ask HN: Examples of agentic LLM systems in production?

#74

Post by arresin »

SebaSeba wrote: post #10
bronco21016 wrote: post #3

With all the agencies and the YouTube demos of n8n and Make.com they should be everywhere. I look at my workplace and I see places where they might fit in but if the reliability isn’t 99.5% they won’t be trusted and I think that’s a problem. I made a toy in n8n that collects transactions in YNAB via API and matches them to Amazon orders in GMail. It then uses GPT-4o with vision to categorize the product pictures acco…

Interesting! To me 80% hitrate sounds actually pretty good and awesome if it actually improves productivity, though understandably not something that could be left on it's own devices. I had no idea about Make.com or n8n, they seem interesting. Thanks for the tip! Will check them out.

also: nodered, windmill.

And non ui flow diagram but essentially the same thing: inngest, hatchet.

Re: Ask HN: Examples of agentic LLM systems in production?

#75

Post by authorfly »

ilaksh wrote: post #34
bluejay2387 wrote: post #11

As a side note, while I know of several language model based systems that have been deployed in companies, some companies don't want to talk about it: 1. Its still perceived as an issue of competitive advantage 2. There is a serious concern about backlash. The public's response to finding out that companies have used AI has often not been good (or even reasonable) -- particularly if there was worker replacement relat…

There is a very simple and obvious definition: it's agentic if it uses tool calls to accomplish a task. This is the only one that makes sense. People want to conflate it with their random vague conceptions of AGI or ASI or make some kind of vague requirement for a certain level of autonomy, but that doesn't make sense. An agent is an agent and an autonomous agent is an autonomous agent, but a fully autonomous agent i…

Agentic AI comes out of historical AI, systems computing and further back biological/philosophical discussion. It's not about tool use although ironically, animal tool use is a fascinating subject not yet corrupted by the hype around intelligence.

I implore you to look into that to see how some people relate it to autonomy or AGI or ASI(wrongly, imo - I think shoehorning OOP and UML diagrams plus limited database like memory/context is not a path to AGI. Clever use of final layers, embeddings and how you store/weight them (and even more interesting combinations) may yield interesting results because we can (buzzword warning) transcend written decoding paradigms - the Human brain clearly does not rely on language).

However what gets marketed today is, as you say, not capable of any real agent autonomy like in academia - they are just self-recursive ChatGPT prompts with additional constraining limits. One day it might be more, but libraries now are all doing that from my eye. And recursion has pros but emphasizes the unreliability con/negative of LLMs.

Re: Ask HN: Examples of agentic LLM systems in production?

#76

Post by isoprophlex »

BrandiATMuhkuh wrote: post #71
bradarner wrote: post #41

Earlier quoted context omitted.

The hype of Agentic AI is to LLMs what an MBA is to business. Overcomplicating something with language that is pretty common sense. I've implement countless LLM based "agentic" workflows over the past year. They are simple. It is a series of prompts that maintain state with a targeted output. The common association with "a floating R2D2" is not helpful. They are not magic. The core elements I'm seeing so far are: the…

Do you have some advice on how to build the structure on how to move from one prompt to the next? Are you using a separate state manager + function calling so the LLM knows where it is?

You can model it as a state machine, where the LLM decides to what state it wants to advance. In terms of developer ergonomics, strongly typed outputs help. You can for example force a function call at each step, where one of the call arguments is an enum specifying the state to advance to.

Shoot me an email if you want to discuss specifics!

Re: Ask HN: Examples of agentic LLM systems in production?

#77

Post by isoprophlex »

moltar wrote: post #65
isoprophlex wrote: post #29

An anecote that helps you maybe: I do contracting work, we're building a text-to-sql automated business analyst. It's quite well-rounded: it tries to recover from errors, allows automatic creation of appropriate visualisations, has a generic "faq" component to help the user understand how to use the tool. The tool is available to some 10.000 b2b users. It's just a bunch of prompts conditionally slapped together in a…

Is the tool public? We are looking for a solid text to sql tool that works with Athena.

Sadly, no, it's a walled-off customer facing tool integrated into one of my client's B2B business intelligence portals.

Hope you can find a tool; the big data players are of course jumping on this (snowflake, databricks, they all talk about their text-to-sql tools).

If you have the budget and want something bespoke built that has some magic sauce tuned to your exact problem field, send me an email!

Re: Ask HN: Examples of agentic LLM systems in production?

#78

Post by isoprophlex »

philipodonnell wrote: post #57
isoprophlex wrote: post #29

An anecote that helps you maybe: I do contracting work, we're building a text-to-sql automated business analyst. It's quite well-rounded: it tries to recover from errors, allows automatic creation of appropriate visualisations, has a generic "faq" component to help the user understand how to use the tool. The tool is available to some 10.000 b2b users. It's just a bunch of prompts conditionally slapped together in a…

I’ve been doing a lot of work on semantic data architecture that better supports LLM analytics, did you use any framework or methodology to decide how exactly to present the data/metadata to the LLM context to allow it to make decisions?

A pre-processing phase does a lot of heavy lifting, where we stuff the table and column comments, additional metadata, and some hand-tuned heuristics into a graph-like structure. Basically using LLMs itself to preprocess the schema metadata.

Everything is very boring tech-wise, using vanilla postgres/pgvector and a few hundred lines of python. Every RAG-searchable text field (mostly column descriptions and a list of LLM-generated example queries) is linked to nodes holding metadata, at most 2 hops out. The tool is available to 10.000 users, but load is only a few queries per minute at peak... so performance wise it's fine.

Re: Ask HN: Examples of agentic LLM systems in production?

#79

Post by philipodonnell »

isoprophlex wrote: post #78
philipodonnell wrote: post #57

Earlier quoted context omitted.

I’ve been doing a lot of work on semantic data architecture that better supports LLM analytics, did you use any framework or methodology to decide how exactly to present the data/metadata to the LLM context to allow it to make decisions?

A pre-processing phase does a lot of heavy lifting, where we stuff the table and column comments, additional metadata, and some hand-tuned heuristics into a graph-like structure. Basically using LLMs itself to preprocess the schema metadata. Everything is very boring tech-wise, using vanilla postgres/pgvector and a few hundred lines of python. Every RAG-searchable text field (mostly column descriptions and a list of…

Enhancing the comments on the existing data model seems to be the most common approach for sure. I'm implementing this as a data architecture at several clients and I've found creating a whole new logical structure designed for the LLM is really effective. Not being bound by the original data model lets you solve several problems related to the "n-hops" question, avoiding needing the comments, and the semantics of how data engineers define columns. Some more details here [1], but obviously you can implement this totally yourself by hand.

[1] (https://github.com/eloquentanalytics/pyeloquent/blob/main/RE...)

Re: Ask HN: Examples of agentic LLM systems in production?

#80

Post by mejutoco »

austinkhale wrote: post #12
bluejay2387 wrote: post #11

As a side note, while I know of several language model based systems that have been deployed in companies, some companies don't want to talk about it: 1. Its still perceived as an issue of competitive advantage 2. There is a serious concern about backlash. The public's response to finding out that companies have used AI has often not been good (or even reasonable) -- particularly if there was worker replacement relat…

This has been my experience. Lots of companies are implementing LLMs but are not advertising it. There's virtually no upside to being public about it.

Investors at throwing money at ai projects. That is one upside.
Post reply on HN