Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: What are you actually using LLMs for in production?

news.ycombinator.com

Post reply on HN
Print view View original

61–70 of 70 posts

Re: Ask HN: What are you actually using LLMs for in production?

#61

Post by scarface_74 »

Chatbots with some nuance. I work with voice and chat call centers hosted on Amazon Connect - the AWS version of the call center that Amazon uses internally.

Traditionally and still how it works in most call centers, you have to explicitly list out the things you can handle (intents), what sentences trigger them (utterances) and slots - ie “I want to get a flight from {origin} to {destination}” the variable parts would be the slots

Anyway, absolutely no company would or should trust an LLM to generate output to a customer. It never ends well. I use Gen ai to categorize free text input from a customer into a set of intents the system can handle and fill in the slots. But the output is very much on rails

It works a lot better than the old school method.

Re: Ask HN: What are you actually using LLMs for in production?

#62

Post by scarface_74 »

captainbland wrote: post #25
petercooper wrote: post #16

Analyzing firehoses of data. RSS feeds, releases, stuff like that. My job involves curating information and while I still do that process by hand, LLMs make my net larger and help me find more signals. This means hallucinations or mistakes aren't a big deal, since it all ends up with me anyway. I'm quite bullish on using LLMs as extra eyes, rather than as extra hands where they can run into trouble.

Is cost a major consideration for you here? Like if you're dealing with firehose data which I'm assuming is fairly high throughput, do you see an incentive for potentially switching to a more specific NLP classifier model rather than sticking with generative LLMs? Or is it that this is good enough/the ROI of switching isn't attractive? Or is the generative aspect adding something else here?

I can’t tell you what I’m working on but I can give you a real world example of where traditional models don’t work well.

Sentiment analysis is like the “Hello World” when you’re using Machine Learning.

But I had a use case similar to a platform like Uber eats where someone can be critical of the service provider or be critical of the platform itself. I needed to be able to distinguish sentiment about the platform based on reviews and sentiment about someone on the platform.

No matter what you do, people are going to conflate the reviews.

As far as costs, I mentioned in another comment that I work with online call centers sometimes. There anytime a person has to answer a call, it costs the company from $2-$5.

One call deflection that saves the company $5 can pay for a lot of inference. It’s literally 100x cheaper at least to use an LLM.

Re: Ask HN: What are you actually using LLMs for in production?

#63

Post by mulmboy »

We operate a saas where a common step is inputting rates of widgets in $/widget, $/widget/day, $/1kwidgets, etc etc. These are incredibly tedious and error prone to enter. And usually the source of these rates is an invoice which presents them in ambiguous ways e.g. rows with "quantity" and "charge" from which you have to back calculate the rate. And these invoices are formatted in all different ways.

We offer a feature to upload the invoice and we pull out all the rates for you. Uses LLMs under the hood. Fundamentally it's a "chatgpt wrapper" but there's a massive amount of work in tweaking the prompts based on evals, splitting things up into multiple calls, etc.

And it works great! Niche software, but for power users were saving them tens of minutes of monotonous work per day and in all likelihood entering things more accurate. This complements the manual entry process with full ability to review the results. Accuracy is around 98-99 percent.

Re: Ask HN: What are you actually using LLMs for in production?

#65

Post by miketery »

dartharva wrote: post #38
miketery wrote: post #37

I built a SQL agent with detailed database context and a set of tools. It’s been a huge lift for me and the team in generating rather complex queries that would take non trivial time to construct, even if using cursor or ChatGPT.

I'm in the process of building one too. Handing off SQL queries to LLMs feels like a no-brainer.

Awesome! Let me know how your experience is or if you have any questions.

I went pretty simple, used OpenAI agent sdk and built a couple of tools like “run_query” with read only connection. Initially I also had a tool for getting the join path from A to B, but the context I wrote out was sufficient.

I think main challenge with this agent is how to keep the context up to date.

Re: Ask HN: What are you actually using LLMs for in production?

#66

Post by PestoDiRucola »

jackthetab wrote: post #56

Which LLMs and plans are you guys using for all of these cool ideas? ATM I use ChatGPT Plus for everything except coding inside my Jetbrains IDEs. I'm starting to look around at other LLMs for non-coding purposes (brainstorming, docs, being a project manager, summarizing, learning new subjects, etc.).

Claude Sonnet 4 is pretty good for programming stuff, albeit pretty expensive. It's also good for other things as well. I like using it as a conversational partner when developing the systems architecture whenever I'm working on something new.

Gemini 2.5 is pretty cheap and has a huge context window, although not as good as Claude for programming. For that reason I would suggest to use is through the API if you're building a product that has an LLM step.

Re: Ask HN: What are you actually using LLMs for in production?

#67

Post by slake »

gametorch wrote: post #49

1. Pre-prompting for image and video generation. Gives you way better results for less than a cent of added cost. Although many image models do this thing for you; you have to understand each individual model and apply this judiciously. 2. I build REPLs into any manual workflow that makes use of LLMs. Instead of just being like "F@ck, it didn't work!" you can instead tell the LLM why it didn't work and help it get th…

How do you do the REPL integration into the manual workflow? Could you explain that a bit?

Re: Ask HN: What are you actually using LLMs for in production?

#68

Post by incomingpain »

My startup, https://mapleintel.ca has an original threatfeed that's 100% reliable, all ip addresses have directly attacked me.

The new AI threatfeed is everything above + I'm using AI to make rapid decisions for me. I can pull info from sources like dnsbl etc to help judge. If I were to do it manually, maybe 1 ip per 30 seconds? Phi4, omg 1 every second.

Re: Ask HN: What are you actually using LLMs for in production?

#70

Post by careful_ai »

That thread nails a common clash: AI tools promise scale, but often just shift complexity to human coordination.

What I’ve noticed in my own projects is similar: every shiny AI integration spawns a hidden cost—coordination overhead, new edge cases, unexpected governance needs—everything that sits between "works in demo" and "works at scale."

We should be wary of framing AI as efficiency silver bullets. Instead, the real work is in system integration—making AI enhancements feel seamless, not another silo.

Post reply on HN