Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Small language models are the future of agentic AI

arxiv.org

Post reply on HN
Print view View original

31–40 of 47 posts

Re: Small language models are the future of agentic AI

#31

Post by incrudible »

iagooar wrote: post #18

I think that part of the beauty of LLMs is their versatility in so many different scenarios. When I build my agentic pipeline, I can plug in any of the major LLMs, add a prompt to it, and have it go off to do its job. Specialized, fine-tuned models sit somewhere in between LLMs and traditional procedural code. The fine-tuning process takes time and is a risk if it goes wrong. In the meantime, the LLMs by major provid…

> In the meantime, the LLMs by major providers get smarter every day.

Are they though? Or are they just getting better at gaming benchmarks?

Subjectively, there has been modest progress in the past year, but I'm curious to hear other anecdotes from people that aren't firmly invested in the hype.

Re: Small language models are the future of agentic AI

#32

Post by chesterhunt20 »

Absolutely agree-small language models are a crucial step toward scalable, efficient agentic AI. Their reduced computational footprint enables faster, on-device reasoning, which is essential for real-time decision-making in autonomous agents. Platforms like Legittmate AI are already exploring how small models can be fine-tuned for secure, task-specific workflows in legal and contract automation. With enhanced fine-tuning and optimization, SLMs are becoming increasingly capable—without the heavy cost and latency of larger models. The future of agentic AI isn't just about power-it's about precision, privacy, and portability.

Re: Small language models are the future of agentic AI

#33

Post by bglazer »

chesterhunt20 wrote: post #32

Absolutely agree-small language models are a crucial step toward scalable, efficient agentic AI. Their reduced computational footprint enables faster, on-device reasoning, which is essential for real-time decision-making in autonomous agents. Platforms like Legittmate AI are already exploring how small models can be fine-tuned for secure, task-specific workflows in legal and contract automation. With enhanced fine-tu…

Posting LLM generated ads on HN is the fastest way for me to lose amy respect or interest in a company

Re: Small language models are the future of agentic AI

#34

Post by iagooar »

incrudible wrote: post #31
iagooar wrote: post #18

I think that part of the beauty of LLMs is their versatility in so many different scenarios. When I build my agentic pipeline, I can plug in any of the major LLMs, add a prompt to it, and have it go off to do its job. Specialized, fine-tuned models sit somewhere in between LLMs and traditional procedural code. The fine-tuning process takes time and is a risk if it goes wrong. In the meantime, the LLMs by major provid…

> In the meantime, the LLMs by major providers get smarter every day. Are they though? Or are they just getting better at gaming benchmarks? Subjectively, there has been modest progress in the past year, but I'm curious to hear other anecdotes from people that aren't firmly invested in the hype.

If you have used Sonnet 3.5, 3.7 and 4 in the last few months, you know how much the model has improved. I am achieving 3-5x complexity with latest Sonnet as compared to what was possible with the earlier versions.

They are getting much much better.

Re: Small language models are the future of agentic AI

#35

Post by DebtDeflation »

automatic6131 wrote: post #6
bryant wrote: post #3

A few weeks ago, I processed a product refund with Amazon via agent. It was simple, straightforward, and surprisingly obvious that it was backed by a language model based on how it responded to my frustration about it asking tons of questions. But in the end, it processed my refund without ever connecting me with a human being. I don't know whether Amazon relies on LLMs or SLMs for this and for similar interactions,…

You can (used to?) get a refund on Amazon with normal CRUD app flow. Putting an SLM and a conversational interface over it is a backwards step.

Likewise, we've been building conversational interfaces for stuff like this for over a decade using traditional NLP techniques and orchestration rather than LLMs. Intent classification, named entity extraction, slot filling, and API calling.

Processing returns is pretty standardized - identify the order and the item within it being returned, capture the reason for the return, check eligibility, capture whether they want a refund or a new item, and execute either. When you have a fully deterministic workflow with 5-6 steps, maybe 1 or 2 if-then branches, and then a single action at the end, I don't see the value of running an LLM in a loop, burning a crazy amount of tokens, and hoping it works at least 80% of the time when there are far simpler and cheaper ways of doing it that will work almost 100% of the time.

Re: Small language models are the future of agentic AI

#36

Post by scarface_74 »

bryant wrote: post #3

A few weeks ago, I processed a product refund with Amazon via agent. It was simple, straightforward, and surprisingly obvious that it was backed by a language model based on how it responded to my frustration about it asking tons of questions. But in the end, it processed my refund without ever connecting me with a human being. I don't know whether Amazon relies on LLMs or SLMs for this and for similar interactions,…

That’s not really an LLM problem. Even sentiment analysts is like the ML version of “Hello World”.

At most that could be done with pre LLM chatbots like classic Alexa - or the AWS equivalent Anazon Lex (which has Sentiment Analysis built in)

Re: Small language models are the future of agentic AI

#37

Post by scarface_74 »

torginus wrote: post #8
bryant wrote: post #3

A few weeks ago, I processed a product refund with Amazon via agent. It was simple, straightforward, and surprisingly obvious that it was backed by a language model based on how it responded to my frustration about it asking tons of questions. But in the end, it processed my refund without ever connecting me with a human being. I don't know whether Amazon relies on LLMs or SLMs for this and for similar interactions,…

I just had my first experience with a customer service LLM. I needed to get my account details changed, and for that I needed to use the customer support chat. The LLM told me what sort of information they need, and what is the process, after which I followed through the whole thing. After I went through the whole thing it reassured me everything is in order, and my request is being processed. For two weeks, nothing…

That has nothing to do with sn LLM. Any chah based system whether LLM or not is going to interpret the human input and convert it to a standardized request for backend processing. This is just a badly written system.

Re: Small language models are the future of agentic AI

#38

Post by zsyllepsis »

DebtDeflation wrote: post #35
automatic6131 wrote: post #6

Earlier quoted context omitted.

You can (used to?) get a refund on Amazon with normal CRUD app flow. Putting an SLM and a conversational interface over it is a backwards step.

Likewise, we've been building conversational interfaces for stuff like this for over a decade using traditional NLP techniques and orchestration rather than LLMs. Intent classification, named entity extraction, slot filling, and API calling. Processing returns is pretty standardized - identify the order and the item within it being returned, capture the reason for the return, check eligibility, capture whether they w…

True, we have been building conversational interfaces with traditional NLP. In my experience, they’ve been fairly fragile.

Extending the example you gave, nicely packaged, fully deterministic workflows work great in demos. Then customers start going off the paved path. They ask about returning 3 items all at once, or a whole order. They get confused and provide a shipping number instead of an order number. They switch language part of the way through the conversation because they get frustrated by all these follow-up questions.

All of these absolutely can be handled through traditional NLP, but require system designers to account for them, model the conversation, and design their system accordingly to react accordingly. And suddenly the 5-6 step deterministic workflows with a couple of if-branches… isn’t.

Re: Small language models are the future of agentic AI

#39

Post by sillytwice »

sillytwice wrote: post #28
automatic6131 wrote: post #6

Earlier quoted context omitted.

You can (used to?) get a refund on Amazon with normal CRUD app flow. Putting an SLM and a conversational interface over it is a backwards step.

Recently I sent a product on guarantee to Amazon for reparation using a tag label and paying 42€, and the next day they cancelled the label of the product (they are investigating why) and the product was rejected in the Amazon store. Now, following indications, I have to wait the product to come back and resend it paying another 42 € that they promise to refund me later. The product is an air conditioned system for a…

Solving the stream of problems with a repl (read-eval-print-loop) is like in python:

   while life: 
     problem = read(bureaucracy, bad_luck, bad_ai)
     response = eval(problem, resources = few)
     print(response) # Spoiler '404 Solution not found'

Re: Small language models are the future of agentic AI

#40

Post by hiatus »

nurettin wrote: post #20
quietbritishjim wrote: post #17

Earlier quoted context omitted.

Air Canada famously lost a court case recently (though the actual interaction happened in 2022) where their chat bot promised a discount that they didn't actually offer. They tried to argue that the chatbot was a "separate legal entity that is responsible for its own actions"!! It still took that person a court case and countless hours to get the discount so it's hardly a victory really. https://www.bbc.co.uk/travel/…

This is why law in it's current form is wrong in every country and jurisdiction. We need "cumulative cases" that work like this: you submit your complaints to existing cumulative cases or open a new one, these are vetted by prosecutors. They accumulate evidence over time and once it is a respectable sum, a court case is opened (paid by the corporation) everyone receives what they are owed if/when the case is won. If…

Do class action lawsuits match what you envision?
Post reply on HN