Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adaptive LLM routing under budget constraints

arxiv.org

Post reply on HN
Print view View original

21–30 of 83 posts

Re: Adaptive LLM routing under budget constraints

#21

Post by nicce »

jibal wrote: post #13
andrewflnr wrote: post #3

Is this really the frontier of LLM research? I guess we really aren't getting AGI any time soon, then. It makes me a little less worried about the future, honestly. Edit: I never actually expected AGI from LLMs. That was snark. I just think it's notable that the fundamental gains in LLM performance seem to have dried up.

LLMs are not on the road to AGI, but there are plenty of dangers associated with them nonetheless.

Just 2 days ago Gemini 2.5 Pro tried to recommend me tax evasion based on non-existing laws and court decisions. The model was so charming and convincing, that even after I brought all the logic flaws and said that this is plain wrong, I started to doubt myself, because it is so good at pleasing, arguing and using words.

And most would have accept the recommendation because the model sold it as less common tactic, while sounding very logical.

Re: Adaptive LLM routing under budget constraints

#22

Post by ctoth »

andrewflnr wrote: post #3

Is this really the frontier of LLM research? I guess we really aren't getting AGI any time soon, then. It makes me a little less worried about the future, honestly. Edit: I never actually expected AGI from LLMs. That was snark. I just think it's notable that the fundamental gains in LLM performance seem to have dried up.

Is a random paper from Fujitsu Research claiming to be the frontier of anything?

Re: Adaptive LLM routing under budget constraints

#23

Post by _heimdall »

kenjackson wrote: post #7
andrewflnr wrote: post #3

Is this really the frontier of LLM research? I guess we really aren't getting AGI any time soon, then. It makes me a little less worried about the future, honestly. Edit: I never actually expected AGI from LLMs. That was snark. I just think it's notable that the fundamental gains in LLM performance seem to have dried up.

First, I don't think we will ever get to AGI. Not because we won't see huge advances still, but AGI is a moving ambiguous target that we won't get consensus on. But why does this paper impact your thinking on it? It is about budget and recognizing that different LLMs have different cost structures. It's not really an attempt to improve LLM performance measured absolutely.

So you don't expect AGI to be possible ever? Or is your concern mainly with the wildly different definitions people use for it and that we'll continue moving goal posts rather than agree we got there?

Re: Adaptive LLM routing under budget constraints

#24

Post by andrewflnr »

yahoozoo wrote: post #17
andrewflnr wrote: post #3

Is this really the frontier of LLM research? I guess we really aren't getting AGI any time soon, then. It makes me a little less worried about the future, honestly. Edit: I never actually expected AGI from LLMs. That was snark. I just think it's notable that the fundamental gains in LLM performance seem to have dried up.

That and LLMs are seemingly plateauing. Earlier this year, it seemed like the big companies were releasing noticeable improvements every other week. People would joke a few weeks is “an eternity” in AI…so what time span are we looking at now?

That's just the thing. There don't seem to have been any breakthroughs in model performance or architecture, so it seems like we're back to picking up marginal reductions in cost to make any progress.

Re: Adaptive LLM routing under budget constraints

#25

Post by roywiggins »

nicce wrote: post #21
jibal wrote: post #13

Earlier quoted context omitted.

LLMs are not on the road to AGI, but there are plenty of dangers associated with them nonetheless.

Just 2 days ago Gemini 2.5 Pro tried to recommend me tax evasion based on non-existing laws and court decisions. The model was so charming and convincing, that even after I brought all the logic flaws and said that this is plain wrong, I started to doubt myself, because it is so good at pleasing, arguing and using words. And most would have accept the recommendation because the model sold it as less common tactic, wh…

> even after I brought all the logic flaws and said that this is plain wrong

Once you've started to argue with an LLM you're already barking up the wrong tree. Maybe you're right, maybe not, but there's no point in arguing it out with an LLM.

Re: Adaptive LLM routing under budget constraints

#26

Post by andrewflnr »

jibal wrote: post #13
andrewflnr wrote: post #3

Is this really the frontier of LLM research? I guess we really aren't getting AGI any time soon, then. It makes me a little less worried about the future, honestly. Edit: I never actually expected AGI from LLMs. That was snark. I just think it's notable that the fundamental gains in LLM performance seem to have dried up.

LLMs are not on the road to AGI, but there are plenty of dangers associated with them nonetheless.

Agreed, broadly. I never really thought they were, but seeing people work on stuff like this instead of even trying to improve the architecture really makes it obvious.

Re: Adaptive LLM routing under budget constraints

#27

Post by andrewflnr »

ctoth wrote: post #22
andrewflnr wrote: post #3

Is this really the frontier of LLM research? I guess we really aren't getting AGI any time soon, then. It makes me a little less worried about the future, honestly. Edit: I never actually expected AGI from LLMs. That was snark. I just think it's notable that the fundamental gains in LLM performance seem to have dried up.

Is a random paper from Fujitsu Research claiming to be the frontier of anything?

Not just this paper, but model working shenanigans also seem to have been a big part of GPT-5, which certainly claims to be frontier work.

Re: Adaptive LLM routing under budget constraints

#29

Post by KTibow »

pqtyw wrote: post #14
pbd wrote: post #4

GPT-4 at $24.7 per million tokens vs Mixtral at $0.24 - that's a 100x cost difference! Even if routing gets it wrong 20% of the time, the economics still work. But the real question is how you measure 'performance' - user satisfaction doesn't always correlate with technical metrics.

> GPT-4 at $24.7 per million tokens While technically true why would you want to use it when OpenAI itself provides a bunch of many times cheaper and better models?

RouterBench is from March 2024.

Re: Adaptive LLM routing under budget constraints

#30

Post by nutjob2 »

_heimdall wrote: post #23
kenjackson wrote: post #7

Earlier quoted context omitted.

First, I don't think we will ever get to AGI. Not because we won't see huge advances still, but AGI is a moving ambiguous target that we won't get consensus on. But why does this paper impact your thinking on it? It is about budget and recognizing that different LLMs have different cost structures. It's not really an attempt to improve LLM performance measured absolutely.

So you don't expect AGI to be possible ever? Or is your concern mainly with the wildly different definitions people use for it and that we'll continue moving goal posts rather than agree we got there?

There's no concrete evidence AGI is possible mostly because it has no concrete definition.

It's mostly hand waving, hype and credulity, and unproven claims of scalability right now.

You can't move the goal posts because they don't exist.

Post reply on HN