Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Adaptive LLM routing under budget constraints

arxiv.org

Post reply on HN
Print view View original

61–70 of 83 posts

Re: Adaptive LLM routing under budget constraints

#61

Post by 9dev »

ACCount37 wrote: post #45
kenjackson wrote: post #7

Earlier quoted context omitted.

First, I don't think we will ever get to AGI. Not because we won't see huge advances still, but AGI is a moving ambiguous target that we won't get consensus on. But why does this paper impact your thinking on it? It is about budget and recognizing that different LLMs have different cost structures. It's not really an attempt to improve LLM performance measured absolutely.

I can totally see "it's not really AGI because it doesn't consistently outperform those three top 0.000001% outlier human experts yet if they work together". It'll be a while until the ability to move the goalposts of "actual intelligence" is exhausted entirely.

Well right now, my niece of 7 years outperforms all LLM contenders in drawing a Pelican on a bicycle

Re: Adaptive LLM routing under budget constraints

#62

Post by ACCount37 »

muldvarp wrote: post #46
ACCount37 wrote: post #42

Earlier quoted context omitted.

Wait until it ramps up so much that people will say "it's a plateau, for real this time" when they go 3 days without a +10% capability jump.

I mean I wish there were a plateau, without one we're well onto our way into techno-feudalism. I just don't see it.

That's what it is: wishful thinking. A lot of people really, really want AI tech to fail - because they don't like the alternative.

Re: Adaptive LLM routing under budget constraints

#64

Post by kenjackson »

9dev wrote: post #61
ACCount37 wrote: post #45

Earlier quoted context omitted.

I can totally see "it's not really AGI because it doesn't consistently outperform those three top 0.000001% outlier human experts yet if they work together". It'll be a while until the ability to move the goalposts of "actual intelligence" is exhausted entirely.

Well right now, my niece of 7 years outperforms all LLM contenders in drawing a Pelican on a bicycle

I know this was a joke, but LLMs are quite good at this now. If your niece draws better then she’s a good artist.

Re: Adaptive LLM routing under budget constraints

#65

Post by worm00111 »

aspect8445 wrote: post #38
nutjob2 wrote: post #35

Earlier quoted context omitted.

Does anyone working in an individual capacity actually end up paying for Gemini (Flash or Pro)? Or does Google boil you like a frog and you end up subscribing?

I've used Gemini in a lot of personal projects. At this point I've probably made tens of thousands of requests, sometimes exceeding 1k per week. So far, I haven't had to pay a dime!

How come you don't need to pay? Do you get it for free somehow?

Re: Adaptive LLM routing under budget constraints

#66

Post by KETHERCORTEX »

worm00111 wrote: post #65
aspect8445 wrote: post #38

Earlier quoted context omitted.

I've used Gemini in a lot of personal projects. At this point I've probably made tens of thousands of requests, sometimes exceeding 1k per week. So far, I haven't had to pay a dime!

How come you don't need to pay? Do you get it for free somehow?

There's free tier for API.

Re: Adaptive LLM routing under budget constraints

#67

Post by _heimdall »

nutjob2 wrote: post #30
_heimdall wrote: post #23

Earlier quoted context omitted.

So you don't expect AGI to be possible ever? Or is your concern mainly with the wildly different definitions people use for it and that we'll continue moving goal posts rather than agree we got there?

There's no concrete evidence AGI is possible mostly because it has no concrete definition. It's mostly hand waving, hype and credulity, and unproven claims of scalability right now. You can't move the goal posts because they don't exist.

Got it, and yeah I agree with you there. I've been frustrated by a different view of it though, many people seem to have a definition and they are often wildly different.

Re: Adaptive LLM routing under budget constraints

#68

Post by hackathonguy »

I'm very curious whether a) anecdotally, anyone has encountered a real enterprise cost-cutting effort focused on LLM APIs and b) empirically, whether anyone has done any research on price elasticity in LLMs of different performance scales.

So far, my experience has been that it's just too early for most people / applications to worry about cost - at most, I've seen AI to be accountable for 10% of cloud costs. But very curious if others have other experiences.

Re: Adaptive LLM routing under budget constraints

#69

Post by drittich »

KETHERCORTEX wrote: post #66
worm00111 wrote: post #65

Earlier quoted context omitted.

How come you don't need to pay? Do you get it for free somehow?

There's free tier for API.

"When you use Unpaid Services, including, for example, Google AI Studio and the unpaid quota on Gemini API, Google uses the content you submit to the Services and any generated responses to provide, improve, and develop Google products and services and machine learning technologies, including Google's enterprise features, products, and services, consistent with our Privacy Policy.

To help with quality and improve our products, human reviewers may read, annotate, and process your API input and output. Google takes steps to protect your privacy as part of this process. This includes disconnecting this data from your Google Account, API key, and Cloud project before reviewers see or annotate it. Do not submit sensitive, confidential, or personal information to the Unpaid Services."

Reference: https://ai.google.dev/gemini-api/terms

Re: Adaptive LLM routing under budget constraints

#70

Post by abalashov »

ashirviskas wrote: post #43
nutjob2 wrote: post #30

Earlier quoted context omitted.

There's no concrete evidence AGI is possible mostly because it has no concrete definition. It's mostly hand waving, hype and credulity, and unproven claims of scalability right now. You can't move the goal posts because they don't exist.

Well, if a human is GI, we just need to make it Artificial. Easy.

I like to say that it's not AI -- it's just A.
Post reply on HN