Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

"Token anxiety", a slot machine by any other name

jkap.io

Post reply on HN
Print view View original

91–100 of 249 posts

Re: "Token anxiety", a slot machine by any other name

#91

Post by Drakim »

YetAnotherNick wrote: post #83
crystal_revenge wrote: post #82

Earlier quoted context omitted.

> What's Anthropic's optimization target??? Getting you the right answer as fast as possible! What makes you believe this? The current trend in all major providers seem to be: get you to spin up as many agents as possible so that you can get billed more and their number of requests goes up. > Slot machines have variable reward schedules by design LLMs by all major providers are optimized used RLHF where they are opti…

Bill is unrelated to their cost. If they can produce answer in 1/10th of the token, they can charge 10x more per token, likely even more.

That is simply not true, token price is largely determined by the token price of their rival services (even before their own operational costs). If everybody else charges about $1 per millions of tokens, then they will also charge about $1 per millions of tokens (or slightly above/below) regardless of how many answers per token they can provide.

Re: "Token anxiety", a slot machine by any other name

#92

Post by tokioyoyo »

crystal_revenge wrote: post #82
ctoth wrote: post #11

The gambling analogy completely falls apart on inspection. Slot machines have variable reward schedules by design — every element is optimized to maximize time on device. Social media optimizes for engagement, and compulsive behavior is the predictable output. The optimization target produces the addiction. What's Anthropic's optimization target??? Getting you the right answer as fast as possible! The variability in…

> What's Anthropic's optimization target??? Getting you the right answer as fast as possible! What makes you believe this? The current trend in all major providers seem to be: get you to spin up as many agents as possible so that you can get billed more and their number of requests goes up. > Slot machines have variable reward schedules by design LLMs by all major providers are optimized used RLHF where they are opti…

> What makes you believe this?

Simply, cut-throat competition. Given multiple nations are funding different AI-labs, quality of output and speed are one of the most important things.

Re: "Token anxiety", a slot machine by any other name

#93

Post by mh2266 »

ctoth wrote: post #11

The gambling analogy completely falls apart on inspection. Slot machines have variable reward schedules by design — every element is optimized to maximize time on device. Social media optimizes for engagement, and compulsive behavior is the predictable output. The optimization target produces the addiction. What's Anthropic's optimization target??? Getting you the right answer as fast as possible! The variability in…

Anthropic themselves have described CC as a slot machine:

https://www-cdn.anthropic.com/58284b19e702b49db9302d5b6f135a...

(cmd-f "slot machine")

Re: "Token anxiety", a slot machine by any other name

#95

Post by sixtyj »

Drakim wrote: post #91
YetAnotherNick wrote: post #83

Earlier quoted context omitted.

Bill is unrelated to their cost. If they can produce answer in 1/10th of the token, they can charge 10x more per token, likely even more.

That is simply not true, token price is largely determined by the token price of their rival services (even before their own operational costs). If everybody else charges about $1 per millions of tokens, then they will also charge about $1 per millions of tokens (or slightly above/below) regardless of how many answers per token they can provide.

This applies when there is a large number of competitors.

Now companies are fighting for the attention of a finite number of customers, so they keep their prices in line with those around them.

I remember when Google started with PPC - because few companies were using it, it cost a fraction of recent prices.

And the other issue to solve is future lack of electricity for land data centers. If everyone wants to use LLM… but data centers capacity is finite due to available power -> token prices can go up. But IMHO devs will find an innovative approach for tokens, less energy demanding… so token prices will probably stay low.

Re: "Token anxiety", a slot machine by any other name

#96

Post by delusional »

tokioyoyo wrote: post #92
crystal_revenge wrote: post #82

Earlier quoted context omitted.

> What's Anthropic's optimization target??? Getting you the right answer as fast as possible! What makes you believe this? The current trend in all major providers seem to be: get you to spin up as many agents as possible so that you can get billed more and their number of requests goes up. > Slot machines have variable reward schedules by design LLMs by all major providers are optimized used RLHF where they are opti…

> What makes you believe this? Simply, cut-throat competition. Given multiple nations are funding different AI-labs, quality of output and speed are one of the most important things.

How is nation-states funding private corporations "cut-throat competition"?

Re: "Token anxiety", a slot machine by any other name

#97

Post by hnbad »

crystal_revenge wrote: post #82
ctoth wrote: post #11

The gambling analogy completely falls apart on inspection. Slot machines have variable reward schedules by design — every element is optimized to maximize time on device. Social media optimizes for engagement, and compulsive behavior is the predictable output. The optimization target produces the addiction. What's Anthropic's optimization target??? Getting you the right answer as fast as possible! The variability in…

> What's Anthropic's optimization target??? Getting you the right answer as fast as possible! What makes you believe this? The current trend in all major providers seem to be: get you to spin up as many agents as possible so that you can get billed more and their number of requests goes up. > Slot machines have variable reward schedules by design LLMs by all major providers are optimized used RLHF where they are opti…

> The current trend in all major providers seem to be: get you to spin up as many agents as possible so that you can get billed more and their number of requests goes up.

I was surprised when I saw that Cursor added a feature to set the number of agents for a given prompt. I figured it might be a performance thing - fan out complex tasks across multiple agents that can work on the problem in parallel and get a combined solution. I was extremely disappointed when I realized it's just "repeat the same prompt to N separate agents, let each one take a shot and then pick a winner". Especially when some tasks can run for several minutes, rapidly burning through millions of tokens per agent.

At that point it's just rolling dice. If an agent goes so far off-script that its result is trash, I would expect that to mean I need to rework the instructions and context I gave it, not that I should try the same thing again and hope that entropy fixes it. But editing your prompt offline doesn't burn tokens, so it's not what makes them money.

Re: "Token anxiety", a slot machine by any other name

#98

Post by reasonableklout »

hnbad wrote: post #97
crystal_revenge wrote: post #82

Earlier quoted context omitted.

> What's Anthropic's optimization target??? Getting you the right answer as fast as possible! What makes you believe this? The current trend in all major providers seem to be: get you to spin up as many agents as possible so that you can get billed more and their number of requests goes up. > Slot machines have variable reward schedules by design LLMs by all major providers are optimized used RLHF where they are opti…

> The current trend in all major providers seem to be: get you to spin up as many agents as possible so that you can get billed more and their number of requests goes up. I was surprised when I saw that Cursor added a feature to set the number of agents for a given prompt. I figured it might be a performance thing - fan out complex tasks across multiple agents that can work on the problem in parallel and get a combin…

Cursor and others have a subagent feature, which sounds like what you wanted. However, there has to be some decision making around how to divide up a prompt into tasks. This is decided by the (parent) model currently.

The best-of-N feature is a bit like rolling N dice instead of one. But it can be quite useful if you use different models with different strengths and weaknesses (e.g. Claude/GPT-5/Gemini), rather than assigning all to N instances of Claude, for example. I like to use this feature in ask mode when diving into a codebase, to get an explanation a few different ways.

Re: "Token anxiety", a slot machine by any other name

#100

Post by pjc50 »

ctoth wrote: post #11

The gambling analogy completely falls apart on inspection. Slot machines have variable reward schedules by design — every element is optimized to maximize time on device. Social media optimizes for engagement, and compulsive behavior is the predictable output. The optimization target produces the addiction. What's Anthropic's optimization target??? Getting you the right answer as fast as possible! The variability in…

> "person who likes making things chooses making things over Netflix"

This is subtly different. It's not clear that the people depicted like making things, in the sense of enjoying the process. The narrative is about LLMs fitting into the already-existing startup culture. There's already a blurry boundary between "risky investment" and "gambling", given that most businesses (of all types, not just startups) have a high failure rate. The socially destructive characteristic identified here is: given more opportunity to pull the handle on the gambling machine, people are choosing to do that at the expense of other parts of their life.

But yes, this relies on a subjective distinction between "building, but with unpredictable results" and "gambling, with its associated self-delusions".

Post reply on HN