Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

businessinsider.com

Post reply on HN
Print view View original

41–50 of 360 posts

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#41

Post by saghm »

chihuahua wrote: post #12

It's amazing that it took months to figure this out. "Well we thought that if engineers are told to maximize costs through AI use, to consume as much as possible of a resource that costs us money, then obviously good things will happen. Imagine my surprise when it didn't turn out that way." Imagine if engineers were ranked based on their AWS spend. People allocate VMs and fill databases with terabytes of random bits,…

Someday maybe Goodhart's Law will be intuitive to people making decisions like this, but not any time soon I guess

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#42

Post by adrianN »

this_user wrote: post #31
chihuahua wrote: post #12

It's amazing that it took months to figure this out. "Well we thought that if engineers are told to maximize costs through AI use, to consume as much as possible of a resource that costs us money, then obviously good things will happen. Imagine my surprise when it didn't turn out that way." Imagine if engineers were ranked based on their AWS spend. People allocate VMs and fill databases with terabytes of random bits,…

The point of this was always to explore what is possible with AI as quickly as possible. Obviously, there is going to be a lot of waste, but the 5-10% of employees who are truly thinking about it and discovering novel applications are what you are truly after. Because right now, you effectively have a giant, as of yet poorly explored space of potential uses. Anyone who can find the actually valuable portions of the s…

What’s the huge advantage though? Adopting workflows that give big productivity gains is relatively easy even for big corporations. It’s only an advantage if you can keep it secret.

OTOH maybe we’re in for a future of patenting prompts.

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#43

Post by fsloth »

solenoid0937 wrote: post #25
chihuahua wrote: post #12

It's amazing that it took months to figure this out. "Well we thought that if engineers are told to maximize costs through AI use, to consume as much as possible of a resource that costs us money, then obviously good things will happen. Imagine my surprise when it didn't turn out that way." Imagine if engineers were ranked based on their AWS spend. People allocate VMs and fill databases with terabytes of random bits,…

You say "amazing that it took months to figure this out" as if the answer to the question is obvious. But it's not. Some FAANGs are doing amazing things with unlimited tokens. Other companies have no clue what to do with tokens, they've just told their engineers to max them. It really depends on how you're using the tokens . If you're just using them for Codex and Claude Code - yeah, tokenmaxxing is incredibly dumb.

> Some FAANGs are doing amazing things with unlimited tokens.

Would love to know what things!

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#44

Post by dghlsakjg »

gigatexal wrote: post #2

I find it useful that if they cut the use altogether I will pay for it out of pocket.

Would you decide its usefulness based on how high the bill is, or how many things you get done while using it?

The former is the issue, and how many companies have been operating. It's like a trucking company ranking driver effectiveness by fuel used instead of by cargo moved.

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#45

Post by BadBadJellyBean »

7777777phil wrote: post #9

As soon as tokens stop stop being subsidized, heavy agentic use will become as least as expensive than paying an (entry level) employee. When this happens many companies will trade off havy tolen usage for (maybe a bit slower, bit less accurate) employees again.

I have been saying the same for while. Someone always says "but Anthropic is making money on their API" or "But it's inference will get cheaper". But I don't believe it. first all the investments have to payed off at some point and second of all there are other things that cost money. I don't believe that any of them have a positive balance sheet.

I also don't think that blitz scaling will work like with Uber. The engineers are still there. We can work without the LLM tools.

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#47

Post by fsloth »

Rohunyyy wrote: post #27

Now we are going to get a new profession. Token Engineer! They will be experts on tokenmaxxing! The job growth that the billionaire CEOs promised us from AI is finally here!

Well there are already offerings like githits (https://news.ycombinator.com/item?id=46105112) that sort of promise optimize bang-per-buck of inference

Re: Uber’s COO says it’s getting harder to justify money spent on tokenmaxxing

#50

Post by tekno45 »

izanton wrote: post #29

What if... we stop for a moment, and then, after thinking for a moment, we stop hammering nails with a microscope, and stop using token usage as a metric of productivity? I know it's sounds stupid, but what if

Not very Billion Dollar Valuation of you.
Post reply on HN