Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta caps internal AI token spending

mlq.ai

Post reply on HN
Print view View original

21–30 of 162 posts

Re: Meta caps internal AI token spending

#21

Post by wpasc »

4yfr wrote: post #18
andsoitis wrote: post #12

measure outcomes (impact), not effort (token usage, lines of code, code coverage, hours worked, etc.)

What outcomes though? The ones I’ve seen posted are still nonsensical metrics that a publicly traded firm absolutely doesn’t care about. It wants to see faster R&D, higher revenues from existing assets, greater operating margins, higher sales to invested capital ratio and so on… The best way to measure that for a software firm is up-time of services, usage and project completion duration

measuring uptime? I've seen Anthropic's status page, and they are a >$1 Trillion dollar company who "largely solved" coding. so clearly you aren't correct. /s

Re: Meta caps internal AI token spending

#22

Post by VygmraMGVl »

qwertytyyuu wrote: post #6
simonw wrote: post #3

"The leaderboard, which ranked employees and teams by token consumption, inadvertently incentivized usage volume over productive output." Who could possibly have predicted that happening?

I know right? What did the leadership think would happen when they give some of the worlds greatest software engineers (supportably), a easily quantifiable metric to target?

The leaderboard wasn't leadership generated, it was engineer generated from internally available data. The leadership target is "impact" from ai tools.

Re: Meta caps internal AI token spending

#23

Post by nojito »

wpasc wrote: post #15
dwoosley wrote: post #8

I’d be curious to see the breakdown on spending by use case. I’ve heard it said that the majority of tokenmaxing comes from none technical uses like reading PDFs, creating PowerPoints, generating graphics/images… ect. But I’ve never heard any actual proof to that.

One thing I find fascinating as a software engineer who talks to non software engineers who use AI tools is how "reading PDFs" is not more of a solved problem. What I mean is that uploading a PDF into a chatbot tool seems to be an extraordinarily obvious use case that non technical (and technical) users would want to do. IMO claude, chatgpt/codex, etc should be able to optimize the PDF use case to be extremely token…

The best way to parse pdfs is to convert them to images and feed them into the llm.

This workflow is highly optimized.

Re: Meta caps internal AI token spending

#24

Post by Aurornis »

simonw wrote: post #3

"The leaderboard, which ranked employees and teams by token consumption, inadvertently incentivized usage volume over productive output." Who could possibly have predicted that happening?

A past employer thought it was a good idea to put up a leaderboard of who sent the most Slack messages. They celebrated the people at the top for being so active.

Predictably, everyone started talking in Slack like their jobs depended on it. Everyone was responding to everything. Instead of writing out a complete message and pressing enter, they'd send each fragment of the sentence as a new line.

The Slack leaderboard was never shown again. Unfortunately the habit remained because people were afraid they were going to be secretly judged by how much Slack activity they generated.

I expect the same thing is going to happen at companies who had token leaderboards. Once you've instilled that fear in people, they internalize the expectation.

Re: Meta caps internal AI token spending

#25

Post by 4yfr »

wpasc wrote: post #21
4yfr wrote: post #18

Earlier quoted context omitted.

What outcomes though? The ones I’ve seen posted are still nonsensical metrics that a publicly traded firm absolutely doesn’t care about. It wants to see faster R&D, higher revenues from existing assets, greater operating margins, higher sales to invested capital ratio and so on… The best way to measure that for a software firm is up-time of services, usage and project completion duration

measuring uptime? I've seen Anthropic's status page, and they are a >$1 Trillion dollar company who "largely solved" coding. so clearly you aren't correct. /s

[flagged]

Re: Meta caps internal AI token spending

#26

Post by wpasc »

nojito wrote: post #23
wpasc wrote: post #15

Earlier quoted context omitted.

One thing I find fascinating as a software engineer who talks to non software engineers who use AI tools is how "reading PDFs" is not more of a solved problem. What I mean is that uploading a PDF into a chatbot tool seems to be an extraordinarily obvious use case that non technical (and technical) users would want to do. IMO claude, chatgpt/codex, etc should be able to optimize the PDF use case to be extremely token…

The best way to parse pdfs is to convert them to images and feed them into the llm. This workflow is highly optimized.

For sure there are very optimized ways to do it. My point is that a non technical user will drag and drop a pdf into a chatbot. and from a UX/product perspective, they should have to think about it more than that IMO. but seemingly, that's very much an expensive, inefficient way of doing it (burning through a whole context window try to read it, reloading it multiple times per conversation, etc.).

Re: Meta caps internal AI token spending

#27

Post by mattnewton »

wpasc wrote: post #15
dwoosley wrote: post #8

I’d be curious to see the breakdown on spending by use case. I’ve heard it said that the majority of tokenmaxing comes from none technical uses like reading PDFs, creating PowerPoints, generating graphics/images… ect. But I’ve never heard any actual proof to that.

One thing I find fascinating as a software engineer who talks to non software engineers who use AI tools is how "reading PDFs" is not more of a solved problem. What I mean is that uploading a PDF into a chatbot tool seems to be an extraordinarily obvious use case that non technical (and technical) users would want to do. IMO claude, chatgpt/codex, etc should be able to optimize the PDF use case to be extremely token…

Because PDFs are a nightmare of a format and the only thing that’s is reasonably guaranteed about them is they will render to an image that people can read, the parsing of which will be much less token efficient than the equivalent text

Re: Meta caps internal AI token spending

#29

Post by SpicyLemonZest »

0cf8612b2e1e wrote: post #16
simonw wrote: post #3

"The leaderboard, which ranked employees and teams by token consumption, inadvertently incentivized usage volume over productive output." Who could possibly have predicted that happening?

Now come on, there was a recent post where the author argued that infallible management knew this would happen, but was part of the double-secret-probation strategy to get the cogs to finally start using AI.

I still think this is true and it’s not obvious to me from the source article that Meta believes otherwise. I couldn’t find the full memo, do they claim the leaderboard or “tokenmaxxing” era was a mistake?

Re: Meta caps internal AI token spending

#30

Post by 4yfr »

downrightmike wrote: post #9
smrtinsert wrote: post #5

That is insane. I'm sure companies will learn the absolute wrong lesson from this, and attempt to centralize and kneecap token usage.

Tokens are less valuable than the eyeball metric of the Dotcom era. At least the eyeballs were real then. I'd argue most of the AI value is related to how 'Dead' the internet is.

This talk of tokens is wasteful.

Ultimately the spend on tokens has to benefit the firm financially or it won’t continue spending on it.

Post reply on HN