Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta caps internal AI token spending

mlq.ai

Post reply on HN
Print view View original

51–60 of 162 posts

Re: Meta caps internal AI token spending

#51

Post by tyre »

mattnewton wrote: post #27
wpasc wrote: post #15

Earlier quoted context omitted.

One thing I find fascinating as a software engineer who talks to non software engineers who use AI tools is how "reading PDFs" is not more of a solved problem. What I mean is that uploading a PDF into a chatbot tool seems to be an extraordinarily obvious use case that non technical (and technical) users would want to do. IMO claude, chatgpt/codex, etc should be able to optimize the PDF use case to be extremely token…

Because PDFs are a nightmare of a format and the only thing that’s is reasonably guaranteed about them is they will render to an image that people can read, the parsing of which will be much less token efficient than the equivalent text

For anyone needing to do this, the answer is to convert it to an image first. Far smaller, LLMs work well with them (even in some pretty insane use cases I've seen), and, along with human review, it can be a huge productivity gain that results in structured data.

Re: Meta caps internal AI token spending

#52

Post by TheOtherHobbes »

SpicyLemonZest wrote: post #29
0cf8612b2e1e wrote: post #16

Earlier quoted context omitted.

Now come on, there was a recent post where the author argued that infallible management knew this would happen, but was part of the double-secret-probation strategy to get the cogs to finally start using AI.

I still think this is true and it’s not obvious to me from the source article that Meta believes otherwise. I couldn’t find the full memo, do they claim the leaderboard or “tokenmaxxing” era was a mistake?

Would they admit it if it was? Or would they try to find a plausible rationalisation for wasting billions without any return?

Re: Meta caps internal AI token spending

#54

Post by lokar »

Aurornis wrote: post #24
simonw wrote: post #3

"The leaderboard, which ranked employees and teams by token consumption, inadvertently incentivized usage volume over productive output." Who could possibly have predicted that happening?

A past employer thought it was a good idea to put up a leaderboard of who sent the most Slack messages. They celebrated the people at the top for being so active. Predictably, everyone started talking in Slack like their jobs depended on it. Everyone was responding to everything. Instead of writing out a complete message and pressing enter, they'd send each fragment of the sentence as a new line. The Slack leaderboar…

I worked somewhere that made time from PR being sent for review and ready to merge be a metric for the reviewers. Not time to add feedback in each round. Total time elapsed.

Insanity

Re: Meta caps internal AI token spending

#55

Post by veber-alex »

andsoitis wrote: post #12

measure outcomes (impact), not effort (token usage, lines of code, code coverage, hours worked, etc.)

It's not flashy.

When shit just works for months or years no one is going to come and praise you for stuff you did a while back.

You are better off breaking stuff and then fixing them to show how useful you are.

Re: Meta caps internal AI token spending

#58

Post by morpheos137 »

Aurornis wrote: post #24
simonw wrote: post #3

"The leaderboard, which ranked employees and teams by token consumption, inadvertently incentivized usage volume over productive output." Who could possibly have predicted that happening?

A past employer thought it was a good idea to put up a leaderboard of who sent the most Slack messages. They celebrated the people at the top for being so active. Predictably, everyone started talking in Slack like their jobs depended on it. Everyone was responding to everything. Instead of writing out a complete message and pressing enter, they'd send each fragment of the sentence as a new line. The Slack leaderboar…

What is about silicon valley leaders not understanding basic economics or business management? These kind of cargo cult tactics would not fly in any other industry.

Re: Meta caps internal AI token spending

#59

Post by goldenarm »

jghn wrote: post #10
simonw wrote: post #3

"The leaderboard, which ranked employees and teams by token consumption, inadvertently incentivized usage volume over productive output." Who could possibly have predicted that happening?

> Who could possibly have predicted that happening? Charles Goodhart :-)

https://en.wikipedia.org/wiki/Goodhart%27s_law

Re: Meta caps internal AI token spending

#60

Post by Avicebron »

darth_avocado wrote: post #47
simonw wrote: post #3

"The leaderboard, which ranked employees and teams by token consumption, inadvertently incentivized usage volume over productive output." Who could possibly have predicted that happening?

> Who could possibly have predicted that happening? Everyone except the executives who get paid millions to predict exactly that.

Not a problem. There are thousands of employees standing by, willing to sacrifice their jobs for their vision.

It's a hard job, someone has to not pay consequences for bad decisions.

Post reply on HN