Live data from Hacker News

Meta caps internal AI token spending

mlq.ai

31–40 of 162 posts

Re: Meta caps internal AI token spending

#31
post #25
post #21

Earlier quoted context omitted.

measuring uptime? I've seen Anthropic's status page, and they are a >$1 Trillion dollar company who "largely solved" coding. so clearly you aren't correct. /s

[flagged]

my friend, I was being sarcastic before, and I am agreeing with you. LoC, token spend, etc as metrics are horrible measures. Software uptime is a great metric. I'm merely lamenting that in the age we're in, uptimes are getting worse and worse

Re: Meta caps internal AI token spending

#32
post #23
post #15

Earlier quoted context omitted.

One thing I find fascinating as a software engineer who talks to non software engineers who use AI tools is how "reading PDFs" is not more of a solved problem. What I mean is that uploading a PDF into a chatbot tool seems to be an extraordinarily obvious use case that non technical (and technical) users would want to do. IMO claude, chatgpt/codex, etc should be able to optimize the PDF use case to be extremely token…

The best way to parse pdfs is to convert them to images and feed them into the llm. This workflow is highly optimized.

Absolutely this. Never try to parse a native PDF document with any expectation of coherence or consistency.

Re: Meta caps internal AI token spending

#34

Clearly no one is using Meta’s customer facing AI products. Why aren’t they using their own gpu/compute for development?

that is a fair point. The contrast between Meta and Apple could not be bigger here. Apple has billions of devices and yet they decided to use 3rd party models from OpenAI and later Google to build their AI features rather than building foundational models in house. Yet Meta did opposite: they built models (spending billions of $$$ and firing 10% of the company) for billions of users who rather would not use Meta AI features.

Re: Meta caps internal AI token spending

#35
post #3

"The leaderboard, which ranked employees and teams by token consumption, inadvertently incentivized usage volume over productive output." Who could possibly have predicted that happening?

A past employer thought it was a good idea to put up a leaderboard of who sent the most Slack messages. They celebrated the people at the top for being so active. Predictably, everyone started talking in Slack like their jobs depended on it. Everyone was responding to everything. Instead of writing out a complete message and pressing enter, they'd send each fragment of the sentence as a new line. The Slack leaderboar…

You have to realize that if you set a measure, you're actually setting a goal for your employees. There is no such thing as a meaningless metric; why else would you measure it?

No amount of "this isn't used for anything" will change that. It's inherent in human nature in the 21st century to believe any and all metrics will be used against them, and therefore must be gamed.

It's why you also have to set UNBELIEVABLY clear goals and have incentives tied to those goals. Incentives meaning money. If you want to measure things, measure them. But have clear, consistent, and meaningful goals tied to bonuses or something if you want a thing done correctly.

Re: Meta caps internal AI token spending

#36

Earlier quoted context omitted.

I know right? What did the leadership think would happen when they give some of the worlds greatest software engineers (supportably), a easily quantifiable metric to target?

The leaderboard wasn't leadership generated, it was engineer generated from internally available data. The leadership target is "impact" from ai tools.

Budget impact is technically impact.

Re: Meta caps internal AI token spending

#37
post #15

Earlier quoted context omitted.

One thing I find fascinating as a software engineer who talks to non software engineers who use AI tools is how "reading PDFs" is not more of a solved problem. What I mean is that uploading a PDF into a chatbot tool seems to be an extraordinarily obvious use case that non technical (and technical) users would want to do. IMO claude, chatgpt/codex, etc should be able to optimize the PDF use case to be extremely token…

Because PDFs are a nightmare of a format and the only thing that’s is reasonably guaranteed about them is they will render to an image that people can read, the parsing of which will be much less token efficient than the equivalent text

I agree with you, but every non-engineer I know using these tools 100% will drag and drop a PDF into a chatbot. Anthropic and OpenAI as companies who are selling their products to all sorts of businesses should have a much better means of handling this nightmare of a format because it is so pervasive and so obviously what so many of their customers are going to drop into the product.

Re: Meta caps internal AI token spending

#38
post #35

Earlier quoted context omitted.

A past employer thought it was a good idea to put up a leaderboard of who sent the most Slack messages. They celebrated the people at the top for being so active. Predictably, everyone started talking in Slack like their jobs depended on it. Everyone was responding to everything. Instead of writing out a complete message and pressing enter, they'd send each fragment of the sentence as a new line. The Slack leaderboar…

You have to realize that if you set a measure, you're actually setting a goal for your employees. There is no such thing as a meaningless metric; why else would you measure it? No amount of "this isn't used for anything" will change that. It's inherent in human nature in the 21st century to believe any and all metrics will be used against them, and therefore must be gamed. It's why you also have to set UNBELIEVABLY c…

Kinda.

The answer is simpler on the surface: focus.

Generally the problem is the larger the firm’s operations, the harder it is to focus.

Apple is the only firm that has done well on this consistently and doesn’t have a huge grave yard of failures to show for it.

Re: Meta caps internal AI token spending

#39
Within a few weeks of telling people at our company that if they don’t use AI they will be replaced by someone who does, they just announced that their allocation with ChatGPT has reset and are now panicking as they blew through their million token allocation for this month in under six hours - you can’t make this shit up.

Re: Meta caps internal AI token spending

#40
post #15

Earlier quoted context omitted.

One thing I find fascinating as a software engineer who talks to non software engineers who use AI tools is how "reading PDFs" is not more of a solved problem. What I mean is that uploading a PDF into a chatbot tool seems to be an extraordinarily obvious use case that non technical (and technical) users would want to do. IMO claude, chatgpt/codex, etc should be able to optimize the PDF use case to be extremely token…

I hope someday we can get out of this local maxima of PDF documents. The format is terrible, but was right place, right time and might be impossible to dislodge.

The problem is that for 99% of people in 99% of cases they work fine. It's hard for people to understand that they're trash.

Source; my last job working with accessibility and that nightmare.

Post reply on HN