Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Meta caps internal AI token spending

mlq.ai

Post reply on HN
Print view View original

81–90 of 162 posts

Re: Meta caps internal AI token spending

#81

Post by tiahura »

wpasc wrote: post #37
mattnewton wrote: post #27

Earlier quoted context omitted.

Because PDFs are a nightmare of a format and the only thing that’s is reasonably guaranteed about them is they will render to an image that people can read, the parsing of which will be much less token efficient than the equivalent text

I agree with you, but every non-engineer I know using these tools 100% will drag and drop a PDF into a chatbot. Anthropic and OpenAI as companies who are selling their products to all sorts of businesses should have a much better means of handling this nightmare of a format because it is so pervasive and so obviously what so many of their customers are going to drop into the product.

I think they’ve just decided that vision gives the best results and the token issue will take care of itself.

Re: Meta caps internal AI token spending

#83

Post by wonderwonder »

I have never worked there and I am likely very unqualified to ever work there and Zuck has more money than I could dream of so take my comment with that in mind.

Meta sounds like a cluster-F of a place to work. Massive reorgs around wild ideas like the metaverse and everything Ai all the time. Employees terrified of being fired. Incentivizing token spending and then cutting it off. While the overall company may be fine, the dev department sounds rudderless and absolutely miserable.

Re: Meta caps internal AI token spending

#84

Post by evdubs »

wpasc wrote: post #15
dwoosley wrote: post #8

I’d be curious to see the breakdown on spending by use case. I’ve heard it said that the majority of tokenmaxing comes from none technical uses like reading PDFs, creating PowerPoints, generating graphics/images… ect. But I’ve never heard any actual proof to that.

One thing I find fascinating as a software engineer who talks to non software engineers who use AI tools is how "reading PDFs" is not more of a solved problem. What I mean is that uploading a PDF into a chatbot tool seems to be an extraordinarily obvious use case that non technical (and technical) users would want to do. IMO claude, chatgpt/codex, etc should be able to optimize the PDF use case to be extremely token…

You don't need to use an online service to do this; you get to avoid spending money on tokens doing it offline.

Gemma 4 works perfectly well offline on limited hardware (I have an 8GB video card) and can handle extracting text from image-based PDFs just fine.

Take a PDF -> run it through MarkItDown [1], using the OCR plugin if you need (point it to Gemma 4) -> now you can ask Gemma 4 questions about the (markdown) document.

I am sure Gemma 4 could even create a GUI to make this process very simple for a non technical user.

[1] https://github.com/microsoft/markitdown

Re: Meta caps internal AI token spending

#85

Post by John23832 »

simonw wrote: post #3

"The leaderboard, which ranked employees and teams by token consumption, inadvertently incentivized usage volume over productive output." Who could possibly have predicted that happening?

Given that Meta has run 5ish layoffs at this point, and everyone is in survival mode, what did they expect? Everyone wants to juice whatever numbers possible to keep their jobs.

Re: Meta caps internal AI token spending

#87

Post by janalsncm »

Trasmatta wrote: post #28

All those billions spent on tokens by Meta, and not a single iota of value generated by any of it

I can’t tell if you’re complaining that Meta isn’t saving the whales or that their products aren’t good. If it’s the latter, you should double check their financials.

Re: Meta caps internal AI token spending

#88

Post by giancarlostoro »

simonw wrote: post #3

"The leaderboard, which ranked employees and teams by token consumption, inadvertently incentivized usage volume over productive output." Who could possibly have predicted that happening?

I still don't understand how Mark Zuckerberg has any serious investors, he went on this AI tangent and has absolutely nothing to show for it, despite FB / Meta having built some key tech in the space. He needs to stop trying to do something "different" and literally try and build a serious coding agent he can sell, he could have probably had something worthwhile in that space by now.

He started being drastically more serious into AI in 2022, and 2023 and he has nothing to show for it.

Heck, he could have rented GPUs the way Elon did at this point and either mended the bleeding or stopped it, not sure how many he has, but it beats losing this badly.

If he doesn't wake up and learn how to business, I suspect he will lose his empire he's built up for himself.

Re: Meta caps internal AI token spending

#89

Post by Freedom2 »

Loughla wrote: post #35
Aurornis wrote: post #24

Earlier quoted context omitted.

A past employer thought it was a good idea to put up a leaderboard of who sent the most Slack messages. They celebrated the people at the top for being so active. Predictably, everyone started talking in Slack like their jobs depended on it. Everyone was responding to everything. Instead of writing out a complete message and pressing enter, they'd send each fragment of the sentence as a new line. The Slack leaderboar…

You have to realize that if you set a measure, you're actually setting a goal for your employees. There is no such thing as a meaningless metric; why else would you measure it? No amount of "this isn't used for anything" will change that. It's inherent in human nature in the 21st century to believe any and all metrics will be used against them, and therefore must be gamed. It's why you also have to set UNBELIEVABLY c…

I wonder if this should be codified as a rule of thumb, or an unofficial "law"? One perhaps we can reference easily among our peers.

Re: Meta caps internal AI token spending

#90

Post by countcol »

tyre wrote: post #56
Trasmatta wrote: post #28

All those billions spent on tokens by Meta, and not a single iota of value generated by any of it

I love how confidently you say this, with no evidence provided (and I doubt you have any.) Just a pristine comment section yap.

The only thing I’ve seen Meta release recently are spy glasses, and every employee who has worked on that product should be in prison (with a live 24/7 feed where the world gets to watch them wallow).

The times I’ve been asked to evaluate a prospective candidate and I see that product on their résumé, it’s been an instant veto, in the same category as working at Palantir.

Post reply on HN