Meta Spends $14B to Hire a Single Guy

The Llama 4 benchmarks published by Meta were proven to be fake immediately after it launched.

I'd push back on a couple things here.

The notion that Scale AI's data is of secondary value to Wang seems wrong: data-labeling in the era of agentic RL is more sophisticated than the pejorative view of outsourcing mechanical turk work at slave wages to third world workers, it's about expert demonstrations and work flows, the shape of which are highly useful for deducing the sorts of RL environments frontier labs are using for post-training. This is likely the primary motivator.

> LLMs are pretty easy to make, lots of people know how to do it — you learn how in any CS program worth a damn.

This also doesn't cohere with my understanding. There's only a few hundred people in the world that can train competitive models at scale, and the process is laden with all sorts of technical tricks and trade secrets. It's what made the deepseek reports and results so surprising. I don't think the toy neural network one gets assigned to create in an undergrad course is a helpful comparison.

Relatedly, the idea that progress in ML is largely stochastic and so horizontal orgs are the only sensible structure seems like a weird conclusion to draw from the record. Saying Schmidhuber is a one hit wonder, or "The LLM paper was written basically entirely by folks for whom "Attention is All You Need" is their singular claim to fame" neglects a long history of foundational contributions in the case of the former, and misses the prolific contributions of Shazeer in the latter. Alec Radford is another notable omission as a consistent superstar researcher. To the point about organizational structure, OpenAI famously made concentrated bets contra the decentralized experimentation of Google and kicked off this whole race. Deepmind is significantly more hierarchical than Brain was and from comments by Pichai, that seemed like part of the motivation for the merger.

Meta has a culture issue.

There are three centres of "AI" gravity: GenAI, FAIR & RL-R

Fair is fucked, they've been passed about, from standalone, to RL-R then to "production" under industrial dipshit Cox. A lot of people have left, or been kicked out. It was a power house, and PSC (the 6month performance charade killed it)

GenAI was originally a nice tight and productive team. Then the facebook disease of doubling the team every 2 months took over. Instead of making good products and dealing with infra scaling issues, 80% of the staff are trying to figure out what they are supposed to be doing. Moreover most of the leadership have no fucking clue how to do applied ML. Also they don't know what the product will be. So the answer is A/B testing what ever coke dream Cox dreamt up that week.

RL-R has the future, but they are tied to either Avatars, which is going to bomb. It'll bomb because its run by an prick who wants perfect rather than deliverable. Moreover splats perform way better than the dumbarse fully ML end-to-end system they spend the last 15 billion trying to make.

Then there is the hand interaction org, which has burnt through not quite as much cash as Avatars, but relies on a wrist device that has to be so tight it feels like a fucking handcuff. That and they've not managed to deliver any working prototype at scale.

The display team promised too much and wildly underdelivered, meaning that orion wasn't possible as a consumer product. Which lets the write team off the hook for not having a comfortable system.

Then there is the mapping team who make research glasses that hovers up any and all personal information with wild abandon.

RL-R had lots of talent. But the "hire to fire" system means that you can't actually do any risky research, unless you have the personal favour of your VP. Plus, even if you do perfect research. getting it to product is a nightmare.

Why should they be in that market at all? I don't understand why we've got to this place where every big tech company needs to have it's fingers in every pie or "they're falling behind". AI / LLMs look to be on their way to being infrastructure more than product in and of themselves. A patient huge company could build on top of what others are doing and wait for the market to settle, then buy for vertical integration.

It takes special kind of loser to pay to wear a video and microphone live feed to Facebook.