201–210 of 661 posts
I expect the next breakthroughs to be all about efficiency. Granted, that could be tomorrow, or in 5 years, and the AI companies have to stay all at in the meantime.
Earlier quoted context omitted.
This is the fastest-growing company by revenue, jumping from $1B to $3B in just five months. Hitting $10B is only a matter of time, which would put its valuation at a reasonable ~18x sales multiple. It doesn't even matter where we are in the AI hype cycle - AI adoption will keep increasing, it's not even a question at this point. From a technical perspective, they manage to attract top talent - Google / OpenAI lose a…
My baby grew from 9 pounds to 18 pounds in a 3 months! Hitting 10000 lbs is only a matter of time.
Impressive round but it seems unlikely this game can go on much longer before something implodes. Given the amount of cash you need to set of fire to stay relevant it’s becoming nearly impossible for all but a few players to stay competitive, but those players have yet to demonstrate a viable business model. With all these models converging, the big players aren’t demonstrating a real technical innovation moat. Every…
They can afford to burn a good chunk of global wealth so that they can have even more global wealth.
Even at the current rates of insanity, the wealthy have spent a tiny fraction of their wealth on AI.
Bezos could put up this $13 billion himself and remain a top five richest man in the world.
(Remember Elon cost himself $40 billion because of a tweet and still was fine!)
This is a technology that could replace a sizable fraction of humamkind as a labor input.
I'm sure the rich can dig much deeper than this.
Every round Anthropic raises twists the knife deeper in SBF. If only he could have survived the downturn his Antropic investment alone probably could have papered over the other loses.
So the difference between criminal fraud, and precient genius investor is a difference of a year or so. We should all try to remember this the next time we vote to cut taxes on billionaires.
The compute moat is getting absolutely insane. We're basically at the point where you need a small country's GDP just to stay in the game for one more generation of models. What gets me is that this isn't even a software moat anymore - it's literally just whoever can get their hands on enough GPUs and power infrastructure. TSMC and the power companies are the real kingmakers here. You can have all the talent in the w…
>You can have all the talent in the world but if you can't get 100k H100s and a dedicated power plant, you're out. I really have to wonder, how long will it be before the competition moves into who has the most wafer-scale engines. I mean, surely the GPU is a more inefficient packaging form factor than large dies with on-board HBM, with a massive single block cooler?
Is it worth it if I have an AI related idea to try and get it built ? It'll take a solid year and about 30k. Any chance of even talking to a VC as an outsider?
The compute moat is getting absolutely insane. We're basically at the point where you need a small country's GDP just to stay in the game for one more generation of models. What gets me is that this isn't even a software moat anymore - it's literally just whoever can get their hands on enough GPUs and power infrastructure. TSMC and the power companies are the real kingmakers here. You can have all the talent in the w…
It's not clear to me that each new generation of models is going to be "that" much better vs cost. Anecdotally moving from model to model I'm not seeing huge changes in many use cases. I can just pick an older model and often I can't tell the difference... Video seems to be moving forward fast from what I can tell, but it sounds like the back end cost of compute there is skyrocketing with it raising other questions.
Probably because you're doing things that are hitting mostly the "well-established" behaviors of these models — the ones that have been stable for at least a full model-generation now, that the AI bigcorps are currently happy keeping stable (since they achieved 100% on some previous benchmark for those behaviors, and changing them now would be a regression per those benchmarks.)
Meanwhile, the AI bigcorps are focusing on extending these models' capabilities at the edge/frontier, to get them to do things they can't currently do. (Mostly this is inside-baseball stuff to "make the model better as a tool for enhancing the model": ever-better domain-specific analysis capabilities, to "logic out" whether training data belongs in the training corpus for some fine-tune; and domain-specific synthesis capabilities, to procedurally generate unbounded amounts of useful fine-tuning corpus for specific tasks, ala AlphaZero playing unbounded amounts of Go games against itself to learn on.)
This means that the models are getting constantly bigger. And this is unsustainable. So, obviously, the goal here is to go through this as a transitionary bootstrap phase, to reach some goal that allows the size of the models to be reduced.
IMHO these models will mostly stay stable-looking for their established consumer-facing use-cases, while slowly expanding TAM "in the background" into new domain-specific use-cases (e.g. constructing novel math proofs in iterative cooperation with a prover) — until eventually, the sum of those added domain-specific capabilities will turn out to have all along doubled as a toolkit these companies were slowly building to "use models to analyze models" — allowing the AI bigcorps to apply models to the task of optimizing models down to something that run with positive-margin OpEx on whatever hardware that would be available at that time 5+ years down the line.
And then we'll see them turn to genuinely improving the model behavior for consumer use-cases again; because only at that point will they genuinely be making money by scaling consumer usage — rather than treating consumer usage purely as a marketing loss-leader paid for by the professional usage + ongoing capital investment that that consumer usage inspires.
Earlier quoted context omitted.
The wildest part is that the frontier models have a lifespan of 6 months or so. I don't see how it's sustainable to keep throwing this kind of money at training new models that will be obsolete in the blink of an eye. Unless you believe that AGI is truly just a few model generations away and once achieved it's game over for everyone but the winner. I don't.
It is being played like a winner-takes-it-all right now (it may or may not be such a market). So it is a game of being the one that is left standing, once the others fall off. In this kind of game, speeding more is done as a strategy to increase the chances of other competitors running out of cash or otherwise hitting a wall. Sustainability is the opposite of the goal being pursued... Whether one reaches "AGI" is not…