141–150 of 308 posts
If I am correct (and I hope that I am wrong!) this will drastically increase the cost of building these new data centers.
Earlier quoted context omitted.
That’s PR hype. They built it quickly, but they didn’t go from deciding they wanted a data center to having it running in weeks. You can’t even get the hardware at that scale without months or years of order lead time. NVidia doesn’t have warehouses full of compute hardware waiting for someone to come get it. They also reused an existing building. Basically, they put 100,000 GPUs into a building and attached the nece…
Why does this matter? The deal is supposed to last 10 years. If you don't pay AWS to order Nvidia GPUs for you, Nvidia won't have to deliver them to AWS, they will have exactly the same quantity of GPUs, but this time they can deliver to you.
If you build datacenters, you have to spend that money now.
They're also not paying amazon to order GPUs, they're paying for compute usage of whatever hardware they have.
If you think you need to spend $100B, does using a third-party cloud provider still make sense? It doesn’t matter what sweet deal Amazon is pitching—in that scenario, you’d want to own your stack. Especially in a hyper-competitive field like this, where margins are going to matter a lot soon. It feels like these hyperscalers are just raising as much as they can giving extremely rosy projections becauses these sooner…
I do think a ton of businesses would benefit from running their own hardware, but they're not getting five billion dollars to stay on the cloud.
Earlier quoted context omitted.
Here’s the answer to your queation (from the article) > The Anthropic deal specifically covers Trainium2 through Trainium4 chips, even though Trainium4 chips are not currently available. The latest chip, Trainium3, was released in December. On top of that, Anthropic has secured the option to buy capacity on future Amazon chips as they become available.
Ah. So it's a scalper situation where an unethetical entity buys up all the supply and then resells it for a greater price.
If you think you need to spend $100B, does using a third-party cloud provider still make sense? It doesn’t matter what sweet deal Amazon is pitching—in that scenario, you’d want to own your stack. Especially in a hyper-competitive field like this, where margins are going to matter a lot soon. It feels like these hyperscalers are just raising as much as they can giving extremely rosy projections becauses these sooner…
Earlier quoted context omitted.
The problem is that at that scale, the alternative is building your own data centers. You'd probably want at least 2 in the US, 2 in Europe, 2 in Asia, maybe 1 in Africa and 1 in LATAM. So 8-10, and you need at least half of them ready "on time." What does "on time" mean? You'll need to negotiate with local authorities, some friendly, some not. Data centers aren't exactly popular neighbors these days. Then negotiate…
Other than data sovereignty, does the data center location really matter that much? Current inference systems are not exactly low latency.
Earlier quoted context omitted.
For AI inference you don't need to geographically distribute your data centers. Latency, throughput, and routes don't matter here. When it's 10 seconds for the first token and then a 1KB/sec streamed response, whatever is fine. You can serve Australia from the US and it'll barely matter. You can find a spot far outside populated areas with cheap power, available water, and friendly leadership, then put all of your da…
latency absolutely matters? this is such a weird thing to say. for training sure, but customers absolutely want low latency
Earlier quoted context omitted.
What does "fully caught up" mean in the context of an ever evolving technology? I think I'm in support of open weight models (though there are safety implications), but these things aren't cheap to train and run. This fact alone gives no incentive for leading labs to release cutting edge open weight models. Why spend the money then give the product for free? Now if "fully caught up" means today's level of intelligenc…
Yeah I don't understand it, it's a marathon with three companies perpetually a minute ahead, and people keep saying "I expect the stragglers to catch up". The only thing I can see them meaning is what you said, "in a minute the stragglers will be where the leaders were a minute ago", which, yeah, sure.
Or AGI hits and this theory collapses, but that's feeling less likely every day.
Does anyone feel that the jig is almost up? Surely the returns aren’t anywhere close to what investors expect with the sheer amount of cash at this point in time. Are Anthropic and OpenAI rushing to IPO for immediate cash so they can delay the inevitable? Surely this cycle of robbing Peter to pay Paul to pay John to pay Tim must end. We are only just now getting a taste of the “true cost” of these tokens. Then there…
This reads to me like Anthropic anticipating demand and making a commitment to acquire supply. Not unlike airlines committing to future jet fuel purchases, or Apple committing to future DRAM volume.