Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

techcrunch.com

Post reply on HN
Print view View original

121–130 of 308 posts

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#121

Post by electroly »

IMTDb wrote: post #106
shubhamjain wrote: post #11

If you think you need to spend $100B, does using a third-party cloud provider still make sense? It doesn’t matter what sweet deal Amazon is pitching—in that scenario, you’d want to own your stack. Especially in a hyper-competitive field like this, where margins are going to matter a lot soon. It feels like these hyperscalers are just raising as much as they can giving extremely rosy projections becauses these sooner…

The problem is that at that scale, the alternative is building your own data centers. You'd probably want at least 2 in the US, 2 in Europe, 2 in Asia, maybe 1 in Africa and 1 in LATAM. So 8-10, and you need at least half of them ready "on time." What does "on time" mean? You'll need to negotiate with local authorities, some friendly, some not. Data centers aren't exactly popular neighbors these days. Then negotiate…

For AI inference you don't need to geographically distribute your data centers. Latency, throughput, and routes don't matter here. When it's 10 seconds for the first token and then a 1KB/sec streamed response, whatever is fine. You can serve Australia from the US and it'll barely matter. You can find a spot far outside populated areas with cheap power, available water, and friendly leadership, then put all of your data centers there. If you're worried about major disasters, you can pick a second city. You definitely don't need a data center in every continent.

You're not wrong about the rest but no AI company would ever build a data center in every continent for this, even if they were prepared to build data centers. AI inference isn't like general purpose hosting.

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#122

Post by sophacles »

amluto wrote: post #112
IMTDb wrote: post #106

Earlier quoted context omitted.

The problem is that at that scale, the alternative is building your own data centers. You'd probably want at least 2 in the US, 2 in Europe, 2 in Asia, maybe 1 in Africa and 1 in LATAM. So 8-10, and you need at least half of them ready "on time." What does "on time" mean? You'll need to negotiate with local authorities, some friendly, some not. Data centers aren't exactly popular neighbors these days. Then negotiate…

Other than data sovereignty, does the data center location really matter that much? Current inference systems are not exactly low latency.

* not every task is waiting on the inference. lowering latency on other, serial tasks, can still have a noticable effect. Login, mcp queries, etc.

* data transit across the world can be very slow when there's network issues (a fiber is cut somewhere, congestion, bgp does it's thing, etc). having something more local can mitigate this

* several countries right now have demented leaders with idiotic cult-like followers. Best not to put all your eggs in those baskets.

* wars, earthquakes, fires, floods, and severe weather rarely affect the whole planet at once, but can have rippling effects across a continent.

And frankly, the real question isn't "why spread out the DCs?", its "what reason is there to put them close to each other?".

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#123

Post by 1738384848 »

jubilanti wrote: post #62
mitchell_h wrote: post #33

Earlier quoted context omitted.

I watched some explain how deepseak got good and the Chinese approach to LLM training. Really wish I could remember it. The premise was China thinks of LLMs not as a thing separate from hardware, but gains efficiencies at each layer of the stack. From Chips to software, it's all integrated and purpose built for training. Wonder if Anthropic is making a mistake by focusing on "consumer" hardware, and not going super s…

So you watched some random video from some random YouTuber, didn't even remember who made it, so much so you didn't even remember that deepseek isn't spelled "deapseak", didn't bother to even find it or verify, and then you go asserting your memory as fact on a serious discussion forum. Comments like yours add nothing to the discussion.

thank you for the aerious discussion my good sir I tip my hat to you

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#124

Post by TSiege »

electroly wrote: post #121
IMTDb wrote: post #106

Earlier quoted context omitted.

The problem is that at that scale, the alternative is building your own data centers. You'd probably want at least 2 in the US, 2 in Europe, 2 in Asia, maybe 1 in Africa and 1 in LATAM. So 8-10, and you need at least half of them ready "on time." What does "on time" mean? You'll need to negotiate with local authorities, some friendly, some not. Data centers aren't exactly popular neighbors these days. Then negotiate…

For AI inference you don't need to geographically distribute your data centers. Latency, throughput, and routes don't matter here. When it's 10 seconds for the first token and then a 1KB/sec streamed response, whatever is fine. You can serve Australia from the US and it'll barely matter. You can find a spot far outside populated areas with cheap power, available water, and friendly leadership, then put all of your da…

latency absolutely matters? this is such a weird thing to say. for training sure, but customers absolutely want low latency

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#125

Post by zozbot234 »

JumpCrisscross wrote: post #91
bwfan123 wrote: post #78

Earlier quoted context omitted.

> consumer grade local models are getting good enough for local inference I am waiting for that. Perhaps a taalas kind of high-performance custom hw coding llm engine paired with an open-source coding-agent. Priced like a high-end graphics card which would be pay off over time. It will be a replay of the ibm-mainframe to PC transition of a previous era.

> I am waiting for that Same, and I think we're close. "The original 1984 128k Mac model was $2,495, and the 1985 512k Mac was $2,795" [1]. That's $8 to 9 thousand today. About the price of a 32-core, 80-GPU M3 Ultra Mac Studio with 256 GB RAM. [1] https://blog.codinghorror.com/a-lesson-in-apple-economics/ [2] https://www.bls.gov/data/inflation_calculator.htm

The maxed out 512GB RAM Mac Studio is no longer available from Apple and is now pushing $20 thousand in the secondary market. And we might not even see a new Mac Studio release from Apple before October.

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#126

Post by muyuu »

iot_devs wrote: post #7

Someone can explain to me what's the expectations for these AI labs? I mostly see their products as commodity at this point, with strong open source contenders. Eventually it will become hard to justify the premium on these models.

the prospect that any of those big players will be able to pay back 100s of billions with profit on top sounds fantastical to me

it will be interesting to see it unfold

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#127

Post by tren_hard »

Culonavirus wrote: post #32
shubhamjain wrote: post #11

If you think you need to spend $100B, does using a third-party cloud provider still make sense? It doesn’t matter what sweet deal Amazon is pitching—in that scenario, you’d want to own your stack. Especially in a hyper-competitive field like this, where margins are going to matter a lot soon. It feels like these hyperscalers are just raising as much as they can giving extremely rosy projections becauses these sooner…

Only Google and xAI build their own, no? I don't think it's that easy to vertically integrate massive datacenters into a software company. Both Google and xAI (Tesla, SpaceX) have a massive wealth of experience when it comes to building factories.

Facebook and Oracle also build their own, at least before the last couple years where they’ve financed out to new bag holders.

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#128

Post by kitsune1 »

hmmmmmmmmmmmmmm wrote: post #12
iot_devs wrote: post #7

Someone can explain to me what's the expectations for these AI labs? I mostly see their products as commodity at this point, with strong open source contenders. Eventually it will become hard to justify the premium on these models.

None of them have any moat, OpenAI already lost the lead [1] and no one is "winning". It is just a race to the bottom as they burn through GPUs that won't even last that long. [1] https://x.com/kenshii_ai/status/2046111873909891151/photo/2

[dead]

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#129

Post by electroly »

TSiege wrote: post #124
electroly wrote: post #121

Earlier quoted context omitted.

For AI inference you don't need to geographically distribute your data centers. Latency, throughput, and routes don't matter here. When it's 10 seconds for the first token and then a 1KB/sec streamed response, whatever is fine. You can serve Australia from the US and it'll barely matter. You can find a spot far outside populated areas with cheap power, available water, and friendly leadership, then put all of your da…

latency absolutely matters? this is such a weird thing to say. for training sure, but customers absolutely want low latency

They want it, sure. Customers want everything if it's free, but this is about what they value with their money. In this thought experiment, you're Anthropic, not the customer. You're making a choice that's best for Anthropic. Will Anthropic lose customers because the latency is higher? No way. Customers want low cost and lots of usage more than they want low latency. In a cutthroat race to the bottom, there's no room to "give away" massively expensive freebies like a data center near every population center when the customer doesn't value those extras with actual money. It's the same reason we all tolerate the relatively slow batched token generation rate--the batching dramatically lowers the cost, and we need low cost inference more than we want fast generation. If the cost goes up we'll actually leave, for real.

After the initial announcement of "fast mode" in Claude Code, did you ever hear about anyone using it for real? I didn't. Vanishingly few people are willing to pay extra for faster inference.

Remember that the time-to-first-token is dominated by the time to process the prompt. It's orders of magnitude more latency than the network route is adding. An extra 200 milliseconds of network delay on a 5-10 second time-to-first-token is not even noticeable; it's within the normal TTFT jitter. It would be foolish to spend billions of dollars to drop data centers around the world to reduce the 200 milliseconds when it's not going to reduce the 5-10 seconds. Skip the exotic locales and put your data centers in Cheap Power Tax Haven County, USA. Perhaps run the numbers and see if Free Cooling City, Sweden is cheaper.

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#130

Post by CamperBob2 »

johnbarron wrote: post #51
iot_devs wrote: post #7

Someone can explain to me what's the expectations for these AI labs? I mostly see their products as commodity at this point, with strong open source contenders. Eventually it will become hard to justify the premium on these models.

Please, some of us are long NVIDIA...let us cope in peace. :-) Here is the thing nobody wants to say out loud or they are too dumb to realize. AI is intelligence, and intelligence has almost never been the binding constraint on productivity. So you will get no productivity increase from the AI bubble. Yes, you read that correctly. The test is simple, if raw brainpower were the bottleneck, you could 10x any company by…

Here is the thing nobody wants to say out loud or they are too dumb to realize. AI is intelligence, and intelligence has almost never been the binding constraint on productivity.

Exactly. We don't use the intelligence we already have! That seems to be the real problem with the "AGI" concept. Given such a capability, we'll just nerf it, gatekeep it, and/or bias it. There's no reason to think we'll actually use it to benefit humanity as a whole. It will be shaped into an instrument to enforce our prejudices.

Post reply on HN