Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

techcrunch.com

Post reply on HN
Print view View original

271–280 of 308 posts

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#271

Post by jcgrillo »

twoodfin wrote: post #256
jcgrillo wrote: post #225

Earlier quoted context omitted.

Yes, it's amazing how quickly so many tech companies have hitched their tooling to these big AI vendors seemingly without any thought towards whether they'll still exist a year or three or five from now. Insane behavior. To the (debatable!) extent that AI coding tools are useful at all wouldn't it be a hell of a lot smarter to self-host? At least that way you have some control over QoS, and a stable, predictable resu…

The basic business math is (to start) software companies realizing that spending $10k, $20k, $50k (more ?) per year, per developer for current models at current token rates might not be particularly insane, given the value return. Models are likely going to keep getting better, and as costs go down, demand is likely to rise faster .

> as costs go down

Huh? Why would that happen? Indications are that costs will likely go up, especially if currently vendors are selling tokens at a loss.

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#272

Post by kgeist »

electroly wrote: post #121
IMTDb wrote: post #106

Earlier quoted context omitted.

The problem is that at that scale, the alternative is building your own data centers. You'd probably want at least 2 in the US, 2 in Europe, 2 in Asia, maybe 1 in Africa and 1 in LATAM. So 8-10, and you need at least half of them ready "on time." What does "on time" mean? You'll need to negotiate with local authorities, some friendly, some not. Data centers aren't exactly popular neighbors these days. Then negotiate…

For AI inference you don't need to geographically distribute your data centers. Latency, throughput, and routes don't matter here. When it's 10 seconds for the first token and then a 1KB/sec streamed response, whatever is fine. You can serve Australia from the US and it'll barely matter. You can find a spot far outside populated areas with cheap power, available water, and friendly leadership, then put all of your da…

>Latency, throughput, and routes don't matter here. When it's 10 seconds for the first token and then a 1KB/sec streamed response, whatever is fine. You can serve Australia from the US and it'll barely matter.

This may be true for simpler cases where you just stream responses from a single LLM in some kind of no-brain chatbot. If the pipeline is a bit more complex (multiple calls to different models, not only LLMs but also embedding models, rerankers, agentic stuff, etc.), latencies quickly add up. It also depends on the UI/UX expectations.

Funny reading this, because the feature I developed can't go live for a few months in regions where we have to use Amazon Bedrock (for legal reasons), simply because Bedrock has very poor latency and stakeholders aren't satisfied with the final speed (users aren't expected to wait 10-15 seconds in that part of the UI, it would be awkward). And a single roundtrip to AWS Ireland from Asia is already like at least 300ms (multiply by several calls in a pipeline and it adds up to seconds, just for the roundtrips), so having one region only is not an option.

Funny though, in one region we ended up buying our own GPUs and running the models ourselves. Response times there are about 3x faster for the same models than on Bedrock on average (and Bedrock often hangs for 20+ seconds for no reason, despite all the tricks like cross-region inference and premium tiers AWS managers recommended). For me, it's been easier and less stressful to run LLMs/embedders/rerankers myself than to fight cloud providers' latencies :)

>then put all of your data centers there

>You definitely don't need a data center in every continent.

Not always possible due to legal reasons. Many jurisdictions already have (or plan to have) strict data processing laws. Also many B2B clients (and government clients too), require all data processing to stay in the country, or at least the region (like EU), or we simply lose the deals. So, for example, we're already required to use data centers in at least 4 continents, just 2 more continents to go (if you don't count Antarctica :)

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#273

Post by jcgrillo »

sassymuffinz wrote: post #269
czhu12 wrote: post #229

Earlier quoted context omitted.

I don’t necessarily disagree but to provide some counter points: 1. Model providers are currently profitable when just counting the cost to serve tokens for inference[1]. They lose money training the next generation of models. 2. Open models don’t work nearly as well. Given that tokens are still relatively cheap, and hallucinations are expensive, I’ve not seen a huge up tick in open model usage for coding agents yet.…

If I start a business making a really special beef sandwich where I have to buy a farm every year for $1mil dollars, and then sell the sandwiches for $5, I can't get away with saying that my sandwiches turn a profit if the raw margin on the bread, the lettuce and the technical value of the weight of the beef is $3. Sure my gross margin might be $2 on each sammie sold but I need to sell 500,000 sandwiches just to brea…

They're literally hoping to make it up on volume. The AGI thing is a boondoggle that I doubt any serious person actually believes or takes seriously. But let's say for the sake of the hypothetical that tomorrow Microsoft Tay or whatever they call it now wakes up and becomes superintelligent? So what? Would everyone's head simultaneously explode like the aliens in Mars Attacks? No. It wouldn't collapse the global economy, people still need to eat and work--a really smart silicon brain in a box can't raise livestock or pick lettuce. It's not even clear whether the superintelligent Tay would have any economic utility at all? The whole "AGI changes everything" narrative seems like total bullshit. It might be scientifically or philosophically interesting, maybe.. But I share your wonderment at why anyone would invest in this space, it's perplexing af.

EDIT: I spent most of the day today pulling an 8/3 cable through conduit and routing it through a crawlspace to run 240V service to my barn for a workshop. If Tay wakes up tomorrow and becomes AGI, how will that help me finish the wiring job? Now extrapolate to almost every single other thing humans do. Even if Tay can write all the world's computer programs forever, it barely means anything for the vast majority of people, and therefore the global economy.

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#274

Post by waterloser »

svnt wrote: post #222
mlinsey wrote: post #213

Earlier quoted context omitted.

You're observing that: a) effective price-per-token is rising b) there is insufficient compute to meet the demand. And your conclusion is that the industry is circling the drain and due to collapse?

They are different observations, I think, though the phrasing confuses it: a) cost per successful task is rising — eg claude max allocation is functionally shrinking b) is there enough potential cost reduction in the queue to make up the gap c) if open models converge on a more efficient but slightly-less capable point (which has effectively happened) what is the actual moat?

Nice em-dash there bro

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#275

Post by svnt »

waterloser wrote: post #274
svnt wrote: post #222

Earlier quoted context omitted.

They are different observations, I think, though the phrasing confuses it: a) cost per successful task is rising — eg claude max allocation is functionally shrinking b) is there enough potential cost reduction in the queue to make up the gap c) if open models converge on a more efficient but slightly-less capable point (which has effectively happened) what is the actual moat?

Nice em-dash there bro

Thanks I am the source. em-dashing since 1997

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#276

Post by [deleted] »

Argonaut998 wrote: post #131

Does anyone feel that the jig is almost up? Surely the returns aren’t anywhere close to what investors expect with the sheer amount of cash at this point in time. Are Anthropic and OpenAI rushing to IPO for immediate cash so they can delay the inevitable? Surely this cycle of robbing Peter to pay Paul to pay John to pay Tim must end. We are only just now getting a taste of the “true cost” of these tokens. Then there…

[deleted]

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#277

Post by QuiEgo »

kgwgk wrote: post #208
giancarlostoro wrote: post #182

Earlier quoted context omitted.

> Surely this cycle of robbing Peter to pay Paul to pay John to pay Tim must end. I think a LOT of companies really never needed to be on the public market, and its a darn shame that so many go on the stock market, we have this obnoxious culture where you have to fire tons of people if you have a bad quarter just to show you're stopping the bleeding. Companies literally fire and hire x number of people every quarter…

> I wish there were an alternative to the stock market where you invest for the long haul, and you cannot take your money out in x number of years. That exists already! People often complain as well when a company ends its golden years because of some economic hurdle and ends up being acquired by a bigger fish who is _not_ on the stock market.

Isn’t that just called a bond?

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#278

Post by stanfordkid »

Argonaut998 wrote: post #131

Does anyone feel that the jig is almost up? Surely the returns aren’t anywhere close to what investors expect with the sheer amount of cash at this point in time. Are Anthropic and OpenAI rushing to IPO for immediate cash so they can delay the inevitable? Surely this cycle of robbing Peter to pay Paul to pay John to pay Tim must end. We are only just now getting a taste of the “true cost” of these tokens. Then there…

They are bringing in $30B in revenue with 3X YoY growth. Why do you think it is a "jig"? I do think the US economy could implode, but thats because of war and wealth inequality in the midst of hyper-inflation. AI models aren't very useful when you have penniless consumers that can't buy the products they help build. All this is to say: the models are valuable, the companies building and providing them are very valuable.

The biggest risk to AI companies IMO is further optimization and distillation of the capabilities into smaller and more efficient models. The moat these companies have right now is that higher intelligence requires more specialized and expensive compute. If you can do that for cheap then it kind of negates their business model. Everything is moving fast, we also yet to see world models/embodied AI and how that impacts thing. I think we've reached the peak with regards to capabilities of pure text trained LLMs.

Re: Anthropic takes $5B from Amazon and pledges $100B in cloud spending in return

#279

Post by sassymuffinz »

jcgrillo wrote: post #273
sassymuffinz wrote: post #269

Earlier quoted context omitted.

If I start a business making a really special beef sandwich where I have to buy a farm every year for $1mil dollars, and then sell the sandwiches for $5, I can't get away with saying that my sandwiches turn a profit if the raw margin on the bread, the lettuce and the technical value of the weight of the beef is $3. Sure my gross margin might be $2 on each sammie sold but I need to sell 500,000 sandwiches just to brea…

They're literally hoping to make it up on volume. The AGI thing is a boondoggle that I doubt any serious person actually believes or takes seriously. But let's say for the sake of the hypothetical that tomorrow Microsoft Tay or whatever they call it now wakes up and becomes superintelligent? So what? Would everyone's head simultaneously explode like the aliens in Mars Attacks? No. It wouldn't collapse the global econ…

I think the economic issue as you say isn't on physical labour it's on the fact that most of the jobs in the west are now in some sort of service industry or office work. Here in the UK we basically rely on the service industry to employ a large amount of the workforce. If MechaHitler Grok becomes sentient then that's a lot of middle-class earners out of a job and the economy goes with it.

You're absolutely right that an AGI isn't running a cable or digging a hole any time soon, but you're going to have 100 people trying to get their hands on the shovel to get paid for the digging - depressing the wages in those hands on jobs.

Post reply on HN