Is this why Google decided to release their article explaining how AI spend makes sense to the plebes?
I missed that, link please?
sure, here you go https://news.ycombinator.com/item?id=49021006
Alphabet's cash burn raises alarm for Big Tech as AI spending climbs
191–200 of 306 posts
Re: Alphabet's cash burn raises alarm for Big Tech as AI spending climbs
#192The current commitment by hyperscalers is around 1.7T USD, reported liabilities 1.3T and this year global debt related to AI is 570B. So that’s around 3T total. For this to make sense AI must generate 2T in new revenue per year by the end of the decade. And that would be only a 10% ROIC. For context ROIC for big tech is around 35% so at 10% they will be barely breaking even. The SP500 gives 10-12%. With 10% ROIC from…
What's the risk of NOT doing this? That's the problem. That's the risk that few (if any) hyperscalers want to take.
Apple might be a good counter example of what happens if you don't focus entirely on AI. Right now it seems to be doing ok.
Re: Alphabet's cash burn raises alarm for Big Tech as AI spending climbs
#193Earlier quoted context omitted.
They hold value as there is insane demand. The same reason a consumer RTX4090 costs more today than bew in 2021. Once the tide drops enough for hardware lead times to shorten to weeks, they will go the way of other used DC hardware - written off after 5 years.
> Once the tide drops enough for hardware lead times to shorten to weeks Which will not be any time soon according to SK Hynix CEO: > We still forecast that customer demand will remain higher than our supply capacity even beyond 2030 https://www.reuters.com/world/asia-pacific/sk-hynix-ceo-sees...
For what its worth, SK Hynix CEO has every incentive to show that RAM prices will remain high for as long as possible because that is the only thing which is floating their evaluation to such astronomical amounts.
Independent estimates sort of show around 2027-28 from what I remember.
I remember reading some article which said that RAM prices are already going down from its peak slowly (IIRC I can be wrong, I usually am but 3-5% month from its absolute peak) but the current RAM prices are still astronomical given past rates but the RAM prices will slow down hopefully sooner rather than later.
Re: Alphabet's cash burn raises alarm for Big Tech as AI spending climbs
#194Earlier quoted context omitted.
That sounds reasonable, it's "just" $1k/yr for 2B workers (there are about 1.2B total "knowledge workers" in the world including gig drivers), or $10k/yr for 200M workers (there are 70M office and technical workers in the US). /s https://www.dpeaflcio.org/factsheets/the-professional-and-te... In 4 years it better be 10x more important to have than a cell phone is today, or 10x more important than having internet/moni…
The math looks good on paper, but in reality enterprise AI is hard, most companies are realizing they are actually not seeing ROI from AI. One of my customers took about 8 months to rollout an AI initiative that by the time it launched and people got trained on it, it was already legacy. Also if you are 10x more productive with AI that doesn’t necessarily increases your billable output. There could be some super mode…
Yes, I'm agreeing with you. There need to be 200 companies willing to pay $10B/yr for this. What is the ROI? That's the pay roll of ~half the work force of the largest 200 companies. Unless you can fire %50 your employees, everything else is a sunk cost you already own.
Re: Alphabet's cash burn raises alarm for Big Tech as AI spending climbs
#195Earlier quoted context omitted.
H100 is nearing five years and costs more to buy a used one now than a new one when it was released :) You are completely missing the bet these companies are making. They think can outlast their competitors and capture a larger portion of the pie while the cost of inference keeps going down dramatically. If you haven't been paying attention, the cost is about 1/100th of what it was in 2024. This is the trajectory pre…
You just stated yourself that it costs more now used than when they were new. If everyone's running local then why are these larger companies dumping cash into data centres?
Economies of scale.
You need a cluster of 8-12 H100s to run the largest models locally.
It doesn't make sense to run these locally yet unless your use case also involves making it available for several dozen concurrent users.
Re: Alphabet's cash burn raises alarm for Big Tech as AI spending climbs
#196Earlier quoted context omitted.
H100 is nearing five years and costs more to buy a used one now than a new one when it was released :) You are completely missing the bet these companies are making. They think can outlast their competitors and capture a larger portion of the pie while the cost of inference keeps going down dramatically. If you haven't been paying attention, the cost is about 1/100th of what it was in 2024. This is the trajectory pre…
>H100 is nearing five years and costs more to buy a used one now than a new one when it was released :) Because everyone is buying as they want to run their own models and not pay for a cloud service?
Because there's no supply, data centers with these GPUs are running reasonably well.
Re: Alphabet's cash burn raises alarm for Big Tech as AI spending climbs
#197Earlier quoted context omitted.
Diversify! Historically, the average length of a recession has been 12-24 months. So set up a system whereby you won’t screw’s yourself over by selling when things are low, but instead you can weather the storm. Build a rainy day fund. Determine how much cash you will need if you are out of a job and how long you think that will last, allocate some portion of that amount into low risk bonds. Russ way if you need cash…
But diversify into what? If we assume this takes down the US economy and bonds, what then? International bonds/stocks? Won't those also be too entangled? Precious metals?
Nobody can tell you what to diversify into without information about what you are concentrated in.
Sure, spread investments across stocks and bonds and treasuries from different markets.
But you can also diversify more broadly beyond economic capital to cultural and social capital. Learn new skills and build networks of generalized reciprocity with others before you (or they) need help.
Re: Alphabet's cash burn raises alarm for Big Tech as AI spending climbs
#198There was an excellent article about AI DC value and depreciation yesterday [1] (discussion [2]). The effective life of GPUs in paticular is a huge unknown. One of my big questions has always been "what will happen to existing GPUs when new GPUs come out?" My guess is that the life of these things isn't as long as the depreciation schedules for some of these companies would have you believe. IIRC Meta was using an 8 year schedule whereas Google is using 4-6, which seems more realistic.
I believe that performance-per-Watt is going to be the only metric that matters. We already have 6 year old hardware (A100) that cannot run the latest models. There will also be new capabilities (eg quantization methods).
I'm not concerned with Alphabet's cash burn rate to be honest. These tech companies are typically shielding themselves from the consequences of this by using Special Purpose Vehicles ("SPVs") where the GPUs themselves are the secured assets for the loans. Even the physical buildings and infrastructure isn't owned by the SPV. Those are rented from another vehicle. So investors are pouring money in to buy GPUs for Google, Amazon, etc. Even SpaceX is partly-insulated by using an xAI SPV.
All of this is I think is a huge risk for OpenAI and Anthropic. The risk to SpaceX is a stock collapse because the AI aspect was always overstated (IMHO).
I think Google will be fine. What is funny is that this is almost using Private Equity type tactics against other investors. Things like the structcures in which the real estate and physical buildings are held in separate entities and the SPVs end up off balance sheet.
[1]: https://ciphertalk.substack.com/p/nobody-knows-what-a-used-g...
Re: Alphabet's cash burn raises alarm for Big Tech as AI spending climbs
#199Earlier quoted context omitted.
The cost of the land, building, mechanical equipment, etc is a very small fraction of the total cost of a DC.
The upfront cost of facility is ~30%, network infra 10-15%, land/utilities is small percentage, power could be significant for an AI DC. The servers are ~50-60% only.
That was not my experience, but that was pre-Covid
Re: Alphabet's cash burn raises alarm for Big Tech as AI spending climbs
#200Earlier quoted context omitted.
> Everyone is in too deep to now admit that there’s a problem I'm not sure how to square this with the dramatic improvement in LLM capabilities in the last 8-9 months. If anything, it makes the earlier investments look prescient?
The problem is that the dramatic improvement in capabilities is not translating to a dramatic increase in revenue.
Ans so far, the dramatic improvements have come with an increase in API costs.
Even if, hypothetically, Fable or a Fable-class model could seriously replace some headcount, it'll only gain further traction of it's actually cheaper than hiring humans. $50/MTok is expensive. Wouldn't be unreasonable to expect somewhere between ~$3k-$5k/month/developer in spend. Cheaper than a Junior in the HCoL areas (in the US), but not much cheaper in lower-to-average COL areas. Most acceleration will come from having the headcount + giving said headcount $3k-$5k/month in token budget, so now it just becomes a very expensive dev tool rather than a headcount replacement tool.
The idea that a $30k/year API bill will replace 2 $100k developers falls part outside of SFC/NYC. No CFO of a mid-market company in a LCOL area is signing off on $3k/month/dev API bills. They'll just hire juniors and cap their spend at $200/month.