Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

xAI is looking more like a datacentre REIT than a frontier lab

martinalderson.com

Post reply on HN
Print view View original

461–470 of 580 posts

Re: xAI is looking more like a datacentre REIT than a frontier lab

#461

Post by fer »

fragmede wrote: post #421
mikepurvis wrote: post #384

Earlier quoted context omitted.

Are current datacenter deployments structured in such a way that the memory can later be moved to newer GPU dies? Or is it all packaged together as on consumer graphics cards? I assumed the latter and therefore that the memory is depreciating along with the GPU cores it's soldered onto PCBs with. ... or is it a different argument being made, perhaps that depreciation for GPUs has slowed because rising demand will kee…

Removing RAM chips off old cards is uneconomical, until it isn't. With things going the way they are, if you've got a card with soldered on RAM that could be transplanted to a newer card, I think you'll start seeing that happening.

It has already become economic. While not exactly the same, the NVIDIA 2080 11GB cards are notorious for being upcycled with extra RAM: https://www.reddit.com/r/nvidia/comments/146us12/nvidia_gefo...

Re: xAI is looking more like a datacentre REIT than a frontier lab

#462

Post by adjejmxbdjdn »

nl wrote: post #309
adjejmxbdjdn wrote: post #251

Earlier quoted context omitted.

Compute is also a rapidly depreciating asset. I want to make a comparison with a car rental business and say that it would be like valuing Hertz entirely on the basis of the number of cars they own, as opposed to how many they rent out, but cars have a much longer depreciation period, if there are no customers they’re not costing you more money, unlike your computer which you are using for training and sucking up mas…

> Compute is also a rapidly depreciating asset. That's the default assumption but in the new GPU+Memory constrained age isn't true. Time on 4 year old H100 servers costs more now than when they were new (!!)

> Time on 4 year old H100 servers costs more now than when they were new (!!)

There are several confounding factors.

We’ve seen massive inflation since then. So some growth in cost was expected.

More importantly, the current Tech industry almost always starts by selling things at a loss. The increased cost could simply be the industry choosing to not subsidize that particular service anymore.

But also, I don’t think that’s a realistic comparison. Rented out GPUs are likely not a similar use profile as compute used for training LLMs. The latter is likely closer to the cryptocurrency GPUs that are running at full tilt 24/7.

And those things physically burn out.

Re: xAI is looking more like a datacentre REIT than a frontier lab

#463

Post by EdiX »

SecretDreams wrote: post #38
treis wrote: post #14

Earlier quoted context omitted.

There's no realistic way for the music to stop. The demand for LLMs is staggering and the big providers are charging full freight for inference. They might not make back the money from training but these data centers are definitely going to be fully utilized for at least the next 5 years.

Look, there's two things: * LLMs are useful * Company valuations around LLMs are not realistic Both can be true, much like they were during the Dotcom bubble. The internet turned out to be a pretty real thing. A couple examples below might feel familiar in the next couple months/years. > Blucora (then InfoSpace): Founded by Naveen Jain, at its peak its market cap was $31 billion and was the largest Internet business…

Here's my theory about the dotcom bubble. The market correctly identified the internet as hugely valuable and correctly identified search engines as being able to capture a large share of this value. Consequently early search engines, chiefly Yahoo!, obtained (merited) high valuations. What Yahoo! did with their stock (they IPOd in 1996) was go on a big startup buying spree. This is what actually started the bubble: invest in some random dotcom crap company in the hopes that Yahoo! swoops in, buys and you get a big payday.

What caused the crash was Yahoo! being unable to do anything with their acquisitions and Google coming out with a better search engine, undermining Yahoo!'s core product. Google basically pulled the rug from under the dot com bubble.

The situation we're in now with LLMs is different, if I'm right we're actually pre-bubble, the bubble hasn't even started yet.

Re: xAI is looking more like a datacentre REIT than a frontier lab

#464

Post by _heimdall »

TSiege wrote: post #6

> And Google is a major shareholder in SpaceX, so they certainly have incentive to juice the valuation of the IPO. Google own 5-6% of the shares of SpaceX. SpaceX is seeking a valuation of $1.77T which means Google's shares would be worth $88.5B-$106.2B. I'm not a skeptic of AI/LLMs but this makes me deeply suspicious of these circular deals. What happens when the music stops?

Given all the rules changes related to IPOs that also just went I to effect, I can only assume their hope is that the public is holding the bag when the music stops and that they think it is going to happen quick.

Re: xAI is looking more like a datacentre REIT than a frontier lab

#465

Post by Traubenfuchs »

leetharris wrote: post #15
hawkice wrote: post #2

They have developed an LLM, so they are an AI lab, but the quality of that model suggests they're not a frontier anything.

I have the pro account for ChatGPT, Claude, Gemini, and Grok. They all have various strengths and weaknesses. My favorite is still ChatGPT, then Gemini/Claude, then Grok. Grok often feels 1-2 generations behind the competition in general use, but it has three things that I love: 1. It seems to be the best at understanding current events. Maybe due to X integration, or some other tool call optimization in the backend?…

> Anthropic is getting here too.

I almost exclusively use claude for all my professional and private needs. In my experience it's really good at adhering to my wishes in regards to sycophancy and pushing back. If you really want to you can tell it to systematically push back on anything where pushback makes sense until it continues with the flow of conversation.

In my first therapy session, the answers were too long and contained multiple questions, spawning multiple threads of conversation. I told it to tone it down and only ever ask one question back, maybe two, if they are related. The answers got too short. I told it to make them "slightly longer" again and reached a sweet spot.

The conversation is yours to form! You need to find the "system prompts" and guidelines to give it that work for you.

Re: xAI is looking more like a datacentre REIT than a frontier lab

#466

Post by donaldjbiden »

paulhebert wrote: post #390
frognumber wrote: post #380

Earlier quoted context omitted.

It's very unclear to me. The key question is on direction of LLMs. Right now, LLMs are taking over human jobs. If the cost of silicon+power If this applies to SWEs, lawyers, business analysts, many research scientists, .... this situation could persist for a long, long time. While capital costs less than the inputs of labor (nominal food, housing, etc.), there is no need for labor. The key question is about continued…

> what rational reason is there to employ a human being? To maintain a functioning society and social contract? Is wanting low unemployment in our society not rational?

[dead]

Re: xAI is looking more like a datacentre REIT than a frontier lab

#467

Post by donaldjbiden »

fragmede wrote: post #421
mikepurvis wrote: post #384

Earlier quoted context omitted.

Are current datacenter deployments structured in such a way that the memory can later be moved to newer GPU dies? Or is it all packaged together as on consumer graphics cards? I assumed the latter and therefore that the memory is depreciating along with the GPU cores it's soldered onto PCBs with. ... or is it a different argument being made, perhaps that depreciation for GPUs has slowed because rising demand will kee…

Removing RAM chips off old cards is uneconomical, until it isn't. With things going the way they are, if you've got a card with soldered on RAM that could be transplanted to a newer card, I think you'll start seeing that happening.

Chinese recyclers already do this with laptops

Re: xAI is looking more like a datacentre REIT than a frontier lab

#468

Post by ForHackernews »

JumpCrisscross wrote: post #383
542458 wrote: post #223

Earlier quoted context omitted.

Is there anything to read on how the economics of an orbital datacenter make any sense? Because I don't really see how blasting a server into space solves any of the typical issues associated with datacentres beyond easier access to solar.

> anything to read on how the economics of an orbital datacenter make any sense? I'll do a write-up at some point. But the core drivers are launch cost, permitting delays for terrestrial datacentres and interest rates. The balance is between, on one hand, the financing cost of the permiting delays against, on the other hand, the cost of launching radiators. (Chips are light. Solar panels without glass cladding are su…

> permitting delays for terrestrial datacentres

"Well, you didn't want a data centre in the field near your town, so instead we'll rain astrocentere debris across the western hemisphere and set off a Kessler syndrome cascade. Thank god we didn't have to wait for a permit."

Re: xAI is looking more like a datacentre REIT than a frontier lab

#469

Post by jstanley »

skybrian wrote: post #65
Qhemlomo wrote: post #10

Earlier quoted context omitted.

At least Alphabet, Microsoft and Amazon can afford it. Nvidia is not losing anything if their stock falls. So whats left? The typical candidates of course: We poor people. 401k, ETF, etc. we pay the bill.

If the S&P 500 dropped 20%, that's about a year's growth. Long-term investors who bought before that would be poorer than they thought they were, but they're not worse off than they started and there wouldn't be any particular bill to pay. If they're a long term investor then they can wait for it to come back. (A similar argument could be made for larger drops.) The real suffering comes from whatever effect there is…

> If the S&P 500 dropped 20%, that's about a year's growth.

But the S&P 500 is currently trading at over 2x its average long-term CAPE: https://www.multpl.com/shiller-pe

So it can reasonably be expected to drop more than 50% to return to average long-term valuation levels.

And the "nonfinancial market cap to gross-value-added" ratio is even more insane, I have a site tracking this number: https://sharperatios.com/market-cap-gva.html

Re: xAI is looking more like a datacentre REIT than a frontier lab

#470

Post by sawjet »

emodendroket wrote: post #211
leetharris wrote: post #15

Earlier quoted context omitted.

I have the pro account for ChatGPT, Claude, Gemini, and Grok. They all have various strengths and weaknesses. My favorite is still ChatGPT, then Gemini/Claude, then Grok. Grok often feels 1-2 generations behind the competition in general use, but it has three things that I love: 1. It seems to be the best at understanding current events. Maybe due to X integration, or some other tool call optimization in the backend?…

> 1. It seems to be the best at understanding current events. Maybe due to X integration, or some other tool call optimization in the backend? I don't know, but I often ask about things going on, and the other models have outdated info, give unhelpful answers, etc. That makes sense, but occasionally you ask about an issue where it's clearly received political instruction from the commissar and it acts totally lobotom…

Do you have an example of this?
Post reply on HN