Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

After the Bubble

tbray.org

Post reply on HN
Print view View original

61–70 of 86 posts

Re: After the Bubble

#61

Post by spwa4 »

pantalaimon wrote: post #29
jsnell wrote: post #22

> Nobody who is doing this is willing to come clean with hard numbers but there are data points, for example from Meta and (very unofficially) Google. The Meta link does not support the point. It's actually implying a MTBF of over 5 years at 90% utilizization even if you assume there's no bathtub curve. Pretty sure that lines up with the depreciation period. The Google link is even worse. It links to https://www.toms…

On top of that, Google isn't using NVIDIA GPUs, they have their own TPU.

Google is using nVidia GPUs. More than that, I'd expect Google to still be something like 90% on nVidia GPUs. You can't really check of course. Maybe I'm an idiot and it's 50%.

But you can see how that works: go to colab.research.google.com. Type in some code ... "!nvidia-smi" for instance. Click on the down arrow next to "connect", and select change runtime type. 3 out of 5 GPU options are nVidia GPUs.

Frankly, unless you rewrite your models you don't really have a choice but using nVidia GPUs, thanks to, ironically, Facebook (authors of pytorch). There is pytorch/XLA automatic translation to TPU but it doesn't work for "big" models. And as a point of advice: you want stuff to work on TPUs? Do what Googlers do: use Jax ( https://github.com/jax-ml/jax ), oh, and look at the commit logs of that repository to get your mind blown btw.

In other words, Google rents out nVidia GPUs to their cloud customers (with the hardware physically present in Google datacenters).

Re: After the Bubble

#62

Post by periodjet »

evanelias wrote: post #56
hackernews_7364 wrote: post #48

There's many questions about the overall economics of AI, its value, is it overvalued, is it not, etc. but this is a very poor article I suspect made by someone with little to no financial or accounting knowledge with a strong "uh big tech bad" bias. > When companies buy expensive stuff, for accounting purposes they pretend they haven’t spent the money; instead they “depreciate” it over a few years. There's no preten…

> I suspect made by someone with little to no financial or accounting knowledge with a strong "uh big tech bad" bias. Actually the author has worked for Google, Amazon (VP-level), Sun, and DEC; and was a co-creator of XML.

[flagged]

Re: After the Bubble

#63

Post by littlestymaar »

spwa4 wrote: post #61
pantalaimon wrote: post #29

Earlier quoted context omitted.

On top of that, Google isn't using NVIDIA GPUs, they have their own TPU.

Google is using nVidia GPUs. More than that, I'd expect Google to still be something like 90% on nVidia GPUs. You can't really check of course. Maybe I'm an idiot and it's 50%. But you can see how that works: go to colab.research.google.com. Type in some code ... "!nvidia-smi" for instance. Click on the down arrow next to "connect", and select change runtime type. 3 out of 5 GPU options are nVidia GPUs. Frankly, unle…

> Frankly, unless you rewrite your models you don't really have a choice but using nVidia GPUs, thanks to, ironically, Facebook (authors of pytorch). There is pytorch/XLA automatic translation to TPU but it doesn't work for "big" models. And as a point of advice: you want stuff to work on TPUs?

I don't understand what you mean, most models aren't anywhere near big in terms of code complexity, once you have the efficient primitives to build on (like you have an efficient hardware-accerated matmul, backprop, flash attention, etc.) these models are in the sub-thousand LoC territory and you can even vibe-convert from one environment to another.

That's kind of a shock to realize how simple the logic behind LLMs is.

I still agree with you, Google is most likely still using Nvidia chips in addition to TPUs.

Re: After the Bubble

#64

Post by keeda »

evanelias wrote: post #56
hackernews_7364 wrote: post #48

There's many questions about the overall economics of AI, its value, is it overvalued, is it not, etc. but this is a very poor article I suspect made by someone with little to no financial or accounting knowledge with a strong "uh big tech bad" bias. > When companies buy expensive stuff, for accounting purposes they pretend they haven’t spent the money; instead they “depreciate” it over a few years. There's no preten…

> I suspect made by someone with little to no financial or accounting knowledge with a strong "uh big tech bad" bias. Actually the author has worked for Google, Amazon (VP-level), Sun, and DEC; and was a co-creator of XML.

1. Being a VP in these companies does not imply they have an understanding of financing, accounting or data-center economics unless their purview covered or was very close to the teams procuring and running the infrastructure.

2. That level of seniority does, on the other hand, expose them to a lot of the shenanigans going on in those companies, which could credibly lead them to develop a "big tech bad" mindset.

Re: After the Bubble

#66

Post by keeda »

John23832 wrote: post #57
heathrow83829 wrote: post #51

Earlier quoted context omitted.

once again, it's not "fire", it's lay off. there is a difference.

People are being removed from their livelihood. Equivocating about what YOU comfortably would prefer to call it is wasted effort that I don't care to engage in.

I think the point is simply that "firing" indicates wrongdoing or lack of performance by the workers, whereas "layoffs" indicates the company unilaterally decided to remove them. It may sound nitpicky, but it is pertinent to your point that people are losing their livelihood through no fault of their own.

Re: After the Bubble

#67

Post by evanelias »

keeda wrote: post #64
evanelias wrote: post #56

Earlier quoted context omitted.

> I suspect made by someone with little to no financial or accounting knowledge with a strong "uh big tech bad" bias. Actually the author has worked for Google, Amazon (VP-level), Sun, and DEC; and was a co-creator of XML.

1. Being a VP in these companies does not imply they have an understanding of financing, accounting or data-center economics unless their purview covered or was very close to the teams procuring and running the infrastructure. 2. That level of seniority does, on the other hand, expose them to a lot of the shenanigans going on in those companies, which could credibly lead them to develop a "big tech bad" mindset.

So on the one side, we have a famous widely-respected 70-year-old software engineer with a lengthy wikipedia bio and history of industry-impactful accomplishments. His statements on depreciation here are aligned with things that have been discussed on HN numerous times from my recollection; here are a couple discussions that I found quickly: https://news.ycombinator.com/item?id=29005669 (search "depreciation"), https://news.ycombinator.com/item?id=45310858

On the other side, we have the sole comment ever by a pseudonymous HN user with single-digit karma.

Personally I'll trust the former over the latter.

Re: After the Bubble

#68

Post by keeda »

dragonwriter wrote: post #59
kevin061 wrote: post #54

Earlier quoted context omitted.

The dotcom bubble was a bunch of Silicon Valley types buying fancy domain names and getting showered in money before they even released anything remotely useful. AI companies are releasing useful things right this second, even if they still require human oversight, they are also able to significantly accelerate many tasks.

> The dotcom bubble was a bunch of Silicon Valley types buying fancy domain names and getting showered in money before they even released anything remotely useful. The AI bubble involves a lot of that, too. > AI companies are releasing useful things right this second, even if they still require human oversight, they are also able to significantly accelerate many tasks. So were the Googles and other leading firms in t…

I feel like this bubble actually has two bubbles happening:

1. The infra build out bubble: this is mostly the hypescalers and Nvidia.

2. The AI company valuation bubble: this includes the hyperscalers, pure-play AI companies like OpenAI and Anthropic, and the swarm of startups that are either vaporware or just wrappers on top of the same set of APIs.

There will probably be a pop in (2), especially the random startups that got millions in VC funding just because they have a ".ai" in their domain name. This is also why the OpenAI and Anthropic are getting into the infra game by trying to own their own datacenters, that may be the only moat they have.

However, when people talk about trillions, it's mostly (1) that they are thinking of. Given the acceleration of demand that is being reported, I think (1) will not really pop, maybe just deflate a bit when (2) pops.

Re: After the Bubble

#69

Post by hackernews_7364 »

evanelias wrote: post #56
hackernews_7364 wrote: post #48

There's many questions about the overall economics of AI, its value, is it overvalued, is it not, etc. but this is a very poor article I suspect made by someone with little to no financial or accounting knowledge with a strong "uh big tech bad" bias. > When companies buy expensive stuff, for accounting purposes they pretend they haven’t spent the money; instead they “depreciate” it over a few years. There's no preten…

> I suspect made by someone with little to no financial or accounting knowledge with a strong "uh big tech bad" bias. Actually the author has worked for Google, Amazon (VP-level), Sun, and DEC; and was a co-creator of XML.

There are many legitimate concerns about the financial implications of these huge investments in AI. In fact the podcast that he references is great at providing _informed_ and _nuanced_ observations about all of this - Paul Kedrosky is great.

BUT (my point)

Is that the article is terrible at reflecting all of that and makes wrong and misleading comments about it.

The idea that companies depreciating assets is them "pretending they haven't spent the money" or that "management gets to pick your depreciation period" is simply wrong.

Do you think any of those two statements are accurate?

P.S. Maybe you make a good point, I said that I suspected based on those statements that he had little financial knowledge. tbh I didn't know the author, hence the "suspect". But now that you say that it might be that he is so biased in this particular topic that he can't make a fair representation of his point. Irrespective of that, I will say it again: statements like the ones I've commented are absurd.

Re: After the Bubble

#70

Post by evanelias »

hackernews_7364 wrote: post #69
evanelias wrote: post #56

Earlier quoted context omitted.

> I suspect made by someone with little to no financial or accounting knowledge with a strong "uh big tech bad" bias. Actually the author has worked for Google, Amazon (VP-level), Sun, and DEC; and was a co-creator of XML.

There are many legitimate concerns about the financial implications of these huge investments in AI. In fact the podcast that he references is great at providing _informed_ and _nuanced_ observations about all of this - Paul Kedrosky is great. BUT (my point) Is that the article is terrible at reflecting all of that and makes wrong and misleading comments about it. The idea that companies depreciating assets is them "…

Those statements are quite clearly simplifications, provided for readers who lack accounting experience (e.g. the majority of readers, since it is an engineering-focused blog). That's entirely reasonable in context. I don't know why you're laser-focusing on these two phrases as some kind of smoking gun that discredits the entire piece.

On the "management gets to pick your depreciation period" one in particular, despite being a massive over-simplification, there is substantial underlying truth to the statement. The author's comment about "I can remember one of the big cloud vendors announcing they were going to change their fleet depreciation from three to four years and that having an impact on their share price" is specifically referring to Alphabet's 2021 Q3 earnings statement. See the HN discussion I linked in reply to the sibling comment.

Post reply on HN