Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Calculating the cost of a Google DeepMind paper

152334h.github.io

Post reply on HN
Print view View original

41–50 of 160 posts

Re: Calculating the cost of a Google DeepMind paper

#41

Post by Sebb767 »

c-linkage wrote: post #12
sigmoid10 wrote: post #8

This is calculation is pretty pointless and the title is flat out wrong. It also gets lost in finer details while totally missing the bigger picture. After all, the original paper written by people either working for Google or at Google. So you can safely assume they used Google resources. That means they wouldn't have used H100s, but Google TPUs. Since they design and own these TPUs, you can also safely assume that…

Reproducibility is a key element of the scientific process How is anyone else going to reproduce the experiment if it's going to cost them $10 million because they don't work at Google and would have to rent the infrastructure?

But what's the solution here? Not doing the (possibly) interesting research because it's hard to reproduce? That doesn't sound like a better situation.

That being said, yes, this is hard to reproduce for your average Joe, but there are also a lot of companies (like OpenAI, Facebook, ...) that are able to throw this amount of hardware at the problem. And in a few years you'll probably be able to do it on commodity hardware.

Re: Calculating the cost of a Google DeepMind paper

#43

Post by bombcar »

thaumasiotes wrote: post #30
bombcar wrote: post #15

Earlier quoted context omitted.

This is the side effect of underutilized capital and it’s present in many cases. For example, if YOU want to rent a backhoe to do some yard rearrangement it’s going to cost you. But Bob who owns BackHoesInc has them sitting around all the time when they’re not being rented or used; he can rearrange his yard wholesale or almost free.

> This is the side effect of underutilized capital and it’s present in many cases. "Underutilized" isn't the right word here. There's some value in putting your capital to productive use. But, once immediate needs are satisfied, there's more value in having the capital available to address future needs quickly than there would be in making sure that everything necessary to address those future needs is tied up in low…

Yeah, airlines make "more return on capital" by faster turn-around of planes to a point - if they are utilizing their airframes above 80 or 90 or whatever percent, the airline itself becomes extremely fragile and unable to handle incidents that impact timing.

We saw the same thing with JIT manufacturing during Covid.

Re: Calculating the cost of a Google DeepMind paper

#44

Post by Bjartr »

nathancahill wrote: post #33
thaumasiotes wrote: post #30

Earlier quoted context omitted.

> This is the side effect of underutilized capital and it’s present in many cases. "Underutilized" isn't the right word here. There's some value in putting your capital to productive use. But, once immediate needs are satisfied, there's more value in having the capital available to address future needs quickly than there would be in making sure that everything necessary to address those future needs is tied up in low…

Same effect when leasing companies let office space sit unoccupied for years on end. The future value is higher than the marginal value of reducing the price to fill it with a tenant.

That may be part of it for spaces properties left unleased for years, but I believe it's not the only part.

I believe the larger factor, and someone correct me if they have a better understanding of this, is that for commercially rented properties the valuation used to determine the mortgage terms you get takes into account what you claim to be able to get from rent. Renting for less than that reduces the valuation and can put you upside down on the mortgage. But the bank will let you defer mortgage payments, effectively taking each month of mortgage duration and moving it from now to after the last month of the mortgage duration, extending the time they earn interest for.

So if no one want to lease the space at that price after a prior lessee leaves for whatever reason, it's better for the property owner financially to leave the space vacant, sometimes for years, until someone willing to pay that price comes along, than to lower the rent and get a tenant.

Re: Calculating the cost of a Google DeepMind paper

#45

Post by pintxo »

BartjeD wrote: post #5

If this ran on google's own cloud it amounts to internal bookkeeping. The only cost is then the electricity and used capacity. Not consumer pricing. So negligible. It is rather unfortunate that this sort of paper is hard to reproduce. That is a BIG downside, because it makes the result unreliable. They invested effort and money in getting an unreliable result. But perhaps other research will corroborate. Or it may gi…

> They chose to publish. So they are interested in seeing it reproduced or improved upon.

Not necessarily, publishing also ensure that the stuff is no longer patentable.

Re: Calculating the cost of a Google DeepMind paper

#46

Post by franga2000 »

thaumasiotes wrote: post #30
bombcar wrote: post #15

Earlier quoted context omitted.

This is the side effect of underutilized capital and it’s present in many cases. For example, if YOU want to rent a backhoe to do some yard rearrangement it’s going to cost you. But Bob who owns BackHoesInc has them sitting around all the time when they’re not being rented or used; he can rearrange his yard wholesale or almost free.

> This is the side effect of underutilized capital and it’s present in many cases. "Underutilized" isn't the right word here. There's some value in putting your capital to productive use. But, once immediate needs are satisfied, there's more value in having the capital available to address future needs quickly than there would be in making sure that everything necessary to address those future needs is tied up in low…

In the case of compute, you can evict low-priority jobs nearly instantly, so the compute capacity running spot instances and internal side-projets is just as available for unexpected bursts as it would be if sitting idle.

Re: Calculating the cost of a Google DeepMind paper

#47

Post by bombcar »

Bjartr wrote: post #44
nathancahill wrote: post #33

Earlier quoted context omitted.

Same effect when leasing companies let office space sit unoccupied for years on end. The future value is higher than the marginal value of reducing the price to fill it with a tenant.

That may be part of it for spaces properties left unleased for years, but I believe it's not the only part. I believe the larger factor, and someone correct me if they have a better understanding of this, is that for commercially rented properties the valuation used to determine the mortgage terms you get takes into account what you claim to be able to get from rent. Renting for less than that reduces the valuation a…

This is mostly correct. People assume commercial loan terms are like single-family homes "but larger" but they're not. They basically are all custom financial deals with multiple banks and may be over multiple properties. As long as total vacancy isn't below a cutoff the banks will be happy, but lowering rents "just to get a tenant" can harm the valuation and trigger terms.

Part of the reason things like Halloween Superstores can pop in is the terms often exclude "short term leases" which are under six months.

Also when you're leasing to companies, they are VERY quick to jump at lower prices if available, which means that if you drop the lease for one tenant, the others are sure to follow, sometimes even before lease terms are up.

Re: Calculating the cost of a Google DeepMind paper

#48

Post by Sebb767 »

vishnugupta wrote: post #7
willis936 wrote: post #3

Earlier quoted context omitted.

Any company of any size that doesn't learn the right lessons from a $10M mistake will be out of business before long.

https://killedbygoogle.com/ I’m confident each one of them were multiple of $10M investments. And this is just what we know because they were launched publicly.

The point the parent made is not to not make mistakes, but to learn from them. Which they probably did not from all of them, as indicated by the sheer amount of messenger apps on this list, but there's definitely a lot to learn from this list.

Re: Calculating the cost of a Google DeepMind paper

#49

Post by khafra »

Bjartr wrote: post #44
nathancahill wrote: post #33

Earlier quoted context omitted.

Same effect when leasing companies let office space sit unoccupied for years on end. The future value is higher than the marginal value of reducing the price to fill it with a tenant.

That may be part of it for spaces properties left unleased for years, but I believe it's not the only part. I believe the larger factor, and someone correct me if they have a better understanding of this, is that for commercially rented properties the valuation used to determine the mortgage terms you get takes into account what you claim to be able to get from rent. Renting for less than that reduces the valuation a…

Land Value Tax would fix this.

Re: Calculating the cost of a Google DeepMind paper

#50

Post by 152334H »

pama wrote: post #40

3USD/hour on the H100 is much more expensive than a reasonable amortized full ownership cost, unless one assumes the GPU is useless within 18 months, which I find a bit dramatic. The MFU can be above 40% and certainly well above the 35% in the estimate, also for small models with plain pytorch and trivial tuning [1] I didnt read the linked paper carefully but I seriously doubt the google team used vocab embedding lay…

You are correct on true H100 ownership costs being far lower. As I mention in the H100 blurb, the H100 numbers are fungible and I don't mind if you halve them.

MFU can certainly be improved beyond 40%, as I mention. But on the point of small models specifically: the paper uses FSDP for all models, and I believe a rigorous experiment should not vary sharding strategy due to numerical differences. FSDP2 on small models will be slow even with compilation.

The paper does not tie embeddings, as stated. The readout layer does lead to 6DV because it is a linear layer of D*V, which takes 2x for a forward and 4x for a backward. I would appreciate it if you could limit your comments to factual errors in the post.

Post reply on HN