[flagged]
Calculating the cost of a Google DeepMind paper
31–40 of 160 posts
Re: Calculating the cost of a Google DeepMind paper
#32Earlier quoted context omitted.
They are not saying it doesn't happen. They are saying: The companies that don't learn from these mistakes will go out of business before long.
In principle, for some other company, sure. Google makes ~$300b a year in profit . They could make a $10m mistake every day and barely make a dent in it.
They do not, they made ~90 billion in profit. So no one would notice a 10 mil mistake, but no they didn't make 300b in profits.
Re: Calculating the cost of a Google DeepMind paper
#33Earlier quoted context omitted.
This is the side effect of underutilized capital and it’s present in many cases. For example, if YOU want to rent a backhoe to do some yard rearrangement it’s going to cost you. But Bob who owns BackHoesInc has them sitting around all the time when they’re not being rented or used; he can rearrange his yard wholesale or almost free.
> This is the side effect of underutilized capital and it’s present in many cases. "Underutilized" isn't the right word here. There's some value in putting your capital to productive use. But, once immediate needs are satisfied, there's more value in having the capital available to address future needs quickly than there would be in making sure that everything necessary to address those future needs is tied up in low…
Same effect when leasing companies let office space sit unoccupied for years on end. The future value is higher than the marginal value of reducing the price to fill it with a tenant.
Re: Calculating the cost of a Google DeepMind paper
#34Earlier quoted context omitted.
Can others also buy the “best effort” tier? If the job could easily run for weeks, even when you could buy your way for doing it in a day. Then have a bidding on this “best effort” resource, where they factor in electricity at any given time
Is the "best effort" tier similar to AWS spot instances?
At every cloud provider there's probably a tier below "spot" (or whatever the equivalent is called at AWS's competitors) that is used for the low-priority jobs of the cloud provider itself.
Re: Calculating the cost of a Google DeepMind paper
#35If this ran on google's own cloud it amounts to internal bookkeeping. The only cost is then the electricity and used capacity. Not consumer pricing. So negligible. It is rather unfortunate that this sort of paper is hard to reproduce. That is a BIG downside, because it makes the result unreliable. They invested effort and money in getting an unreliable result. But perhaps other research will corroborate. Or it may gi…
Is the electricity cost negligible? It's a pretty compute intensive application.
Of course it would be a tiny fraction of the $10m figure here, but even 1% would be $100,000. Negligible to Google, but for Google even $10 million is couch cushion money.
Re: Calculating the cost of a Google DeepMind paper
#36Content aside. This is hands down my favorite blog format.
Re: Calculating the cost of a Google DeepMind paper
#37Earlier quoted context omitted.
Reproducibility is a key element of the scientific process How is anyone else going to reproduce the experiment if it's going to cost them $10 million because they don't work at Google and would have to rent the infrastructure?
Cheap compared to some high energy physic experiments.
I was thinking this too.
Splitting the atom, and various space program experiments would also be difficult to reproduce if someone wanted to try.
Re: Calculating the cost of a Google DeepMind paper
#38If this ran on google's own cloud it amounts to internal bookkeeping. The only cost is then the electricity and used capacity. Not consumer pricing. So negligible. It is rather unfortunate that this sort of paper is hard to reproduce. That is a BIG downside, because it makes the result unreliable. They invested effort and money in getting an unreliable result. But perhaps other research will corroborate. Or it may gi…
Opportunity cost is cost. What you could have earned by selling the resources to customers instead of using them yourself is what the resources are worth.
Not if you’re only using the resources when they’re available because no customer has paid to use them.
Re: Calculating the cost of a Google DeepMind paper
#39Earlier quoted context omitted.
> This is the side effect of underutilized capital and it’s present in many cases. "Underutilized" isn't the right word here. There's some value in putting your capital to productive use. But, once immediate needs are satisfied, there's more value in having the capital available to address future needs quickly than there would be in making sure that everything necessary to address those future needs is tied up in low…
Same effect when leasing companies let office space sit unoccupied for years on end. The future value is higher than the marginal value of reducing the price to fill it with a tenant.
Real estate is a playground for irrationally hopeful or stubborn participants.
Re: Calculating the cost of a Google DeepMind paper
#403USD/hour on the H100 is much more expensive than a reasonable amortized full ownership cost, unless one assumes the GPU is useless within 18 months, which I find a bit dramatic. The MFU can be above 40% and certainly well above the 35% in the estimate, also for small models with plain pytorch and trivial tuning [1] I didnt read the linked paper carefully but I seriously doubt the google team used vocab embedding layers with 2 D V parameters stated in the link, because this would be suboptimal by not tying the weights of the token embedding layer in the decoder architecture (even if they did double the params in these layers, it would not lead to 6 D V compute because the embedding input is indexed). To me these assumptions suggested a somewhat careless attitude towards the cost estimation and so I stopped reading the rest of this analysis carefully. My best guess is that the author is off by a large factor in the upward direction, and a true replication with H100/200 could be about 3x less expensive.
[1] if the total cost estimate was relatively low, say less than 10k, then of course the lowest rental price and a random training codebase might make some sense in order to reduce administrative costs; once the cost is in the ballpark of millions of USD, it feels careless to avoid optimizing it further. There exist H100s in firesales or Ebay occasionally, which could reduce the cost even more, but the author already mentions 2USD/gpu/hour for bulk rental compute, which is better than the 3USD/gpu/hour estimate they used in the writeup.