Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't rent the cloud, own instead

blog.comma.ai

Post reply on HN
Print view View original

11–20 of 516 posts

Re: Don't rent the cloud, own instead

#11

Post by Onavo »

comrade1234 wrote: post #8

15-years ago or so a spreadsheet was floating around where you could enter server costs, compute power, etc and it would tell you when you would break-even by buying instead of going with AWS. I think it was leaked from Amazon because it was always three-years to break-even even as hardware changed over time.

Well, somebody should recreate it. I smell a potential startup idea somewhere. There's a ton of "cloud cost optimizers" software but most involve tweaking AWS knobs and taking a cut of the savings. A startup that could offload non critical service from AWS to colo and traditional bare metal hosting like Hetzner has a strong future.

One thing to keep in mind is that the curve for GPU depreciation (in the last 5 years at least) is a little steeper than 3 years. Current estimates is that the capital depreciation cost would plunge dramatically around the third year. For a top tier H100 depreciation kicks in around the 3rd year but they mentioned for the less capable ones like the A100 the depreciation is even worse.

https://www.silicondata.com/use-cases/h100-gpu-depreciation/

Now this is not factoring cost of labour. Labor at SF wages is dreadfully expensive, now if your data center is right across the border in Tijuana on the other hand..

Re: Don't rent the cloud, own instead

#12

Post by hyperbovine »

langarus wrote: post #5

This is a great solution for a very specific type of team but I think most companies with consistent GPU workloads will still just rent dedicated servers and call it a day.

I agree, and cloud compute is poised to become even more commoditized in the coming years (gazillion new data centers + AI plateauing + efficiency gains, the writing is on the wall). There’s no way this makes sense for most companies.

Re: Don't rent the cloud, own instead

#13

Post by simianwords »

The reason companies don’t go with on premises even if cloud is way more expensive is because of the risk involved in on premises.

You can see it quite clearly here that there’s so many steps to take. Now a good company would concentrate risk on their differentiating factor or the specific part they have competitive advantage in.

It’s never about “is the expected cost in on premises less than cloud”, it’s about the risk adjusted costs.

Once you’ve spread risk not only on your main product but also on your infrastructure, it becomes hard.

I would be vary of a smallish company building their own Jira in house in a similar way.

Re: Don't rent the cloud, own instead

#14

Post by d1sxeyes »

simianwords wrote: post #13

The reason companies don’t go with on premises even if cloud is way more expensive is because of the risk involved in on premises. You can see it quite clearly here that there’s so many steps to take. Now a good company would concentrate risk on their differentiating factor or the specific part they have competitive advantage in. It’s never about “is the expected cost in on premises less than cloud”, it’s about the r…

It’s also opex vs capex, which is a battle opex wins most of the time.

Re: Don't rent the cloud, own instead

#15

Post by mbreese »

hbogert wrote: post #9

Datacenters need cool dry air? No, low isn't good perse. I worked in a datacenter which in winters had less than 40%, ram was failing all over the place. Low humidity causes static electricity.

Low is good if you are also adding more humidity back in. If you want to maintain 45-50% (guessing), then you would want It is much cheaper to use external air for cooling if you can.

Re: Don't rent the cloud, own instead

#17

Post by simianwords »

d1sxeyes wrote: post #14
simianwords wrote: post #13

The reason companies don’t go with on premises even if cloud is way more expensive is because of the risk involved in on premises. You can see it quite clearly here that there’s so many steps to take. Now a good company would concentrate risk on their differentiating factor or the specific part they have competitive advantage in. It’s never about “is the expected cost in on premises less than cloud”, it’s about the r…

It’s also opex vs capex, which is a battle opex wins most of the time.

I think it wins because opex is seen as stable recurring cost and capex is seen as the money you put in your primary differentiation for long term gains.

Re: Don't rent the cloud, own instead

#18

Post by NitpickLawyer »

hyperbovine wrote: post #12
langarus wrote: post #5

This is a great solution for a very specific type of team but I think most companies with consistent GPU workloads will still just rent dedicated servers and call it a day.

I agree, and cloud compute is poised to become even more commoditized in the coming years (gazillion new data centers + AI plateauing + efficiency gains, the writing is on the wall). There’s no way this makes sense for most companies.

> AI plateauing

Ummm is that plateauing with us in the room?

The advantage of renting vs. owning is that you can always get the latest gen, and that brings you newer capabilities (i.e. fp8, fp4, etc) and cheaper prices for current_gen-1. But betting on something plateauing when all the signs point towards the exact opposite is not one of the bets i'd make.

Re: Don't rent the cloud, own instead

#19

Post by TonyStr »

comrade1234 wrote: post #8

15-years ago or so a spreadsheet was floating around where you could enter server costs, compute power, etc and it would tell you when you would break-even by buying instead of going with AWS. I think it was leaked from Amazon because it was always three-years to break-even even as hardware changed over time.

Azure provides their own "Total Cost of Ownership" calculator for this purpose [0]. Notably, this makes you estimate peripheral costs such as cost of having a server administrator, electricity, etc.

[0] - https://azure-int.microsoft.com/en-us/pricing/tco/calculator...

Re: Don't rent the cloud, own instead

#20

Post by d1sxeyes »

simianwords wrote: post #17
d1sxeyes wrote: post #14

Earlier quoted context omitted.

It’s also opex vs capex, which is a battle opex wins most of the time.

I think it wins because opex is seen as stable recurring cost and capex is seen as the money you put in your primary differentiation for long term gains.

True, but for a lot of companies “our servers are on-prem” is not a primary differentiator.
Post reply on HN