Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't rent the cloud, own instead

blog.comma.ai

Post reply on HN
Print view View original

111–120 of 516 posts

Re: Don't rent the cloud, own instead

#111

Post by iso1631 »

bayindirh wrote: post #33
d1sxeyes wrote: post #14

Earlier quoted context omitted.

It’s also opex vs capex, which is a battle opex wins most of the time.

Opex is faster. Login, click, SSH, get a tea. Capex needs work. A couple of years, at least. If you are willing to put in the work. Your mundane computer is always better than the shiny one you don't own.

That's because of company policies. An SME owner will buy a server and have it in the rack the next day.

Of course creating a VM is still a teraform commit away (you're not using clickops in prod surely)

Re: Don't rent the cloud, own instead

#112

Post by aequitas »

bpavuk wrote: post #83
adamcharnock wrote: post #74

This is an industry we're[0] in. Owning is at one end of the spectrum, with cloud at the other, and a broadly couple of options in-between: 1 - Cloud – This is minimising cap-ex, hiring, and risk, while largely maximising operational costs (its expensive) and cost variability (usage based). 2 - Managed Private Cloud - What we do. Still minimal-to-no cap-ex, hiring, risk, and medium-sized operational cost (around 50%…

if someone on the DevOps team knows Nix, option 3 becomes a lot cheaper time-wise! yeah, Nix flakes still need maintenance, especially on the `nixos-unstable` branch, but you get the quickest disaster recovery route possible! plus, infra flexibility removes random constraints that e.g. Cloudflare Workers have

This is what we do, I gave a talk about our setup earlier this week at CfgMgmtCamp: https://www.youtube.com/watch?v=DBxkVVrN0mA&t=8457s

Re: Don't rent the cloud, own instead

#113

Post by theodric »

DyslexicAtheist wrote: post #88
adamcharnock wrote: post #74

This is an industry we're[0] in. Owning is at one end of the spectrum, with cloud at the other, and a broadly couple of options in-between: 1 - Cloud – This is minimising cap-ex, hiring, and risk, while largely maximising operational costs (its expensive) and cost variability (usage based). 2 - Managed Private Cloud - What we do. Still minimal-to-no cap-ex, hiring, risk, and medium-sized operational cost (around 50%…

this is what we did in the 90ies into mid 2000: > Buy and colocate the hardware yourself – Certainly the cheapest option if you have the skills back then this type of "skill" was abundant. You could easily get sysadmin contractors who would take a drive down to the data-center (probably rented facilities in a real-estate that belonged to a bank or insurance) to exchange some disks that died for some reason. such a pe…

> ancient skills https://youtu.be/ZtYU87QNjPw?&t=10

It baffles me that my career trajectory somehow managed to insulate me from ever having to deal with the cloud, while such esoteric skills as swapping a hot swap disk or racking and cabling a new blade chassis are apparently on the order of finding a COBOL developer now. Really?

I can promise you that large financial institutions still have datacenters. Many, many, many datacenters!

Re: Don't rent the cloud, own instead

#115

Post by belorn »

DyslexicAtheist wrote: post #88
adamcharnock wrote: post #74

This is an industry we're[0] in. Owning is at one end of the spectrum, with cloud at the other, and a broadly couple of options in-between: 1 - Cloud – This is minimising cap-ex, hiring, and risk, while largely maximising operational costs (its expensive) and cost variability (usage based). 2 - Managed Private Cloud - What we do. Still minimal-to-no cap-ex, hiring, risk, and medium-sized operational cost (around 50%…

this is what we did in the 90ies into mid 2000: > Buy and colocate the hardware yourself – Certainly the cheapest option if you have the skills back then this type of "skill" was abundant. You could easily get sysadmin contractors who would take a drive down to the data-center (probably rented facilities in a real-estate that belonged to a bank or insurance) to exchange some disks that died for some reason. such a pe…

I wonder how much companies pay yearly in order to avoid having an employee pick up a drive from a local store, drive to the data center, pull the disk drive, screw out the failing hard drive and put in the new one, add it in the raid, verify the repair process has started, and then return to the office.

Re: Don't rent the cloud, own instead

#116

Post by noosphr »

adamcharnock wrote: post #89
mgaunard wrote: post #80

Earlier quoted context omitted.

you're missing 5, what they are doing. There is a world of difference between renting some cabinets in an Equinix datacenter and operating your own.

Fair point! 5 - Datacenter (DC) - Like 4, except also take control of the space/power/HVAC/transit/security side of the equation. Makes sense either at scale, or if you have specific needs. Specific needs could be: specific location, reliability (higher or lower than a DC), resilience (conflict planning). There are actually some really interesting use cases here. For example, reliability: If your company is in a phys…

5 was a great option for ml work last year since colo rented didn't come with a 10kW cable. With ram, sd and GPU prices the way they are now I have no idea what you'd need to do.

Thank goodness we did all the capex before the OpenAI ram deal and expensive nvidia gpus were the worst we had to deal with.

Re: Don't rent the cloud, own instead

#117

Post by slyall »

bpavuk wrote: post #83
adamcharnock wrote: post #74

This is an industry we're[0] in. Owning is at one end of the spectrum, with cloud at the other, and a broadly couple of options in-between: 1 - Cloud – This is minimising cap-ex, hiring, and risk, while largely maximising operational costs (its expensive) and cost variability (usage based). 2 - Managed Private Cloud - What we do. Still minimal-to-no cap-ex, hiring, risk, and medium-sized operational cost (around 50%…

if someone on the DevOps team knows Nix, option 3 becomes a lot cheaper time-wise! yeah, Nix flakes still need maintenance, especially on the `nixos-unstable` branch, but you get the quickest disaster recovery route possible! plus, infra flexibility removes random constraints that e.g. Cloudflare Workers have

There are a bunch of ways to manage bare metal servers apart from Nix. People have been doing it for years. Kickstart, theforeman, maas, etc, [0]. Many to choose from according to your needs and layers you want them to manage.

Reality is these days you just boot a basic image that runs containers

[0] Longer list here: https://github.com/alexellis/awesome-baremetal

Re: Don't rent the cloud, own instead

#118

Post by adamcharnock »

weavie wrote: post #110
adamcharnock wrote: post #74

This is an industry we're[0] in. Owning is at one end of the spectrum, with cloud at the other, and a broadly couple of options in-between: 1 - Cloud – This is minimising cap-ex, hiring, and risk, while largely maximising operational costs (its expensive) and cost variability (usage based). 2 - Managed Private Cloud - What we do. Still minimal-to-no cap-ex, hiring, risk, and medium-sized operational cost (around 50%…

What is the upper limit of Hertzner? Say you have an AWS bill in the $100s of millions, could Hertzner realistically take on that scale?

An interesting question, so time for some 100% speculation.

It sounds like they probably have revenue in the €500mm range today. And given that the bare metal cost of AWS-equivalent bills tends to be a 90% reduction, we'll say a €10mm+ bare metal cost.

So I would say a cautious and qualified "yes". But I know even for smaller deployments of tens or hundreds of servers, they'll ask you what the purpose is. If you say something like "blockchain," they're going to say, "Actually, we prefer not to have your business."

I get the strong impression that while they naturally do want business, they also aren't going to take a huge amount of risk on board themselves. Their specialism is optimising on cost, which naturally has to involve avoiding or mitigating risk. I'm sure there'd be business terms to discuss, put it that way.

Re: Don't rent the cloud, own instead

#120

Post by rasjani »

rvz wrote: post #51

Not long ago Railway moved from GCP to their own infrastructure since it was very expensive for them. [0] Some go for a Oxide rack [1] for a full stack solution (both hardware and software) for intense GPU workloads, instead of building it themselves. It's very expensive and only makes sense if you really need infrastructure sovereignty. It makes more sense if you're profitable in the tens of millions after raising h…

I was under impression that Oxide rack does not currently ship with GPU's - at least with buildin. . Has this changed recently ?
Post reply on HN