Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Don't rent the cloud, own instead

blog.comma.ai

Post reply on HN
Print view View original

201–210 of 516 posts

Re: Don't rent the cloud, own instead

#201

Post by alpinisme »

infecto wrote: post #194
bojangleslover wrote: post #179

Earlier quoted context omitted.

The complexity is what gets you. One of AWS's favorite situations is 1) Senior engineer starts on AWS 2) Senior engineer leaves because our industry does not value longevity or loyalty at all whatsoever (not saying it should, just observing that it doesn't) 3) New engineer comes in and panics 4) Ends up using a "managed service" to relieve the panic 5) New engineer leaves 6) Second new engineer comes in and not only…

It’s all anecdotal but in my experiences it’s usually opposite. Bored senior engineer wants to use something new and picks a AWS bespoke service for a new project. I am sure it happens a multitude of ways but I have never seen the case you are describing.

I’ve seen your case more than the ransom scenario too. But also even more often: early-to-mid-career dev saw a cloud pattern trending online, heard it was a new “best practice,” and so needed to find a way to move their company to using it.

Re: Don't rent the cloud, own instead

#202

Post by huntaub »

mrbluecoat wrote: post #157

Stopped reading at "Our main storage arrays have no redundancy". This isn't a data center, it's a volatile AI memory bank.

This turns out to be a more and more important primitive for companies who are building their own models [1].

[1] https://si.inc/posts/the-heap/

Re: Don't rent the cloud, own instead

#203

Post by objektif »

Lucasoato wrote: post #95
adamcharnock wrote: post #74

This is an industry we're[0] in. Owning is at one end of the spectrum, with cloud at the other, and a broadly couple of options in-between: 1 - Cloud – This is minimising cap-ex, hiring, and risk, while largely maximising operational costs (its expensive) and cost variability (usage based). 2 - Managed Private Cloud - What we do. Still minimal-to-no cap-ex, hiring, risk, and medium-sized operational cost (around 50%…

Hetzner is definitely an interesting option. I’m a bit scared of managing the services on my own (like Postgres, Site2Site VPN, …) but the price difference makes it so appealing. From our financial models, Hetzner can win over AWS when you spend over 10~15K per month on infrastructure and you’re hiring really well. It’s still a risk, but a risk that definitely can be worthy.

No amount of money will make me maintain my own dbs. We tried it at first and it was a nightmare.

Re: Don't rent the cloud, own instead

#204

Post by apothegm »

This also depends so much on your scaling needs. If you need 3 mid-sized ECS/EC2 instances, a load balancer, and a database with backups, renting those from AWS isn’t going to be significantly more expensive for a decent-sized company than hiring someone to manage a cluster for you and dealing with all the overhead of keeping it maintained and secure.

If you’re at the scale of hundreds of instances, that math changes significantly.

And a lot of it depends on what type of business you have and what percent of your budget hosting accounts for.

Re: Don't rent the cloud, own instead

#205

Post by coliveira »

bojangleslover wrote: post #179
torginus wrote: post #151

Earlier quoted context omitted.

I think the issue with this formulation is what drives the cost at cloud providers isn't necessarily that their hardware is too expensive (which it is), but that they push you towards overcomplicated and inefficient architectures that cost too much to run. A core at this are all the 'managed' services - if you have a server box, its in your financial interest to squeeze as much per out of it as possible. If you're us…

The complexity is what gets you. One of AWS's favorite situations is 1) Senior engineer starts on AWS 2) Senior engineer leaves because our industry does not value longevity or loyalty at all whatsoever (not saying it should, just observing that it doesn't) 3) New engineer comes in and panics 4) Ends up using a "managed service" to relieve the panic 5) New engineer leaves 6) Second new engineer comes in and not only…

The end result of all this is that the percentage of people who know how to implement systems without AWS/Azure will be a single digit. From that point on, this will be the only "economic" way, it doesn't matter what the prices are.

Re: Don't rent the cloud, own instead

#206

Post by acdha »

infecto wrote: post #200
iso1631 wrote: post #140

Earlier quoted context omitted.

> I’m a bit scared of managing the services on my own (like Postgres, Site2Site VPN, …) Out of interest, how old are you? This was quite normal expectation of a technical department even 15 years ago.

No it was not. 15 years ago Heroku was the rage. Even the places that had bare metal usually had someone running something similar to devops and at least core infrar was not being touched. I am sure places existed but 15 years while far away was already pretty far along from what you describe. At least in SV.

Heroku was popular with startups who didn’t have infrastructure skills but the price was high enough that anyone who wasn’t in that triangle of “lavish budget, small team, limited app diversity” wasn’t using it. Things like AWS IaaS were far more popular due to the lower cost and greater flexibility but even that was far from a majority service class.

Re: Don't rent the cloud, own instead

#207

Post by coliveira »

infecto wrote: post #197
mrweasel wrote: post #171

Earlier quoted context omitted.

Just this week a friend of mine was spinning up some AWS managed service, complaining about the complexity, and how any reconfiguration took 45 minutes to reload. It's a service you can just install with apt, the default configuration is fine. Not only is many service no longer cheaper in the cloud, the management overhead also exceed that of on-prem.

What managed service? Curious, I don’t use the full suite of aws services but wondering what would take 45mins, maybe it was a large cluster of some sort that needed rolling changes?

My observation is that all these services are exploding in complexity, and they justify saying that there are more features now, so everyone needs to accept spending more and more time and effort for the same results.

Re: Don't rent the cloud, own instead

#208

Post by infecto »

apothegm wrote: post #204

This also depends so much on your scaling needs. If you need 3 mid-sized ECS/EC2 instances, a load balancer, and a database with backups, renting those from AWS isn’t going to be significantly more expensive for a decent-sized company than hiring someone to manage a cluster for you and dealing with all the overhead of keeping it maintained and secure. If you’re at the scale of hundreds of instances, that math changes…

I also thinks it’s risk model too. Every time I see these kind of posts I think it misses the point there is a balance not only on cost like you describe but risk as well. You are paying to offload some of the risk from yourself.

Re: Don't rent the cloud, own instead

#209

Post by objektif »

geocar wrote: post #123
weavie wrote: post #110

Earlier quoted context omitted.

What is the upper limit of Hertzner? Say you have an AWS bill in the $100s of millions, could Hertzner realistically take on that scale?

Who are you thinking of? Netflix might be spending as much as $120m (but probably a little less), and I thought they were probably Amazon's biggest customer. Does someone (single-buyer) spend more than that with AWS? Hertzner's revenue is somewhere around $400m, so probably a little scary taking on an additional 30% revenue from a single customer, and Netflix's shareholders would probably be worried about risk relyin…

Figma apparently spends around 300-400k/day on AWS. I think this puts them up there.

Re: Don't rent the cloud, own instead

#210

Post by jermaustin1 »

adamcharnock wrote: post #89
mgaunard wrote: post #80

Earlier quoted context omitted.

you're missing 5, what they are doing. There is a world of difference between renting some cabinets in an Equinix datacenter and operating your own.

Fair point! 5 - Datacenter (DC) - Like 4, except also take control of the space/power/HVAC/transit/security side of the equation. Makes sense either at scale, or if you have specific needs. Specific needs could be: specific location, reliability (higher or lower than a DC), resilience (conflict planning). There are actually some really interesting use cases here. For example, reliability: If your company is in a phys…

When I worked IT for a school district at the beginning of my career (2006-2007), I was blown away that every school had a MASSIVE server room (my office at each school - the MDF). 3-5 racks filled (depending on school size and connection speed to the central DC - data closet) 50-75% was networking equipment (5 PCs per class hardwired), 10% was the Novell Netware server(s) and storage, the other 15% was application storage for app distributions on login.
Post reply on HN