Live data from Hacker News

Move Back to Datacenters

rakkhi.substack.com

71–80 of 97 posts

Re: Move Back to Datacenters

#71
post #57

Why is the choice always portrayed as if it's either cloud, or on-prem / colocation ? Those are two extremes. At work I use dedicated physical machines from Hetzner. If I need an extra one they deliver it in a few minutes, and thanks to Ansible it's provisioned within a few minutes more. Hetzner keeps an eye on the hardware and replaces disks, PSU's and the like if needed. I wouldn't often advocate to do colocation o…

I think your example falls under the cloud use case, it's a form of IaaS. The reason it doesn't scale is at some point it becomes cheaper to own the hardware or you need better scaling/redundancy and switch to managed cloud VMs/containers/functions

Re: Move Back to Datacenters

#72
I thought for a second I was hearing a.suggestion that by running your own hardware you can turn off all mitigations. Which should save you a ton of money; for the same core you may be getting more than double the performance. And since you're not running any unknown foreign workloads, no one would have access to be running timing attacks on you.

The article pretty quickly moved off that point. And it seems to at points be saying that making it hard to setup infrastructure is a feature, will help the bottom line. Which has some truth but at enormous emotional cost to your teams.

I do think this shift needs to happen and I appreciate such lengthy sets of concerns being brought out. But I'm pretty lukewarm on this analysis.

Re: Move Back to Datacenters

#73
post #19

Earlier quoted context omitted.

In general when people mention "cost" as an issue for cloud, they're narrowly comparing what a CPU core or GB of RAM or disk costs on paper between cloud and datacenter/onprem. This completely misses the business case for cloud, and is why these sorts of analyses don't convince that many business folk, and why cloud use continues to grow fast. The business case is that someone else is doing the infrastructure managem…

The real business case for the cloud is it replaces capex with opex. Which is great if you have limited capital (bootstrapping, new startup, etc), or are the CEO of a large company that wants to show better cash flow to earn a larger bonus. In reality, the idea that the cloud somehow means you can outsource your infrastructure team rarely works out in practice. At best you’ll get much more expensive developers having…

The REAL business case is cargo culting.

Re: Move Back to Datacenters

#74
post #48

Earlier quoted context omitted.

Genuinely feels like there’s a constant trickle of arguments from people looking to do all they can to shift the tides such that their skill set is more in demand. Or maybe it’s just an ideological obsession. I don’t know. I feel the same way whenever the “use C for everything” people catch a whiff of performance mattering and very strongly suggest that the only way to dig ourselves out of this mess is to move back t…

I believe both arguments held some merits. It all depend on the scale. If you are a startup and/or have a limited/specialized business scope* Cloud can be the best choice. For organizations of 4000+ people that manage 100+ applications (from different vendors) with a 90+ people IT department this is maybe not the silver bullet. At this scale you need to keep complexity and security manageable and a more traditional r…

Yeah, at that scale your devs end up bogged down in cloud concerns. We can't keep anyone at big companies very long because of all the arduous infrastructure bullshit that they have to deal with. Lots of it ends up being done outside of Jira. Total mess.

Re: Move Back to Datacenters

#75

With IAM you can restrict all dynamodb endpoint access to a VPCE/private link. But the insecurity of the public facing endpoints is vastly overstated even without using VPCE. The rest is a bunch of FUD - I spent years going through these points with some of the worlds best security teams to secure some of the most systemically important workloads. These arguments are fairly tired. I’ll tackle another one - speculativ…

> worlds best security teams > the most systemically important workloads Big claims require big evidence. How did you end up with these superlatives?

A long career working hard and building a reputation for delivering hard stuff well and building the right network that I used to good effect.

Re: Move Back to Datacenters

#76
> in traditional datacenters, with Infrastructure and Support teams separate from Development (anti DevOps), there are/were strong human checks and balances. If your Devs wanted to make an API Internet accessible and connect it to what they thought was a “sanitized” database, they probably had to raise a change, submit some firewall rules, maybe talk to a DBA to get credentials.

This is just romanticizing things. In every deployment - whether in it's "in cloud" or anywhere else - there is always the quick on off change that someone makes. Probabably for a valid reason, e.g. solving a production problem quickly. Chances are high that will go unnoticed, until another problem manifests (and that is hopefully not a security issue!).

I would argue that in cloud setups the chances for that to happen are actually slightly lower, because teams are incentivized to use immutable and declarative infrastructure. And that there might be an audit log in place which tells what changes have been made - although that still requires people to look at it, which again happens only in the case problems show up.

Re: Move Back to Datacenters

#77

The conclusion completely handwaves the massive overheads that come with not only owning your own infrastructure, but also having to manage a complex platform stack and its security. The security points, which the main points hinges on, seem hyperfocused and in most cases misguided. Redshift is not internet facing. If your Redshift is internet facing, you've messed up somewhere. The CPU attack example given is for AM…

It handwaves all the things EC2 and all services built on top do.

The amount of engineering needed just to migrate droplets live is enormous - it’s an art (according to a friend in hypervisors), and that ignores everything else regarding hypervisors and security.

Scalability is also a thing you get “for free” with serverless computing.

Re: Move Back to Datacenters

#78
post #60
post #56

Earlier quoted context omitted.

> The business case is that someone else is doing the infrastructure management for you. Many business people believe that moving to cloud can reduce the headcount needed for managing the infra, but that is usually not what's happening. You will still need more or less the same amount of people to patch the OS and configure the networking, but with a slightly different skill set -- instead of Cisco IOS commands, they…

If you're treating cloud as a sort of 1:1 replacement of hardware infrastructure, then what you're saying makes sense. But we don't ever "patch the OS." Our cloud provider does that for us. Upgrades and patching are automatic within the maintenance window. We deploy containers, the OS is just a platform layer that the cloud provider manages. As for needing "more or less the same amount of people", that simply isn't t…

And now you need to patch the containers, too!

Re: Move Back to Datacenters

#79
post #19

The conclusion completely handwaves the massive overheads that come with not only owning your own infrastructure, but also having to manage a complex platform stack and its security. The security points, which the main points hinges on, seem hyperfocused and in most cases misguided. Redshift is not internet facing. If your Redshift is internet facing, you've messed up somewhere. The CPU attack example given is for AM…

In general when people mention "cost" as an issue for cloud, they're narrowly comparing what a CPU core or GB of RAM or disk costs on paper between cloud and datacenter/onprem. This completely misses the business case for cloud, and is why these sorts of analyses don't convince that many business folk, and why cloud use continues to grow fast. The business case is that someone else is doing the infrastructure managem…

Do the math and think again.

Infra engineers needed to run your "managed" cloud stack cost much more than generic sysadmins for commodity open-source on-prem solutions.

Re: Move Back to Datacenters

#80
post #27

The conclusion completely handwaves the massive overheads that come with not only owning your own infrastructure, but also having to manage a complex platform stack and its security. The security points, which the main points hinges on, seem hyperfocused and in most cases misguided. Redshift is not internet facing. If your Redshift is internet facing, you've messed up somewhere. The CPU attack example given is for AM…

The other thing they keep missing. Where are the people with the skills to do this work? I wrote a whole post about it, but it is not like people with that knowledge exist in troves just waiting to be hired. There are no path to learn this stuff either. Do we really expect every single company out there to build teams of sysadmin and operators from a non existing pool of talents? It is easy to say that people can mov…

AWS and Azure is a much more niche skill set.

E.g., there's probably three orders of magnitude more engineers familiar with Postgres out there in the world than with DynamoDB.

Post reply on HN