Oh, "just" run Kubernetes. Gotcha.
Move Back to Datacenters
61–70 of 97 posts
Re: Move Back to Datacenters
#62I'd say the attack surface on rolling your own tech in a datacenter is substantially higher. Yes DynamoDB has an API, but I'd wager a AWS engineer with good security skills has looked at it carefully. Do you have an equally skilled security expert on hand to look at the datacenters stack and then same for whatever you're deploying on it? Not all internet exposure is equal. Moving out of cloud often makes sense, but s…
> Yes DynamoDB has an API, but I'd wager a AWS engineer with good security skills has looked at it carefully. Do you have an equally skilled security expert on hand to look at the datacenters stack and then same for whatever you're deploying on it? Is that AWS security engineer looking specifically at how your systems interact with DynamoDB? And how you have specifically configured your DynamoDB instances? And someon…
Again, not all configuration is equal. In hosted services a lot of the decisions have been made for you, that reduces customizability and flexibility but also footguns.
>They’re using software from other companies which also have excellent security engineers that have looked into how you can safely deploy and use their software.
Of course. Even with well designed software the risk is much higher for the end user though. Deploying a cloud load balancer is pretty idiot proof. If I do the same via local nginx...configuration files, local firewall, file permission, maybe my distro repackaged it, did I set up the user right, am I updating it regularly, what about my underlying operating system security etc. And on top of that it's a once off configuration...it isn't battle tested by thousands running the same load balancer.
It's not that it can't be done, but you do pay the price in complexity and engineering hours.
>And how you have specifically configured your DynamoDB instances?
Well yeah - I'd expect them to have thought through the possible permutations in reasonable detail, which is feasible given the limited customizability. If you're self deploying the possible combinations are endless.
It's a bit like testing on android vs iphone. You can reasonably test against the 30 or whatever common iphones, but you're never testing against the 1000s of funky android hardware combinations out there.
Re: Move Back to Datacenters
#63The analogy of this article would be: Stop using airbnbs when traveling, just build a house.
I think people overestimate the expenses related to maintaining their own infrastructure. I work for a large org that does both. We have a lot of workloads running in the public cloud, and many more on premises (for various reasons, including compliance). Having worked on these for years, building solutions, diagnosing problems and so on I'd risk saying we have reached the point where the cloud is slowly getting more…
> anybody who has maintained a large repository of CloudFormation code will tell you its not a panacea.
CloudFormation is one of the worst IaCs out there, so it's not really a good example.
One really nice model is managed Kubernetes with IaC/GitOps via tools like ArgoCD or Flux CD. This eliminates the need for things like CloudFormation, Terraform, Ansible, Puppet, OS version and patch management, etc. - most of the stuff that those tools are used to manage is replaced by the cloud provider's management of the cluster nodes. This lets you focus on the application layer, which is where the business value is. IaC in this context is pretty seamless. It also makes you much less dependent on the cloud provider's specific APIs - Kubernetes becomes your interface to all that.
Re: Move Back to Datacenters
#64Earlier quoted context omitted.
Sure they do for "serverless workloads"
I can’t find specific documentation that indicates that AWS deploys firecracker with core isolation (though I would be pretty surprised if they didn’t given their documented caution around shared-core VMs). Google uses gVisor with Linux Core Scheduling to prevent this for GKE sandbox, but I can’t find any documentation that they do the same for Cloud Functions. In any case ”don’t use serverless functions” is a very f…
That is not the only possible vulnerability.
Re: Move Back to Datacenters
#65Important point they miss is that unless your monthly spend with AWS is couple of mil a month you are nothing to them.
Re: Move Back to Datacenters
#66No. We rolled our own stack for 10 years, until 2018, well after AWS and Azure were around. We switched to Azure in 2018 and never looked back. Sure you trade security (do you really, though?) in exchange for: - not needing to head to the DC because a power supply failed and a rando who was in the cage never plugged in the redundant one - not having to be way over-capacity in scalability, or suddenly under-capacity a…
Then there are options that are probably better for you. I've been doing colocation for many years, but I've never seen the inside of the datacenter.
Remote hands set up the hardware for me and replace failed hardware. The service is so rarely needed that it's basically free.
In recent years I've started to rent the hardware I want as I want more frequent changes. I tell my host what hardware I want and they'll acquire it and install it.
If you're running very old hardware that you can't even buy replacement drives for then you are doing something very wrong.
Need to scale fast? Then I don't have to wait for servers to arrive, I use their cloud offering that's in the same network.
I agree that it's a solved problem, but the solution to anything somewhat permanent is not the cloud. It's not really any harder to run your own hardware.
Re: Move Back to Datacenters
#67The analogy of this article would be: Stop using airbnbs when traveling, just build a house.
I think people overestimate the expenses related to maintaining their own infrastructure. I work for a large org that does both. We have a lot of workloads running in the public cloud, and many more on premises (for various reasons, including compliance). Having worked on these for years, building solutions, diagnosing problems and so on I'd risk saying we have reached the point where the cloud is slowly getting more…
People 100 % overestimate the complexity and cost of running infrastructure on their own.
Re: Move Back to Datacenters
#68With IAM you can restrict all dynamodb endpoint access to a VPCE/private link. But the insecurity of the public facing endpoints is vastly overstated even without using VPCE. The rest is a bunch of FUD - I spent years going through these points with some of the worlds best security teams to secure some of the most systemically important workloads. These arguments are fairly tired. I’ll tackle another one - speculativ…
> worlds best security teams
> the most systemically important workloads
Big claims require big evidence. How did you end up with these superlatives?
Re: Move Back to Datacenters
#69The conclusion completely handwaves the massive overheads that come with not only owning your own infrastructure, but also having to manage a complex platform stack and its security. The security points, which the main points hinges on, seem hyperfocused and in most cases misguided. Redshift is not internet facing. If your Redshift is internet facing, you've messed up somewhere. The CPU attack example given is for AM…
In general when people mention "cost" as an issue for cloud, they're narrowly comparing what a CPU core or GB of RAM or disk costs on paper between cloud and datacenter/onprem. This completely misses the business case for cloud, and is why these sorts of analyses don't convince that many business folk, and why cloud use continues to grow fast. The business case is that someone else is doing the infrastructure managem…
> It's sort of like saying, instead of taking cars from your corporate fleet to a repair shop when they break, repair them yourself! Just hire some mechanics, build or rent a garage and equipment, and voila! And also, why are you buying electricity from the electric company anyway? Just build a generator next to your office building! Maybe make it nuclear just like some of the big electric companies!
I think this is an unfair analogy.
It would be more akin to saying instead of hiring Ubers and taxi's and spending several tens of thousands a quarter, buy a couple of cars for the company. But instead of needing a single person to onboard people onto the company's taxi account and paying the invoice each month, you instead need someone to buy the cars, ensure they're taxed and certified, create an internal hiring system, etc.
For some companies it does make financial sense to have your own fleet of cars, in others it makes more sense to get Ubers.
It's exactly the same as cloud vs not-cloud. If you're not taking advantage of the cloud (e.g. serverless technologies) why pay that premium when a small number of dedicated machines and a simpler architecture/stack will do the job better and cheaper? They're only as difficult to maintain as complex cloud environments.
Re: Move Back to Datacenters
#70Earlier quoted context omitted.
> The business case is that someone else is doing the infrastructure management for you. Many business people believe that moving to cloud can reduce the headcount needed for managing the infra, but that is usually not what's happening. You will still need more or less the same amount of people to patch the OS and configure the networking, but with a slightly different skill set -- instead of Cisco IOS commands, they…
If you're treating cloud as a sort of 1:1 replacement of hardware infrastructure, then what you're saying makes sense. But we don't ever "patch the OS." Our cloud provider does that for us. Upgrades and patching are automatic within the maintenance window. We deploy containers, the OS is just a platform layer that the cloud provider manages. As for needing "more or less the same amount of people", that simply isn't t…
I see, so you are using PaaS and I was talking about IaaS offerings. The term cloud is so vaguely defined that it can mean so many things.
If the PaaS works for your scale and budget then it’s perfectly fine. I just want to mention that it is probably no longer the case for scaled-ups.