41–50 of 52 posts
I guess people really have forgotten how to run datacenters. The only people who should be shocked in this thread are the people who have been hoodwinked into thinking operations is so hard you need thousands of staff. I know AWS/GCP/Azure like to charge us as if we were hiring an army of sysadmins, but the truth is that day-to-day DC ops does not require so many people. Hardware failures are more rare than you think…
I guess people really have forgotten how to run datacenters. The only people who should be shocked in this thread are the people who have been hoodwinked into thinking operations is so hard you need thousands of staff. I know AWS/GCP/Azure like to charge us as if we were hiring an army of sysadmins, but the truth is that day-to-day DC ops does not require so many people. Hardware failures are more rare than you think…
You actually need thousands of people for operations at major cloud provider scale (and much more for development), but it scales at some point where you only need people for hands-on tasks at satellite plants and the rest sits at HQ.
I guess people really have forgotten how to run datacenters. The only people who should be shocked in this thread are the people who have been hoodwinked into thinking operations is so hard you need thousands of staff. I know AWS/GCP/Azure like to charge us as if we were hiring an army of sysadmins, but the truth is that day-to-day DC ops does not require so many people. Hardware failures are more rare than you think…
But what about the hundreds of local jobs they promise?
I guess people really have forgotten how to run datacenters. The only people who should be shocked in this thread are the people who have been hoodwinked into thinking operations is so hard you need thousands of staff. I know AWS/GCP/Azure like to charge us as if we were hiring an army of sysadmins, but the truth is that day-to-day DC ops does not require so many people. Hardware failures are more rare than you think…
The management's way of thinking is: "Well, let's just pay for the peace of mind." Except that this famous peace of mind never comes, because the cloud gets more and more complex each year and it's hard to keep up. Heck, even Amazon can't keep up: for example, officially they depreciate bucket policies but internally they are using it for example in the Cloud Formation templates for the Control Tower. But now it's to…
In this case they can't even get blamed for their vendor choice because both AWS and Azure are now so big that they're in "nobody ever got fired for buying IBM" territory.
I know very little on the datacenter operations side of things - I guess 3 people is not a lot, but what is normal? How many operations people are at say AWS US-East-1? I presume it doesn't scale with number of servers, that would not scale well. What is a 'normal' level? 10? 100? It can't be more than 100, can it?
They go on site to geographically adjacent DCs and outside that just travel onsite for special projects.
I guess people really have forgotten how to run datacenters. The only people who should be shocked in this thread are the people who have been hoodwinked into thinking operations is so hard you need thousands of staff. I know AWS/GCP/Azure like to charge us as if we were hiring an army of sysadmins, but the truth is that day-to-day DC ops does not require so many people. Hardware failures are more rare than you think…
Earlier quoted context omitted.
The management's way of thinking is: "Well, let's just pay for the peace of mind." Except that this famous peace of mind never comes, because the cloud gets more and more complex each year and it's hard to keep up. Heck, even Amazon can't keep up: for example, officially they depreciate bucket policies but internally they are using it for example in the Cloud Formation templates for the Control Tower. But now it's to…
I don't believe that S3 Bucket Policies are deprecated. They are powerful, effective, and consistent with almost everything else at AWS (Resource Policy). Perhaps you are thinking of ACLs?
Earlier quoted context omitted.
I believe the real secret reason the cloud is so popular among developers (based on 10+ years of experience) is that cloud providers are so much nicer and faster to deal with than your company IT department. Also on the price side, I'm not comparing the price of cloud vs colo, but the price of cloud vs what the company IT department charges my department for being allowed to use one of 'their' colo servers, and that…
Also the vast array of managed services. Managed databases, message queues, infinite storage, data warehouses, caches , etc etc. Many of which are very complicated to host well yourself and operationize (failovers, monitoring , backups etc) This idea that you can build a DC that competes on cost for rented cloud compute - it might be technically true but it’s mostly missing the point of why modern shops prefer the cl…
Oh you are hilarious.
Time for your daily reminder that failovers, monitoring and backups DO NOT EXIST in the cloud UNLESS (a) you configure and manage them (b) deploy your services in multiple zones (and spend $$$$$ along the way).
Lots of people cannot do (a) properly and it is regularly demonstrated by AWS US-East-1 and others that not many people do (b) fully, or in many cases, don't do it at all.
So yeah, the cloud is still "complicated", it's just a different sort of complicated. And if you do failovers, monitoring and backups properly, the cloud is still "expensive", its just a different sort of expensive.
Earlier quoted context omitted.
I doubt any amount of staffing could address lack of specialism in dealing with power or air conditioning issues, both likely involve infrequent maintenance by external vendors. 20 people blowing up the phone to a vendor doesn't fix a problem any faster than 1 etc.
Amazon goes to the extreme of putting its own custom firmware on switchgear because the choices that vendor makes in theirs doesn’t align with their objectives. I don’t think AWS is blowing up a vendors phone when something goes wrong in one of their facilities. [1] https://www.datacenterknowledge.com/archives/2017/04/07/how-...
All the same gear is still there, its control functions are just ceded to Amazon panels. And integrated to the point of even removing some PLC like devices.