681–690 of 870 posts
Earlier quoted context omitted.
Jesus. One would think you'd have some safeguards for that. Even Dropbox will give you an alert if you try to nuke over 1,000 files. More reasons to COLOR CODE your work environments, if possible.
Yes but that was eons ago. The safeguards are well and truly in-place now. Not just one, several in fact.
Earlier quoted context omitted.
I remember how one of our engineers had his docker daemon connected to production instead of his local one and casually did a docker rm -f $(docker ps -aq) . Same thing happened to me but with CI, which felt bad enough already.
No Engineer should have production access from their workstation. Period. source: am Engineer =).
Earlier quoted context omitted.
For a second I legit thought Google banned my account for some reason. And I don't like it that I feel relieved..
Underrated comment. If the government freezes your bank account, at least you can take them to court. If Google mistakenly disables your account, there does not seem to be any legal recourse. Considering the extent to which people depend on their services, perhaps there should be a more elaborate appeal process.
At this point the only reason I use it is because I grandfathered in on an old plan it's still free, if that changes I'll go elsewhere.
Earlier quoted context omitted.
> I do not envy Google engineers right now. A few years ago I released a bug in production that prevented users from logging into our desktop app. It affected about ~1k users before we found out and rolled back the release. I still remember a very cold feeling in my belly, barely could sleep that night. It is difficult to imagine what the people responsible for this are feeling right now.
Same. At AWS, I once took an entire AZ down of a public-facing production service (with a mis-typed command), but that was nothing compared to when I accidentally deleted an entire region via internal console (too many browser tabs). Thank goodness turned out to be unused / unlaunched, non-production stack. I felt horrible for hours despite zero impact (in both the cases).
That's even more surprising to me.
Earlier quoted context omitted.
why?
In this example: you get free days. Which depending on your business might be worthless if you have suffered more monetary loss due to the downtime than the free days are worth.
Earlier quoted context omitted.
Oh man, you're right. Bloated gmail loaded instantly. What's going on? It's loading almost 2x to 3x faster.
I always thought gmail being slow is because of me using firefox but now it's surprisingly snappy. What the hell is going on?
Earlier quoted context omitted.
Nope, especially considering the implications of this, with the amount of people working remotely. Google Meet, Classroom, etc. are down. This is probably literally costing billions every minute just in loss of productivity.
Total world economic output is ~$150M / minute, so billions every minute is off by few orders of magnitude.
Consider an exactly one minute outage that affects multiple things I use for work.
First, I may not immediately recognize that the outage is actually with some single service provider. If several things are out I'm probably going to suspect it is something on my end, or maybe with my ISP. I might spend several minutes thoroughly checking that possibility out, before noticing that whatever it was seems to have been resolved.
Second, even if I immediately recognize it for what it is and immediately notice when it ends it might take me several minutes to get back to where I was. Not everything is designed to automatically and transparently recover from disruptions, and so I might have had things in progress when the outage stuck that will need manual cleanup and restarting.
World economic output is - $150mn/minute
Current world population is 7.8bn.
Google has 4.8bn users. That's approx 61% of the population.
Let's assume about 50% of the users were impacted = 2.4bn (which is 30% of the population).
So, the loss could have been about $50mn/minute.
This is without taking SLA into consideration. There will be losses incurred on that too, wouldn't it?
Earlier quoted context omitted.
People would probably argue a “multi cloud” solution. Have your infrastructure be “cloud agnostic” and this sort of problem would be avoided. There was actually a project called “spinnaker” that was supposed to solve this problem. Whether the cost of paying 2 or more cloud providers is worth it for most companies is up in the air.
"Multi-cloud" only works if you stick with the basics. Like disk storage, compute, and a well-supported database. Once you tie in into a cloud's specific offerings.....
https://aws.amazon.com/hybrid https://azure.microsoft.com/en-us/services/azure-arc https://cloud.google.com/anthos
Full disclosure: I work for Azure. Don't work on Arc tho. Don't have experience being a customer for these products