There is a market opportunity here for serving customers who would rather risk broken generators but ensure constant power.
At Scale, Rare Events Aren’t Rare
51–59 of 59 posts
Re: At Scale, Rare Events Aren’t Rare
#52As a pretty good rule of thumb, a system that fails 1/nth of the time and has n opportunities to fail has ~.63 probability of failure, where n is more than ~10. Graph: http://www.meta-calculator.com/online/?panel-102-graph&data-...
Very nice rule of thumb, honestly I did not expect it to (sort of) converge to ~63%. Does anyone have some intuition for this?
Is this related to the fuel fraction of a rocket that can accelerate to its own exhaust velocity?
Re: At Scale, Rare Events Aren’t Rare
#53Earlier quoted context omitted.
I think by "intuition" the GP meant "for the non-mathematicians" :P
It's always amusing when someone asks for a layman/non-math/intuitive reason why something works out and HN responds with a 3-paragraph long proof that seems to always require university-level math. And it seems those comments almost invariably start with "Oh, you just..."
'pedrosorio gave a nice one upthread[0].
Ultimately, it's hard to give a math-free explanation for something that comes out straight from math. If you break down an explanation into small enough steps, they should be comprehensible for anyone even if they have to take some steps on faith.
--
Re: At Scale, Rare Events Aren’t Rare
#54Earlier quoted context omitted.
It's always amusing when someone asks for a layman/non-math/intuitive reason why something works out and HN responds with a 3-paragraph long proof that seems to always require university-level math. And it seems those comments almost invariably start with "Oh, you just..."
'pedrosorio gave a nice one upthread[0]. Ultimately, it's hard to give a math-free explanation for something that comes out straight from math. If you break down an explanation into small enough steps, they should be comprehensible for anyone even if they have to take some steps on faith. -- [0] - https://news.ycombinator.com/item?id=14040434
He did, yes, I was just amused by the GP's answer!
Re: At Scale, Rare Events Aren’t Rare
#55The part about the switchgear vendor deciding to do something a certain way that the customer didn't want because it can cause a rare failure reminded me of something that happened to me. Way back I bought a 1500VA UPS that was not an APC but still a known brand to protect my home server. The decision was based on cost as it was significantly less money.
One night I was near the server when the power went out. So I sat there waiting to see the auto shutdown. Soon enough the UPS told the server to shutdown and it was well on it's way to power off. Just before it shut off the power came back. And the UPS stopped beeping and went back to normal operation... while the server completed shutdown.
And know I have a server that's off and if I wasn't around I would have not known what happened. When I got the UPS I just did one test to make sure the server shut off and the UPS shut itself off without draining its batteries. This meant that I plugged it back in AFTER the UPS powered off. I never considered that the manufacturer of the UPS would botch the power restored after telling the server to shut down sequence.
I contacted the manufacturer about this. I told them that after telling the server to shut down there was only a brief window where a power restored signal would maybe abort shutdown. Once the UPS monitoring program is terminated during shutdown there's no turning back. Nothing came of that.
So now I buy only APC gear. They do the proper thing that if the AC power comes back after a shutdown command is issued the UPS will continue the shutdown sequence. And when the UPS shuts off it sees the power back on and restarts itself and the server comes back online.
Other manufacturers may do it correctly and the one I dealt with might have clued in and fixed it but I'm not willing to gamble anymore.
Re: At Scale, Rare Events Aren’t Rare
#56In an infinite universe, nothing is rare, by that logic?
Rare is relative, so just because something happens an trillion times it can still be nearly nonexistently rare, given the data sample is trillions of trillions?
Seems like a silly thought..
Re: At Scale, Rare Events Aren’t Rare
#57"You know, the most amazing thing happened to me tonight. I saw a car with the license plate ARW 357. Can you imagine? Of all the millions of license plates in the state, what was the chance that I would see that particular one tonight? Amazing!" -- Richard Feynman
Re: At Scale, Rare Events Aren’t Rare
#58i would have chosen a slightly different title, perhaps "at scale, improbable events are frequent" or something like that
"at scale" of course just means more much frequent sampling; it's not some sort of alternate reality where good become evil, rare becomes not rare, etc
Re: At Scale, Rare Events Aren’t Rare
#59Earlier quoted context omitted.
There was essentially instability and chaos in the big dev heavy divisions at MS when they all worked in one branch, but that led very rapidly to more sophisticated models with more points of validation between the average dev and the common code/builds that qa and everyone else shared and used.
The common pattern nowadays is to code review each commit, build each commit on Jenkins & pass tests, git bisect, etc... What was the procedure back then ?
Do you mean before or after? Before, it was "do your best", which was never enough, not with everyone partying in the same branch. After, it was a matter of breaking things up into different feeder branches down to individual teams. Code that goes "up" into more mainline branches is required to have a higher QC bar in order to get in, such as a full clean build of the entire set of sources (which for something like windows, office, or visual studio / .net could take a very long time) and running the entire set of build verification tests with zero failures (also a long process). Code would go up during an integration window, there would inevitably be some instability that would need cleaned up (test failures, maybe code breaks sometimes) which would get stabilized, then code would flow back "down" to individual teams from those good builds. And code would flow up further into other branches that would be shared and used by more teams around the division/company.
Most dev groups these days use different systems because they tend not to have such monolithic projects, they can leverage automation better, and it's rare to have literally thousands of devs all working on the same software. When you can only get a few builds out per day, or maybe only one, then you have to become a lot more careful at keeping a separation between devs coding away at their desktops and the builds that everyone else depends on.