Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The compiler is your best friend

blog.daniel-beskin.com

Post reply on HN
Print view View original

121–130 of 149 posts

Re: The compiler is your best friend

#121

Post by shakna »

swiftcoder wrote: post #118
shakna wrote: post #113

Earlier quoted context omitted.

Bitflips are waaay more common than you think they are. [0] > A 2011 Black Hat paper detailed an analysis where eight legitimate domains were targeted with thirty one bitsquat domains. Over the course of about seven months, 52,317 requests were made to the bitsquat domains. [0] https://en.wikipedia.org/wiki/Bitsquatting

> Bitflips are waaay more common than you think they are... Over the course of about seven months, 52,317 requests... Your data does not show them to be common - less than 1 in 100,000 computing devices seeing an issue during a 7 month test qualifies as "rare" in my book (and in fact the vast majority of those events seem to come from a small number of server failures). And we know from Google's datacenter research[0…

1 in 100,000 devices is about 1 in about 40,000 customers due to how many devices most people own.

Which means if you're about medium business or above, one of your customers will see this about once a year.

That classifies more as "inevitable" than "rare" in my book.

Re: The compiler is your best friend

#122

Post by graemep »

shakna wrote: post #113
swiftcoder wrote: post #110

Earlier quoted context omitted.

> Until you have a bit flip These are vanishingly unlikely if you mostly target consumer/server hardware. People who code for environments like satellites, or nuclear facilities, have to worry about it, sure, but it's not a realistic issue for the rest of us

Bitflips are waaay more common than you think they are. [0] > A 2011 Black Hat paper detailed an analysis where eight legitimate domains were targeted with thirty one bitsquat domains. Over the course of about seven months, 52,317 requests were made to the bitsquat domains. [0] https://en.wikipedia.org/wiki/Bitsquatting

Link to the paper: https://media.blackhat.com/bh-us-11/Dinaburg/BH_US_11_Dinabu...

Re: The compiler is your best friend

#123

Post by swiftcoder »

shakna wrote: post #121
swiftcoder wrote: post #118

Earlier quoted context omitted.

> Bitflips are waaay more common than you think they are... Over the course of about seven months, 52,317 requests... Your data does not show them to be common - less than 1 in 100,000 computing devices seeing an issue during a 7 month test qualifies as "rare" in my book (and in fact the vast majority of those events seem to come from a small number of server failures). And we know from Google's datacenter research[0…

1 in 100,000 devices is about 1 in about 40,000 customers due to how many devices most people own. Which means if you're about medium business or above, one of your customers will see this about once a year. That classifies more as "inevitable" than "rare" in my book.

> That classifies more as "inevitable" than "rare" in my book.

But also pretty much insignificant. Is any other component in your product achieving 5 9s reliability?

Re: The compiler is your best friend

#124

Post by Yoric »

cozzyd wrote: post #86
thatoneengineer wrote: post #38

Earlier quoted context omitted.

Ideally, if you can convince yourself something cannot happen, you can also convince the compiler, and get rid of the branch entirely by expressing the predicate as part of the type (or a function on the type, etc.) Language support for that varies. Rust is great, but not perfect. Typescript is surprisingly good in many cases. Enums and algebraic type systems are your friend. It'll never be 100% but it sure helps fil…

Until you have a bit flip or a silicon error. Or someone changed the floating point rounding mode.

Of course, any attempt at safety or security requires defense in depth.

But usually, any effort spent on making one layer sturdy is worth it.

Re: The compiler is your best friend

#125

Post by shakna »

swiftcoder wrote: post #123
shakna wrote: post #121

Earlier quoted context omitted.

1 in 100,000 devices is about 1 in about 40,000 customers due to how many devices most people own. Which means if you're about medium business or above, one of your customers will see this about once a year. That classifies more as "inevitable" than "rare" in my book.

> That classifies more as "inevitable" than "rare" in my book. But also pretty much insignificant. Is any other component in your product achieving 5 9s reliability?

We're not talking 5 9s, here.

> ... A new consumer grade machine with 4GiB of DRAM, will encounter 3 errors a month, even assuming the lowest estimate of 120 FIT per megabit.

The guarantees offered by our hardware suppliers today, is not "never happens" but "accounted for in software".

So, if you ignore it, and start to operate at any scale, you will start to see random irreproducible faults.

Sure, you can close all tickets as user error or unable to reproduce. But it isn't the user at fault. Account for it, and your software has less glitches than the competitor.

Re: The compiler is your best friend

#126

Post by lifeisstillgood »

There is a trade off here (of course) as in anything.

You can write the type heavy language with the nullable-type and the carefully thought through logic. Or you can use the dynamic language with the likelihood that it will crash. The issue is not “you are a bad coder, and should be guilty” but that there is a cost to a crash and a cost to moving wholesale to Haskell or perhaps more realistically to typed python, and those costs are quantifiable- and perhaps sometimes the throwaway code that has made it to production is on the right side of the cost curve.

Re: The compiler is your best friend

#127

Post by SAI_Peregrinus »

svantana wrote: post #56
LegionMammal978 wrote: post #2

> How many times did you leave a comment on some branch of code stating "this CANNOT happen" and thrown an exception? Did you ever find yourself surprised when eventually it did happen? I know I did, since then I at least add some logs even if I think I'm sure that it really cannot happen. I'm not sure what the author expects the program to do when there's an internal logic error that has no known cause and no defini…

> At some level, the simplest thing to do is to give up and crash if things are no longer sane. The problem with this attitude (that many of my co-workers espouse) is that it can have serious consequences for both the user and your business. - The user may have unsaved data - Your software may gain a reputation of being crash-prone If a valid alternative is to halt normal operations and present an alert box to the us…

- The user may have unsaved data

That should not need to be a consideration. Crashing should restore the state from just before the crash. This isn't the '90s, users shouldn't have to press "save" constantly to avoid losing data.

Re: The compiler is your best friend

#128

Post by swiftcoder »

shakna wrote: post #125
swiftcoder wrote: post #123

Earlier quoted context omitted.

> That classifies more as "inevitable" than "rare" in my book. But also pretty much insignificant. Is any other component in your product achieving 5 9s reliability?

We're not talking 5 9s, here. > ... A new consumer grade machine with 4GiB of DRAM, will encounter 3 errors a month, even assuming the lowest estimate of 120 FIT per megabit. The guarantees offered by our hardware suppliers today, is not "never happens" but "accounted for in software". So, if you ignore it, and start to operate at any scale, you will start to see random irreproducible faults. Sure, you can close all…

> We're not talking 5 9s, here.

1 in 40,000 customer devices experiencing a failure annually is considerable better than 4 9s of reliability. So we are debating whether going from 4 9s to 5 9s is worth it.

And like, sure, if the rest of your stack is sufficiently polished (and your scale is sufficiently large) that the once-a-year bit flip event becomes a meaningful problem... then by all means do something about it.

But I maintain that the vast majority of software developers will never actually reach that point, and there are a lot of lower-hanging fruit on the reliability tree

Re: The compiler is your best friend

#129

Post by addaon »

Lvl999Noob wrote: post #116
addaon wrote: post #97

Earlier quoted context omitted.

> Usually the only sensible thing to do is crash. Correct. But how are you testing that you successfully crash in this case, instead of corrupting on-disk data stores or propagating bad data? That needs a test.

You don't. Assertions are assumptions. You don't explicitly write recovery paths for individual assumptions being wrong. Even if you wanted to, you probably wouldn't have a sensible recovery in the general case (what will you do when the enum that had 3 options suddenly comes in with a value 1000?). I don't think any C programmer (where assert() is just debug_assert!() and there is no assert!()) is writing code like:…

Anyone writing with a standard that requires 100% decision-point coverage will either not write that code (because NDEBUG is insane and assert should have useful semantics), or will have to both write and test that code.

Re: The compiler is your best friend

#130

Post by bccdee »

GabrielBRAA wrote: post #9
LegionMammal978 wrote: post #2

> How many times did you leave a comment on some branch of code stating "this CANNOT happen" and thrown an exception? Did you ever find yourself surprised when eventually it did happen? I know I did, since then I at least add some logs even if I think I'm sure that it really cannot happen. I'm not sure what the author expects the program to do when there's an internal logic error that has no known cause and no defini…

Heh, recently I had to fix a bug in some code that had one of these comments. Feels like a sign of bad code or laziness. Why make a path that should not happen? I can get it when it's on some while loop that should find something to return, but on a if else sequence it feels really wrong.

It's much better to have a `panic!("this should never happen")` statement than to let your program get into an inconsistent state and then keep going. Ideally, you can use your type system to make inconsistent states impossible, but type systems can only express so much. Even Haskell can't enforce typeclass laws at the compiler level.

A program that never asserts its invariants is much more likely to be a program that breaks those invariants than a program that probably doesn't.

Post reply on HN