The Servers Are Burning • Hacker News Board

Re: The Servers Are Burning

#91

Post by auganov » Thu, Sep 06, 2018, 10:19 PM UTC

mreome wrote: post #20

noelwelsh wrote: post #10

How could such a tiny change have such an outsized impact on the site? “That same story happened so many different times,” my old boss David told me. “Someone launched a small, relatively innocuous change that did one of the millions of unexpected things it could have done, which then happened to break some part of the site, and then bring it all down—sometimes bring it down to the point where we couldn’t recover it…

While I can understand some of the benefits of functional programming within a single sub-system, or within academic research, I struggle to understand how pure functional programming can address the need for the large amounts of state information and abstraction layers required for a large scale system. I work with systems where (large/numerous) sub components require very specialized domain knowledge to understand…

Before you spend some time doing mostly functional it's easy to ignore just how much meaningless state you might have floating around. You're right that any non-trivial endeavour will necessitate quite a lot of information being kept around. But in my experience most of the state you produce in languages that don't support functional patterns isn't even related to the domain. You're often forced to put thing into variables, objects and all sorts of things that might not make a lot of sense.

Re: The Servers Are Burning

#92

Post by zbentley » Thu, Sep 06, 2018, 10:20 PM UTC

bunderbunder wrote: post #44

> in order to write effective tests, a programmer had to know all of the ways that a piece of software could fail in order to write tests for those cases No. In order to write effective tests, a programmer has to think of the piece of software's entire input domain, carve it up into a set of equivalence classes, and then determine what the expected behavior should be for a piece of input from each of those classes. W…

> In order to write effective tests, a programmer has to think of the piece of software's entire input domain, carve it up into a set of equivalence classes, and then determine what the expected behavior should be for a piece of input from each of those classes.

OK, so let's revise the statement to "in order to write effective tests, you must know how your software should behave in the face of all kinds of inputs".

In my mind, that's an equivalent, or nearly equivalent* proposition. It's still above and beyond the capabilities or available resources of people developing very complex software. That's not excusing anyone. Just saying that your correction isn't that different from the original statement.

* Sure, "someone unplugged the computer while it was running my code" is a failure mode that's not a function of (what most people would consider to be) the input domain. Genuinely external failures like that, while damaging, are rare. What you usually end up defending against are the downstream results of those failures (i.e. "a network partition meant that half of my input dataset was null pointers").

Re: The Servers Are Burning

#93

Post by T2_t2 » Thu, Sep 06, 2018, 10:33 PM UTC

blacksmith_tb wrote: post #12

danesparza wrote: post #5

Earlier quoted context omitted.

The OP is about technical process in the software world. I'm not sure how a company's valuation comes into the conversation...

I assume the implication is "if OKC can fly by the seat of their pants and still make millions, anyone could!" Which is true, but still not advisable...

I think the implication is that they achieved the goal they had, to grow a company. Note that goal doesn't mention code at all.

It depends what you think software engineering is - a means to an end, e.g. to grow a company, or as a craft you want to perfect and do for your whole career.

I think there are more craft coders, because most jobs are at companies that have survived and there is more to lose from errors than gain from moving fast. The craft attitude may not be ideal when you want to go from zero to something, and avoid going back to zero very quickly. Or, perhaps more importantly, if you want to go from zero -> something wrong -> something wrong again -> something almost correct -> Product Market fit.

Re: The Servers Are Burning

#94

Post by wdewind » Thu, Sep 06, 2018, 10:46 PM UTC

bunderbunder wrote: post #44

> in order to write effective tests, a programmer had to know all of the ways that a piece of software could fail in order to write tests for those cases No. In order to write effective tests, a programmer has to think of the piece of software's entire input domain, carve it up into a set of equivalence classes, and then determine what the expected behavior should be for a piece of input from each of those classes. W…

> has to think of the piece of software's entire input domain

If you're a startup this is not clear and changes frequently. Practicing TDD in this environment has very real downsides that exist less in mature companies with clearer understanding of business requirements.

Re: The Servers Are Burning

#95

Post by [deleted] » Thu, Sep 06, 2018, 11:07 PM UTC

spamizbad wrote: post #87

gred wrote: post #75

> My first employer, the online dating site OkCupid, didn’t harp on testing either. In part, this was because our company was so small. Only seven fellow engineers and I maintained all the code running on our servers—and making tests work was time-consuming and error-prone. “We can’t sacrifice forward momentum for technical debt,” then-CEO Mike Maxim told me, referring to the cost of engineers building behind-the-sce…

This is a smart way to think of testing and will discover bugs when your understanding of the code is the most lucid. But it does require some nuance. Unfortunately, some people think writing tests or test harnesses are wasteful because they think of shops that insist on 90%+ coverage and have test suites that are easily 5-10x the size of the application's codebase.... so they decide to forgo them completely. When in…

[deleted]

Re: The Servers Are Burning

#96

Post by chrisweekly » Thu, Sep 06, 2018, 11:09 PM UTC

bunderbunder wrote: post #78

pjc50 wrote: post #67

Earlier quoted context omitted.

> think of the piece of software's entire input domain, carve it up into a set of equivalence classes, and then determine what the expected behavior should be for a piece of input from each of those classes This is an extremely good way of putting it. Doing some of this at the requirements stage if possible can be very helpful too, because you can detect nonsensical or conflicting requirements before you even write a…

I'm rather fond of a book, Specification by Example by Gojko Adzic, that's partially a manifesto for, and partially a guide to, doing this.

Thanks for the rec! Just bought for $5 (kindle ed).

Re: The Servers Are Burning

#97

Post by chrisweekly » Thu, Sep 06, 2018, 11:14 PM UTC

kbenson wrote: post #57

fhood wrote: post #40

Earlier quoted context omitted.

You can't possibly say that with any confidence. I literally just finished reproducing a memory leak caused by an insane combination of circumstances. This was not something valgrind or any similar tool would find. It was not something that any rational human being would have thought to write tests for. Once code is no longer synchronous, and tasks are being juggled around and swapped between, it becomes virtually im…

Tests only catch what you test for. In this case "memory leak" can be replaced with "site functions as expected", and the same argument you made applies, and yet it's still useful . Just because you can't determine the exact cause of the leak, or even detect all of them, doesn't mean that having some indication of memory usage initially and under test loads of different (reproducible) amounts at different intervals a…

> "Tests only catch what you test for."

Er, sort of. Integration and e2e tests often catch problems nobody specifically, explicitly envisioned. That's part of what makes them so valuable! Right?

https://m.youtube.com/watch?v=0GypdsJulKE

Re: The Servers Are Burning

#98

Post by user5994461 » Thu, Sep 06, 2018, 11:18 PM UTC

jhayward wrote: post #33

jschwartzi wrote: post #27

Earlier quoted context omitted.

> However, the moral of the story was essentially: "software is so complicated that it is bound it break, so you have to be good at fixing it". While that is certainly true, I think that developers have a responsibility to use whatever tools they can to write high-quality software. I actually disagree that we have a responsibility to write high-quality software. As engineers our job is to create software that is good…

> As engineers our job is to create software that is good enough for the task at hand This is the essence of engineering. As the saying goes, "Any fool can build a bridge that won't fall down. It takes an engineer to build a bridge that just barely won't fall down." In other words an engineer's contribution is not perfection, it is being able to know the difference between "good enough" and "needs more work" and to a…

Every bridge will fall down, it's just a matter of time.

Re: The Servers Are Burning

#99

Post by shady-lady » Thu, Sep 06, 2018, 11:22 PM UTC

chrisweekly wrote: post #96

bunderbunder wrote: post #78

Earlier quoted context omitted.

I'm rather fond of a book, Specification by Example by Gojko Adzic, that's partially a manifesto for, and partially a guide to, doing this.

Thanks for the rec! Just bought for $5 (kindle ed).

what site? i can only see manning for $36. not on amazons.

Re: The Servers Are Burning

#100

Post by Sophistifunk » Thu, Sep 06, 2018, 11:48 PM UTC

TeMPOraL wrote: post #63

jerkstate wrote: post #53

Earlier quoted context omitted.

I'm not a great programmer, I can't usually write code that does exactly what I expect the first time, so I use unit tests with a debugger in my IDE to run small portions of my new code until it works the way I envisioned it before I started writing it. This is a lot faster than my old method of writing code that doesn't quite work, and running the whole program over and over with small changes and print statements e…

> so I use unit tests with a debugger in my IDE to run small portions of my new code until it works the way I envisioned it before I started writing it. In languages like Lisp, this is what you use REPL for. It's an insanely more efficient way of working than the usual edit->recompile->run the whole app again. However, couple of years working with Common Lisp and (recently) Clojure taught me that, even with a good RE…

REPLs are a great tool for exploring your ideas, and poking your software to test it "right now" but once you're happy with a small piece of code, nothing beats a suite of unit tests to help you refactor with confidence as your implementation gets a bit too hairy and you need to rejigger it in order to progress in a direction you didn't see coming.