Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Servers Are Burning

logicmag.io

Post reply on HN
Print view View original

81–90 of 158 posts

Re: The Servers Are Burning

#81

Post by scns »

mreome wrote: post #20
noelwelsh wrote: post #10

How could such a tiny change have such an outsized impact on the site? “That same story happened so many different times,” my old boss David told me. “Someone launched a small, relatively innocuous change that did one of the millions of unexpected things it could have done, which then happened to break some part of the site, and then bring it all down—sometimes bring it down to the point where we couldn’t recover it…

While I can understand some of the benefits of functional programming within a single sub-system, or within academic research, I struggle to understand how pure functional programming can address the need for the large amounts of state information and abstraction layers required for a large scale system. I work with systems where (large/numerous) sub components require very specialized domain knowledge to understand…

Maybe the blogpost "Functional architecture is ports and adapters" by Mark Seemann may be of help to you. "fsharp for fun and profit" by Scott Wlashin is really approachable. Cant post links since on mobile.

Re: The Servers Are Burning

#82

Post by nradov »

megaman8 wrote: post #58
JackFr wrote: post #2

Wow. 1. We don't test. 2. We don't code review (or rather if we do, we do it so poorly swallowed exceptions don't raise red flags.) That's an outrageously unprofessional software process.

You have to judge it in context. If it's entirely a senior team then the value of code reviews is greatly diminished. You rarely catch a bug in a bug review when you're reviewing a Senior software engineer's code. those bugs aren't caught until you do testing.

I've seen plenty of defects in senior developers' code that were caught in reviews. Please the purpose of code reviews isn't just to catch defects, it's also to disseminate knowledge of that module across the team in order to reduce the "bus factor".

Re: The Servers Are Burning

#83

Post by tluyben2 »

JackFr wrote: post #2

Wow. 1. We don't test. 2. We don't code review (or rather if we do, we do it so poorly swallowed exceptions don't raise red flags.) That's an outrageously unprofessional software process.

In my experience this ‘outrageously unprofessional‘ process is very normal. Most companies are not software companies and they are not going to pay for whatever is not adding to the bottomline. And even in software companies that ofcourse happens. It is unfortunate but the shock and awe are a bit over the top; when I talk with partners (ctos is large corps; banks, insurance, retail), they generally glaze over and mumble about their dreams of doing tests and code reviews but unfortunately it is not possible ‘at the moment’.

Re: The Servers Are Burning

#84

Post by toss1 »

gred wrote: post #75

> My first employer, the online dating site OkCupid, didn’t harp on testing either. In part, this was because our company was so small. Only seven fellow engineers and I maintained all the code running on our servers—and making tests work was time-consuming and error-prone. “We can’t sacrifice forward momentum for technical debt,” then-CEO Mike Maxim told me, referring to the cost of engineers building behind-the-sce…

Yup, classic.

"...no time/resources to do it right in the first place, but plenty of time/resources to fix it when the customers complain..."

I always thought it better to find the bugs in-house before shipping, but so many others don't see it...

Re: The Servers Are Burning

#85

Post by 3pt14159 »

adrianN wrote: post #46
3pt14159 wrote: post #41

I appreciated this article because I so infrequently get to hear from people that hold this view of testing, which I do not share. To me, the importance of tests are a function of the consequence of failure and the likelihood that failure will happen. It's a question of hazard and risk. One thing to realize is that code that lasts longer is more likely to fail because the people and libraries that support it are pron…

Without automated deploys, how do you know that what you deploy is actually what you tested?

Personally I have automated deploys, I was just reaching for an example that was available and relevant for a wide range of projects from inception. I think the thinking is that early on everything is on master so you just do the deploy by hand, run the migrations / tests, and bring the server back online.

Re: The Servers Are Burning

#86

Post by 3pt14159 »

ozim wrote: post #65
3pt14159 wrote: post #41

I appreciated this article because I so infrequently get to hear from people that hold this view of testing, which I do not share. To me, the importance of tests are a function of the consequence of failure and the likelihood that failure will happen. It's a question of hazard and risk. One thing to realize is that code that lasts longer is more likely to fail because the people and libraries that support it are pron…

I think just SSHing to box and doing deploy is not function of users as you noticed but function of how many devs are working on project. For 1-2 devs manual deploy might be ok, when 3rd comes in, it is time to get rid of any magic that might happen when deploying by hand. Because you know someone will do something special and not document that, when he updates automated deploy he does not have to document or talk wi…

I agree. Function of devs that have deploy power.

Re: The Servers Are Burning

#87

Post by spamizbad »

gred wrote: post #75

> My first employer, the online dating site OkCupid, didn’t harp on testing either. In part, this was because our company was so small. Only seven fellow engineers and I maintained all the code running on our servers—and making tests work was time-consuming and error-prone. “We can’t sacrifice forward momentum for technical debt,” then-CEO Mike Maxim told me, referring to the cost of engineers building behind-the-sce…

This is a smart way to think of testing and will discover bugs when your understanding of the code is the most lucid.

But it does require some nuance. Unfortunately, some people think writing tests or test harnesses are wasteful because they think of shops that insist on 90%+ coverage and have test suites that are easily 5-10x the size of the application's codebase.... so they decide to forgo them completely. When in reality, the answer is probably something like: you have at least 25% of your codebase that would greatly benefit from testing, even if they're just "smoke tests"

Re: The Servers Are Burning

#88

Post by skybrian »

bunderbunder wrote: post #44

> in order to write effective tests, a programmer had to know all of the ways that a piece of software could fail in order to write tests for those cases No. In order to write effective tests, a programmer has to think of the piece of software's entire input domain, carve it up into a set of equivalence classes, and then determine what the expected behavior should be for a piece of input from each of those classes. W…

These are great techniques and should be done more. But I'll add a caveat: they don't cover all the ways software can fail and we probably shouldn't pretend they do.

The environment in which the software runs is also an input, and you often don't get to pick that. Consider new browser releases, browser addons, and browser bugs.

Performance is a global property of a system that crosses most abstraction boundaries. Outside hard real-time systems, we get few performance guarantees.

Some domains are simple on one level and very complicated on another. The set of all Still, being able to decompose problems and treat bugs as either functionality or performance (and not a complicated mixture of both) is pretty helpful.

Re: The Servers Are Burning

#89

Post by bunderbunder »

noelwelsh wrote: post #10

How could such a tiny change have such an outsized impact on the site? “That same story happened so many different times,” my old boss David told me. “Someone launched a small, relatively innocuous change that did one of the millions of unexpected things it could have done, which then happened to break some part of the site, and then bring it all down—sometimes bring it down to the point where we couldn’t recover it…

By definition, pure functions can't read from or write to disk, can't read from or write to the database, and can't read from or write to a socket.

If you aren't able to do any of those things, there is no "at scale" - you're trapped in a single process on a single computer, spinning away but not able to communicate with anyone else.

Pure functional programming helps within an individual component by letting you quarantine the statefulness at defined areas (e.g., in Haskell, at the program's entry point), but it doesn't let you make it disappear. Sooner or later, you're gonna have to deal with it.

At scale, you've got tens or hundreds or thousands of different processes, and each of them has its own statefulness that it ultimately has to deal with. When you're looking at the big picture like that, I'm not sure it makes sense to worry about whether it's been quarantined, Haskell-style, or not. It's still there, and, in the big picture, it's still distributed more-or-less homogeneously throughout the system. Even a process that doesn't use disk or talk to the DB or anything like that is still not referentially transparent, because it's still subject to side effects by virtue of the network being unreliable, the OOM killer being unpredictable, and all that fun stuff.

Re: The Servers Are Burning

#90

Post by rhizome »

adrianN wrote: post #46
3pt14159 wrote: post #41

I appreciated this article because I so infrequently get to hear from people that hold this view of testing, which I do not share. To me, the importance of tests are a function of the consequence of failure and the likelihood that failure will happen. It's a question of hazard and risk. One thing to realize is that code that lasts longer is more likely to fail because the people and libraries that support it are pron…

Without automated deploys, how do you know that what you deploy is actually what you tested?

Use tags?
Post reply on HN