Live data from Hacker News

Best practices can slow your application down

stackoverflow.blog

61–70 of 187 posts

Re: Best practices can slow your application down

#61
post #20

Somewhere down the line, developers will start to realize that single applications running in the same server memory and CPU cache will run much faster than Micro services applications that makes api calls over the network. Depending on your organization scale of course, but for many smaller shops, there is a significant performance overhead of going over the network. Ie a Single binary running Rust/Go/Java/NodeJs vs…

I like RDS because I don't have to think about backups, replication etc but if I colocate the app server (web server, application code) and the database server on the same machine I see 1-2 orders of magnitude performance improvement in database-intensive apps.

Replication is the same as usual; have another machine setup to replicate onto. The restriction is you can't fan out hundreds of application servers and half a dozen database servers if you need that - but I don't need to.

I haven't done this with clients but I'm doing it with my own (cost-sensitive) side projects and appreciate removing the internal network latency.

Re: Best practices can slow your application down

#62
post #19

Earlier quoted context omitted.

I think that the “we can go quicker by not testing” is a fallacy. If you do TDD right it will make you quicker, not slower in the short term. Not just the long term...

Emphasis on " I think ". Unfortunately, your thinking hasn't had a whole lot of support, as studies have found both findings in favor of and arguments against TDD, both short term and long term. Furthermore, I would ask you to name the contexts in which you believe TDD to be better in the short term. Things differ when a lot of critical and non-trivial data-modifying code exists compared to an interface which changes…

I wouldn't have thought "testing is always good" would be a controversial claim! I could make the claim that dogs shouldn't eat potting soil and most folk would probably agree with me. I don't have any proof that it's actually bad though. My dog gobbled up some potting soil today and he will probably be just fine. I still think he's a doofus, though.

I personally write significantly higher quality code, faster, when I spend a few hours thinking through the problem in a document beforehand. One of the things to think about is how to best test it, and most of the time that means a unit test. Knowing that when starting to write the code influences the interfaces and structures that I come up with.

Anecdotally, one recent project at work required writing a relatively small, 100% self-contained program with significantly less serious / more silly requirements than the typical type of code I write. I initially didn't plan on writing any unit tests, and about 500 lines of code later I had the program working 99% correctly. There was one bit of overtly complex code that was a bottleneck, though, and it made the overall program 30x too slow for its purpose. The fix was to replace a conceptually simple iterative solver with some brain-melting closed form equations. About 8 hours of iteration and manual testing later, I was pretty sure I had correctly handled all the edge cases. My brain was melted, though, and I couldn't even remember half of the edge cases I had encountered. It took me another 20 minutes to write a unit test that compared the output of the iterative solver with the output of the closed-form solver. Had I done that from the beginning, I probably would have saved 4 hours and not melted my brain.

The thing with writing code and testing is that, if you're in the habit of it, unit tests typically take a less-than-linear amount of effort vs. writing the actual code. Once you have tests, they decouple code complexity from development and maintenance, and you can therefore achieve a more sophisticated overall solution than otherwise with the same resources. If you don't develop testing as a skill, though, then those benefits don't materialize because you're inefficient at it. So, while it may be true that a given project or team wouldn't benefit from prioritizing test coverage, it's seemingly due to lack of expertise rather than a lack of obvious benefit. If I'm working on a home improvement project and my toddler wants to help tighten a screw, I wouldn't hand her my impact driver and expect her to successfully use it let alone not hurt herself, despite it being an unambiguously more efficient tool for driving screws than her little green plastic hammer.

Re: Best practices can slow your application down

#63
I get a bit annoyed by this sort of post, blanket statements like:

"With our code base, you won’t be able to easily do test driven development or similar practices that the industry seems to love."

Why does the industry seem to love testing? Does the industry love testing? Does this mean you do or don't love testing?

It also feels like they wrote this blog post to show off: we write our c# code as if it was c code because we are better than all of you.

The fact is that every application is different, they chose to use a language that back in 2000's wasn't designed for performance and so they paid the price of using it for performance and didn't want to pay the scale out price.

Is their codebase testable, yes completely. Will adding unit tests to their code destroy their performance? no, there are always ways to make it testable and fast. Maybe in a few years we will see a "We thought testing was impossible but we made it work and also keep our codebase fast".

Re: Best practices can slow your application down

#65
post #59

These sound like some poor excuses to me. The idea that you can have either best practices or performance is a false dichotomy. Testing can be used to help track down performance issues. It can help you write code faster and prevent breakages. It should not slow you down. Best practices inside a codebase should not sacrifice performance. If you have to do that to conform to some structural ideal, you don’t do it. I w…

> The idea that you can have either best practices or performance is a false dichotomy. Testing can be used to help track down performance issues. It can help you write code faster and prevent breakages. It should not slow you down.

It orks the other way too. Benchmarking can show that your bottleneck is virtcalls/boxing where you have used an interface field instead of a static instance to a concrete dependency. The argument isn't "performance vs. tests" it's really "performance vs modularization". You can always move the tests upwards until you test the whole monolith. This isn't a common tradeoff but I also wouldn't dismiss it as something that never occurs and "best practices" wrt. modularization and unit testing always enables similar performance. It doesn't.

Re: Best practices can slow your application down

#66
I think the problem with this article and their approach will be more visible when super star developers start to leave the company. If they are all the same people from the very beginning, yes, throw lots of practices out of the window because people know the codebase very well. But once they are out, there will be a mountain of tech-debt. They even try to make it testable.

Re: Best practices can slow your application down

#67
post #31

One thing that stood out to me was they wouldn't need necessarily need to optimize for performance if they didn't choose a stack whose costs increase massively as you scale. In many applications things like having a testable, easy-to-read, well-structured codebase can trump performance, and it's often worth sacrificing a bit of performance if it means it's easier to refactor and build new features.

I don't know why they went for SQL Server, .NET has had good support for other databases even back to 2009. I imagine that was a huge amount of their licensing cost.

With .NET Core running so well on Linux now they would have no Windows Server licenses to worry about too. The first production version of .NET core was released in 2016.

Even back in 2009 it was possible to run web apps on mono on linux too, I know a lot of shops around that era that used windows for more complex parts of the app but then used mono for 'simple' high volume parts to save $$$ on windows licensing.

Re: Best practices can slow your application down

#68

I get a bit annoyed by this sort of post, blanket statements like: "With our code base, you won’t be able to easily do test driven development or similar practices that the industry seems to love." Why does the industry seem to love testing? Does the industry love testing? Does this mean you do or don't love testing? It also feels like they wrote this blog post to show off: we write our c# code as if it was c code be…

Adding test coverage might indeed destroy their performance, in that they might expose where their system's bottlenecks are and realize that they've been focusing on the wrong things. When it comes to performance optimizations, folks' intuitions are almost always wrong. It's important to actually test and measure stuff like that.

I think it's true in general that modern CPUs are insanely fast at executing code within a single process, and context switches are orders of magnitude slower due to their overhead. So, monolithic systems are going to be intrinsically "fast" compared to systems that pass messages through an operating system. They seemingly made a good decision for their project there, whether or not they made that decision rationally.

It's possible they invested a lot in infrastructure for runtime performance profiling and monitoring and therefore know exactly what pieces of code to optimize. Reading the article, though, they're seemingly sweating the overhead of polymorphic function calls. The overhead of instrumenting the code would likely be higher than the overhead of polymorphism in a language like C#, so if their code was actually that performance sensitive, they wouldn't have it instrumented in production. If they did have it instrumented, they would likely have realized that their code wouldn't be bottlenecked by a little bit of function call indirection. Also, all the talented folk I know that stress the importance of runtime instrumentation also stress the importance of regression testing, so I suspect it would be pretty rare for a team to value one but not the other.

Re: Best practices can slow your application down

#69

I get a bit annoyed by this sort of post, blanket statements like: "With our code base, you won’t be able to easily do test driven development or similar practices that the industry seems to love." Why does the industry seem to love testing? Does the industry love testing? Does this mean you do or don't love testing? It also feels like they wrote this blog post to show off: we write our c# code as if it was c code be…

I also found the "or similar practices that the industry seems to love" coming across as a bit vague and even arrogant.

On the other hand, I think that they have earned bragging rights - we're talking about a site that has massive scale and reach, and has a reputation for being a very efficient system. I appreciate that people like them share their views, even if they are contrarian and a touch arrogant.

> they chose to use a language that back in 2000's wasn't designed for performance and so they paid the price of using it for performance and didn't want to pay the scale out price.

In terms of more performant languages at the time what could have they picked? C++? It sounds like they would've lost a lot of the conveniences that a framework like .net gave them. Isn't the fact that they avoided paying the scale out price remarkable?

Re: Best practices can slow your application down

#70
post #45
post #20

Somewhere down the line, developers will start to realize that single applications running in the same server memory and CPU cache will run much faster than Micro services applications that makes api calls over the network. Depending on your organization scale of course, but for many smaller shops, there is a significant performance overhead of going over the network. Ie a Single binary running Rust/Go/Java/NodeJs vs…

Architectures and scalability, although related, are two different conversations. You could build a monolith with an internal messaging queue, this obviously limits scaling possibilities horizontally but may be able to scale vertically. If you externalize the queue and use something like Rabbit, Redis, or NATS then you have more options but you also have more/new problems. Not all things need to scale, and not all th…

Scaling vertically is problematic in the cloud where it’s very common to have instances and even entire regions come and go at any time.

It’s one of the big reasons why horizontally architected applications have thrived in the cloud era.

Post reply on HN