StackOverflow Update: 560M Pageviews a Month, 25 Servers • Hacker News Board

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#131

Post by electrum » Mon, Jul 21, 2014, 10:22 PM UTC

shurcooL wrote: post #74

kbenson wrote: post #33

> The cost of inefficient code can be higher than you think. Efficient code stretches hardware further, reduces power usage, makes code easier for programmers to understand. I'm curious what the reasoning is for "Efficient code ... makes code easier for programmers to understand". To my mind, efficient code (in this case, I assume coding to the hardware, as they mention elsewhere), has many benefits, but making it ea…

Sometimes "efficient" and "easy to understand and be sure is correct" don't have to be mutually exclusive; see this example [1] of Java and Go. Note that the code is autogenerated, so it should be equally efficient. The Go version also happens to be very simple and no different than most humans would write by hand (without trying very hard to optimize). [1] https://gist.github.com/shurcooL/9f94bbd021b4693cf584

The generated Java code is just bad. You can simplify it by adding imports and removing the redundant variable: https://gist.github.com/electrum/d2f7668f2a7eb95420a9

Of course, as others have mentioned, it is validating the UTF-8 but the Go version is not.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#132

Post by buckbova » Mon, Jul 21, 2014, 10:23 PM UTC

habosa wrote: post #5

> With their SQL Servers loaded with 384 GB of RAM and 2TB of SSD, AWS would cost a fortune. I have next to zero experience with server administration, but 384GB seems like a lot to me. Is that common for production servers for popular web services? Do you need a customized OS to address that much memory? Seems like you'd really need to beef up the cache hierarchy make 0.38TB of RAM fast.

SQL Server only reads data from memory. When something is requested it is first looked for in RAM (or virtual) and if not found is copied from disk to RAM.

The more RAM you have the better without exception here.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#133

Post by guiomie » Mon, Jul 21, 2014, 10:29 PM UTC

Can someone help me understand what they mean by: "Garbage collection driven programming. SO goes to great lengths to reduce garbage collection costs, skipping practices like TDD, avoiding layers of abstraction, and using static methods. While extreme, the result is highly performing code."

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#134

Post by ck2 » Mon, Jul 21, 2014, 10:29 PM UTC

This is the fascinating part to me, their SSD have not failed:

Failures have not been a problem, even with hundreds of intel 2.5" SSDs in production, a single one hasn’t failed yet. One or more spare parts are kept for each model, but multiple drive failure hasn't been a concern.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#135

Post by louthy » Mon, Jul 21, 2014, 10:33 PM UTC

guiomie wrote: post #133

Can someone help me understand what they mean by: "Garbage collection driven programming. SO goes to great lengths to reduce garbage collection costs, skipping practices like TDD, avoiding layers of abstraction, and using static methods. While extreme, the result is highly performing code."

I assume it means using C# more as a functional language than an OO language, so that most garbage collection is of short-lifetime objects and therefore cheaper to collect (generation 0).

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#136

Post by mtdewcmu » Mon, Jul 21, 2014, 10:42 PM UTC

guiomie wrote: post #133

Can someone help me understand what they mean by: "Garbage collection driven programming. SO goes to great lengths to reduce garbage collection costs, skipping practices like TDD, avoiding layers of abstraction, and using static methods. While extreme, the result is highly performing code."

That sounds like a roundabout way of saying procedural.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#137

Post by ugh123 » Mon, Jul 21, 2014, 10:55 PM UTC

daigoba66 wrote: post #7

lemcoe9 wrote: post #3

Makes me wonder why smaller website teams need dozens of engineers to keep their infinitesimally smaller app running.

I personally think it's due to overly complex applications and systems with many moving (and breaking) parts. Also StackOverflow often practices "scale-up" instead of "scale-out".

Also due to political managers that want to increase their head-count to give the impression they have more power.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#138

Post by samstave » Mon, Jul 21, 2014, 11:02 PM UTC

AlisdairO wrote: post #105

Stack Overflow is the example I'm forever using when arguing against premature scale-out. For non-trivial applications scale-out has substantial complexity costs attached to it, and the overwhelming majority of applications will never truly need it . It's frustrating to see time wasted obsessing over trying to maintain eventual consistency (or chasing bugs when you failed to do so) on systems that could quite happily…

>For non-trivial applications scale-out has substantial complexity costs attached to it

Forgie me if I am misunderstanding you - but non-trivial applications can actually require scale out.

From my perspective, StackExchange is not techinically that complex. They have built a very efficient, cost-effective and performant stack for their singular application and that works very well for them, but the complexity of their forum is not an extraordinarily complex problem.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#139

Post by mschuster91 » Mon, Jul 21, 2014, 11:14 PM UTC

duderific wrote: post #98

tiernano wrote: post #68

Earlier quoted context omitted.

Stack Exchange dont own their CDN. If you really start couting external servers, you would be adding DNS, possible client side proxy servers, client side network infrastructure, etc... these are things that OTHER people do, not SO.

I would say though, CDN doesn't fall in the same category as DNS. Serving static assets off other networks does take a huge load off the SE infrastructure and should probably be mentioned in the article.

Only in terms of bandwidth, though. The CPU load by processing is the interesting figure, not side noise by images/js/css.

Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers

#140

Post by mtdewcmu » Mon, Jul 21, 2014, 11:22 PM UTC

abritishguy wrote: post #75

itry wrote: post #42

Earlier quoted context omitted.

users=db.get("SELECT name FROM users WHERE group='admin'") is more efficient and understandable then: users=[]; tmp=db.get("SELECT name,group FROM users") for (i=0;i

But who in their right mind would do the second one? Obviously you can make code both less efficient and less readable. But starting with code that a competent programmer has written, I find it seldom makes it more readable when you make it more efficient.

>>But who in their right mind would do the second one?

Surely you jest. I don't know who, but their handiwork is everywhere.