Also the SSDs are funny to me because I recently had someone who was supposed to be an expert say that SSDs were a fatal mistake with a high likelihood of massive failure.
StackOverflow Update: 560M Pageviews a Month, 25 Servers
261–270 of 278 posts
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#262StackOverflow shows just how powerful using a fast language can be. Compare the TechEmpower benchmarks for, say, Java vs Rails on a very simple JSON serialization benchmark (we can assume .Net would achieve comparable performance to the JVM): http://www.techempower.com/benchmarks/ The Java servers get around 900K requests/s on beefy hardware while Rails squeezes out 6K. That's a 150x difference! Any real application…
> StackOverflow shows just how powerful using a fast language can be. Compare the TechEmpower benchmarks [...] (we can assume .Net would achieve comparable performance to the JVM) Except those TechEmpower benchmarks show .NET is not nearly as fast as Java. I think StackExchange prove that the platform is NOT the most important: it's much more important to make performance a priority in all engineering decisions, to b…
To more accurately judge the request routing performance of C# tests on Windows, see Round 8 data. For example, see Round 8 limited to Java, C#, and Ruby tests on i7 hardware [1].
Other important notes on that data:
* The http-listener test is a low-level test implementation.
* The rack-jruby test is running on Torqbox (TorqueBox 4), which is based on Undertow and has very little Ruby code [2]. This test is mostly about Undertow/Torqbox, with a bit of JRuby.
Another interesting test is Fortunes on i7 in Round 8 [3]. Fortunes involves request routing, a database query, ORM translation of multiple rows, dynamic addition of rows, in-memory sorting, server-side templating, XSS countermeasures, and UTF-8 encoding. Here you will see Java frameworks at about 45K rps, ASP.NET on Windows at about 15K rps, and Ruby at about 2.5K rps.
[1] http://www.techempower.com/benchmarks/#section=data-r8&hw=i7...
[2] https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast...
[3] http://www.techempower.com/benchmarks/#section=data-r8&hw=i7...
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#263Earlier quoted context omitted.
The don't have servers with that much RAM, but a 244GB RAM server with 8x800GB SSDs and 32 cores costs <$5k a month ad-hoc and about $1300/month with a 3 year commitment.
$1300/month w/ a 3 year commitment is $46.8k over the lifespan. Which is more then 4x more then the actual hardware cost. Your ROI on a physical server with the same stats at that price tag would be 8 months.
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#264> Redis has 2 slaves, SQL has 2 replicas, tag engine has 3 nodes, elastic has 3 nodes - any other service has high availability as well (and exists in both data centers) Does this mean the 2 Redis slaves, 2 SQL replicates etc. are in a secondary data center?
We have 2 SQL Clusters. In the primary data center (New York) there is usually 1 master and 1 read-only replica in each cluster. There's also 1 read-only replica (async) in the DR data center (Oregon). If we're running in Oregon then the primary is there and both of the New York replicas are read-only and async. For redis both data centers have 2 servers, the chain would look like this typically: ny-redis01 - ny-redi…
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#265Earlier quoted context omitted.
It can also be caused by too much embrace of polyglot tools. "We use the best tools for the job" can easily mean an architecture with Redis, Memcached, Mongo, MySQL, Angular, JQuery, Node, and several dozen third-party libraries, all glued together with Thrift or JSON-RPC. If all you're doing is a messaging app, you might be much better off with a single in-process Java app. Usually you want your solution to be just…
I wish more people considered this alternative when recommending the usage of queues etc. I always remember of this post: http://williamedwardscoder.tumblr.com/post/18065079081/cogs-...
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#266Earlier quoted context omitted.
The don't have servers with that much RAM, but a 244GB RAM server with 8x800GB SSDs and 32 cores costs <$5k a month ad-hoc and about $1300/month with a 3 year commitment.
$1300/month w/ a 3 year commitment is $46.8k over the lifespan. Which is more then 4x more then the actual hardware cost. Your ROI on a physical server with the same stats at that price tag would be 8 months.
Still, I agree re. the difference in price. I'd buy too
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#267Anyone else not quite expecting StackExchange to be using a Microsoft stack?
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#268StackOverflow shows just how powerful using a fast language can be. Compare the TechEmpower benchmarks for, say, Java vs Rails on a very simple JSON serialization benchmark (we can assume .Net would achieve comparable performance to the JVM): http://www.techempower.com/benchmarks/ The Java servers get around 900K requests/s on beefy hardware while Rails squeezes out 6K. That's a 150x difference! Any real application…
Java servers get around 900K requests/s [...] while Rails squeezes out 6K
Yes, but that's for the /json test, which serves a static object that the framework converts to json.I'm not sure it is really representative of the overall speed you'll get with whatever framework you'll be using: how often do you have a static page that's best served by your framework instead of leaving nginx serve it directly?
I prefer watching the test results that actually involve the database (/db, /queries, /fortunes and /updates), as it shows less raw speed for serving static things and more overall speed for your dynamic pages.
With /queries for example, Java does ~11.3K, while php (on hhvm) does ~10.7K, Python is at ~7.7K and Dart ~12.8K (For the fastest framework of each language. Rails still does bad though).
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#269StackOverflow shows just how powerful using a fast language can be. Compare the TechEmpower benchmarks for, say, Java vs Rails on a very simple JSON serialization benchmark (we can assume .Net would achieve comparable performance to the JVM): http://www.techempower.com/benchmarks/ The Java servers get around 900K requests/s on beefy hardware while Rails squeezes out 6K. That's a 150x difference! Any real application…
Java servers get around 900K requests/s [...] while Rails squeezes out 6K Yes, but that's for the /json test, which serves a static object that the framework converts to json. I'm not sure it is really representative of the overall speed you'll get with whatever framework you'll be using: how often do you have a static page that's best served by your framework instead of leaving nginx serve it directly? I prefer watc…
I prefer the /json test because it gives me a performance ceiling. I know that's the max performance I can expect, and it's up to me to design my system to get as close to that as I can.
On the other hand the db tests don't really tell me much. Does my data and data access patterns match theirs? Probably not. So it is difficult to generalise from their results. If the systems you build are always stateless web servers talking to a relational db then I can see it might be more useful.
Re: StackOverflow Update: 560M Pageviews a Month, 25 Servers
#270Earlier quoted context omitted.
IME that always happens when you try to perform tdd in combination with Javaesque encapsulation. The good solution to the problem is to not be so afraid of classes seeing each others internals. The bad solution is to add, factory patterns, dependency injectors and other useless layers just to try and keep your design both well encapsulated and testable.
You do realize that most efficient DI framework only inject the dependency once. Separating Controller, Repository, and Services are good practices as well and let's be honest, we're looking at 3 methods layer at most. Here's what happened in Java: 1. When you deploy your WAR, the DI will inject necessary component _once_ (and these components only instantiated _once_ for the whole web-app so there you go, Singleton…