Note that HN, a top-1000 site in the US, runs on a single box via a single racket process. "The key to performance is elegance, not battalions of special cases."
[deleted]
Learn how to design large-scale systems
61–70 of 199 posts
Re: Learn how to design large-scale systems
#62Earlier quoted context omitted.
A single box?! Do we know its specs?
As of 2015: https://news.ycombinator.com/item?id=9222006 FreeBSD 10.1 Nginx 1.7.10 Racket 6.1.1 with some HN and FreeBSD specific patches. 2x 3GHz Intel Xeon-IvyBridge (E5-2690-V2-DecaCore) 8x16GB Kingston 16GB DDR3 2Rx4 SuperMicro X9DRI-LN4F+_R1.2A Adaptec 71605 9x 1000GB Western Digital WD RE4 2x 200GB Smart XceedIOPS SSD
This "single box" has a lot of computing units as on AWS EC2.
Re: Learn how to design large-scale systems
#63Note that HN, a top-1000 site in the US, runs on a single box via a single racket process. "The key to performance is elegance, not battalions of special cases."
What would happen if the cleaner accidentally unplugged that box? Would it failover to a backup?
I guess the advantage of not relying on your site for profit is that you can afford to not worry about a bit of down time so much...
Re: Learn how to design large-scale systems
#64Earlier quoted context omitted.
As of 2015: https://news.ycombinator.com/item?id=9222006 FreeBSD 10.1 Nginx 1.7.10 Racket 6.1.1 with some HN and FreeBSD specific patches. 2x 3GHz Intel Xeon-IvyBridge (E5-2690-V2-DecaCore) 8x16GB Kingston 16GB DDR3 2Rx4 SuperMicro X9DRI-LN4F+_R1.2A Adaptec 71605 9x 1000GB Western Digital WD RE4 2x 200GB Smart XceedIOPS SSD
This "single box" has a lot of computing units as on AWS EC2.
That should tell you a lot about the price you pay for using AWS.
Re: Learn how to design large-scale systems
#65Note that HN, a top-1000 site in the US, runs on a single box via a single racket process. "The key to performance is elegance, not battalions of special cases."
What would happen if the cleaner accidentally unplugged that box? Would it failover to a backup? I guess the advantage of not relying on your site for profit is that you can afford to not worry about a bit of down time so much...
Hacker News has been up and running for like 10 years now, isn’t it? Instead of hypothetical question, I guess we have actual data on the uptime for such question.
Re: Learn how to design large-scale systems
#66Note that HN, a top-1000 site in the US, runs on a single box via a single racket process. "The key to performance is elegance, not battalions of special cases."
What would happen if the cleaner accidentally unplugged that box? Would it failover to a backup? I guess the advantage of not relying on your site for profit is that you can afford to not worry about a bit of down time so much...
I think redundancy is much easier if the thing being made redundant is extremely simple? Imagine having to manage 50 different services (and sometimes servers) as opposed to 1 machine.
Re: Learn how to design large-scale systems
#67Note that HN, a top-1000 site in the US, runs on a single box via a single racket process. "The key to performance is elegance, not battalions of special cases."
While I agree with you, a lack of knowledge about this stuff will unfortunately not help you pass any interviews.
Disagree! Myself (and I hope most companies) will jump at the devs who build simple solutions that work.
Re: Learn how to design large-scale systems
#68I'd add a section on using TLA+ as a design tool. Diagrams and rules of thumb are useful but they don't catch errors or help you discover the correct architecture. See the Amazon paper [0] on their use of TLA+ in designing (and trouble-shooting) services. [0] https://lamport.azurewebsites.net/tla/formal-methods-amazon....
I feel TLA+ would be too much to ask in a system interview which is what this site is about. In case anybody is interested, there is a nice talk by Hillel Wayne on youtube ( https://www.youtube.com/watch?v=_9B__0S21y8 ) that provides a high-level overview on what TLA+ is about.
I feel TLA+ would be too much to ask
Re: Learn how to design large-scale systems
#69Earlier quoted context omitted.
I think it's due to the trend against hiring DBAs. Nobody wants to put so much work into their databases anymore. I feel like that was half of the NoSQL craze, let's stop thinking about how to use databases because it's a pain. After a while of slowly modifying and lumping more crap on a database, and it becomes a slow PITA that everyone is too afraid to touch, the usual result is to lift everything onto a new DB. DA…
Our DBAs loved the DAL. There is a trend today to let developers run all over production doing what they want and it has direct impact on systems. The complexity of the modern stack is ridiculous. You run java containers inside docker containers inside virtual machines and call it optimized.
I think that's a result of the systems getting more complex and development models changing over time. You can either design one monolithic behemoth that can do everything by itself, or you use so many abstractions that you can assemble a different kind of behemoth piece by piece.
Re: Learn how to design large-scale systems
#70Note that HN, a top-1000 site in the US, runs on a single box via a single racket process. "The key to performance is elegance, not battalions of special cases."
I am running top-1000 site in one of EU countries on one 4 core machine with 20-30% load. Around 1000 http/https reqs/s. Most of those requests do couple of postgres reqs (read and write) and couple of redis reqs.
Elasticsearch - for searching/recommendations
Redis - hot data (certain data is only kept in redis)
Postgres - for the rest of data
Clickhouse - analytics
Most of the system is written in Go. Whole system was tuned for performance from day one. As to latency, data from the last 21 million requests today:
p99: 17.37 ms
p95: 6.86 ms
avg: 2.37 ms