Live data from Hacker News

Learn how to design large-scale systems

github.com

71–80 of 199 posts

Re: Learn how to design large-scale systems

#71

Earlier quoted context omitted.

A single box?! Do we know its specs?

As of 2015: https://news.ycombinator.com/item?id=9222006 FreeBSD 10.1 Nginx 1.7.10 Racket 6.1.1 with some HN and FreeBSD specific patches. 2x 3GHz Intel Xeon-IvyBridge (E5-2690-V2-DecaCore) 8x16GB Kingston 16GB DDR3 2Rx4 SuperMicro X9DRI-LN4F+_R1.2A Adaptec 71605 9x 1000GB Western Digital WD RE4 2x 200GB Smart XceedIOPS SSD

A minor upgrade these days:

  FreeBSD 11.1-RELEASE-p4
  CPU: Intel(R) Xeon(R) CPU E5-2637 v4 @ 3.50GHz

Re: Learn how to design large-scale systems

#73

Oh interesting, I have never seen Anki ( https://apps.ankiweb.net/ ) being used for large blocks of source code. Anki is an open source application (desktop + mobile) for spaced repetition learning (aka flashcards). It's a very popular tool among people who want to learn languages (and basically anything else you want to remember). There are many shared decks ( https://ankiweb.net/shared/decks/ ). Creating and format…

I've been experimenting with Anki recently. I loaded up a deck of popular fonts, with the goal being to memorize them to the point where I could recognize them in the real world. Each card contains the sentence, "The quick brown fox jumped over the lazy dog," and you have to identify what font it's written in.

The first day was really tough; I missed cards over and over again. 20 new cards (the default) is probably too many for this type of study. But I kept at it, once per day, and today (a week later) I can recognize nearly every font in the deck, and the ones that I have trouble with are very similar to other fonts (which is a useful thing to know in its own right; you can start to group fonts into "families" with a common ancestor). Pretty cool!

There's just one problem: so far, this hasn't translated into any ability to recognize fonts in the real world. I can think of a few reasons why. First, there are a LOT of fonts out there; even the "most popular" ones don't show up all that frequently. This is especially true for business logos, which like to use unusual fonts to make themselves stand out. Secondly, I think studying by memorizing a single sentence has caused me to "overfit!"

For example, there's a font that I can instantly recognize (Minion Pro) by how the 'T' and 'h' look together at the start of the sentence. I don't pay attention to anything else about the font, because that single feature is enough to distinguish it from the rest of the deck. And this turns out to be true for most fonts: Today Sans has a funny-looking 'w', Syntax has a funny-looking 'x', etc. So if I see a logo written in Today Sans, but it doesn't contain a 'w', I can't recognize it! Similarly, because the cards only contain the one sentence, which is entirely lowercase except for the 'T', I can't identify any fonts from an uppercase writing sample. What I can do is say, "Hmm, I don't know what that font is, but it definitely has a lot in common with Myriad..." and then I look it up and find out that the actual font is Warnock, which was designed by the same guy (Robert Slimbach) who designed Myriad.

So yeah, Anki is pretty cool, but an unintended side effect is that it can give you a striking sense of how a classification algorithm "feels" from the inside. :)

Re: Learn how to design large-scale systems

#74

Note that HN, a top-1000 site in the US, runs on a single box via a single racket process. "The key to performance is elegance, not battalions of special cases."

> runs on a single box via a single racket process Incomplete. They are fronted by one of the largest CDNs in the world, on whom they rely for most traffic. Well, I actually don't know the numbers, but I do know that for popular posts HN admins (a) try to break the conversation up over multiple posts and (b) plead with users to log out so as to allow Cloudflare to handle the load. The biggest lesson HN teaches for de…

[deleted]

Re: Learn how to design large-scale systems

#75
I hoped this would help me with this problem I have - I'm coding a web app with a smallish database (<1GB for the next few years, <1% writes). I need low latencies for accessing it. And I would like to have multiple servers over the world sharing the database.

Re: Learn how to design large-scale systems

#76

Note that HN, a top-1000 site in the US, runs on a single box via a single racket process. "The key to performance is elegance, not battalions of special cases."

Heh, elegance like "There is a story on the front page getting lots of attention, please log out so we can serve you from cache."

This is elegance.

Balancing between user experience and cost.

Re: Learn how to design large-scale systems

#77

Oh interesting, I have never seen Anki ( https://apps.ankiweb.net/ ) being used for large blocks of source code. Anki is an open source application (desktop + mobile) for spaced repetition learning (aka flashcards). It's a very popular tool among people who want to learn languages (and basically anything else you want to remember). There are many shared decks ( https://ankiweb.net/shared/decks/ ). Creating and format…

Michael Nielsen just this month published an essay on using spaced repetition (and especially Anki) to augment long-term memory. It seems to be part of his work with YC Research.

I highly recommend it if this kind of thing interests you. It gives a solid overview of the theory and practical lessons from his daily use of Anki over the last few years:

http://augmentingcognition.com/ltm.html

Re: Learn how to design large-scale systems

#78

Note that HN, a top-1000 site in the US, runs on a single box via a single racket process. "The key to performance is elegance, not battalions of special cases."

HN is only a top-1000 for the US, not the world and it barely makes it into the top-1000 for the US.

When HN breaks into the top-100 for the world, let us know what it runs on.

Re: Learn how to design large-scale systems

#80
post #70

Note that HN, a top-1000 site in the US, runs on a single box via a single racket process. "The key to performance is elegance, not battalions of special cases."

I am running top-1000 site in one of EU countries on one 4 core machine with 20-30% load. Around 1000 http/https reqs/s. Most of those requests do couple of postgres reqs (read and write) and couple of redis reqs. Elasticsearch - for searching/recommendations Redis - hot data (certain data is only kept in redis) Postgres - for the rest of data Clickhouse - analytics Most of the system is written in Go. Whole system w…

are you using any specific Go web framework or just the standard library?
Post reply on HN