Live data from Hacker News

Redis explained

architecturenotes.co

21–30 of 104 posts

Re: Redis explained

#21
post #4

Really love this style of writing. Pairing the diagrams/illustrations with the easy to grok copy is really helpful for folks like myself who have been mainly focused on the front-end. What tool do you use for your diagramming, is it all hand-drawn?

Its hand drawn with some fonts for the titles.

Re: Redis explained

#22

> Send 1KB over a 1GBps network This is said to have a 10μs latency in the chart. But I'm fairly sure that is a calculation of bandwidth based on 1KB / 1GBps 10μs is about 3Km, so at most a 1.5Km round-trip. For a chart labelled latency, I'm surprised to see bandwidth calculations included. Any network hop would actually have far greater latency, if nothing else because communication typically involves more than a si…

Fair point! Will update! I think the focus was on pure line latency. Check out the more detailed post here. https://gist.github.com/jboner/2841832

Re: Redis explained

#23

> Send 1KB over a 1GBps network This is said to have a 10μs latency in the chart. But I'm fairly sure that is a calculation of bandwidth based on 1KB / 1GBps 10μs is about 3Km, so at most a 1.5Km round-trip. For a chart labelled latency, I'm surprised to see bandwidth calculations included. Any network hop would actually have far greater latency, if nothing else because communication typically involves more than a si…

Distance is of course a factor, but at fixed distance size matters a lot, and most applications are at more or less a fixed latency.

Re: Redis explained

#24
post #19
post #8

I'm not too familiar with redis and this may well help, so thank you. I see some data-types on the right. It surprises me that redis doesn't have a numeric data type. I understand that at its heart it is just a key-value store and doesn't ever need to do range-based lookup but it still surprises me. One consequence of "everything is a string" I've run into (although probably a sign I'm "doing it wrong"), is serialisa…

Numbers in redis can be natively represented using BITFIELDS. > BITFIELD player:1:stats SET u32 #0 1000 1) (integer) 0 > BITFIELD player:1:stats INCRBY u32 #0 -900 1) (integer) 100 > BITFIELD player:1:stats GET u32 #0 1) (integer) 100

OK, that's helpful thank you.

That said, all the keys themselves are still strings and therefore you can't have a SET of numbers or bitfields.

Re: Redis explained

#25
post #8

I'm not too familiar with redis and this may well help, so thank you. I see some data-types on the right. It surprises me that redis doesn't have a numeric data type. I understand that at its heart it is just a key-value store and doesn't ever need to do range-based lookup but it still surprises me. One consequence of "everything is a string" I've run into (although probably a sign I'm "doing it wrong"), is serialisa…

How would a native number type avoid some serialization overhead that using e.g. 4 byte BE keys yourself must pay?

Re: Redis explained

#26
post #8

I'm not too familiar with redis and this may well help, so thank you. I see some data-types on the right. It surprises me that redis doesn't have a numeric data type. I understand that at its heart it is just a key-value store and doesn't ever need to do range-based lookup but it still surprises me. One consequence of "everything is a string" I've run into (although probably a sign I'm "doing it wrong"), is serialisa…

[deleted]

Re: Redis explained

#27
I think a few more concrete use cases would help.

First, a key limitation that every architect should pay attention. Redis reaches the limits of what you can do in well-written single-threaded C. One of those limits is that you really, really, *really* don't want to go outside of RAM. Think about what is stored, and be sure not to waste space. (It is surprisingly easy to leak memory.)

Second, another use case. Replication in Redis is cheap. If your data is small and latency is a concern (eg happened to me with an adserver), then you can locate read-only Redis replicas everywhere. The speed of querying off of your local machine is not to be underestimated.

And third, it is worth spending time mastering Redis data structures. For example suppose you have a dynamic leaderboard for an active game. A Redis sorted set will happily let you instantly display any page of that leaderboard, live, with 10 million players and tens of thousands of updates per second. There are a lot of features like that which will be just perfect for the right scenario.

Re: Redis explained

#28
post #14
post #9

Earlier quoted context omitted.

It's a new article so it's relatively easy to explain: HN automatically combines submissions so that subsequent submissions count as upvotes for the first submission. If a popular source posts a new article, users will "rush" to post it to HN to reap that sweet karma and the winner will "catch" the upvotes of the others.

That could explain it. Thanks! Is this behavior documented anywhere on news.ycombinator.com?

There isn't a ton of documentation per-se about HN behavior. There is:

https://news.ycombinator.com/newsguidelines.html

and

https://news.ycombinator.com/newsfaq.html

and a handful of posts by dang, sama, pg, etc. over the course of the years. most of the rest is what long-time users have just figured out through observation. There's a Git repo[1] out there that aggregates a lot of that stuff, but keep in mind that it's technically unofficial. That said, I think most of what's there is widely considered to be correct.

[1]: https://github.com/minimaxir/hacker-news-undocumented

Re: Redis explained

#29
post #14

Earlier quoted context omitted.

That could explain it. Thanks! Is this behavior documented anywhere on news.ycombinator.com?

There isn't a ton of documentation per-se about HN behavior. There is: https://news.ycombinator.com/newsguidelines.html and https://news.ycombinator.com/newsfaq.html and a handful of posts by dang, sama, pg, etc. over the course of the years. most of the rest is what long-time users have just figured out through observation. There's a Git repo[1] out there that aggregates a lot of that stuff, but keep in mind that it…

Thanks, that's a good summary of what I've seen referenced throughout my years here.

I can't find any reference to something like "combine the scores of new submissions of the same URL to the first submission's score" though. I guess that's either new information or incorrect.

Re: Redis explained

#30
post #27

I think a few more concrete use cases would help. First, a key limitation that every architect should pay attention. Redis reaches the limits of what you can do in well-written single-threaded C. One of those limits is that you really, really , *really* don't want to go outside of RAM. Think about what is stored, and be sure not to waste space. (It is surprisingly easy to leak memory.) Second, another use case. Repli…

> One of those limits is that you really, really, really don't want to go outside of RAM. Think about what is stored, and be sure not to waste space. (It is surprisingly easy to leak memory.)

You can have massive amounts of RAM these days. You’re sooner to hit big-O limits from bad architectural decisions than run out of memory. If you do get to that point you likely have enough value in your usage to justify scaling out further and sharding.

> And third, it is worth spending time mastering Redis data structures.

Bingo. The true secret to properly using Redis: understanding the big-O complexity of each operation (…and ensuring that none of your interactions are more than logarithmic).

Post reply on HN