Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis explained

architecturenotes.co

Post reply on HN
Print view View original

91–100 of 104 posts

Re: Redis explained

#91

Post by [deleted] »

jbboehr wrote: post #64
btilly wrote: post #33

Earlier quoted context omitted.

You can have massive amounts of RAM these days. You’re sooner to hit big-O limits from bad architectural decisions than run out of memory. If you do get to that point you likely have enough value in your usage to justify scaling out further and sharding. Absolute disagreement. It is very easily to accidentally leak a few hundred MB per week in a busy Redis system. The code will look and work fine...at first. It is co…

You can also "leak" rows in a traditional RDBMS or even a filesystem. Why is this particular notable for Redis?

[deleted]

Re: Redis explained

#92

Post by btilly »

YetAnotherNick wrote: post #84
btilly wrote: post #59

Earlier quoted context omitted.

No. Replication time was measured in hundredths of a second, and Redis operations are atomic. So all queries got a consistent view of the data, and the lag to update was very reasonable.

It depends on definition of consistency, but it is not strongly consistent in theoretical terms[0]. But the ordering of update is guaranteed to be same, so if master is guaranteed to be internally consistent, so is the replica. And that property is enough for almost all usecases, except for maybe transactions. [0]: https://redis.io/docs/manual/scaling/#:~:text=Redis%20Cluste... .

I don't trust Redis clustering to actually work. I only trust the single threaded, single server. Potentially with lots of replicas.

See the following for why I don't trust their various attempts to scale to a truly distributed system.

https://aphyr.com/posts/283-jepsen-redis https://aphyr.com/posts/307-jepsen-redis-redux https://jepsen.io/analyses/redis-raft-1b3fbf6

Re: Redis explained

#93

Post by Groxx »

rfrey wrote: post #67

A question so noob I'm almost shy to ask it: The simplest scenario in the article is a single Redis instance residing on the same machine as the application. What's the benefit to this versus just storing data directly within the application?

Storing it in a different process at the very least lets you restart/deploy changes to/crash your application without losing that data. Or building your own replication/in-place upgrader/well you're just screwed if it crashes.

Re: Redis explained

#94

Post by mnutt »

tomnipotent wrote: post #85
itake wrote: post #70

Earlier quoted context omitted.

My concern is how fast it takes a CPU to scan through all of that memory.

What "scanning"? That's not how memory access works in a K/V store, and Redis does very little work that demands much of the CPU.

There are workloads that will saturate a redis instance's CPU: using it as an LRU cache, eventually you will hit the configured memory limits and adding new keys will require finding old keys to delete. Eventually it may also require redis to do memory defragmentation which can be fairly intensive.

Re: Redis explained

#95

Post by tomnipotent »

mnutt wrote: post #94
tomnipotent wrote: post #85

Earlier quoted context omitted.

What "scanning"? That's not how memory access works in a K/V store, and Redis does very little work that demands much of the CPU.

There are workloads that will saturate a redis instance's CPU: using it as an LRU cache, eventually you will hit the configured memory limits and adding new keys will require finding old keys to delete. Eventually it may also require redis to do memory defragmentation which can be fairly intensive.

> There are workloads that will saturate a redis instance's CPU

I might imagine this scenario if you're excessively using smembers and a few other slow ops, but I have yet to see CPU issues outside of bad eval's.

> require finding old keys to delete

LRU/LFU eviction is not particularly CPU intensive.

> redis to do memory defragmentation which can be fairly intensive

Active defrag has relatively negligible overhead, and assuming jemalloc even more so.

Re: Redis explained

#96

Post by lalwanivikas »

googletron wrote: post #10

My personal nightmare happened and accidentally published a rough draft! It has since been updated! Apologies!

I haven't read it, but the presentation is really beautiful! Is it possible to make it printer friendly by any chance? Default browser PDF is a mess. Long form content like this is much easier to read printed.

Re: Redis explained

#98

Post by intelVISA »

halotrope wrote: post #90
rfrey wrote: post #67

A question so noob I'm almost shy to ask it: The simplest scenario in the article is a single Redis instance residing on the same machine as the application. What's the benefit to this versus just storing data directly within the application?

Sometimes your application has multiple instances even on the same machine. Scripting languages like Node or Python don‘t share memory. With Redis they can share state in a high-performance manner.

Database-as-IPC makes me feel uneasy... even if it's kinda OK with Redis.

Re: Redis explained

#99

Post by googletron »

omarshammas wrote: post #35

Very informative and love the illustrations. I'm building a new website and am using sidekiq for background job processing which relies on redis behind the scenes to store all the job data. I configured a high availability redis instance with `maxmemory-policy noeviction` to ensure no data is lost. The website is still in its infancy so not thinking about scale for the next little while but curious if you have any ti…

I would ensure that the data size is managed if you hit the limits due to your policy Redis will stop responding to ensure the data it has isn’t lost. I would also turn on some sort of persistence for data recovery in case of catastrophic failure. Early on this is totally fine and I would setup some monitors in redis data size relative to memory and try to keep 20% overhead weird things start to happen when systems are memory constrained.

Re: Redis explained

#100

Post by mnutt »

tomnipotent wrote: post #95
mnutt wrote: post #94

Earlier quoted context omitted.

There are workloads that will saturate a redis instance's CPU: using it as an LRU cache, eventually you will hit the configured memory limits and adding new keys will require finding old keys to delete. Eventually it may also require redis to do memory defragmentation which can be fairly intensive.

> There are workloads that will saturate a redis instance's CPU I might imagine this scenario if you're excessively using smembers and a few other slow ops, but I have yet to see CPU issues outside of bad eval's. > require finding old keys to delete LRU/LFU eviction is not particularly CPU intensive. > redis to do memory defragmentation which can be fairly intensive Active defrag has relatively negligible overhead, a…

Nothing but lots of small (~100b) pipelined SETs and a small number of GETs here and there. Only 10MB/s but at 100k SETs/sec redis’s CPU core sits at 60-70%. Active defrag can easily send it into a death spiral.
Post reply on HN