Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis explained

architecturenotes.co

Post reply on HN
Print view View original

81–90 of 104 posts

Re: Redis explained

#81

Post by btilly »

anonymousDan wrote: post #75
btilly wrote: post #59

Earlier quoted context omitted.

No. Replication time was measured in hundredths of a second, and Redis operations are atomic. So all queries got a consistent view of the data, and the lag to update was very reasonable.

So in other words, potentially yes since there is some lag :)?

For that application, there really wasn't. The results of the read were not used for writes, and the latency from when information was published to available was on par with the time a request to the master would have taken. The time from data published to available was faster than the time to switch tabs in a browser and manually check.

But your requirements will depend on the application. Financial transactions need explicit locking logic and atomic operations. Such as is provided by SELECT ... FOR UPDATE in SQL. So another application could have more demanding requirements. Which is why, in addition to answering whether I encountered problems, I gave the actual performance characteristics. So that anyone planning their application can know whether this is a good enough solution for you.

Re: Redis explained

#82

Post by sidlls »

jbboehr wrote: post #64
btilly wrote: post #33

Earlier quoted context omitted.

You can have massive amounts of RAM these days. You’re sooner to hit big-O limits from bad architectural decisions than run out of memory. If you do get to that point you likely have enough value in your usage to justify scaling out further and sharding. Absolute disagreement. It is very easily to accidentally leak a few hundred MB per week in a busy Redis system. The code will look and work fine...at first. It is co…

You can also "leak" rows in a traditional RDBMS or even a filesystem. Why is this particular notable for Redis?

Redis starts to have issues at high scale, even on sophisticated hardware, that can be quite difficult to debug without a lot of additional effort and storage. It’s not just memory, but odd behavior (e.g. randomly dropped connections) with a lot of connected clients, or hot keys/nodes in a cluster configuration, etc.

These issues can exist in any system, but in my experience it’s especially tough (relatively) to identify and diagnose them with Redis. Once you add lua script usage it can get even worse.

Re: Redis explained

#83

Post by wokwokwok »

LewisVerstappen wrote: post #41
btilly wrote: post #27

I think a few more concrete use cases would help. First, a key limitation that every architect should pay attention. Redis reaches the limits of what you can do in well-written single-threaded C. One of those limits is that you really, really , *really* don't want to go outside of RAM. Think about what is stored, and be sure not to waste space. (It is surprisingly easy to leak memory.) Second, another use case. Repli…

> Replication in Redis is cheap. If your data is small and latency is a concern (eg happened to me with an adserver), then you can locate read-only Redis replicas everywhere. The speed of querying off of your local machine is not to be underestimated. Do you face any consistency issues with doing this?

It’s not a daft question, because redis clusters do have consistently issues (https://redis.io/docs/manual/scaling/#redis-cluster-consiste...), but for simple replication it is not really an issue.

Re: Redis explained

#84

Post by YetAnotherNick »

btilly wrote: post #59
LewisVerstappen wrote: post #41

Earlier quoted context omitted.

> Replication in Redis is cheap. If your data is small and latency is a concern (eg happened to me with an adserver), then you can locate read-only Redis replicas everywhere. The speed of querying off of your local machine is not to be underestimated. Do you face any consistency issues with doing this?

No. Replication time was measured in hundredths of a second, and Redis operations are atomic. So all queries got a consistent view of the data, and the lag to update was very reasonable.

It depends on definition of consistency, but it is not strongly consistent in theoretical terms[0]. But the ordering of update is guaranteed to be same, so if master is guaranteed to be internally consistent, so is the replica. And that property is enough for almost all usecases, except for maybe transactions.

[0]: https://redis.io/docs/manual/scaling/#:~:text=Redis%20Cluste....

Re: Redis explained

#85

Post by tomnipotent »

itake wrote: post #70
tomnipotent wrote: post #54

Earlier quoted context omitted.

Redis is single-threaded and will have no problem saturating a 10G NIC with a single socket.

My concern is how fast it takes a CPU to scan through all of that memory.

What "scanning"? That's not how memory access works in a K/V store, and Redis does very little work that demands much of the CPU.

Re: Redis explained

#86

Post by bcjordan »

tpmx wrote: post #55
bcjordan wrote: post #38

Earlier quoted context omitted.

I've been using UpStash's serverless Redis offering and it's worked super well for my needs. Scales to zero/free which was nice for getting started, and using their http SDK didn't need to worry about concurrent connection limits when calling from simultaneous cloud functions. & not a second of downtime in the few months I've used it so far. Want to move more of my app's datastore to Redis now that I've learned more…

Quoted post unavailable.

To be clear, I am not affiliated with that web service other than as a now happy paying user. I only replied with my experience getting started running Redis on it since that was GP's question and I found it useful while first learning Redis and now in production.

Re: Redis explained

#87

Post by bcjordan »

tpmx wrote: post #55
bcjordan wrote: post #38

Earlier quoted context omitted.

I've been using UpStash's serverless Redis offering and it's worked super well for my needs. Scales to zero/free which was nice for getting started, and using their http SDK didn't need to worry about concurrent connection limits when calling from simultaneous cloud functions. & not a second of downtime in the few months I've used it so far. Want to move more of my app's datastore to Redis now that I've learned more…

Quoted post unavailable.

BTW, would recommend reviewing the HN comment etiquette guidelines at https://news.ycombinator.com/newsguidelines.html

In particular:

> Assume good faith.

> Be kind. Don't be snarky.

Re: Redis explained

#88

Post by adra »

YetAnotherNick wrote: post #84
btilly wrote: post #59

Earlier quoted context omitted.

No. Replication time was measured in hundredths of a second, and Redis operations are atomic. So all queries got a consistent view of the data, and the lag to update was very reasonable.

It depends on definition of consistency, but it is not strongly consistent in theoretical terms[0]. But the ordering of update is guaranteed to be same, so if master is guaranteed to be internally consistent, so is the replica. And that property is enough for almost all usecases, except for maybe transactions. [0]: https://redis.io/docs/manual/scaling/#:~:text=Redis%20Cluste... .

During partions you may as well throw the play book away. You could have minority writes on both sides of the cluster and a big nadda to reconcile the two when they're mended. Redis is a great ssytem for what its built for and for the trade-offs that it makes to keep itself fast and lean. Redis is not CP and it will probably never care to support it. If data resiliency and correctness is important to you, Redis alone isn't sufficient. Several years ago, we tried sentinels mostly to avoid large costly rebuilds when an instance went down, and though it usually worked just fine, we certainly had single network disruptions large enough to throw off the cluster enough that required a manual rebuild.

Re: Redis explained

#90

Post by halotrope »

rfrey wrote: post #67

A question so noob I'm almost shy to ask it: The simplest scenario in the article is a single Redis instance residing on the same machine as the application. What's the benefit to this versus just storing data directly within the application?

Sometimes your application has multiple instances even on the same machine. Scripting languages like Node or Python don‘t share memory. With Redis they can share state in a high-performance manner.
Post reply on HN