I had a jr dev connect and typed 'flushall' because he thought it would refresh the dataset to disk. thankfully it was on a staging env, I think he's at google now.
Ways to shoot yourself in the foot with Redis
71–80 of 85 posts
Re: Ways to shoot yourself in the foot with Redis
#72Earlier quoted context omitted.
As the other guy says the lock is released by the server. If you don't have a mechanism to release it after a timeout, what happens if a node fails?
RedLock automatically releases a lock after a given timeout. The server can just release it early or refresh it also.
But now you have a released lock and a client that thinks they have the lock.
Re: Ways to shoot yourself in the foot with Redis
#73Earlier quoted context omitted.
Thanks, I'll update the post and link to your comment for attribution.
"Footgun" unfortunately does have some "prior art" as blaming unexpected behavior of a technology. PS Love the aesthetic of your blog!
(I also wonder if "shoot yourself in the foot" is an idiom that doesn't translate well; at least in the UK I think it's fairly well understood to put blame squarely on the person doing the shooting, rather than the firearm they happen to be holding at the time)
Re: Ways to shoot yourself in the foot with Redis
#74Change the default `stop-writes-on-bgsave-error` to "no" or you're asking for trouble... a ticking time bomb.
Finally, check out Redis-compatible alternatives that don't require the data set to fit in RAM. [0]
Re: Ways to shoot yourself in the foot with Redis
#75Earlier quoted context omitted.
This. Configure private vlans and/or Wireguard or whatever VPN software you prefer.
And what about mTLS?
Re: Ways to shoot yourself in the foot with Redis
#76I had a jr dev connect and typed 'flushall' because he thought it would refresh the dataset to disk. thankfully it was on a staging env, I think he's at google now.
You can use rename-command to help avoid these kinds of mistakes: # To disable: rename-command FLUSHALL "" # To rename: rename-command FLUSHALL DANGER_WILL_ROBINSON_FLUSH_ALL
Re: Ways to shoot yourself in the foot with Redis
#77Earlier quoted context omitted.
That doesn't make any sense. the timeout is how long to block for and retry, now how long to block for and continue.
Instance A grabs the lock and makes an API call that takes 120 seconds. Instance B sees the lock but considers it expired after the lock times out at 100 seconds. Instance B falsely concludes A died, overwrites A's lock so the system doesn't dead lock waiting for A, and makes its own request. Unfortunately, A's request was still processing and B's accidentally concurrent request cause corruption.
Re: Ways to shoot yourself in the foot with Redis
#78> One common mistake is serialising objects to JSON strings before storing them in Redis. This works for reading and writing objects as atomic units but is inefficient for reading or updating individual properties within an object I would love to see some numbers on this. My intuition says there are probably some workloads where JSON strings are better and some where one key per property is better.
Re: Ways to shoot yourself in the foot with Redis
#79My team manages a handful of clusters at work and I wrote on an internal redis client proxy (it's on my todo list to opensource). A few things I tell other teams to set them up for success (we use Elasticache): - Connection pooling / pipelining and circuit breaking is a must at scale. The clients are a lot better than they used to be but it's important developers understand the behavior of the client library they are…
You need two line breaks after each item so they'll show stacked as a list
Re: Ways to shoot yourself in the foot with Redis
#80> One common mistake is serialising objects to JSON strings before storing them in Redis. This works for reading and writing objects as atomic units but is inefficient for reading or updating individual properties within an object I would love to see some numbers on this. My intuition says there are probably some workloads where JSON strings are better and some where one key per property is better.
Depends on at which level you need atomic updates. At the entire document level or at individual property level