Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Redis is fast – I'll cache in Postgres

dizzy.zone

Post reply on HN
Print view View original

91–100 of 311 posts

Re: Redis is fast – I'll cache in Postgres

#91

Post by [deleted] »

dizzyVik wrote: post #47
jamesblonde wrote: post #43

Why do we promote articles like this that have nice graphs and are well written, when they should get a grade 'F' as an actual benchmark study. The way it is presented, a casual reader would think Postgres is 2/3rds the performance of Redis. Good god. He even admits Postgres maxxed out its 2 cores, but Redis was bottlenecked by the HTTP server. We need more of an academic, not a hacker, culture for benchmarks.

There's a reason this is on my blog and not a paper in a journal. This isn't supposed to show the absolute speed of either tool, the benchmark is not set up for that. I do state that redis has more performance on the table in the blog post.

[deleted]

Re: Redis is fast – I'll cache in Postgres

#92

Post by motorest »

maxbond wrote: post #76
motorest wrote: post #73

Earlier quoted context omitted.

> If we don't use a TTL, aren't we going to have to either accept that our cache will grow without bounds (...) Do you have a bound? I mean, with Redis you do, but that's primarily a cost-driven bound. Nevertheless, I think you're confusing the point of a TTL. TTLs are not used to limit how much data you cache. The whole point of a TTL is to be able to tell whether a cache entry is still fresh or it is stale and must…

A TTL doesn't really tell you if it's stale though. It gives you an upper bound on how long it can have been stale. But something becomes stale when the underlying resource is written to, which can happen an hour or an instant after you cache it. You should probably evict it when the write comes in. In my mind, it's for evicting things that aren't in use (to free up memory).

> A TTL doesn't really tell you if it's stale though (...)

Non-sequitur,and imaterial to the discussion.

> You should probably evict it when the write comes in.

No. This is only required if memory is maxed out and there is no more room to cache your entry. Otherwise you are risking cache misses by evicting entries that are still relatively hot.

Re: Redis is fast – I'll cache in Postgres

#93

Post by motorest »

re-thc wrote: post #87
hvb2 wrote: post #74

Earlier quoted context omitted.

I've done this many times in AWS leveraging dynamodb. Comes with ttl support (which isn't precise so you still need to check expiration on read), and can support long TTLs as there's essentially no limit to the storage. All of this at a fraction of the cost of HA redis Only if you need that last millisecond of performance and have done all other optimizations should one consider redis imho

> All of this at a fraction of the cost of HA redis This depends on your scale. Dynamodb is pay per request and the scaling isn’t as smooth. At certain scales Redis is cheaper. Then if you don’t have high demand maybe it’s ok without HA for Redis and it can still be cheaper.

> At certain scales Redis is cheaper.

Can you specify in which scenario you think Redis is cheaper than caching things in, say, dynamodb.

Re: Redis is fast – I'll cache in Postgres

#94

Post by necovek »

I'd be curious to see how Postgres behaves with numeric IDs instead of strings (or another built-in, faster-to-index/hash type).

Still leaves you with needing to perfomantly hash your strings into those IDs on top, but mostly as the Postgres plateau of performance compared to purposely built KV DBs.

Re: Redis is fast – I'll cache in Postgres

#95

Post by zer00eyz »

jamesblonde wrote: post #43

Why do we promote articles like this that have nice graphs and are well written, when they should get a grade 'F' as an actual benchmark study. The way it is presented, a casual reader would think Postgres is 2/3rds the performance of Redis. Good god. He even admits Postgres maxxed out its 2 cores, but Redis was bottlenecked by the HTTP server. We need more of an academic, not a hacker, culture for benchmarks.

You might not have been here 25 years ago when the dot com bubble burst.

A lot of us ate shit to stay in the Bay Area, to stay in computing. I have stories of great engineers doing really crappy jobs and "contracting" on the side.

I couldn't really have a 'startup' out of my house and a slice of rented hosting. Hardware was expensive and nothing was easy. Today I can set up a business and thrive on 1000 users at 10 bucks a month. Thats a viable and easy to build business. It's an achievable metric.

But Im not going to let amazon and its infinite bill you for everything at 2012 prices so it can be profitable hosting be my first choice. Im not going to do that when I can get fixed cost hosting.

For me, all the interesting things going on in tech aren't coming out of FB, Google and hyperscalers. They aren't AI or ML. We dont need another Kubernetes or Kafka or react (no more Conways law projects). There is more interesting work going on down at the bottom. In small 2 and 3 man shops solving their problems on limited time and budget with creative "next step" solutions. Their work is likely more applicable to most people reading HN than another well written engineering blog from cloud flare about their latest massive rust project.

Re: Redis is fast – I'll cache in Postgres

#96

Post by dizzyVik »

vasco wrote: post #80
dizzyVik wrote: post #47

Earlier quoted context omitted.

There's a reason this is on my blog and not a paper in a journal. This isn't supposed to show the absolute speed of either tool, the benchmark is not set up for that. I do state that redis has more performance on the table in the blog post.

It's not a paper or a journal but you could at least try to run a decent benchmark. As it is this serves no purpose other than reinforcing whatever point you started with. Didn't even tweak postgres buffers, literally what's the point.

I still end up recommending using postgres though, don't I?

Re: Redis is fast – I'll cache in Postgres

#97

Post by hvb2 »

re-thc wrote: post #87
hvb2 wrote: post #74

Earlier quoted context omitted.

I've done this many times in AWS leveraging dynamodb. Comes with ttl support (which isn't precise so you still need to check expiration on read), and can support long TTLs as there's essentially no limit to the storage. All of this at a fraction of the cost of HA redis Only if you need that last millisecond of performance and have done all other optimizations should one consider redis imho

> All of this at a fraction of the cost of HA redis This depends on your scale. Dynamodb is pay per request and the scaling isn’t as smooth. At certain scales Redis is cheaper. Then if you don’t have high demand maybe it’s ok without HA for Redis and it can still be cheaper.

You would need to get to insane read counts pretty much 24/7 for this to work out.

For HA redis you need at least 6 instances, 2 regions * 3 AZs. And you're paying for all of that 24/7.

And if you truly have 24/7 use then just 2 regions won't make sense as the latency to get to those regions from the other side of the globe easily removes any caching benefit.

Re: Redis is fast – I'll cache in Postgres

#98

Post by rplnt »

qu4z-2 wrote: post #12
throwup238 wrote: post #9

Earlier quoted context omitted.

“Dropping” something from a “new” project is premature optimization? Wherever you go, there you are.

Presumably adding Redis to a new project with no performance issues (yet?) is the premature optimisation.

If you are optimizng for simplicity it may not be, as the use as a cache is much (much) more straightforward. Also, for a new project, I'd go with in-memory service-level cache as it outperforms both (in any metric) and can be easily replaced once the need arises.

Re: Redis is fast – I'll cache in Postgres

#99

Post by pcthrowaway »

dizzyVik wrote: post #96
vasco wrote: post #80

Earlier quoted context omitted.

It's not a paper or a journal but you could at least try to run a decent benchmark. As it is this serves no purpose other than reinforcing whatever point you started with. Didn't even tweak postgres buffers, literally what's the point.

I still end up recommending using postgres though, don't I?

"I'll use postgres" was going to be your conclusion no matter what I guess?

I mean what if an actual benchmark showed Redis is 100X as fast as postgres for a certain use case? What are the constraints you might be operating with? What are the characteristics of your workload? What are your budgetary constraints?

Why not just write a blog post saying "Unoptimized postgres vs redis for the lazy, running virtualized with a bottleneck at the networking level"

I even think that blog post would be interesting, and might be useful to someone choosing a stack for a proof of concept. For someone who to scale to large production workloads (~10,000 requests/second or more), this isn't a very useful article, so the criticism is fair, and I'm not sure why you're dismissing it off hand.

Re: Redis is fast – I'll cache in Postgres

#100

Post by nasretdinov »

mj2718 wrote: post #89

My laptop running OOTB Postgres in docker does 100k requests per second with ~5ms latency. How are you getting 20x slower?

That's what I'm struggling with too. Redis can also serve roughly 500k-1m QPS using just ~4-8 cores, so on two cores it should be about 100k-200k at least
Post reply on HN