Live data from Hacker News

Redis is fast – I'll cache in Postgres

dizzy.zone

261–270 of 311 posts

Re: Redis is fast – I'll cache in Postgres

#261
post #252

Earlier quoted context omitted.

you can become a db expert with the right prompts

You can learn how to pour a drink in 1 minute, that is why most bartenders earn minimum wage. You can't become a db expert with a promt. I hope you make a lot of money with your lies and good luck.

You can become a DB expert by reading books, forums and practicing hard.

These days you can replace those books and forums with a top tier LLM, but you still need to put in the practice yourself. Even with AI assistance that's still a lot of work.

Re: Redis is fast – I'll cache in Postgres

#262

> Both postgres and redis are used with the out of the box settings Ugh. I know this gives the illusion of fairness, but it's not how any self-respecting software engineer should approach benchmarks. You have hardware. Perhaps you have virtualized hardware. You tune to the hardware. There simply isn't another way, if you want to be taken seriously. Some will say that in a container-orchestrated environment, tuning go…

Yep. I worked in a famous-big-company that had a 15 years old service that was dogslow, systemd restarts would take multiple hours.

Everyone was talking about C++ optimizations, mutex everywhere etc - which was in fact a problem.

However.. I seemed to be the first person to actually try to debug what the database was doing, and it was going to disk all the time with a very small cache.. weird..

I see the MySQL settings on a 1TB ram machine and they were... out-of-the-box settings.

With small adjustments I improved the performance of this core system an order of magnitude.

Re: Redis is fast – I'll cache in Postgres

#263
post #253
post #242

Earlier quoted context omitted.

I agree that people like to hire profesionals. That is why I hire db experts to work on our infra, not prompt engenieers. Saying that anybody can learn to unblock a sink by watching youtube is your tipical HN mentality of stating opinons as facts.

"Saying that anybody can learn to unblock a sink by watching youtube is your tipical HN mentality of stating opinons as facts." I don't understand what you mean. Are you saying that it's not true that anyone could learn to unblock a sink by watching YouTube videos?

Yes I do think not all people could fix it with Youtube. My grandma couldn't for example. I had a neigbor come for help with something like that too.

Is not that hard to understan mate. Maybe put my comment in the LLM so you can get it.

What is your point again?

Re: Redis is fast – I'll cache in Postgres

#264

Earlier quoted context omitted.

> If you, like the whole world, consume Redis through a network connection, it should be obvious to you that network is in fact the bottleneck. Not to be annoying - but... what? I specifically _do not_ use Redis over a network. It's wildly fast. High volume data ingest use case - lots and lots of parallel queue workers. The database is over the network, Redis is local (socket). Yes, this means that each server runnin…

that is an interesting use case, I hadn't thought about a setup like this with a local redis cache before. Is it the typical advantages of using a db over a filesystem the reason to use redis instead of just reading from memory mapped files?

> Is it the typical advantages of using a db over a filesystem the reason to use redis instead of just reading from memory mapped files?

Eh - while surely not everyone has the benefits of doing so, I'm running Laravel and using Redis is just _really_ simple and easy. To do something via memory mapped files I'd have to implement quite a bit of stuff I don't want/need to (locking, serialization, ttl/expiration, etc).

Redis just works. Disable persistence, choose the eviction policy that fits the use, config for unix socket connection and you're _flying_.

My use case is generally data ingest of some sort where the processing workers (in my largest projects I'm talking about 50-80 concurrent processes chewing through tasks from a queue (also backed by redis) and are likely to end up running the same queries against the database (mysql) to get 'parent' records (ie: user associated with object by username, post by slug, etc) and there's no way to know if there will be multiples (ie: if we're processing 100k objects there might be 1 from UserA or there might be 5000 by UserA - where each one processing will need the object/record of UserA). This project in particular there's ~40 million of these 'user' records and hundreds of millions of related objects - so can't store/cache _all_ users locally - but sure would benefit from not querying for the same record 5000 times in a 10 second period.

For the most part, when caching these records over the network, the performance benefits were negligible (depending on the table) compared to just querying myqsl for them. They are just `select where id/slug =` queries. But when you lose that little bit of network latency and you can make _dozens_ of these calls to the cache in the time it would take to make a single networked call... it adds up real quick.

PHP has direct memory "shared memory" but again, it would require handling/implementing a bunch of stuff I just don't want to be responsible for - especially when it's so easy and performant to lean on Redis over a unix socket. If I needed to go faster than this I'd find another language and likely do something direct-to-memory style.

Re: Redis is fast – I'll cache in Postgres

#265

Earlier quoted context omitted.

> copy pasting some configuration I might not really understand Uh, yea... why would you? Do you do that for configurations you found that weren't from LLMs? I didn't think so. I see takes like this all the time and I'm really just mind-boggled by it. There are more than just the "prompt it and use what it gives me" use cases with the LLMs. You don't have to be that rigid. They're incredible learning and teaching too…

My comment is mainly in opposition to the "five minutes" part from parent. If you have 5 minutes then you can't as you say : > Then I will dig deeper into those things myself ... So my point is I don't care if it's coming from LLM or a random blog, you won't have time to know if it's really working (ideally you would want to benchmark the change). If you can't invest the time better to stay with the defaults, which i…

Yea, I guess in that case I'd say it's likely a bad move in every direction if you're constrained to 5 min to deploy something you don't understand.

Re: Redis is fast – I'll cache in Postgres

#266
post #243

Earlier quoted context omitted.

Sometimes, a cache is all about reducing expense: I.e, free cache query vs expensive API query.

Sometimes people host software on a server they own or rent, the server is plenty fast, and it costs literally nothing to issue those queries at the scale on which they’re needed.

Yes, that is true, but the original poster said getting rid of caches was always a good idea, when in reality the answer (as usual with engineering) is “it depends.”

Re: Redis is fast – I'll cache in Postgres

#267
post #172

Earlier quoted context omitted.

> "If we don't need performance, we don't need caches" feels like a great broader takeaway here. I don't think this holds true. Caches are used for reasons other than performance. For example, caches are used in some scenarios for stampede protection to mitigate DoS attacks. Also, the impact of caches on performance is sometimes negative. With distributed caching, each match and put require a network request. Even wh…

DoS is a performance problem, if your server was infinitely fast with infinite storage they wouldnt be an issue.

It is actually a financial problem too. Servers stop working when the bill goes unpaid. Sad but true.

Re: Redis is fast – I'll cache in Postgres

#268
post #254
post #242

Earlier quoted context omitted.

I agree that people like to hire profesionals. That is why I hire db experts to work on our infra, not prompt engenieers. Saying that anybody can learn to unblock a sink by watching youtube is your tipical HN mentality of stating opinons as facts.

Personally I'd like to hire a DB expert who also knows how to drive an agentic coding system to help them accelerate their work. AI tools, used correctly, act as an amplifier of existing knowledge and experience.

As far as I know nobody has really came up with proof that LLMs act as an amplifier of existing knoledge.

It does make people FEEL more productive.

Re: Redis is fast – I'll cache in Postgres

#269
post #155

Earlier quoted context omitted.

No need to be so combative. Take a chill pill, zoom out and look at the reliability of the entire system and its services rather than the db in isolation. If postgres has issues, it can affect the reliability of the service further if it's also running the cache. Besides, having the cache on separate hardware can reduce the impact on the db on spikes, which can also factor into reliability. Having more headroom for m…

> No need to be so combative. You're confusing being "combative" with asking you to substantiate your extraordinary claims. You opted to make some outlandish and very broad sweeping statements, and when asked to provide any degree of substance, you resorted to talk about "chill pills"? What does that say about the substance if your claims? > If postgres has issues, it can affect the reliability of the service further…

> You're confusing being "combative" with asking you to substantiate your extraordinary claims. You opted to make some outlandish and very broad sweeping statements, and when asked to provide any degree of substance, you resorted to talk about "chill pills"?

what are these "extraordinary claims" you speak of? I believe it's you who are confusing me with someone else. I am not GP. You appear to be fighting windmills.

Re: Redis is fast – I'll cache in Postgres

#270
post #129

> Both postgres and redis are used with the out of the box settings Ugh. I know this gives the illusion of fairness, but it's not how any self-respecting software engineer should approach benchmarks. You have hardware. Perhaps you have virtualized hardware. You tune to the hardware. There simply isn't another way, if you want to be taken seriously. Some will say that in a container-orchestrated environment, tuning go…

I disagree. They found that Postgres, without tuning, was easily fast enough on low level hardware and would come with the benefit of not deploying another service. Additionally tuning it isn’t really relevant. If the defaults are fine for a use case then unless I want to tune it for personal interest it’s either a poor use of my fun time or a poor use of my clients funds.

> They found that Postgres, without tuning, was easily fast enough on low level hardware

Is that production? When you basket it into "low level" it sounds like a base case but it really isn't.

In production you don't have local storage, RAM being used for all kinds of other things, your CPU only available in small slices, network effects and many others.

> If the defaults are fine for a use case

Which I hope isn't the developer's edition of it works on my machine.

Post reply on HN