I’m leaving Redis for SolidQueue
simplethread.com
I’m leaving Redis for SolidQueue
1–10 of 146 posts
Re: I’m leaving Redis for SolidQueue
#2[dead]
Re: I’m leaving Redis for SolidQueue
#3Nice article, I'm just productionising a Rails 8 app and was wondering whether it was worth switching from SolidQueue (which has given me no stress in dev) to Redis ... maybe not.
Re: I’m leaving Redis for SolidQueue
#4Every time some production environment can be simplified, it is good news in my opinion. The ideal situation with Rails would be if there is a simple way to switch back to Redis, so that you can start simple, and as soon as you hit some fundamental issue with using SolidQueue (mostly scalability, I guess, in environments where the queue is truly stressed -- and you don't want to have a Postgres scalability problem because of your queue), you have a simple upgrade path. But I bet a lot of Rails apps don't have high volumes, and having to maintain two systems can be just more complexity.
Re: I’m leaving Redis for SolidQueue
#5Postgres will eat the world
Re: I’m leaving Redis for SolidQueue
#6For Node.js, my startup used to use [Graphile Worker](https://github.com/graphile/worker) which utilised the same "SKIP LOCKED" mechanism under the hood.
We ran into some serious issues in high throughput scenarios (~2k jobs/min currently, and ~5k job/min during peak hours) and switched to Redis+BullMQ and have never looked back ever since. Our bottleneck was Postgres performance.
I wonder if SolidQueue runs into similar issues during high load, high throughput scenarios...
Re: I’m leaving Redis for SolidQueue
#7Postgres will eat the world
Postgres will eat the world indeed. I'm just waiting for the pg_kernel extension so I can finally uninstall Linux :)
Re: I’m leaving Redis for SolidQueue
#8Postgres will eat the world
RDMS will eat the world.
Turns out it is a matter of feature set.
Re: I’m leaving Redis for SolidQueue
#9For people that does not think it scales. A similar implementation in Elixir is Oban. Their benchmark shows a million jobs per minute on a single node (and I am sure it could be increased further with more optimizations). I bet 99,99999% of apps have less than a million background jobs per minute.
https://oban.pro/articles/one-million-jobs-a-minute-with-oba...
Re: I’m leaving Redis for SolidQueue
#10[deleted]