21–30 of 40 posts
Is there a job/message queue implemented in pure Postgres, so it doesn't depend on Python, Perl, Ruby, ...? So that it can be used from any language that can connect to Postgres?
Is there a job/message queue implemented in pure Postgres, so it doesn't depend on Python, Perl, Ruby, ...? So that it can be used from any language that can connect to Postgres?
My current favourite task queue is http://p3rl.org/Minion and I mention this not in a spirit of competition but because the pg queries to be found in the source code of https://metacpan.org/dist/Minion/source/lib/Minion/Backend/P... are IMO truly beautiful and worthy of the "great artists steal" treatment.
Anything specific worth calling out?
Is there a job/message queue implemented in pure Postgres, so it doesn't depend on Python, Perl, Ruby, ...? So that it can be used from any language that can connect to Postgres?
My opinion is if you need a message queue use a message queue product, not a database.
And when it (the database) cannot serve your needs in terms of performance anymore, then you must choose a domain-specific product dedicated to solve the you challenges.
> premature optimization is the root of all evil.
I’ve been looking for a tool that could replace Celery for my smaller projects and use postgresql as its queuing system. I was pointed this way in the recent thread about message queuing natively in postgresql [1]. I thought it seemed like a project deserving a bit more attention. https://news.ycombinator.com/item?id=30119285
Why this and not celery? And why Postgres and not rabbitmq or redis?
I have a production system running on Celery (on Redis). It's been fine, once I tuned it.
Time for a refactor; wondering: should I consider migration?
This looks nice, I'll give it a try. It's worth mentionning Huey too: https://huey.readthedocs.io/en/latest/ I've been using it for a year on top of sqlite.
It's not open source though it is free.
StarQueue has a pure HTTP API so any language can talk to it.
StarQueue is designed to be a more simple implementation of the SQS way of doing things, without some of the unnecessary stuff that SQS has in its API.
Under the hood it uses Postgres although I actually wrote it to support MySQL and SQL Server also, which are equally capable of doing queueing as Postgres.
Quite timely to see this discussion here. I got to a point on a home project last night where I need to implement a scheduler and was thinking 'man I dont want to setup Celery for this', that can wait for tomorrow. In the past I have actually written my own very little scheduler that borrows the app database. Really just table that is a job queue and a script on a loop that grabs the job when it appears in the table.…
I’ve been looking for a tool that could replace Celery for my smaller projects and use postgresql as its queuing system. I was pointed this way in the recent thread about message queuing natively in postgresql [1]. I thought it seemed like a project deserving a bit more attention. https://news.ycombinator.com/item?id=30119285
I was using Celery for sending emails - nothing else. And Celery was such a nightmare to configure and debug and such overkill for email buffering that in a fit of frustration I wrote the Arnie SMTP buffering server and ditched Celery.
https://github.com/bootrino/arniesmtpbufferserver
It's only 100 lines of code:
https://github.com/bootrino/arniesmtpbufferserver/blob/maste...