Live data from Hacker News

Show HN: Drop-in SQS replacement based on SQLite

github.com

71–80 of 171 posts

Re: Show HN: Drop-in SQS replacement based on SQLite

#71

I love that we're seeing a lot of projects apply the KISS principal and leverage (or take inspiration) from SQLite, like this, PocketBase and even DuckDB. An entire generation of developers (including myself) were tricked into thinking you had to build for scale from day one, or worse, took the path of least resistance right to the most expensive place for cloud services: the middle. I'm hopeful the next generation w…

[deleted]

Re: Show HN: Drop-in SQS replacement based on SQLite

#74
post #19

Good, we need open implementations of all the AWS stuff. I swear they reimplement stuff we have just so there are more places to bill us.

SQS is cheap. 10m requests for $4. You will need hundreds of millions of requests per month for it to be noticable. And can an implementation like this even help you at that point?

Question: is there any feature set (in contrast to pricing) that would make it worthwhile for someone to switch?

Re: Show HN: Drop-in SQS replacement based on SQLite

#76
I may be asking a naive question, but what is the rationale behind disabling foreign key support and using them anyway in the database schema? See https://github.com/poundifdef/SmoothMQ/blob/46f8b22/queue/sq...

The "TODO: check for errors" comment, combined with what seems like disabling foreign key constraint checks, makes me a bit hesitant to try this out.

Re: Show HN: Drop-in SQS replacement based on SQLite

#78
post #76

I may be asking a naive question, but what is the rationale behind disabling foreign key support and using them anyway in the database schema? See https://github.com/poundifdef/SmoothMQ/blob/46f8b22/queue/sq... The "TODO: check for errors" comment, combined with what seems like disabling foreign key constraint checks, makes me a bit hesitant to try this out.

Good question. It was an evolution. Originally I enforced foreign keys, but inserts/updates were unbearably slow. So I updated the connection string to disable them (but, as you point out, I haven't updated the CREATE TABLE statements.)

In practice, I did not find they were necessary - I was only using foreign keys to automatically CASCADE deletes when messages were removed. But instead of relying on sqlite to do that, I do it myself and wrap the delete statements in transactions.

There are many TODOs and error checkings that I will, over time, clean up and remove. I'm glad you've pointed them out - that's the great thing about open source, you at least know what you're getting into and can help shine a light on things to improve!

Re: Show HN: Drop-in SQS replacement based on SQLite

#80
This is very interesting! The self-hosted aspect is something I'll have to consider for certain purposes.

My lab also developed an SQS-esque system based on the filesystem, so no dependencies whatsoever and no need for any operational system other than the OS. It doesn't support all SQS commands (because we haven't needed them), but it also supports commands that SQS doesn't have (like release all messages to visible status).

https://github.com/seung-lab/python-task-queue

Post reply on HN