Live data from Hacker News

Show HN: Drop-in SQS replacement based on SQLite

github.com

61–70 of 171 posts

Re: Show HN: Drop-in SQS replacement based on SQLite

#61
post #50

The whole idea here is superlative. +1 for k8s, kubernetes, cloud native, self-hosted, edge-enabled at low cost, no cost. I ran rq and minio for years on k8s, but been watching sqlite as a drop-in-replacement since most of my work has been early stage at or near the edge. Private cloud matters. This is an enabler. We've done too much already in public cloud where many things don't belong. BTLE sensors are perfectly h…

Really interesting. Question: when it comes to running these software on k8s, do you prefer to manage and host yourself, or do you use managed solutions on top of your own infra? (Do you pay for minio support?) Asking from a business perspective - I of course intend to keep developing this, but am also really trying to think through the business case as well.

Good question!

The answer depends on funding, i.e. in my own never-leaves-my-house case it is always self-host, much like SOC work.

In the case of startup or research lab work (day-job, for lack of a better descriptor). It's frequently a slice of AWS, GCP, or Azure, i.e. 6 figure/mo cloud bills.

I think those two broad cases are worth considering.

Re: Show HN: Drop-in SQS replacement based on SQLite

#62
post #50

The whole idea here is superlative. +1 for k8s, kubernetes, cloud native, self-hosted, edge-enabled at low cost, no cost. I ran rq and minio for years on k8s, but been watching sqlite as a drop-in-replacement since most of my work has been early stage at or near the edge. Private cloud matters. This is an enabler. We've done too much already in public cloud where many things don't belong. BTLE sensors are perfectly h…

Really interesting. Question: when it comes to running these software on k8s, do you prefer to manage and host yourself, or do you use managed solutions on top of your own infra? (Do you pay for minio support?) Asking from a business perspective - I of course intend to keep developing this, but am also really trying to think through the business case as well.

k3s for on premises deployments. I'm also using it for local self hosting side projects.

Re: Show HN: Drop-in SQS replacement based on SQLite

#64
post #17
post #9

This looks great! How were you planning on tackling distributed?

This is a great question and something I've been thinking about a lot. Big picture: Each queue node can operate (mostly) independently, and this is good. As a consumer, I don't really care where my next message comes from, so I can minimize the amount of data that needs to have a "leader". The only data that needs to be synced is the list of queues, which doesn't change often. If one server is full, it should be able…

You're hand-waving away all the complexity in the "nitty gritty."

Enqueue absolutely requires coordination, if not via leader then at least amongst multiple nodes, if you want to guarantee at least once delivery

If you don't guarantee that, cool, but you're not competing with sqs

Re: Show HN: Drop-in SQS replacement based on SQLite

#65

The whole idea here is superlative. +1 for k8s, kubernetes, cloud native, self-hosted, edge-enabled at low cost, no cost. I ran rq and minio for years on k8s, but been watching sqlite as a drop-in-replacement since most of my work has been early stage at or near the edge. Private cloud matters. This is an enabler. We've done too much already in public cloud where many things don't belong. BTLE sensors are perfectly h…

rq as in Redis Queue?

Re: Show HN: Drop-in SQS replacement based on SQLite

#68
post #20
post #10

Keeping questions from scale & benchmarks aside, this is a cool thing for functional/unit testing module that uses SQS, instead of dumb mocks.

Thank you! Yes, you should definitely use this instead of localstack :)

Although I wouldn't recommend using LocalStack's SQS implementation for production workloads, calling it a dumb mock is a bit outdated. It emulates pretty much every SQS behvaior including long polling, delayed messages, visibility timeouts, DLQ redrive, batch send/receive, ...

Re: Show HN: Drop-in SQS replacement based on SQLite

#69
post #44

Not sure about the goal of providing a hosted service cheaper than SQS. SQS is already one of the cheapest services on Earth. It's pretty hard to spend more than a few bucks a month, even if you really try!

That is fair. Two things to validate from a biz perspective: 1. Is there some threshold where this would make sense financially (n billions of messages.) 2. Are the extra developer features (ie, larger message sizes, observability, DAGs) worth it for people to switch? Would love your thoughts - what, if anything, would make you even entertain moving to a different queue system?

Not the GP, but I also think you'll struggle to compete in the hosted service space.

Maybe you could add a web admin GUI as a paid add on?

Re: Show HN: Drop-in SQS replacement based on SQLite

#70
post #19

Good, we need open implementations of all the AWS stuff. I swear they reimplement stuff we have just so there are more places to bill us.

SQS is cheap.

10m requests for $4.

You will need hundreds of millions of requests per month for it to be noticable.

And can an implementation like this even help you at that point?

Post reply on HN