Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Drop-in SQS replacement based on SQLite

github.com

Post reply on HN
Print view View original

91–100 of 171 posts

Re: Show HN: Drop-in SQS replacement based on SQLite

#91

Post by threecheese »

memset wrote: post #74
paulddraper wrote: post #70

Earlier quoted context omitted.

SQS is cheap. 10m requests for $4. You will need hundreds of millions of requests per month for it to be noticable. And can an implementation like this even help you at that point?

Question: is there any feature set (in contrast to pricing) that would make it worthwhile for someone to switch?

Operations. At my $bigCo we are always rolling our own ops tooling for queues, as it’s more than just “reprocess message”; there is always some business logic that separates different types of consumers/destinations, and so q management is never straightforward. Might be a differentiable feature.

Re: Show HN: Drop-in SQS replacement based on SQLite

#92

Post by localfirst »

bumbledraven wrote: post #89
localfirst wrote: post #85

Earlier quoted context omitted.

i didnt ask for a product catalogue, all im asking is to clarify whether you are truly FOSS or not by being transparent about the license. Any type of licensing conditions like AGPL 3.0 to force you to share your code is a no go for many many companies. I understand the desire to make money and you can do that without AGPL 3.0

It's right there in the LICENSE file: https://github.com/poundifdef/SmoothMQ/blob/main/LICENSE What's unclear or not transparent about that?

All I'm asking is to include the type of license in the Show HN title

I had no idea there would be so much push back against this.

Re: Show HN: Drop-in SQS replacement based on SQLite

#93

Post by threecheese »

memset wrote: post #44
slotrans wrote: post #39

Not sure about the goal of providing a hosted service cheaper than SQS. SQS is already one of the cheapest services on Earth. It's pretty hard to spend more than a few bucks a month, even if you really try!

That is fair. Two things to validate from a biz perspective: 1. Is there some threshold where this would make sense financially (n billions of messages.) 2. Are the extra developer features (ie, larger message sizes, observability, DAGs) worth it for people to switch? Would love your thoughts - what, if anything, would make you even entertain moving to a different queue system?

What is your target market? Cloud-native is, like the commenter said, going to be difficult to differentiate based on cost. I can see this being useful for hybrid cloud (onprem instances), functional or local testing, “localfirst” workloads, enthusiasts etc.

Re: Show HN: Drop-in SQS replacement based on SQLite

#94

Post by memset »

threecheese wrote: post #91
memset wrote: post #74

Earlier quoted context omitted.

Question: is there any feature set (in contrast to pricing) that would make it worthwhile for someone to switch?

Operations. At my $bigCo we are always rolling our own ops tooling for queues, as it’s more than just “reprocess message”; there is always some business logic that separates different types of consumers/destinations, and so q management is never straightforward. Might be a differentiable feature.

Can you tell me more about the types of queue things you need to implement? (Feel free to email me too - I’d be grateful for the feedback!)

Re: Show HN: Drop-in SQS replacement based on SQLite

#95

Post by sieabahlpark »

pfix wrote: post #65
davidthewatson wrote: post #46

The whole idea here is superlative. +1 for k8s, kubernetes, cloud native, self-hosted, edge-enabled at low cost, no cost. I ran rq and minio for years on k8s, but been watching sqlite as a drop-in-replacement since most of my work has been early stage at or near the edge. Private cloud matters. This is an enabler. We've done too much already in public cloud where many things don't belong. BTLE sensors are perfectly h…

rq as in Redis Queue?

[dead]

Re: Show HN: Drop-in SQS replacement based on SQLite

#96

Post by memset »

threecheese wrote: post #93
memset wrote: post #44

Earlier quoted context omitted.

That is fair. Two things to validate from a biz perspective: 1. Is there some threshold where this would make sense financially (n billions of messages.) 2. Are the extra developer features (ie, larger message sizes, observability, DAGs) worth it for people to switch? Would love your thoughts - what, if anything, would make you even entertain moving to a different queue system?

What is your target market? Cloud-native is, like the commenter said, going to be difficult to differentiate based on cost. I can see this being useful for hybrid cloud (onprem instances), functional or local testing, “localfirst” workloads, enthusiasts etc.

Now you sound like an investor :)

My own vision is to take a queue that is relatively dumb and make it smarter. I want it to be able to, for example, allow you to rate limit workers without needing to implement this client side. And so on for all of the other bits that one needs to implement in the course of distributed processing.

I’m still figuring out the market. Very large firms spending thousands of queues? Or developers who want a one stop solution built on familiar tech? Or hosting companies who want to offer their own queue as a service?

Re: Show HN: Drop-in SQS replacement based on SQLite

#97

Post by andrewstuart »

I wrote sasquatch which is similar. sasquatch is more simple than SQS and queues are virtual (i.e the queue exists because you put a message in it, and is gone when there's no messages in it).

https://github.com/crowdwave/sasquatch

sasquatch is also a message queue, also written in Golang and also based on sqlite.

sasquatch implements behaviour very similar to SQS but does not attempt to be a dropin replacement.

sqsquatch is not a complete project though, nor even really a prototype, just early code. Likely it does not compile.

HOWEVER - sasquatch is MIT license (versus this project which is AGPL) so you are free to do with it as you choose.

sasquatch is a single file of 700 lines so easy to get your head around: https://raw.githubusercontent.com/crowdwave/sasquatch/main/s...

Just remember as I say it's early code, won't even compile yet but functionally should be complete.

Re: Show HN: Drop-in SQS replacement based on SQLite

#98

Post by andrewstuart »

john-tells-all wrote: post #79

Why not use LocalStack? It has SQS and a lot of AWS services for testing/development. Well documented, open source. https://docs.localstack.cloud/overview/

Because SIMPLE.

single exectuable binary

golang with sqlite so fast

minimal config so no ridiculous hours or days of config and operations

This blitzes other ideas for use cases that do not require distributed queues - and that is likely many use cases.

Re: Show HN: Drop-in SQS replacement based on SQLite

#99

Post by andrewstuart »

Onavo wrote: post #41
jerrygenser wrote: post #7

> In terms of monetization, the goal is to just have a hosted queue system. I believe this can be cheaper than SQS without sacrificing performance. Just as Backblaze and Minio have had success competing in the S3 space, I wanted to take a crack at queues. are you monetizing this as a separate business from: https://www.ycombinator.com/companies/scratch-data

What's the point of AGPL though? The enterprises with the budget for self hosting this sort of software usually have requirements and scale beyond what SQLite can offer out of the box.

Ugh I didn't see that.

My enthusiasm has instantly waned.

Why is AGPL needed? Just be MIT and make it easy for people, espefcially if you're not planning on monetising it.

I won't use AGPL code just on principle.

Re: Show HN: Drop-in SQS replacement based on SQLite

#100

Post by PanMan »

slotrans wrote: post #39

Not sure about the goal of providing a hosted service cheaper than SQS. SQS is already one of the cheapest services on Earth. It's pretty hard to spend more than a few bucks a month, even if you really try!

This depends on your definition of few.. but SQS costs us more than the servers handling the messages… It’s cheap until you do billions of messages…
Post reply on HN