Earlier quoted context omitted.
as far as I can tell NOTIFY is fanout, in the sense that it will send a message to all the LISTENing connections, so it wouldn't make sense in that context anyway. It's not one-to-one, it's about making sure that jobs get picked up in a timely fashion. If you're doing something fancier with event sourcing or equivalent, you can send events via NOTIFY, and have clients decide what to do with those events then. Quoth t…
Notify can be triggered with stored procedures to send payloads related to changes to a table. It can be set up to send the id of a row that was inserted or updated, for example. (But WAL replication is usually better for this)
Show HN: Hatchet – Open-source distributed task queue
191–195 of 195 posts
Re: Show HN: Hatchet – Open-source distributed task queue
#192Earlier quoted context omitted.
Notify can be triggered with stored procedures to send payloads related to changes to a table. It can be set up to send the id of a row that was inserted or updated, for example. (But WAL replication is usually better for this)
Broadcasting the id to a lot of workers is not useful, only one of them should work on the task. Waking up the workers to do a SELECT FOR UPDATE .. SKIP LOCKED is the trick. At best the NOTIFY payload could include the kind of worker that should wake up.
Re: Show HN: Hatchet – Open-source distributed task queue
#193Earlier quoted context omitted.
You might want to look at https://www.inngest.com for that. Disclaimer: I'm a cofounder. We released event-driven step functions about 20 months ago.
Looks cool but looks like it's only typescript. If there is a json payload, couldn't any web server handle it?
Re: Show HN: Hatchet – Open-source distributed task queue
#194Re: Show HN: Hatchet – Open-source distributed task queue
#195Something I really like about some pub/sub systems is Push subscriptions. For example in GCP pub/sub you can have a "subscriber" that is not pulling events off the queue but instead is an http endpoint where events are pushed to. The nice thing about this is that you can use a runtime like cloud run or lambda and allow that runtime to scale based on http requests and also scale to zero. Setting up autoscaling for wor…
You simply define a task using our API and we take care of pushing it to any HTTP endpoint, holding the connection open and using the HTTP status code to determine success/failure, whether or not we should retry, etc.
Happy to answer any questions here or over email james@mergent.co