Live data from Hacker News

Show HN: Hatchet – Open-source distributed task queue

github.com

31–40 of 195 posts

Re: Show HN: Hatchet – Open-source distributed task queue

#31
Looks pretty great! My biggest issue with Celery has been that the observability is pretty bad. Even if you use Celery Flower, it still just doesn’t give me enough insight when I’m trying to debug some problem in production.

I’m all for just using Postgres in service of the grug brain philosophy.

Will definitely be looking into this, congrats on the launch!

Re: Show HN: Hatchet – Open-source distributed task queue

#32
post #31

Looks pretty great! My biggest issue with Celery has been that the observability is pretty bad. Even if you use Celery Flower, it still just doesn’t give me enough insight when I’m trying to debug some problem in production. I’m all for just using Postgres in service of the grug brain philosophy. Will definitely be looking into this, congrats on the launch!

Appreciate it, thank you! We've spent quite a bit of time in the Celery Flower console. Admittedly it's been a while, I'm not sure if they've added views for chains/groups/etc - it was just a linear task view when I used it.

A nice thing in Celery Flower is viewing the `args, kwargs`, whereas Hatchet operates on JSON request/response bodies, so some early users have mentioned that it's hard to get visibility into the exact typing/serialization that's happening. Something for us to work on.

Re: Show HN: Hatchet – Open-source distributed task queue

#34

Something I really like about some pub/sub systems is Push subscriptions. For example in GCP pub/sub you can have a "subscriber" that is not pulling events off the queue but instead is an http endpoint where events are pushed to. The nice thing about this is that you can use a runtime like cloud run or lambda and allow that runtime to scale based on http requests and also scale to zero. Setting up autoscaling for wor…

I like that idea, basically the first HTTP request ensures the worker gets spun up on a lambda, and the task gets picked up on the next poll when the worker is running. We already have the underlying push model for our streaming feature: https://docs.hatchet.run/home/features/streaming. Can configure this to post to an HTTP endpoint pretty easily.

The daemon feels fragile to me, why not just shut down the worker client-side after some period of inactivity?

Re: Show HN: Hatchet – Open-source distributed task queue

#36

Just pointing out even though this is a "Show HN" they are, indeed, backed by YC. Is this going to follow the "open core" pattern or will there be a different path to revenue?

> path to revenue

There have to be at least 10 different ways between different cloud providers to run a distributed task queue. Amazon, Azure, GCP

Self-hosting RabbitMQ, etc.

I'm curious how they are able to convince investors that there is a sizable portion of market they think doesn't already have this solved (or already has it solved and is willing to migrate)

Re: Show HN: Hatchet – Open-source distributed task queue

#39
post #37

How does this compare to ZeroMQ (ZMQ) ? https://zeromq.org/

Not the OP or familiar with Hatchet, but generally ZeroMQ is a bit lower down in the stack -- it's something you'd build a distributed task queue or protocol on top of, but not something you'd usually reach for if you needed one for a web service or similar unless you had very special requirements and a specific, careful design in mind.

This tool comes with more bells and whistles and presumably will be more constrained in what you can do with it, where ZeroMQ gives you the flexibility to build your own protocol. In principle they have many of the same use cases, like how you can buy ready made whipped cream or whip up your own with some heavy cream and sugar -- one approach is more constrained but works for most situations where you need some whipped cream, and the other is a lot more work and somewhat higher risk (you can over whip your cream and end up with butter), but you can do a lot more with it.

Re: Show HN: Hatchet – Open-source distributed task queue

#40

Just pointing out even though this is a "Show HN" they are, indeed, backed by YC. Is this going to follow the "open core" pattern or will there be a different path to revenue?

> path to revenue There have to be at least 10 different ways between different cloud providers to run a distributed task queue. Amazon, Azure, GCP Self-hosting RabbitMQ, etc. I'm curious how they are able to convince investors that there is a sizable portion of market they think doesn't already have this solved (or already has it solved and is willing to migrate)

There will be space for improvement until every cloud has a managed offering with exactly the same interface. Like docker, postgres, S3.
Post reply on HN