Live data from Hacker News

Show HN: Hatchet – Open-source distributed task queue

github.com

131–140 of 195 posts

Re: Show HN: Hatchet – Open-source distributed task queue

#131
Related, I also wrote my own distributed task queue in Python [0] and TypeScript [1] with a Show HN [2]. Time it took was about a week. I like your features, but it was easy to write my own so I'm curious how you're building a money making business around an open source product. Maybe the fact everyone writes their own means there's no best solution now, so you're trying to be that and do paid closed source features for revenue?

[0] https://github.com/wakatime/wakaq

[1] https://github.com/wakatime/wakaq-ts

[2] https://news.ycombinator.com/item?id=32730038

Re: Show HN: Hatchet – Open-source distributed task queue

#132

Just pointing out even though this is a "Show HN" they are, indeed, backed by YC. Is this going to follow the "open core" pattern or will there be a different path to revenue?

Wasn’t the first Dropbox introduction also a show HN?

I don’t think this is out of place

Re: Show HN: Hatchet – Open-source distributed task queue

#134

Just pointing out even though this is a "Show HN" they are, indeed, backed by YC. Is this going to follow the "open core" pattern or will there be a different path to revenue?

Yep, we're backed by YC in the W24 batch - this is evident on our landing page [1]. We're both second time CTOs and we've been on both sides of this, as consumers of and creators of OSS. I was previously a co-founder and CTO of Porter [2], which had an open-core model. There are two risks that most companies think about in the open core model: 1. Big companies using your platform without contributing back in some way…

[flagged]

Re: Show HN: Hatchet – Open-source distributed task queue

#135
post #58

Earlier quoted context omitted.

I case you’re stuck with Celery for a while: I was hit with this same problem, and solved it by adding a sidecar HTTP server thread to the Python workers that would expose metrics written by the workers into a multithreaded registry. This has been working amazingly well in production for over two years now, and makes it really straightforward to get custom metrics out of a distributed Celery app.

Any chance you could share more specifics about your solution?

Here you go: https://stackoverflow.com/questions/75652326/celery-spawn-si...

Plus some adjacent discussion on GitHub: https://github.com/prometheus/client_python/issues/902

Hope that helps!

Re: Show HN: Hatchet – Open-source distributed task queue

#136
post #81

Earlier quoted context omitted.

It's only a few billion instructions on a decent sized server these days

Damn, I want one of these 100GHz CPUs you have, that sounds great. I think you mean million :)

My ipad has 8 cores executing about 4 to 6 billion instructions a second these days (3GHz at a most ipc of about two)

Re: Show HN: Hatchet – Open-source distributed task queue

#137
Congrats on the launch!

You say Celery can use Redis or RabbitMQ as a backend, but I've also used it with Postgres as a broker successfully, although on a smaller scale (just a single DB node). It's undocumented, so definitely won't recommend anybody using this in production now, but seems to still work fine. [1]

How does Hatchet compare to this setup? Also, have you considered making a plugin backend for Celery, so that old systems can be ported more easily?

[1]: https://stackoverflow.com/a/47604045/1593459

Re: Show HN: Hatchet – Open-source distributed task queue

#138
post #38

With NATS in the stack, what's the advantage over using NATS directly?

I'm assuming specifically you mean Nex functions? Otherwise NATS gives you connectivity and a message queue - it doesn't (or didn't) have the concept of task executions or workflows. With regards to Nex -- it isn't fully stable and only supports Javascript/Webassembly. It's also extremely new, so I'd be curious to see how things stabilize in the coming year.

I recently found Nex in the context of Wasmcloud [0] and ability for it to support long-running tasks/workflows. Impression that indeed Nex needs a good time to mature still. There was also a talk [1] about using Temporal here. For Hatchet it may be interesting to check it out (note: I am not affiliated with Wasmcloud, nor currently using it).

[0] https://wasmcloud.com

[1] https://www.temporal.io/replay/videos/zero-downtime-deploys-...

Re: Show HN: Hatchet – Open-source distributed task queue

#139

I love your vision and am excited to see the execution! I've been looking for exactly this product (postgres-backed task queue with workers in multiple languages and decent built-in observability) for like... 3 years. Every 6 months I'll check in and see if someone has built it yet, evaluate the alternatives, and come away disappointed. One important feature request that probably would block our adoption: one reason…

[deleted]

Re: Show HN: Hatchet – Open-source distributed task queue

#140
post #70

Earlier quoted context omitted.

Funny how this is vision now. I started my career 29 years ago at a company that build exactly this, but based on oracle. The agents would run on Solaris, aix, vax vms, hpux, windows nt, iris, etc. Was also used to create an automated cicd pipeline to build all binaries on all these different systems.

Also basically has existed as an open source (pro version has web dashboard and complex task zoo) drop-in library (no sidecar dependencies outside of postgres) in Elixir for years called Oban.

Yep, it feels like half the show hn launches is for infrastructure tooling that already exist natively or as plug and play libraries for Elixir/Erlang.

I really try to suggest people skip Node and learn a proper backend language with a solid framework with a proven architecture.

Post reply on HN