Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Launch HN: Hatchet (YC W24) – Open-source task queue, now with a cloud version

news.ycombinator.com

Post reply on HN
Print view View original

71–80 of 102 posts

Re: Launch HN: Hatchet (YC W24) – Open-source task queue, now with a cloud version

#71

Post by n00bskoolbus »

abelanger wrote: post #69
n00bskoolbus wrote: post #67

This looks really awesome! We were just discussing at work how we're having a hard time finding a framework for a task queue that supports dependant tasks and has support for Python & TS. I suppose writing that out it does feel like a pretty specific requirement. I'm glad to see this pop up though, feels very relevant to me right now. A question around workflows having just skimmed your docs. Is it possible to define…

Thanks! Yes, our recommended approach is to write a parent workflow which calls child workflows registered on a different worker. We have users who are managing a set of Python functions from a Typescript backend with this approach. It's also possible to have a single DAG workflow (instead of parent/child) that has steps across multiple languages, but you'll need to use a relatively undocumented method called `Regist…

Ah okay that makes sense! Thanks for the reply, will definitely try hatchet out!

Re: Launch HN: Hatchet (YC W24) – Open-source task queue, now with a cloud version

#72

Post by tecoholic »

barrell wrote: post #66

I’ve been through a whole journey with distributed tasks queues - from celery, to arq, to recently hatchet. Not only is hatchet the only solution that doesn’t make me want to tear my hair out, but the visibility the product gives you is amazing! Being able to visually explore logs, props, refrigerate specific queues etc has been a game changer, Also, minor thing, but the granularity around rate limiting and queues al…

After multiple years fighting with Celery, we moved to Prefect last year and have been mostly happy with it. The only sticking point for me has been “tasks can’t start tasks, will have to be sub-flows” part. Did you ever try out Prefect and can share anything from the experience?

Re: Launch HN: Hatchet (YC W24) – Open-source task queue, now with a cloud version

#73

Post by andrewstuart »

I've built several message queues over the years.

I hated the configuration and management complexity of RabbitMQ and Celery and pretty much everything else.

My ultimate goal was to build a message queue that was extremely fast and required absolutely zero config and was HTTP based thus has no requirement for any specific client.

I developed one in Python that was pretty complete but slow, then developing a prototype in Rust that was extremely fast but incomplete.

The latest is sasquatch. Its written in golang, uses sqlite for the db and behaves in a very similar way to Amazon SQS in that connections are HTTP and it uses long polling to wait for messages.

https://github.com/crowdwave/sasquatch

Its only in the very early stages of development at this stage and likely isn't even compiling but most of the code is in place. I'm hoping to get around to next phase of development soon.

I just love the idea of a message queue that is a single static binary and when you run it, you have a fully functioning message queue with nothing more to do - not even fiddling with Postgres.

Absolute zero config, not minutes, hours or days of futzing with configs and blogs and tutorials.

Re: Launch HN: Hatchet (YC W24) – Open-source task queue, now with a cloud version

#75

Post by pizzafeelsright »

smallshen wrote: post #74

I'm wondering what is the difference from https://docs.urlinks.io/gateway-chain/ . There are a lot similar concepts, like very similar. Hatchet feels like same product but with money from VCs.

So there's a market.

Re: Launch HN: Hatchet (YC W24) – Open-source task queue, now with a cloud version

#76

Post by kevdoran »

didip wrote: post #40

I am surprised that there's still money for this type of OSS SaaS companies. Aren't all the money go to AI companies these days (even the unicorns didn't do well with their IPOs. E.g. Hashicorp). That said, I love every single addition to the Go community so thumbs up from me.

There are a lot of AI startups that fall in the category of LLM API consumers (Anthropic/OpenAI wrappers). Or, as I heard the CTO of one of them joking, "we're actually more EC2 wrappers than OpenAI wrappers".

The problem we often hit when building apps on top of LLMs is managing LLM context windows (and sometimes swappable LLM providers). For which you need different types of worker/consumer/queue setups.

TypeScript is amazing for building full-stack web apps quickly. For a decade my go-to was Django, but everything just goes so much faster with endpoints & frontend all in the same place. But, finding a good job/queue service is a little more of a challenge in this world that "just setup Celery". BullMQ is great, but doesn't work with "distributed" Redis providers like Upstash (Vercel's choice).

So, in a roundabout way, an offering like this is in a super-duper position for AI money :)

Re: Launch HN: Hatchet (YC W24) – Open-source task queue, now with a cloud version

#77

Post by plasma »

abelanger wrote: post #59
thevivekshukla wrote: post #51

Seems interesting, what are the plans on Rust SDK?

We'd like to stabilize our existing 3 SDKs and create a proper spec for future SDKs to implement. While we use proto definitions and openapi to generate clients, there are a lot of decisions made while calling these APIs that are undocumented but kept consistent between TS, Python and Go. Once that's done and we consider our core API stable, there's a good chance we'll start tackling a new set of SDKs later this year…

Project looks interesting, would welcome seeing an API (or c# client) to be able to use it.

Re: Launch HN: Hatchet (YC W24) – Open-source task queue, now with a cloud version

#78

Post by bgentry »

acaloiar wrote: post #17

I love seeing commercial activity around using Postgres as a queue. Last year I wrote a post titled "Choose Postgres queue technology" that spent quite a bit of time on the front page here. I don't think it's likely that my post actually sparked new development in this area, but for the people who were already using Postgres queues in their applications, I hope it made them feel more comfortable talking about it in p…

I remember that post and I’ve read it a few times, thank you for it! I was already working on River at the time but it was refreshing to see the case made so strongly by another person who gets it.

- Blake, co-author of riverqueue.com / https://github.com/riverqueue/river :)

Re: Launch HN: Hatchet (YC W24) – Open-source task queue, now with a cloud version

#79

Post by krick »

Can somebody explain why would I use it instead a simple Redis/SQS/Postgres queue implemented in 50 LOC (+ some grafana panel for monitoring) (which is pretty much mandatory even for a wrapper of this or any other service)? I'm not trying to mock it, it's a serious question. What is implied by "task queue" that makes it worth bothering to use a dedicated service?

Re: Launch HN: Hatchet (YC W24) – Open-source task queue, now with a cloud version

#80

Post by altdataseller »

krick wrote: post #79

Can somebody explain why would I use it instead a simple Redis/SQS/Postgres queue implemented in 50 LOC (+ some grafana panel for monitoring) (which is pretty much mandatory even for a wrapper of this or any other service)? I'm not trying to mock it, it's a serious question. What is implied by "task queue" that makes it worth bothering to use a dedicated service?

I also want the answer to this question. Instinctually i want to say if you’re asking this Q it means you don’t need it (just like most people dont need Kubernetes/Snowflake/data lakes)
Post reply on HN