Earlier quoted context omitted.
Just no, your tasks should be idempotent. Distributed transactions are stupid.
They’re not talking about distributed transactions: it’s not about a task being published and consumed atomically, it’s about it being consumed and executed atomically.
Show HN: Hatchet v1 – A task orchestration platform built on Postgres
71–78 of 78 posts
Re: Show HN: Hatchet v1 – A task orchestration platform built on Postgres
#72Nice work on the lite mode, open source, logging, dx interface.
You may want to replace Hello world examples with real world scenarios.
The workflows that involve multiple steps tasks, dag in your terminology - the code simply isn't intuitive.
You now have to get into the hatchets mindset, patterns, terminology. Eg: the random number example is riddled with too many. How many of the logos on your homepage did you have to write code for? Be honest.
Knowing to program should be 90% enough. Eg for js:
// send("hi", user => user.signed_up_today)
// .waitFor("7d")
// .send("upgrade", user => !user.upgraded)
Just made this up, but something like this is more readable. (PS:would love to be proved wrong by an implementation of exactly the above example here in the comments). The whole point of being smart is for your team at hatchet to absorb difficulty at the benefit of an easy interface that looks simple and magic. Your 5 line examples has types to learn, functions to learn, arguments to know, 5-10 kinds of things to learn. It showed little effort to make it easy for customers.An engineering post on what's under the hood makes sense. But customers really don't care about your cloud infra flexes in a post introducing your company pitching the product. It's just koolaid.
Same with complete rewrite so early. I'm glad you are open to change. But the workflow market today with so many options, i don't belive this is the last rewrite or pivot to come.
The DAGs itself aren't very readable. You are better off switching to something like react flow that lets you nocode edit as well.
Focus on automation journeys that are common. Like cookbooks. And allow folks to just import them or change some configurations. like drip marketing, renewals, expired cards, forgot password handlers, shortlink creators, maybe pdf merging, turning a bunch of saved links to a daily blog post, etc
How does a workflow replace a saas they are paying $99 for. That's powerful.
Tough to serialize a worflow to json . Or atleast didn't see it. this makes it easy to have workflows as code, create nocode editors in your own roadmap. You want people to hop from 1 company to another taking their hatchet workflows with them
Good luck, and sorry for coming off as rude. It's just a space I am very passionate about.
Re: Show HN: Hatchet v1 – A task orchestration platform built on Postgres
#73Here is my feedback after spending 15 mins on your docs. Nice work on the lite mode, open source, logging, dx interface. You may want to replace Hello world examples with real world scenarios. The workflows that involve multiple steps tasks, dag in your terminology - the code simply isn't intuitive. You now have to get into the hatchets mindset, patterns, terminology. Eg: the random number example is riddled with too…
Re: Show HN: Hatchet v1 – A task orchestration platform built on Postgres
#74Here is my feedback after spending 15 mins on your docs. Nice work on the lite mode, open source, logging, dx interface. You may want to replace Hello world examples with real world scenarios. The workflows that involve multiple steps tasks, dag in your terminology - the code simply isn't intuitive. You now have to get into the hatchets mindset, patterns, terminology. Eg: the random number example is riddled with too…
Are you saying the JavaScript api shouldn’t use types?
Re: Show HN: Hatchet v1 – A task orchestration platform built on Postgres
#75Earlier quoted context omitted.
Cloudtasks are excellent and I’ve been wanting something similar for years. I’ve been occasionally hacking away at a proof of concept built on riverqueue but have eased off for a while due to performance issues obvious with non-partitioned tables and just general laziness. https://github.com/jarshwah/dispatchr if curious but it doesn’t actually work yet.
Developer of River here ( https://riverqueue.com ). I'm curious if you ran into actual performance limitations based on specific testing and use cases, or if it's more of a hypothetical concern. Modern Postgres running on modern hardware and with well-written software can handle many thousands or tens of thousands of jobs per second (even without partitioning), albeit that depends on your workload, your tuning / auto…
To be clear, I really like the model of riverqueue and will keep going at a leisurely pace since this is a personal time interest at the moment. I’m sick of celery and believe a service is a better model for background tasks than a language-specific tool.
If you guys were to build http ingestion and http targets I’d try and deploy it right away.
Re: Show HN: Hatchet v1 – A task orchestration platform built on Postgres
#76Earlier quoted context omitted.
Developer of River here ( https://riverqueue.com ). I'm curious if you ran into actual performance limitations based on specific testing and use cases, or if it's more of a hypothetical concern. Modern Postgres running on modern hardware and with well-written software can handle many thousands or tens of thousands of jobs per second (even without partitioning), albeit that depends on your workload, your tuning / auto…
Perceived only at this stage, though the kind of volume we’re looking at is 10s to 100s of millions of jobs per day. https://github.com/riverqueue/river/issues/746 talks about some of the same things you mention. To be clear, I really like the model of riverqueue and will keep going at a leisurely pace since this is a personal time interest at the moment. I’m sick of celery and believe a service is a better model for…
> Perceived only at this stage, though the kind of volume we’re looking at is 10s to 100s of millions of jobs per day.
Yeah that's a little over 100 jobs/sec sustained :) Shouldn't be much of an issue on appropriate hardware and with a little tuning, in particular to keep your jobs table from growing to more than a few million rows and to vacuum frequently. Definitely hit us up if you try it and start having any trouble!
Re: Show HN: Hatchet v1 – A task orchestration platform built on Postgres
#77Amazing what you can do when you read the manual, eh?
Seriously though, that’s awesome, and I’m very happy to see someone leaning hard into RDBMS features like triggers instead of shying away.
Re: Show HN: Hatchet v1 – A task orchestration platform built on Postgres
#78One aspect I’d be curious to hear more about (and might be worth expanding on in docs or future posts) is how hatchet holds up operationally in production. For example, what does a typical alerting setup look like for common failure modes? And since the system relies on partitioned tables and tuned schemas, how do you approach migrations or schema changes without downtime?
A lot of open-source job orchestration systems shine at the core execution model but fall short when it comes to observability and smooth day-2 operations. If Hatchet nails that too, it’s a huge win.