The label "Sign in with your corporate ID" for GitHub sign in seems a little odd..
Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
51–60 of 112 posts
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#52Earlier quoted context omitted.
Our goal is to make it easier to write code that handles failures - failed outbound api calls, infrastructure issues like a host dying, problems talking between services. The primitive we offer is that we guarantee that your handlers always run to completion (whether to a result or a terminal error) The way we do that is by writing down what your code is doing, while its doing it, to a store. Then, on any failure, we…
This assumes that the APIs work this way? What if the first call is to get a resource that expires and then the last call fails? Now it will retry but with an expired resource (first call is saved).
I think you would need to validate the response from the first call before determining it to be a success?
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#53How does Restate compare with Apache Airflow or Prefect?
Disclaimer, I am not an Airflow expert and even less of a Prefect expert.
One difference is that Airflow seems geared towards heavier operations, like in data pipelines. In contrast, would be that Restate is not by default spawning any tasks, but it acts more of a proxy/broker for RPC- or event handlers and adds durable retries, journaling, ability to make durable RPCs, etc.
That makes it quite lightweight: If the handlers is fast in a running container, the whole thing results in super fast turnaround times (milliseconds).
You can also deploy the handlers on FaaS and basically get the equivalent of spawning a (serverless task) per step.
The other difference would be the way that the logic is defined, can maintain state, can make exactly-once calls to other handlers.
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#54Being fairly familiar with Temporal, I definitely appreciate your cleaner architectural choices. Add a Go SDK and I’ll definitely give this a try.
Someone already contributed an MVP; in the next few months we'll be adopting it fully and upgrading it to 1.0 (we hired the awesome Azmy after he built it) https://github.com/muhamadazmy/restate-sdk-go
When you do consider adopting it fully, I highly recommend trying to make the state handling as transparent as possible to the end consumer. For example, implementing an HTTP Client that wraps a http.RoundTripper versus what that SDK provides.
Evaluating a selection of these durable workflow SDKs for Go, I'm not keen on being tightly coupled to a vendor and the implementation shouldn't be that crazy to fit into existing Go interfaces.
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#55Earlier quoted context omitted.
Our goal is to make it easier to write code that handles failures - failed outbound api calls, infrastructure issues like a host dying, problems talking between services. The primitive we offer is that we guarantee that your handlers always run to completion (whether to a result or a terminal error) The way we do that is by writing down what your code is doing, while its doing it, to a store. Then, on any failure, we…
So non-response time bound workloads that need to reliably dispatch other processes to completion? Would a good example be something like, automated highway toll collecting? i.e. I drive past a scanner on the highway, my license plate is scanned and several state bound collection events need to be triggered until the toll is ultimately collected?
Yes, definitely, but we can also cover response time bound tasks! Not just async. Typical p90 of a 3-step workflow is 50ms. Our goal is to run on every RPC, anywhere you need reliability
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#56The label "Sign in with your corporate ID" for GitHub sign in seems a little odd..
I think its a cognito default - will take a look!
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#57I still haven't gotten around to adopting Restate yet, but it's on the radar. One thing that Step Functions probably has over Restate is the diagram visualization of your state machine definition and execution history. It's been really neat to be able to zero in on a root cause at the conceptual level instead of the implementation level.
One big hangup for me is that there's only a single node orchestrator as a CDK construct. Having a HA setup would be a must for business critical flows.
I stumbled on Restate a few months ago and left the following message on their discord.
> I was considering writing a framework that would let you author AWS Step Functions workflows as code in a typesafe way when I stumbled on Restate. This looks really interesting and the blog posts show that the team really understands the problem space.
> My own background in this domain was as an early user of AWS SWF internally at AWS many, many years ago. We were incredibly frustrated by the AWS Flow framework built on top of SWF, so I ended up creating a meta Java framework that let you express workflows as code with true type-safety, arrow function based step delegations, and leveraging Either/Maybe/Promise and other monads for expressiveness. The DX was leaps and bounds better than anything else out at the time. This was back around 2015, I think.
> Fast-forward to today, I'm now running a startup that uses AWS Step Functions. It has some benefits, the most notable being that it's fully serverless. However, the lack of type-safety is incredibly frustrating. An innocent looking change can easily result in States.Runtime errors that cannot be caught and ignore all your catch-error logic. Then, of course, is how ridiculous it feels to write logic in JSON or a JSON-builder using CDK. As if that wasn't bad enough, the pricing is also quite steep. $25 for every million state transitions feels like a lot when you need to create so many extra state transitions for common patterns like sagas, choice branches, etc.
> I'm looking forward to seeing how Restate matures!
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#58I still haven't gotten around to adopting Restate yet, but it's on the radar. One thing that Step Functions probably has over Restate is the diagram visualization of your state machine definition and execution history. It's been really neat to be able to zero in on a root cause at the conceptual level instead of the implementation level. One big hangup for me is that there's only a single node orchestrator as a CDK c…
A visualisation/dashboard is a top priority!
Distributed architecture (to support multiple nodes for HA and horizontal scaling) is being actively worked on and will land in the coming months
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#59Earlier quoted context omitted.
Doesn't anything involving requests to other services inherently have to be idempotent because there's still a chance of a communication error resulting in an unknown outcome of the action? You don't know if the "widget order" was successfully placed or not, and therefore there's no way to know if that action can safely be tried again.
https://news.ycombinator.com/item?id=40659968 Absolutely, sorry if im not tight enough with my language. Maybe should be described as 'operation idempotency' vs 'handler idempotency'. IMO, an entire handler re-executing is much harder to reason about and test for than a particular operation re-executing individually, with nothing else changing between executions
A special case is if the operation is calling another Restate service. In this case, Restate will make sure that the callee will be executed exactly once and there is no need for the user to pass an idempotency key or something similar. Only when interacting with the external world from a Restate service, the operation needs to be idempotent.
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#60how do tools like this handle evolving workflows? e.g., if I have a "durable worklflow" that sleeps for a month and then performs its next actions, what do I do if I need to change the workflow during that month? I really like the concept but this seems like an issue for anything except fairly short workflows. If I keep my data and algorithms separate I can modify my event handling code while workflows are "active."
I wrote two blog posts on this! It's a really hard problem https://restate.dev/blog/solving-durable-executions-immutabi... https://restate.dev/blog/code-that-sleeps-for-a-month/ The key takeaways: 1. Immutable code platforms (like Lambda) make things much more tractable - old code being executable for 'as long as your handlers run' is the property you need. This can also be achieved in Kubernetes with some clever con…
ah! this took me a second to grok, but from #2 above: "we just want to send the email service a request that we want to be processed in a month. The thing that hangs around ‘in-flight’ wouldn’t be a journal of a partially-completed workflow, with potentially many steps, but instead a single request message."
I'll have to think through how much that solves, but it's a new insight for me - thanks!
I like that you're working on this. seems tricky, but figuring out how to clearly write workflows using this pattern could tame a lot of complexity.