Do you have anything comparing and contrasting with temporal?
I'm particularly interested in the scaling characteristics, and how your approach to durable storage (seems no external database is required?) differs
Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
101–110 of 112 posts
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#102This seems interesting! I couldn’t find an equivalent of the codec server in temporal that basically encrypts all data in the event log. Is there something similar?
Currently, Restate does not support this functionality out of the box. Since Restate does not need access to input/output messages or state (it ships it as bytes to the service endpoint), you could add your own client-side encryption mechanism. In the foreseeable future, Restate will probably add a more integrated solution for it.
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#103Do you have anything comparing and contrasting with temporal? I'm particularly interested in the scaling characteristics, and how your approach to durable storage (seems no external database is required?) differs
We will create a more detailed comparison to Temporal shortly. Until then @sewen gave a nice summarizing comparison here: https://news.ycombinator.com/item?id=40660568.
And yes, Restate does not have any external dependencies. It comes as a single self-contained binary that you can easily deploy and operate wherever you are used to run your code.
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#104Looks very interesting, but calling it Open Source is misleading. BSL is not "minimal Amazon defense". It effectively prevents any meaningful dynamic functionality to be built on top of it without a commercial subscription.
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#105Any plans for a Python SDK? We’re actively looking for a platform like this but our stack is TS / Python!
We are actively looking for feedback on what SDK to develop next. Quite a few people have voiced interest in Python so far. This will make it more likely that we might tackle this soonish. We'll keep you posted.
Python SDK +1
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#106For context (because he's too good to brag) OP is among the original creators of Apache Flink. Question for OP: I'd bet Flink's Statefuns comes in Restate's story. Could you please comment on this? Maybe Statefuns we're sort of a plugin, and you guys wanted to rebase to the core of a distributed function?
Thank you! Yes, Flink Stateful Functions were a first experiment to build a system for the use cases we have here. Specifically in Virtual Objects you can see that legacy. With Stateful Functions, we quickly realized that we needed something built for transactions, while Flink is built for analytics. That manifests in many ways, maybe most obviously in the latency: Transactional durability takes seconds in Flink (che…
Does the efficiency come from the raft implementation of distributed transactions or something else?
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#107Do you have anything comparing and contrasting with temporal? I'm particularly interested in the scaling characteristics, and how your approach to durable storage (seems no external database is required?) differs
We will create a more detailed comparison to Temporal shortly. Until then @sewen gave a nice summarizing comparison here: https://news.ycombinator.com/item?id=40660568 . And yes, Restate does not have any external dependencies. It comes as a single self-contained binary that you can easily deploy and operate wherever you are used to run your code.
Nice thanks. The ability to use cloud functions/lambdas is certainly intriguing and something I'd hoped would be possible with temporal when I first discovered it.
In a multi-replica / horizontally scaled setup:
- Does each replica get its own independent storage volume?
- How much state is replicated between each volume if so?
- What does a typical workloads journal look like in terms of storage size? How often does this get compacted / is there archival to cold storage?
- How do you manage upgrades to restate in the case of a change to the on-disk format? (Or is this designed to be static between releases)
The fact that it's without external dependencies also makes me wonder if it would be encouraged to have multiple independent deployments of restate managing different independent services/workflows to avoid noisy neighbors and single points of failure - seems like it might be lightweight enough that this is practical?
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#108Looks very interesting, but calling it Open Source is misleading. BSL is not "minimal Amazon defense". It effectively prevents any meaningful dynamic functionality to be built on top of it without a commercial subscription.
We tried to design the additional usage grant (https://github.com/restatedev/restate/blob/39f34753be0e27af8...) as permissive as possible. Our intention is to only prevent the big cloud service providers from offering Restate as a managed service as it has happened in the past with other open source projects. If you find the additional usage grant still too restrictive, then let us talk how to adjust it to enable you while still maintaining our initial intention.
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#109Earlier quoted context omitted.
Thank you! Yes, Flink Stateful Functions were a first experiment to build a system for the use cases we have here. Specifically in Virtual Objects you can see that legacy. With Stateful Functions, we quickly realized that we needed something built for transactions, while Flink is built for analytics. That manifests in many ways, maybe most obviously in the latency: Transactional durability takes seconds in Flink (che…
Does the efficiency come from the raft implementation of distributed transactions or something else?
Both systems pick different trade-offs:
Flink doesn't persist intermediate state synchronously at all. It runs asynchronous global snapshots in the background, which avoid capturing in-flight messages, just store state, aligned through epoch markers (a synchronization step). On a failure, typically seconds of work need to be redone. That's fine, because it is for analytics, and that approach results in good throughput.
Restate won't start step 2 of a sequence before step 1's result is durable, so it needs to make sure that this durability is achieved quickly. It does frequent (batched) log appends, and each partition does that by itself without synchronizing with others. The result is faster (latency) because it is more fine grained and has less coordination, but it is also more work that is done.
Re: Show HN: Restate – Low-latency durable workflows for JavaScript/Java, in Rust
#110I'm not sure "In Rust" serve any marketing value. A product's success rarely has to do with the use of a programming language, if not at all. I understand the arguments made by Paul Graham on the effectiveness of programming languages, but specifically for a workflow manager, a user like me cares literally zero about which programming language the workflow system uses even if I have to hack into the internal of the s…
You are free to ignore it. Personally I like to see new projects be made in Rust, because it means they're easier to contribute to than projects in other unmanaged non-GC languages.