Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Solving durable execution's immutability problem

restate.dev

Post reply on HN
Print view View original

11–20 of 29 posts

Re: Solving durable execution's immutability problem

#11

Post by pavel_pt »

stsffap wrote: post #2

The hardest problem in durable execution, as in many areas of infrastructure, is safe updates.

Not just in durable execution, but in control systems in general! Anything that operates on persisted state in a distributed environment has similar issues. Durable execution frameworks make them more apparent - and may also help us chart a path for how to deal with updates more rigorously.

Re: Solving durable execution's immutability problem

#12

Post by pavel_pt »

whoiskatrin wrote: post #8
JackHopkins wrote: post #6

Earlier quoted context omitted.

Can't you replay pending requests? How can you mitigate the issue of differing side effects generated by the new / old versions?

also would be great to see a deep comparison with Temporal and AWS Func

Are there any aspects in particular that would be of interest?

Re: Solving durable execution's immutability problem

#13

Post by whoiskatrin »

pavel_pt wrote: post #12
whoiskatrin wrote: post #8

Earlier quoted context omitted.

also would be great to see a deep comparison with Temporal and AWS Func

Are there any aspects in particular that would be of interest?

I'd love a deep technical comparison, but it would also be great to understand if Restate is better than Temporal for specific use cases and vice versa. When someone should choose one of them over another

Re: Solving durable execution's immutability problem

#18

Post by p10jkle »

threepointone wrote: post #15

Curious how you think about state that changes outside of a workflow’s scope (like user info in a database)

In general, if you're changing database schemas, you need a code version running that supports old and new. If you have some old code running to serve in-flight requests, you probably need to wait for those requests to complete. This is one of the main areas that makes me feel that requests > a few hours are a real problem (and that we shouldn't be writing workflows this way)

Re: Solving durable execution's immutability problem

#19

Post by stsffap »

danism wrote: post #17
akbirkhan wrote: post #10

this is pretty important when you move to LLMs as microservices (e.g. calling gpt-4 for some plan / then 3.5 for easier sub component). Would be keen for a python implementation

Is there a timeline for a Python SDK?

We are currently gathering feedback on which SDKs to prioritize next. Python has been asked for a couple of times already. Once we decide on the next SDK, we'll let you know.

Re: Solving durable execution's immutability problem

#20

Post by tveita »

p10jkle wrote: post #9
JackHopkins wrote: post #6

Earlier quoted context omitted.

Can't you replay pending requests? How can you mitigate the issue of differing side effects generated by the new / old versions?

if you replay the request on the new version then you might encounter new steps that don't match what you have in the journal. temporal users know this pain well...

Rolling a task over to a new version should be "safe" in that you can detect conflicts and roll back if the sequence of calls does not match the old version.

For a post about "solving" durable execution I would expect both a scale-to-zero way to keep older versions around indeterminately - I guess the Lambda based approach does qualify - and a safe and controlled way to upgrade task versions iff the execution history is compatible.

Post reply on HN