Temporal .NET – Deterministic Workflow Authoring in .NET • Hacker News Board

Re: Temporal .NET – Deterministic Workflow Authoring in .NET

#11

Post by kodablah » Wed, May 03, 2023, 5:26 PM UTC

lukevp wrote: post #7

lorendsr wrote: post #2

For those not familiar with workflows as code, a workflow is a method that is executed in a way that can't fail—each step the program takes is persisted, so that if execution is interrupted (the process crashes or machine loses power), execution will be continued on a new machine, from the same step, with all local/instance variables, threads, and the call stack intact. It also transparently retries network requests…

What happens if you have a workflow in progress that you want to change the implementation of? Eg. If I had a workflow that was waiting for 30 days but then decided I wanted that interval to be 7 days instead?

Temporal determines a workflow is non-deterministic if upon code replay the high-level commands don't match up with what happened during the original code execution. In this case, technically it's safe to change the code this way because both still result timer commands. But what the timer was first created with on existing runs is what applies (there are reset approaches though as mentioned in other comment). However if, say, you changed the implementation do start a child workflow or activity before the timer, the commands would mismatch and you'd get a non-determinism error upon replay.

There are multiple strategies to update live workflows, see https://community.temporal.io/t/workflow-versioning-strategi.... Most often for small changes, people would use the `Workflow.Patched` API.

Re: Temporal .NET – Deterministic Workflow Authoring in .NET

#12

Post by kodablah » Wed, May 03, 2023, 5:31 PM UTC

0xcoffee wrote: post #10

kodablah wrote: post #4

Earlier quoted context omitted.

That's basically what is happening here, except you create `e` ahead of time instead of a do-anything-you-want lambda. But they essentially the same thing, though I think `ExecuteActivityAsync(MyActivities.Ref.DoPurchaseAsync, etc...)` is clearer that you are referencing an activity. And since it's just a delegate that an attribute is on, you can write a helper to do what you have instead.

The main reason I'm curious is because they write 'We solve this problem by allowing users to create instances of the class/interface without invoking anything on it. For classes this is done via FormatterServices.GetUninitializedObject and for interfaces this is done via Castle Dynamic Proxy. This lets us "reference" (hence the name Ref) methods on the objects without actually instantiating them with side effects. M…

> Which sounds like a lot of heavy lifting

How is `e` created for the `e => e.DoPurchaseAsync(purchase)` lambda? You're going to have to do that lifting anyways to create an instance for `e` that isn't really a usable instance. Unless you use source generators which we plan on doing.

I think what you have there is a lot more heavy lifting. Also note that workflows and activities are unrelated to each other. Workflow can invoke any activities. The code you have is a bit confusing because `PurchaseActivities` should be completely unrelated to workflows.

Re: Temporal .NET – Deterministic Workflow Authoring in .NET

#13

Post by mirsadm » Wed, May 03, 2023, 5:32 PM UTC

lorendsr wrote: post #2

For those not familiar with workflows as code, a workflow is a method that is executed in a way that can't fail—each step the program takes is persisted, so that if execution is interrupted (the process crashes or machine loses power), execution will be continued on a new machine, from the same step, with all local/instance variables, threads, and the call stack intact. It also transparently retries network requests…

It is a replica of Amazon Simple Workflow as far as I can see. Which is not a bad thing, SWF is great and does not get much attention.

Re: Temporal .NET – Deterministic Workflow Authoring in .NET

#14

Post by kodablah » Wed, May 03, 2023, 5:32 PM UTC

Merad wrote: post #9

kodablah wrote: post #4

Earlier quoted context omitted.

That's basically what is happening here, except you create `e` ahead of time instead of a do-anything-you-want lambda. But they essentially the same thing, though I think `ExecuteActivityAsync(MyActivities.Ref.DoPurchaseAsync, etc...)` is clearer that you are referencing an activity. And since it's just a delegate that an attribute is on, you can write a helper to do what you have instead.

If you're worried about someone doing something other than calling a method in the lambda it's pretty straightforward to use an Expression > and validate the contents of the expression. Seems like that would make for a much better dev experience than this static Ref property pattern that you don't see anywhere else in C#.

It's not just that worry, it's the other reasons on top of that detailed in the post. But note that GP's post doesn't _call_ the method, it just references it which is basically what we are doing in the post. But for things like Durable Entities which _do_ call the method, yes we can prevent multi-use of the lambda arg but there are other problems (forcing interface/virtual, can't allow other args, sync vs async, etc).

Re: Temporal .NET – Deterministic Workflow Authoring in .NET

#15

Post by lorendsr » Wed, May 03, 2023, 5:33 PM UTC

lukevp wrote: post #7

lorendsr wrote: post #2

For those not familiar with workflows as code, a workflow is a method that is executed in a way that can't fail—each step the program takes is persisted, so that if execution is interrupted (the process crashes or machine loses power), execution will be continued on a new machine, from the same step, with all local/instance variables, threads, and the call stack intact. It also transparently retries network requests…

What happens if you have a workflow in progress that you want to change the implementation of? Eg. If I had a workflow that was waiting for 30 days but then decided I wanted that interval to be 7 days instead?

After you deploy the new workflow code, you can reset [1] a workflow's execution state to before the DelayAsync statement was called, and then the workflow will sleep for 7 days from now.

That doesn’t take into account the time it’s already been waiting. For when you want to do something on a schedule and be able to edit the schedule in future, there’s a Schedules feature that allows you to periodically start a workflow, like a cron but more flexible. [2] In this case, the workflow code would be simpler, like just ChargeCustomer() and SendEmailNotification().

[1] https://docs.temporal.io/cli/workflow#reset

[2] https://docs.temporal.io/workflows#schedule

Re: Temporal .NET – Deterministic Workflow Authoring in .NET

#16

Post by kodablah » Wed, May 03, 2023, 5:33 PM UTC

mirsadm wrote: post #13

lorendsr wrote: post #2

For those not familiar with workflows as code, a workflow is a method that is executed in a way that can't fail—each step the program takes is persisted, so that if execution is interrupted (the process crashes or machine loses power), execution will be continued on a new machine, from the same step, with all local/instance variables, threads, and the call stack intact. It also transparently retries network requests…

It is a replica of Amazon Simple Workflow as far as I can see. Which is not a bad thing, SWF is great and does not get much attention.

That's no coincidence, Temporal is founded by the creators of Amazon Simple Workflow. See https://temporal.io/about.

Re: Temporal .NET – Deterministic Workflow Authoring in .NET

#17

Post by 0xcoffee » Wed, May 03, 2023, 5:39 PM UTC

kodablah wrote: post #12

0xcoffee wrote: post #10

Earlier quoted context omitted.

The main reason I'm curious is because they write 'We solve this problem by allowing users to create instances of the class/interface without invoking anything on it. For classes this is done via FormatterServices.GetUninitializedObject and for interfaces this is done via Castle Dynamic Proxy. This lets us "reference" (hence the name Ref) methods on the objects without actually instantiating them with side effects. M…

> Which sounds like a lot of heavy lifting How is `e` created for the `e => e.DoPurchaseAsync(purchase)` lambda? You're going to have to do that lifting anyways to create an instance for `e` that isn't really a usable instance. Unless you use source generators which we plan on doing. I think what you have there is a lot more heavy lifting. Also note that workflows and activities are unrelated to each other. Workflow…

>You're going to have to do that lifting anyways to create an instance for `e` that isn't really a usable instance.

But this should never happen, these is no reason to create an unusable instance. The real instance should be resolved in the same way however the current workflow resolves it.

Re: Temporal .NET – Deterministic Workflow Authoring in .NET

#18

Post by kodablah » Wed, May 03, 2023, 5:49 PM UTC

borissk wrote: post #5

Wonder how does it compare to https://github.com/UiPath/CoreWF

Hrmm, I admit not knowing a lot about Windows Workflow Foundation, but from a quick glance I suspect they both have roots in the same place - deterministic workflow authoring that invokes activities like Amazon SWF and Azure Durable Task Framework. So they are probably quite similar, yet CoreWF seems to prefer YAML or limited code-based DSL whereas Temporal prefers to provide the full language with determinism constraints.

Re: Temporal .NET – Deterministic Workflow Authoring in .NET

#19

Post by kodablah » Wed, May 03, 2023, 5:53 PM UTC

0xcoffee wrote: post #17

kodablah wrote: post #12

Earlier quoted context omitted.

> Which sounds like a lot of heavy lifting How is `e` created for the `e => e.DoPurchaseAsync(purchase)` lambda? You're going to have to do that lifting anyways to create an instance for `e` that isn't really a usable instance. Unless you use source generators which we plan on doing. I think what you have there is a lot more heavy lifting. Also note that workflows and activities are unrelated to each other. Workflow…

>You're going to have to do that lifting anyways to create an instance for `e` that isn't really a usable instance. But this should never happen, these is no reason to create an unusable instance. The real instance should be resolved in the same way however the current workflow resolves it.

Activities may actually run on completely different systems than where the workflow calls execute on. All "execute activity" is from a workflow perspective is telling Temporal server to execute an activity with a certain string name and serializable arguments. Everything else is sugar. So if you're gonna use a type caller-side to refer to the name and argument types, you can't instantiate it fully (its constructor may have side effects for when it really runs).

You can jump through a bunch of hoops like requiring interfaces which is what some frameworks do. But in our case, we just decided to make it easy to reference the method without invoking it or its instance.

Re: Temporal .NET – Deterministic Workflow Authoring in .NET

#20

Post by 0xcoffee » Wed, May 03, 2023, 6:02 PM UTC

kodablah wrote: post #19

0xcoffee wrote: post #17

Earlier quoted context omitted.

>You're going to have to do that lifting anyways to create an instance for `e` that isn't really a usable instance. But this should never happen, these is no reason to create an unusable instance. The real instance should be resolved in the same way however the current workflow resolves it.

Activities may actually run on completely different systems than where the workflow calls execute on. All "execute activity" is from a workflow perspective is telling Temporal server to execute an activity with a certain string name and serializable arguments. Everything else is sugar. So if you're gonna use a type caller-side to refer to the name and argument types, you can't instantiate it fully (its constructor ma…

Lamba/Func/Expressions are doing exactly this in C#, there is no instantiation required. Creating a unusable Ref object is jumping through hoops.

You can parse an expression to serialize it and run it on a different server etc

See e.g. https://github.com/6bee/Remote.Linq