Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Temporal: open-source microservices orchestration platform

temporal.io

Post reply on HN
Print view View original

31–40 of 53 posts

Re: Temporal: open-source microservices orchestration platform

#31

Post by tsujp »

rytemporal wrote: post #21
sgt wrote: post #12

How would it work with existing HashiCorp tech like Nomad?

It’s a very valid question but it’s just a bit timely since Mitchell tweeted a few days ago about how most of HashiCorp cloud is built on Temporal tech. Disclaimer: head of product at Temporal. Temporal is not container orchestration and is not an infrastructure management tool. In most cases users run Temporal on top of Kubernetes. Temporal provides a distributed experience which is decoupled from the reliability of…

So instead of say Kubernetes I could use Nomad for orchestration of my todo list MVC and Temporal on that same Nomad host, and others, and non-Nomad hosts, for my distributed data pipelines, or cleanup jobs, or email marketing etc right? What I mean to say is: is that scenario one valid/proper use of Temporal as you all envision it especially within the context of tools HashiCorp provides too.

Re: Temporal: open-source microservices orchestration platform

#32

Post by slotrans »

This is super interesting but I have what feels like a basic/dumb question...

Let's say I have some workflow which executes over a period of days or weeks, and I want to deploy a new version of it. How does that work? What happens to workflow instances that are currently executing?

After writing the above I clicked back to the site to see if this was addressed, and found this https://docs.temporal.io/docs/java-versioning and... surely you're not serious?

Re: Temporal: open-source microservices orchestration platform

#33

Post by rytemporal »

tsujp wrote: post #31
rytemporal wrote: post #21

Earlier quoted context omitted.

It’s a very valid question but it’s just a bit timely since Mitchell tweeted a few days ago about how most of HashiCorp cloud is built on Temporal tech. Disclaimer: head of product at Temporal. Temporal is not container orchestration and is not an infrastructure management tool. In most cases users run Temporal on top of Kubernetes. Temporal provides a distributed experience which is decoupled from the reliability of…

So instead of say Kubernetes I could use Nomad for orchestration of my todo list MVC and Temporal on that same Nomad host, and others, and non-Nomad hosts, for my distributed data pipelines, or cleanup jobs, or email marketing etc right? What I mean to say is: is that scenario one valid/proper use of Temporal as you all envision it especially within the context of tools HashiCorp provides too.

Temporal doesn't have an opinion on how you manage your infrastructure. Most users consume our docker images but there is zero reason you can't compile binaries and run on bare metal.

That being said, Temporal backend consists of a few stateless and horizontally scalable services (matching service, frontend service etc). Because these roles experience load differently it often makes sense to scale them separately. Due to this design, users often find it convenient to use an orchestration solution such as Kubernetes, ECS etc. HashiCorp themselves run our technology using Nomad to directly answer your question.

The only thing we are strongly opinionated about is that you run the underlying database in a production-grade manner. Throwing a MySQL container into a helm chart isn't going to cut it for serious usage.

Re: Temporal: open-source microservices orchestration platform

#34

Post by mfateev »

slotrans wrote: post #32

This is super interesting but I have what feels like a basic/dumb question... Let's say I have some workflow which executes over a period of days or weeks, and I want to deploy a new version of it. How does that work? What happens to workflow instances that are currently executing? After writing the above I clicked back to the site to see if this was addressed, and found this https://docs.temporal.io/docs/java-versio…

Your question is actually is pretty important as upgrading long running applications is indeed non trivial problem.

The standard approach of versioning the whole workflow is OK for introducing new features but doesn't really support bug fixing. For example, a workflow is expected to run for three months. A bug is found at the end of this workflow definition. The bug is fixed and deployed as a new version, but all workflows that started up to the fix and used previous versions will keep failing for the next three months. So there is a need to patch workflow without changing its version.

The approach Temporal takes is that every part of the code is versioned independently. This allows deploying changes at any time, even for libraries shared by multiple workflows, and doesn't require running multiple worker versions.

Re: Temporal: open-source microservices orchestration platform

#35

Post by spankalee »

slotrans wrote: post #32

This is super interesting but I have what feels like a basic/dumb question... Let's say I have some workflow which executes over a period of days or weeks, and I want to deploy a new version of it. How does that work? What happens to workflow instances that are currently executing? After writing the above I clicked back to the site to see if this was addressed, and found this https://docs.temporal.io/docs/java-versio…

That seems liked quite a good way to handle things, because it lets you do things like cancel, restart, or migrate an in-flight instance on any way including dynamically based on its state. It also lets you handle any number of versions.

Re: Temporal: open-source microservices orchestration platform

#36

Post by slotrans »

mfateev wrote: post #34
slotrans wrote: post #32

This is super interesting but I have what feels like a basic/dumb question... Let's say I have some workflow which executes over a period of days or weeks, and I want to deploy a new version of it. How does that work? What happens to workflow instances that are currently executing? After writing the above I clicked back to the site to see if this was addressed, and found this https://docs.temporal.io/docs/java-versio…

Your question is actually is pretty important as upgrading long running applications is indeed non trivial problem. The standard approach of versioning the whole workflow is OK for introducing new features but doesn't really support bug fixing. For example, a workflow is expected to run for three months. A bug is found at the end of this workflow definition. The bug is fixed and deployed as a new version, but all wor…

I get the idea but it seems clear that it will result in unmanageable spaghetti code in very little time. This model forces the developer to entangle versioning with logic. Since a big part of Temporal's value proposition is dis-entangling logic from state management and other orchestration, I expect you can see why this is a problem.

Put another way, it's a great big leak in the abstraction.

Re: Temporal: open-source microservices orchestration platform

#37

Post by mfateev »

slotrans wrote: post #36
mfateev wrote: post #34

Earlier quoted context omitted.

Your question is actually is pretty important as upgrading long running applications is indeed non trivial problem. The standard approach of versioning the whole workflow is OK for introducing new features but doesn't really support bug fixing. For example, a workflow is expected to run for three months. A bug is found at the end of this workflow definition. The bug is fixed and deployed as a new version, but all wor…

I get the idea but it seems clear that it will result in unmanageable spaghetti code in very little time. This model forces the developer to entangle versioning with logic. Since a big part of Temporal's value proposition is dis-entangling logic from state management and other orchestration, I expect you can see why this is a problem. Put another way, it's a great big leak in the abstraction.

I see your point. But so far we didn't hear many complaints about this feature from the users. The main problem is in forgetting to version code that changed. But the system has protections against such cases.

It doesn't end up in spaghetti code as old branches are proactively removed after they are not needed anymore. Temporal provides APIs that allow to count number of workflows using each version.

I don't think it is a leaky abstraction as it is indeed a part of the business logic of deciding how old version of the state should become a new one. I don't think there is a generic solution that allows implicitly migrate old state to a new state on any long running computation.

Re: Temporal: open-source microservices orchestration platform

#38

Post by jeffnappi »

Temporal is a company/project to watch. The founders know the problem space with as much depth as might be possible. They were responsible for AWS Simple Workflow, then built Cadence Workflow at Uber to solve Uber's workflow/orchestration issues - and later formed Temporal and forked Cadence to build a company around.

I'm excited for Maxim, Samar and team :)

Re: Temporal: open-source microservices orchestration platform

#39

Post by jacques_chester »

mfateev wrote: post #37
slotrans wrote: post #36

Earlier quoted context omitted.

I get the idea but it seems clear that it will result in unmanageable spaghetti code in very little time. This model forces the developer to entangle versioning with logic. Since a big part of Temporal's value proposition is dis-entangling logic from state management and other orchestration, I expect you can see why this is a problem. Put another way, it's a great big leak in the abstraction.

I see your point. But so far we didn't hear many complaints about this feature from the users. The main problem is in forgetting to version code that changed. But the system has protections against such cases. It doesn't end up in spaghetti code as old branches are proactively removed after they are not needed anymore. Temporal provides APIs that allow to count number of workflows using each version. I don't think it…

You might find the concept of Edit Lenses to be useful. A good writeup is: https://www.inkandswitch.com/cambria.html

Re: Temporal: open-source microservices orchestration platform

#40

Post by jacques_chester »

nrmitchi wrote: post #15
hestefisk wrote: post #13

Has anyone here used it for anything serious? Looks interesting but also a lot of buzzwords...

I suspect it is here on HN (or the timing is a coincidence) because it was mentioned my mitchellh as a component of what is powering the Hashicorp Cloud offerings in a comment here mid-last week.

They also closed a fundraising round recently.
Post reply on HN