Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Temporal Python – A durable, distributed asyncio event loop (2023)

temporal.io

Post reply on HN
Print view View original

41–50 of 56 posts

Re: Temporal Python – A durable, distributed asyncio event loop (2023)

#41

Post by 015a »

no_wizard wrote: post #40
015a wrote: post #33

Earlier quoted context omitted.

We migrated from an in-house redis queuing system. Temporal has its own way of doing things; there's rules about what you can and cant do in workflows, what has to live in activities, etc. Its generally quite easy to adapt existing code work with it. We use typescript. The worst part for us has been error/anomaly handling. Workflows can sometimes hit a state where the status reads in progress and errors aren't report…

Would you not like it if you didn’t self host? If I’m being honest if it is a big issue to self host but it’s value to developers is obvious and apparent why not pay?

Nah we'd probably be fine paying temporal cloud to host the control plane. Their billing is a little weird; I know quite a bit about temporal-the-technology, and the pricing page is literally the first time I've ever seen the word "action" used. I'm familiar with workflows, activities, sinks, codecs, events, but not actions; so when they bill $N/million actions I have no idea what that means, and its surprising to me that that's how they bill it. But I'm sure there's an answer somewhere.

Temporal Cloud is really, really new. Like, it was in some kind of closed beta for a while, with a "contact us" form, as recently as a couple months ago? So, the main reason we don't use it is because it simply wasn't available. It looks like its more widely available now though.

Re: Temporal Python – A durable, distributed asyncio event loop (2023)

#42

Post by addisonj »

It is interesting seeing the comments here, the comments from adopters is there is a lot of value but it taking time to get up to speed. Those new to temporal a lot of questions seeking understanding.

I have spent a lot of time in the adjacent space of event driven systems and there, like here, it seems like some of the biggest challenge is just education.

I wouldn't say that EDA or workflow based systems are preferable to traditional API services with DBs, just that the space they occupy in the industry is so large that I think it is really really hard if you to introducing any different paradigms, even when you focus on domains where API services aren't a great fit (like here with long running, complex operations).

My point with this comment is simply that I think if you are trying to build anything that does things differently, developer education is as important or even more important than design and architecture, but often not considered because those building these systems are already so deep into it that they can't approach the problem as an outsider.

Re: Temporal Python – A durable, distributed asyncio event loop (2023)

#43

Post by BiteCode_dev »

I like the concept.

The nice things is that it abstract the conditions checks on whether something is done, has succeeded or should be retried.

The bad things is that it abstract the conditions checks on whether something is done, has succeeded or should be retried.

It's nice because that's something you do again and again, and that's a lot of code. A lot of ways it can go wrong.

But it's bad because that's a huge chunk of black box magic that may execute remotely. If you need a custom or more optimized behavior anywhere in this logic, you are done for. If there is a bug/problem in this logic, it's game over. I also have to imagine debugging and error reporting is likely not super fun.

One point in particular that strikes me, is that impotence is generically guaranteed with something like "has this task executed without error last time". But usually, what I want is something much more specific, like "has that entries been updated", "has that file been created" and so on. From a bird view, it looks the same, but from a system reliability point of you, they are not at all the same.

Hard to see how they avoid duplicate results, overlapping tasks, etc.

I don't think they really can at that level of abstraction, which means you need to implement it manually.

Eventually it seems it's a huge dep to bring in for the actual practical problem is really solves well.

But I'm willing to be proven wrong on this one, because the tech is really damn cool.

Re: Temporal Python – A durable, distributed asyncio event loop (2023)

#46

Post by ub-volta-toss »

Temporal is really neat but I think its marketed at too many use cases.

After a year of high-scale Temporal work, I found it was only good for low-scale work.

The onboarding and learning curve were insanely difficult and complex. Ultimately it doesn't scale as well as you think. The temporal team invented their own database to get around this limitation.

Re: Temporal Python – A durable, distributed asyncio event loop (2023)

#47

Post by claytonjy »

ub-volta-toss wrote: post #46

Temporal is really neat but I think its marketed at too many use cases. After a year of high-scale Temporal work, I found it was only good for low-scale work. The onboarding and learning curve were insanely difficult and complex. Ultimately it doesn't scale as well as you think. The temporal team invented their own database to get around this limitation.

Would love to hear more about the scale issues you saw. How many workflows or actions was too many? which components started breaking down, what were their failure modes?

Re: Temporal Python – A durable, distributed asyncio event loop (2023)

#48

Post by pm »

7bit wrote: post #22
kodablah wrote: post #21

Earlier quoted context omitted.

Not necessarily "equal" but the basic premise is the same, yes, and there is a common lineage. Azure Durable Functions sits on Azure Durable Task Framework which was created by the co-founder of Temporal ( https://temporal.io/about ). (disclaimer, I'm the author of the post)

Ohh, that's great to hear! I do like ADF, but the Python worker is full of bugs and weird behaviour and tickets stay open for month without progress. I will definitely check that out!

Working on a freelance job 3 years ago, I got sucked down a rabbit-hole for months trying to get Azure Durable Functions to work. Too many bugs, no visibility into its workings and the worker would always grind to a halt. That job did not go well.

Avoid ADF (let alone Azure) until you've got some innovation tokens to spend, or go with another provider.

Re: Temporal Python – A durable, distributed asyncio event loop (2023)

#49

Post by NoThisIsMe »

ub-volta-toss wrote: post #46

Temporal is really neat but I think its marketed at too many use cases. After a year of high-scale Temporal work, I found it was only good for low-scale work. The onboarding and learning curve were insanely difficult and complex. Ultimately it doesn't scale as well as you think. The temporal team invented their own database to get around this limitation.

Invented their own database? They use Cassandra IIRC
Post reply on HN