Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

When "letting it crash" is not enough

flawless.dev

Post reply on HN
Print view View original

81–84 of 84 posts

Re: When "letting it crash" is not enough

#81

Post by vagab0nd »

I happen to have this exact need come up recently. I have a few Bash scripts that I run once in a while. They evolve each time I run them, because often I will discover some missing things, or improve them a bit as I use them. They also fail a lot, due to the experimental nature.

Now the problem is that every time they fail, I need to restart it. But not from the beginning, but where it failed. So I comment out everything before the failure and rerun.

It would be nice to use durable execution for this.

Re: When "letting it crash" is not enough

#82

Post by solatic »

dnr wrote: post #80
solatic wrote: post #6

Flawless sounds a lot like https://temporal.io/ . I'm wondering if it has the same scalability concerns - sticking everything in Postgres is fine at small-ish scale, but what happens when you outgrow Postgres, either because you have higher availability requirements (can't handle primary DB restarts) or because of the sheer volume of the workload?

Temporal can also run on Cassandra, which scales much larger than Postgres (if you put in enough effort). It can also be replicated across regions for high availability. It's already running some pretty huge use cases. (I work at Temporal)

Oh come on, when you click through the setup through to Cassandra the documentation states that cassandra support was deprecated in 1.21 and to migrate to a "supported" database: https://docs.temporal.io/self-hosted-guide/visibility#cassan...

Re: When "letting it crash" is not enough

#84

Post by dnr »

solatic wrote: post #82
dnr wrote: post #80

Earlier quoted context omitted.

Temporal can also run on Cassandra, which scales much larger than Postgres (if you put in enough effort). It can also be replicated across regions for high availability. It's already running some pretty huge use cases. (I work at Temporal)

Oh come on, when you click through the setup through to Cassandra the documentation states that cassandra support was deprecated in 1.21 and to migrate to a "supported" database: https://docs.temporal.io/self-hosted-guide/visibility#cassan...

You're looking at the docs for "visibility". Visibility is a separate eventually-consistent data store off to the side that's used for certain queries so it can be scaled independently of the main data store, and indexed in fancier ways. The main data store for all the stateful and transactional stuff has always, and probably will always, support Cassandra. For visibility, the recommendation for high scalability is currently Elasticsearch.

Temporal may have properties that make it not a good fit for a particular use case, but scalability is really not one.

Post reply on HN