When "letting it crash" is not enough
flawless.dev
When "letting it crash" is not enough
1–10 of 84 posts
Re: When "letting it crash" is not enough
#2I thought the idea of "exactly once" was a very questionable claim given what we know about computing, specially distributed.
Then somewhere else you see this gem.
> Workflows in flawless are written in Rust, in fact they are just regular Rust functions. This means that they can contain arbitrary logic. But instead of native code, the functions are compiled to WebAssembly and executed in a completely deterministic environment.
As much as I love Rust, this sounds like, here is a problem, let me throw fancy Rust and WebAssembly and that should fix it.
Re: When "letting it crash" is not enough
#3The approach of check-pointing computation such that it is restartable sounds similar to a time-traveling debugger, like rr or WinDbg:
https://learn.microsoft.com/windows-hardware/drivers/debugge...
Some Googling found Checkpoint/Restore In Userspace, or CRIU. It’s like Flawless, but for Linux processes:
I bet that Flawless can make better guarantees about reliability due to constraints of the WebAssembly sandbox.
Re: When "letting it crash" is not enough
#4I thought the idea of "exactly once" was a very questionable claim given what we know about computing, specially distributed. Then somewhere else you see this gem. > Workflows in flawless are written in Rust, in fact they are just regular Rust functions. This means that they can contain arbitrary logic. But instead of native code, the functions are compiled to WebAssembly and executed in a completely deterministic en…
The main page (https://flawless.dev/) has a diagram/video that shows how it would work and it basically writes any 'side effects' to a log which is used to track their existence. If they exist in the log, you read them; if they don't you run the code that generates them and then log it.
It's interesting, but I can't imagine how it is going to work at scale, both in terms of managing state across a very large application and in terms of running thousands of instances concurrently.
Re: When "letting it crash" is not enough
#5I thought the idea of "exactly once" was a very questionable claim given what we know about computing, specially distributed. Then somewhere else you see this gem. > Workflows in flawless are written in Rust, in fact they are just regular Rust functions. This means that they can contain arbitrary logic. But instead of native code, the functions are compiled to WebAssembly and executed in a completely deterministic en…
That can absolutely work... if you are confined into a single machine. (In fact, that's roughly how cloud instances work.) I see no way to generalize that into a distributed system. In my previous job I worked on a game server engine which solves a very limited version of this problem via virtual actors and it was still way too hard.
Re: When "letting it crash" is not enough
#6Flawless sounds a lot like https://temporal.io/ .
I'm wondering if it has the same scalability concerns - sticking everything in Postgres is fine at small-ish scale, but what happens when you outgrow Postgres, either because you have higher availability requirements (can't handle primary DB restarts) or because of the sheer volume of the workload?
Re: When "letting it crash" is not enough
#7I thought the idea of "exactly once" was a very questionable claim given what we know about computing, specially distributed. Then somewhere else you see this gem. > Workflows in flawless are written in Rust, in fact they are just regular Rust functions. This means that they can contain arbitrary logic. But instead of native code, the functions are compiled to WebAssembly and executed in a completely deterministic en…
Exactly Once is basically a solved problem. The CAP theorem says you can either get consistency or availability in a system that has network partitions.
If you give up consistency, you end up corrupting data every once in a while.
If you don’t, then you can have exactly once, but it might take a long time if there’s a network failure.
NFSv3 solved this back in the 1980’s. (V2 and V1 may have, but I don’t know.)
It did it without requiring deterministic execution or other programming language innovations, so I share your skepticism about rust and web assembly solving these problems.
(I really like rust, and recommend it for pretty much all new systems code. However, it is not a panacea.)
Re: When "letting it crash" is not enough
#8AFAIK Erlang and Elixir have a way to save the state just before the process is stopped. It should be the `trap_exit` flag on a gen_server that guarantees that the exit message will be managed. In that handler you can save all your state and resume it when the process restarts.
Re: When "letting it crash" is not enough
#9Bit of a cliffhanger style ad. But well set up for the pitch as I know the whole Erlang deal and that part was covered well.
I stopped reading somewhere in the presentation of flawless. Ran out of care for the moment. Might revisit it though. Sounds interesting.
Re: When "letting it crash" is not enough
#10(2023)