Async and Finaliser Deadlocks
tratt.net
Async and Finaliser Deadlocks
1–10 of 22 posts
Re: Async and Finaliser Deadlocks
#2Dupe of [0], though there's only 1 comment on that submission as of this comment.
Re: Async and Finaliser Deadlocks
#3We use async code in two modes which have different very different consequences for concurrency issues.
We have an imperative code flow where we perform a series of tasks that involve IO, and apply the effects sequentially. Here the biggest problem is holding a lock for a long transaction and starving the rest of the system. So we break it up into a finite state machine where the lock is held mostly during the synchronous parts.
The other is asking a lot of questions and then making a decision based on the sum of the answers. These actually happen in parallel, and we often have to relax the effective isolation levels to make this work. But it always seems to work better if the parallel task can be treated as a pure function. Purity removes side effects, which removes the need for write locks, which if applied consistently removes the Dining Philosopher’s problem. “Applied consistently” is the hard part. Because it requires not just personal discipline but team and organizational discipline.
> There is usually not much of a point in writing a finalizer that touches only the object being finalized, since such object updates wouldn’t normally be observable. Thus useful finalizers must touch global shared state.
That seems like an “Abandon hope, all ye who enter here.”
Re: Async and Finaliser Deadlocks
#4Unless I'm missing something, this has nothing to do with asynchronous code. The delete is just synchronous code running, same as if we called a function/closure right there.
This is just about syntax sugar hiding function calls.
Re: Async and Finaliser Deadlocks
#5Unless I'm missing something, this has nothing to do with asynchronous code. The delete is just synchronous code running, same as if we called a function/closure right there. This is just about syntax sugar hiding function calls.
I think it says if your async code holds locks you’re gonna have a bad time. Async and optimistic locks probably should go hand in hand.
I would think finalizers and async code magnify problems that are already there.
Re: Async and Finaliser Deadlocks
#6Unless I'm missing something, this has nothing to do with asynchronous code. The delete is just synchronous code running, same as if we called a function/closure right there. This is just about syntax sugar hiding function calls.
I'm assuming you're referring to the Python finaliser example? If so, there's no syntax sugar hiding function calls to finalisers: you can verify that by running the code on PyPy, where the point at which the finaliser is called is different. Indeed, for this short-running program, the most likely outcome is that PyPy won't call the finaliser before the program completes!
Re: Async and Finaliser Deadlocks
#7We use async code in two modes which have different very different consequences for concurrency issues. We have an imperative code flow where we perform a series of tasks that involve IO, and apply the effects sequentially. Here the biggest problem is holding a lock for a long transaction and starving the rest of the system. So we break it up into a finite state machine where the lock is held mostly during the synchr…
There's a reason Java got rid of finalizers. It forces the programmer to choose between synchronous cleanup (`AutoCloaseable`) or asynchronous cleanup on a thread (`Cleaner`).
Re: Async and Finaliser Deadlocks
#8Unless I'm missing something, this has nothing to do with asynchronous code. The delete is just synchronous code running, same as if we called a function/closure right there. This is just about syntax sugar hiding function calls.
I think it says if your async code holds locks you’re gonna have a bad time. Async and optimistic locks probably should go hand in hand. I would think finalizers and async code magnify problems that are already there.
If you use a single-threaded executor then you don't need locks in your async code. Well, you might use external locks, but not thread synchronization primitives.
When I write async code I use a single-threaded multi-process pattern. Look ma'! No locks!
Well, that's not very fair. The best async code I've written was embarrassingly parallel, no-sync-needed, read-only stuff. If I was writing an RDBMS I would very much need locks, even if using the single-threaded/multi-processed pattern. But also then my finalizers would mainly drop locks rather than acquire them.
Re: Async and Finaliser Deadlocks
#9While I think the problem highlighted in the article is a longstanding problem for Rust[1], I don't think the example, or finalizers was the problem with Futurelock as described by Oxide.
I'm not sure you can write a simple example in Python, because Rust's future's architecture and Python's is different. `futurelock` is an issue of cancellation safety which is a stranger concept (related to finalizers, but not in the way OP has described).
Personally, I think `tokio::select!` is dangerous and I don't use it my code - it's very easy to deadlock yourself or create weird performance issues. I think the interface is too close to Go and if you don't understand what is going on, you can create deadlocks. That said, even if you avoid `tokio::select!`, I think cancellation safety is one of those dragons that exist in async rust.
Re: Async and Finaliser Deadlocks
#10While I think the problem highlighted in the article is a longstanding problem for Rust[1], I don't think the example, or finalizers was the problem with Futurelock as described by Oxide. I'm not sure you can write a simple example in Python, because Rust's future's architecture and Python's is different. `futurelock` is an issue of cancellation safety which is a stranger concept (related to finalizers, but not in th…
The `futurelock` is probably closer to something like:
import threading
mutex = threading.Lock()
def gen_1():
yield 1
print("acquiring")
mutex.acquire();
print("acquired")
yield 2
print("releasing")
mutex.release()
yield 3
def gen_2():
yield "a"
def do_something_else():
print("im gonna do something else")
mutex.acquire()
print("acquired")
mutex.release()
print("done")
a = gen_1();
b = gen_2();
zipped_data = zip(a, b)
for num, letter in zipped_data:
print("output", num, letter)
do_something_else()
print("done")
The problem is before `do_something_else` is called, either `a` must be destroyed or someone has to call `next` on it. However from just reading the code, the fact that this exists can be difficult to see.