Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What Python’s asyncio primitives get wrong about shared state

inngest.com

Post reply on HN
Print view View original

31–40 of 55 posts

Re: What Python’s asyncio primitives get wrong about shared state

#31

Post by pocksuppet »

josephg wrote: post #27
ori_b wrote: post #19

Earlier quoted context omitted.

Async and await is manually scheduling threads. So, if you're quite careful about what functions you call, you can arrange things so that you don't get concurrency when you don't want it. Being careful about what functions you call is quite fragile and tedious, and doesn't compose well: what if a library changes when it adds a yield point? Overall, async/await is a result of people programming like it's 2003, when th…

This works a lot better in JavaScript, which is exactly this model - a single threaded executor with async await. The problem you talk about is solved with function colouring. Async functions are marked as such. In general, sync functions can’t call async functions. (Well, you can invoke them. You just can’t run them to completion before returning). For all the complaints about function colouring, I’m glad JavaScript…

Until every function is async.

Re: What Python’s asyncio primitives get wrong about shared state

#32

Post by rciorba »

dbt00 wrote: post #4

What about a more general message-passing mailbox approach? This works really well in the Erlang/gen_server/gen_fsm world. (and in plenty of other contexts, but Erlang's OTP is still some of the best, simplest incarnation of these things)

I mean, the "one-queue per consumer" they eventually ended up with, is basically an inbox that the sequential process reads from.

Re: What Python’s asyncio primitives get wrong about shared state

#33

Post by samarthr1 »

ggm wrote: post #12
evil-olive wrote: post #8

the title seems unnecessarily clickbaity. rather than "What Python's asyncio primitives get wrong" this seems more like "why we chose one asyncio primitive (queue) instead of others (event and condition)" also, halfway through the post, the problem grows a new requirement: > Instead of waking consumers and asking "is the current state what you want?", buffer every transition into a per-consumer queue. Each consumer d…

yes. I felt something very similar. I do think there is value in pointing out the pitfalls naieve users (me!) can make assuming things which aren't true about ordering of events, states. Queues with lock regions are also really nice because they are (as I understand it) very cheap: so making a thread or other concurrency primitive which writes into a queue under lock, and gets out of the way, aligns nicely with havin…

BTW: what is a lock region?

Re: What Python’s asyncio primitives get wrong about shared state

#34

Post by seanhunter »

A better title would be: “Person who doesn’t know how to write state machines struggles to write a state machine”.

In attempt 2 the old school C way of writing the state machine would work just fine in python, avoid a bunch of the boilerplate and avoid the “state setter needs to know a bunch of stuff” problem. Basically you make the states as a table and put the methods you need in the table so in python a dictionary is convenient. Then you have

   > def set_state(new_state):
   >   state = new_state
   >   events[new_state].set()
Aaand you’re done. When you add a new state, you add an event corresponding to that state into the events table. If the stuff you would put into a conditional in set_state is more complicated, you could make a state transition method and link to it in the table. Or you could make a nested dict or whatever. It’s not hard, and the fact that the author doesn’t know an idomatic way to write a fsm definitely isn’t something that’s wrong with python’s asyncio and shared state.

In general if you’re writing a state machine and you have a lot of “if curr_state == SOME_STATE” logic, chances are it would be better if you used tables.

Re: What Python’s asyncio primitives get wrong about shared state

#35

Post by nine_k »

samarthr1 wrote: post #33
ggm wrote: post #12

Earlier quoted context omitted.

yes. I felt something very similar. I do think there is value in pointing out the pitfalls naieve users (me!) can make assuming things which aren't true about ordering of events, states. Queues with lock regions are also really nice because they are (as I understand it) very cheap: so making a thread or other concurrency primitive which writes into a queue under lock, and gets out of the way, aligns nicely with havin…

BTW: what is a lock region?

A part of code that executes between a lock as taken and released, I suppose.

Re: What Python’s asyncio primitives get wrong about shared state

#36

Post by nine_k »

pocksuppet wrote: post #31
josephg wrote: post #27

Earlier quoted context omitted.

This works a lot better in JavaScript, which is exactly this model - a single threaded executor with async await. The problem you talk about is solved with function colouring. Async functions are marked as such. In general, sync functions can’t call async functions. (Well, you can invoke them. You just can’t run them to completion before returning). For all the complaints about function colouring, I’m glad JavaScript…

Until every function is async.

If a function is not pure, it very likely has to be async.

Re: What Python’s asyncio primitives get wrong about shared state

#40

Post by pocksuppet »

nine_k wrote: post #36
pocksuppet wrote: post #31

Earlier quoted context omitted.

Until every function is async.

If a function is not pure, it very likely has to be async.

Which just brings you back to preemptive multithreading, but without being able to use all the cores you paid for.
Post reply on HN