Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Async Python Is Secretly Deterministic

dbos.dev

Post reply on HN
Print view View original

21–30 of 38 posts

Re: Async Python Is Secretly Deterministic

#21

Post by TeMPOraL »

lexicality wrote: post #2

> This makes it possible to write simple code that’s both concurrent and safe. Yeah, great, my hello world program is deterministic. What happens when you introduce I/O? Is every network call deterministic? Can you depend on reading a file taking the same amount of time and being woken up by the scheduler in the same order every time?

No, but determinism reduces the number of stones you need to turn over when debugging hairy problems such as your program occasionally returning different results for the same inputs. You may not have control over the timing of I/O operations or order of external events (including OS scheduler), but at least you know that your side of the innovation/response is, in isoaltion, behaving predictably.

Re: Async Python Is Secretly Deterministic

#22

Post by stuartjohnson12 »

9dev wrote: post #18
KraftyOne wrote: post #11

Earlier quoted context omitted.

It's been a stable (and documented) behavior of the Python standard library for almost a decade now. It's possible it may change--nothing is ever set in stone--but that would be a large change in Python that would come with plenty of warning and time for adjustment.

And then one day, Astral creates a new Python implementation in Rust or something that is way faster and all the rage, but does this particular thing different than CPython. Whoops, you can’t use that runtime, because you now have cursed parts in your codebase that produce nondeterministic behaviour you can’t really find a reason for.

and then all the serverless platforms will start using Astral's new rust-based runtime to reduce cold starts, and in theory it's identical, except half of packages now don't work and it's very hard to anticipate which ones will and will not and behold! You have achieved Deno

Re: Async Python Is Secretly Deterministic

#23

Post by mort96 »

12_throw_away wrote: post #6

No, determinstic scheduling is not a property of async python. Yes, the stdlib asyncio event loop does have deterministic scheduling, but that's an implementation detail and I would not rely on it for anything critical. Other event loops - for instance trio [1] - explicitly randomize startup order so that you won't accidentally write code that relies on it. [1] https://github.com/python-trio/trio/issues/32

How do you differentiate between something that "happens to work due to an implementation detail" and a "proper feature that's specified to work" in a language without a specification?

Re: Async Python Is Secretly Deterministic

#24

Post by BrenBarn »

mort96 wrote: post #23
12_throw_away wrote: post #6

No, determinstic scheduling is not a property of async python. Yes, the stdlib asyncio event loop does have deterministic scheduling, but that's an implementation detail and I would not rely on it for anything critical. Other event loops - for instance trio [1] - explicitly randomize startup order so that you won't accidentally write code that relies on it. [1] https://github.com/python-trio/trio/issues/32

How do you differentiate between something that "happens to work due to an implementation detail" and a "proper feature that's specified to work" in a language without a specification?

There's still documentation.

Re: Async Python Is Secretly Deterministic

#25

Post by frizlab »

lilyball wrote: post #10
frizlab wrote: post #4

Earlier quoted context omitted.

Interestingly I think things that should not be deterministic should actually forced not to be. Swift for instance will explicitly make iterating on a dictionary not deterministic (by randomizing the iteration), in order to catch weird bugs early if a client relies (knowingly or not) on the specific order the elements of the dictionary are ordered.

This claim sounds vaguely familiar to me (though the documentation on Dictionary does not state any reason for why the iteration order is unpredictable), though the more common reason for languages to have unstable hash table iteration orders is as a consequence of protection against hash flooding, malicious input causing all keys to hash to the same bucket (because iteration order is dependent on bucket order).

Oh yeah you’re right, apparently the main reason was to avoid hash-flooding attacks[1].

I do seem to remember there was a claim regarding the fact that it also prevented a certain class of errors (that I mentioned earlier), but I cannot find the source again, so it might just be my memory playing tricks on me.

[1] https://forums.swift.org/t/psa-the-stdlib-now-uses-randomly-...

Re: Async Python Is Secretly Deterministic

#26

Post by nhumrich »

mort96 wrote: post #23
12_throw_away wrote: post #6

No, determinstic scheduling is not a property of async python. Yes, the stdlib asyncio event loop does have deterministic scheduling, but that's an implementation detail and I would not rely on it for anything critical. Other event loops - for instance trio [1] - explicitly randomize startup order so that you won't accidentally write code that relies on it. [1] https://github.com/python-trio/trio/issues/32

How do you differentiate between something that "happens to work due to an implementation detail" and a "proper feature that's specified to work" in a language without a specification?

In a language without a spec? You don't. But python has a very strong spec.

Re: Async Python Is Secretly Deterministic

#28

Post by LtWorf »

9dev wrote: post #18
KraftyOne wrote: post #11

Earlier quoted context omitted.

It's been a stable (and documented) behavior of the Python standard library for almost a decade now. It's possible it may change--nothing is ever set in stone--but that would be a large change in Python that would come with plenty of warning and time for adjustment.

And then one day, Astral creates a new Python implementation in Rust or something that is way faster and all the rage, but does this particular thing different than CPython. Whoops, you can’t use that runtime, because you now have cursed parts in your codebase that produce nondeterministic behaviour you can’t really find a reason for.

If the python core team cared about not breaking things I wouldn't need to run my tests on all versions of python.

Re: Async Python Is Secretly Deterministic

#30

Post by game_the0ry »

12_throw_away wrote: post #6

No, determinstic scheduling is not a property of async python. Yes, the stdlib asyncio event loop does have deterministic scheduling, but that's an implementation detail and I would not rely on it for anything critical. Other event loops - for instance trio [1] - explicitly randomize startup order so that you won't accidentally write code that relies on it. [1] https://github.com/python-trio/trio/issues/32

I just realized how little I know about how async event loops.
Post reply on HN