Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python 3.15's JIT is now back on track

fidget-spinner.github.io

Post reply on HN
Print view View original

291–300 of 330 posts

Re: Python 3.15's JIT is now back on track

#291

Post by zahlman »

cloudbonsai wrote: post #211
zahlman wrote: post #180

Earlier quoted context omitted.

There is no caching of a "utf-8 representation". You may check for example: >>> x = '日本語'*100000000 >>> import time >>> t = time.time(); y = x.encode(); time.time() - t # takes nontrivial time >>> t = time.time(); y = x.encode(); time.time() - t # not cached; not any faster Generally, the only reason this would happen implicitly is for I/O; actual operations on the string operate directly on the internal representati…

> There is no caching of a "utf-8 representation". No there certainly is. This is documented in the official API documentation: UTF-8 representation is created on demand and cached in the Unicode object. https://docs.python.org/3/c-api/unicode.html#unicode-objects In particular, Python's Unicode object (PyUnicodeObject) contains a field named utf8 . This field is populated when PyUnicode_AsUTF8AndSize() is first call…

The C API may provide for it, but I'm not seeing a way to access that from Python. This sort of thing is provided for people writing C extensions who need to interface to other C code.

(And the code search seems to be broken; it can't find me the definition of `unicode_fill_utf8` although I'm sure it's obvious enough.)

Re: Python 3.15's JIT is now back on track

#292

Post by [deleted] »

gpderetta wrote: post #250
Someone wrote: post #179

Earlier quoted context omitted.

> What's nuts is that the language doesn't guarantee that successive references to the same member value within the same function body are stable. The language supports multiple threads and doesn’t have private fields ( https://docs.python.org/3/tutorial/classes.html#private-vari... ), so the runtime cannot rule out that the value gets changed in-between. And yes, it often is obvious to humans that’s not intended to…

wouldn't a concurrent change without synchronization be UB anyway? Also parent wants to cache the address, not the value (but you have to cache the value if you want to optimize manually)

[deleted]

Re: Python 3.15's JIT is now back on track

#293

Post by zahlman »

thaumasiotes wrote: post #200
zahlman wrote: post #177

Earlier quoted context omitted.

What realistic use case do you have for caring about whether two integers of the same value are distinct objects? Modern versions of Python warn about doing unpredicatble things with `is` exactly because you are not supposed to do those things. Valid use cases for `is` at all are rare.

> Valid use cases for `is` at all are rare. There might not be that many of them, depending on how you count, but they're not rare in the slightest. For example, you have to use `is` in the common case where you want the default value of a function argument to be an empty list.

I assume you refer to the `is None` idiom. That happens often enough, but I count it as exactly one use case, and I think it's usually poorly considered anyway. Again, you probably don't actually want the default value to be an empty list, because it doesn't make a lot of sense to mutate something that the caller isn't actually required to provide (unless the caller never provides it and you're just abusing the default-argument behaviour for some kind of cache).

Using, for example, `()` as a default argument, and cleaning up your logic to not do those mutations, is commonly simpler and more expressive. A lot of the community has the idea that a tuple should represent heterogeneous fixed-length data and a list should be homogeneous; but I consider (im)mutability to be a much more interesting property of types.

Re: Python 3.15's JIT is now back on track

#294

Post by zahlman »

Revisional_Sin wrote: post #204
zahlman wrote: post #175

Earlier quoted context omitted.

We don't call them "arrays". It has nothing to do with whether the list is empty. It has nothing to do with lists at all. It's the behaviour of default arguments. It happens at the time that the function object is created, which is during runtime. You only notice because lists are mutable. You should already prefer not to mutate parameters, and it especially doesn't make sense to mutate a parameter that has a default…

It's best practice to avoid mutable defaults even if you're not planning to mutate the argument. It's just slightly annoying having to work around this by defaulting to None.

You don't need to use `None`. If you indeed aren't planning to mutate the argument, then use something immutable that provides the necessary interface. Typically, this will be `()`, and then your logic doesn't require the special case. I genuinely don't understand, after 20+ years of this, why everyone else has decided that the `None` check should be idiomatic. It's just, ugh. I'm pretty sure I've even seen people do this where a string is expected and `''` is right there staring at them as the obvious option.

Re: Python 3.15's JIT is now back on track

#295

Post by zahlman »

panzi wrote: post #129
mattclarkdotnet wrote: post #125

Python really needs to take the Typescript approach of "all valid Python4 is valid Python3". And then add value types so we can have int64 etc. And allow object refs to be frozen after instantiation to avoid the indirection tax. Sensible type-annotated python code could be so much faster if it didn't have to assume everything could change at any time. Most things don't change, and if they do they change on startup (e…

Isn't rpython doing that, allowing changes on startup and then it's basically statically typed? Does it still exist? Was it ever production ready? I only once read a paper about it decades ago.

It exists in the sense that PyPy exists.

As far as I can tell, it only ever existed to make PyPy possible, and was only defined/specified in terms of PyPy's needs.

Re: Python 3.15's JIT is now back on track

#296

Post by zahlman »

fyrn_ wrote: post #171
rich_sasha wrote: post #136

Earlier quoted context omitted.

I think sadly a lot of Python in the wild relies heavily, somewhere, on the crazy unoptimisable stuff. For example pytest monkey patches everything everywhere all the time. You could make this clean break and call it Python 4 but frankly I fear it won't be Python anymore.

As a person who has spent a lot of time with pytest, I'm ready for testing framework that doesn't do any of that non-obvious stuff. Generally use unittest as much as I can these days, so much less _wierd_ about how it does things. Like jeeze pytest, do you _really_ need to stress test every obscure language feature? Your job is to call tests.

Yeah, I've been thinking about how I'd do it from scratch, honestly. (One of the reasons Pytest could catch on is that it supported standard library `unittest` classes, and still does. But the standard library option is already ugly as sin, being essentially an ancient port of JUnit.)

I think it's not so much that Pytest is using obscure language features (decorators are cool and the obvious choice for a lot of this kind of stuff) but that it wants too much magic to happen in terms of how the "fixtures" automatically connect together. I would think that "Explicit is better than implicit" and "Simple is better than complex" go double for tests. But things like `pytest.mark.parametrize` are extremely useful.

Re: Python 3.15's JIT is now back on track

#297

Post by zahlman »

germandiago wrote: post #281
rich_sasha wrote: post #136

Earlier quoted context omitted.

I think sadly a lot of Python in the wild relies heavily, somewhere, on the crazy unoptimisable stuff. For example pytest monkey patches everything everywhere all the time. You could make this clean break and call it Python 4 but frankly I fear it won't be Python anymore.

If you do that you then have a less productive language for many use cases IMHO. All the dynamism from Python should stay where it is. Just JIT and remember a type maybe, but do not force a type from a type hint or such things. As a minimum, I would say not relying on that is the correct thing. You could exploit it, but not force it to change the semantics.

I think there are ways that it could be reined in quite a bit with most people not noticing. But it would still be a different language.

Re: Python 3.15's JIT is now back on track

#298

Post by asa400 »

teleforce wrote: post #283
rtpg wrote: post #159

Earlier quoted context omitted.

In Python attribute access aren't stable! `self.x` where `x` is a property is not guaranteed to refer to the same thing. And getting rid of descriptors would be a _fundamental change to the language_. An immeense one. Loads of features are built off of descriptors or descriptor-like things. And what you're complaining about is also not true in Javascript world either... I believe you can build descriptor-like things…

>Remember, this is a scripting language, not a compiled language This is the fundamental issue and "elephant in the room" that everyone is seems to be overlooking, and putting under the carpet. The extreme compiled type language guys going gung-ho with very slow to compile and complicated Rust (moreso than C++), while the rest of the world gladly hacking their shiny ML/AI codes in scripting language aka Python "the g…

> The problem is that Python was meant for scripting not properly designed software system engineering.

What something was meant to do has never, ever stopped people. People find creative ways to use tools in unintended ways all the time. It's what we do.

We can call this dumb or get misanthropic about it, or we can try to understand why people all over the world choose to use Python in "weird" ways, and what this tells us about the way people relate to computing.

Re: Python 3.15's JIT is now back on track

#299

Post by asa400 »

gpderetta wrote: post #250
Someone wrote: post #179

Earlier quoted context omitted.

> What's nuts is that the language doesn't guarantee that successive references to the same member value within the same function body are stable. The language supports multiple threads and doesn’t have private fields ( https://docs.python.org/3/tutorial/classes.html#private-vari... ), so the runtime cannot rule out that the value gets changed in-between. And yes, it often is obvious to humans that’s not intended to…

wouldn't a concurrent change without synchronization be UB anyway? Also parent wants to cache the address, not the value (but you have to cache the value if you want to optimize manually)

Not necessarily UB, but absolutely "spooky action" nondeterministic race conditions that make things difficult to understand.

Re: Python 3.15's JIT is now back on track

#300

Post by xenadu02 »

1718627440 wrote: post #289
dundarious wrote: post #287

Earlier quoted context omitted.

This level of dynamism is commonly forgotten/omitted because it is most often not at all needed. "There is no object identity across the values [retrieved by self.x]" is a very curious choice to many.

It's very Pythonic to expose e.g. state via the existence of attributes. This also makes it possible to dynamically expose foreign language interfaces. You can really craft the interface you like, because the interface exposal is also normal code that returns strings and objects. You are right that it is not needed often, but there is often somewhere a part in the library stack that does exactly this, to expose a nic…

This is just an analogy but in Swift String is such a commonly used hot path the type is designed to accommodate different backing representations in a performant way. The type has bits in its layout that indicate the backing storage. eg a constant string is just a pointer to the bytes in the binary and unless the String escapes or mutates incurs no heap allocation at all - it is just a stack allocation and a pointer.

Javascript implementations do their own magic since most objects aren't constantly mutating their prototypes or doing other fun things. They effectively fast-path property accesses and fallback if that assumption proves incorrect.

Couldn't python tag objects that don't need such dynamism (the vast majority) so it can take the fast path on them?

Post reply on HN