Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python 3.13 Gets a JIT

tonybaloney.github.io

Post reply on HN
Print view View original

481–490 of 553 posts

Re: Python 3.13 Gets a JIT

#481

Post by eyegor »

nextaccountic wrote: post #466
remram wrote: post #448

Earlier quoted context omitted.

Pypy actually works really well. It could probably get even farther if people knew about it more. Test your packages on Pypy, people.

The only trouble is C packages, which are really really common, right? So you have a performance hit or something (or so I heard) I guess that now the GIL is going away, pypy will become better at handling packages with native code like numpy

In my experience, pypy works with basically everything these days. I remember having some struggles with a weird fortran based extension module a few years ago, but it might work now too.

Most c extension modules should work in pypy, there's just a performance hit depending on how they're built (cffi is the most compatible).

https://doc.pypy.org/en/latest/faq.html#do-c-extension-modul...

Re: Python 3.13 Gets a JIT

#482

Post by lifthrasiir »

KRAKRISMOTT wrote: post #398
twbarr wrote: post #372

Earlier quoted context omitted.

Ultimately, most good ideas were first implemented by Fabrice Bellard.

Copy and patch goes all the way back to Grace Hopper's original compiler implementation

They were called "template-based" JIT and copy-and-patch approach is not new in this regard. The novel idea of copy-and-patch JIT was an automatic code generation via relocatable objects. (By the way, QEMU is indeed cited as an inspiration for copy-and-patch JIT.)

Re: Python 3.13 Gets a JIT

#483

Post by zbentley »

nyanpasu64 wrote: post #408
zbentley wrote: post #330

Earlier quoted context omitted.

> If you pre-open shared memory in a ProcessPoolExecutor's initializer functions, you can't close them when the worker process exits That's quite surprising to learn, as I didn't think the initializer ran in a specialized context (like a pthread_atfork postfork hook in the child). What happens when you try to close an initializer-allocated SharedMemory object on worker exit?

ProcessPoolExecutor doesn't let you supply a callback to run on worker process exit, only startup. Perhaps I could've looked for and tried something like atexit ( https://docs.python.org/3/library/atexit.html )? In any case I don't want to touch my code at the moment until I regain interest or hear of resource exhaustion, since "it works".

Fair enough. If you ever do decide to touch it to address that, I suggest (in preference order):

- Subclassing ProcessPoolExecutor such that it spawns multiprocessing.Process objects whose runloop function wraps the stdlib "_process_worker" function in a try/finally which runs your at-shutdown logic. That'll be as reliable as any try/finally (e.g. SIGKILL and certain interpreter faults can bypass it).

- Writing custom destructors of objects in your call arguments which are aware of and can do appropriate cleanup actions for associated SharedMemory objects. This is less preferred than subclassing because of the usual issues with custom destructors: no real exception handling, and objects sneaking out into long-lived/global caches can cause destructors to run late (after the interpreter has torn down things your cleanup logic needs) or not at all.

- Atexit, as you suggest. This is least-preferred because the execution context of atexit code is .... weird, to say the least. Much like a signal handler or pthread_atfork callback, it's not a place that I'd put code that does complicated I/O or depends on the rest of the interpreter being in ordinary conditions.

Re: Python 3.13 Gets a JIT

#484

Post by guappa »

pjmlp wrote: post #226
guappa wrote: post #199

Earlier quoted context omitted.

You're naming things that need handling… which take time to be set up and so on.

I am naming things that need learning, pity that it is so big effort nowadays.

The things you personally already know aren't more worthy of learning just by virtue that you personally happen to have learnt them already.

You're just sticking with what you know, calling out the others for not wanting to learn.

Not a nice attitude from where I'm at.

Re: Python 3.13 Gets a JIT

#485

Post by pjmlp »

guappa wrote: post #484
pjmlp wrote: post #226

Earlier quoted context omitted.

I am naming things that need learning, pity that it is so big effort nowadays.

The things you personally already know aren't more worthy of learning just by virtue that you personally happen to have learnt them already. You're just sticking with what you know, calling out the others for not wanting to learn. Not a nice attitude from where I'm at.

We have gotten into a lazy learn attitude society.

Re: Python 3.13 Gets a JIT

#486

Post by guappa »

oblio wrote: post #208
guappa wrote: post #197

Earlier quoted context omitted.

> Do a quick test with the latest JVM. java: 031s python: 013s it's 3x slower to start up… it means that any bash script with a loop calling a java command will run 3x slower. 3x slower isn't my definition of minuscule.

> java: 031s python: 013s I'm confused, what kind of time measurement system is this? Are those 31 seconds versus 13 seconds? 0.31s vs 0.13s? If it's the first case, something is wrong with that machine, Hello World should not take that long. If it's the second, are we talking about 0.18s? See below. > it's 3x slower to start up… it means that any bash script with a loop calling a java command will run 3x slower. I d…

double click selection issue. 0m0,031s vs 0m0,013s.

> I don't know how you write your scripts,

You know how… I put print hello world in them… that was the test you asked for?

> premature optimization

Not using something that has 3x startup time, for a short lived command is just "common sense".

I'm starting to think that the issue here is that you know java but don't know python or C, and you are unconsciously trying to get water to your own watermill.

Re: Python 3.13 Gets a JIT

#487

Post by silviogutierrez »

eternityforest wrote: post #435
acdha wrote: post #397

Earlier quoted context omitted.

Just wait until you see what the enterprise Java developers passing around with type Object and encoded XML blobs. Type checking is really useful but it can be defeated in any language if you don’t have a healthy technical culture.

I wish Python would have native JSON schema support to stop this issue. There's https://pypi.org/project/jsonschema-typed-v2/ but it hasn't been updated in a few years.

What about https://quicktype.io ?

Re: Python 3.13 Gets a JIT

#489

Post by oblio »

guappa wrote: post #486
oblio wrote: post #208

Earlier quoted context omitted.

> java: 031s python: 013s I'm confused, what kind of time measurement system is this? Are those 31 seconds versus 13 seconds? 0.31s vs 0.13s? If it's the first case, something is wrong with that machine, Hello World should not take that long. If it's the second, are we talking about 0.18s? See below. > it's 3x slower to start up… it means that any bash script with a loop calling a java command will run 3x slower. I d…

double click selection issue. 0m0,031s vs 0m0,013s. > I don't know how you write your scripts, You know how… I put print hello world in them… that was the test you asked for? > premature optimization Not using something that has 3x startup time, for a short lived command is just "common sense". I'm starting to think that the issue here is that you know java but don't know python or C, and you are unconsciously trying…

I haven't written Java as my day to day for about 5 years. Instead I write... Python :-)

It's just that I recognize disingenuous comments.

Java is perfectly adequate as a language for writing command line tools. You're likely using one of them weekly and don't even know it.

Just don't write your Java CLIs using Spring or DI frameworks in general :-)

Re: Python 3.13 Gets a JIT

#490

Post by livesinhel »

dlahoda wrote: post #349
pjmlp wrote: post #27

Finally! Regardless of the work being done in PyPy, Jython, GraalPy and IronPython, having a JIT in CPython seems to be the only way beyond "C/C++/Fortran libs are Python" mindset. Looking forward to its evolution, from 3.13 onwards.

rust libs recently. pydantic for example

Also Polars.
Post reply on HN