Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Python 3.13 Gets a JIT

tonybaloney.github.io

Post reply on HN
Print view View original

351–360 of 553 posts

Re: Python 3.13 Gets a JIT

#351

Post by traverseda »

mg wrote: post #205
onetoo wrote: post #96

Earlier quoted context omitted.

The restart isn't expensive in absolute terms, on a human level it's practically instant. You would only do this during development, hopefully your local machine isn't the production environment. It's also very easy, often just adding a CLI flag to your local run command. edit: Regarding performance, Python today can easily handle at least 1k requests per second. The vast vast vast majority of web applications today…

The thing is, I don't run my applications locally with a "local run command". I prefer to have a local system set up just like the production server, but in a container. Maybe using WSGI with MaxConnectionsPerChild=1 could be a solution? But that would start a new (for example) Django instance for every request. Not sure how fast Django starts. Another option might be to send a HUP signal to Apache: apachectl -k rest…

Is the problem you're having that you feel the need to expose a WSGI/ASGI interface instead of just a reverse proxy? Take a look at gunicorn, and for static files server you can use whitenoise.

With those two you can just stand up an python program in a container that serves html, and put it behind whatever reverse proxy you want.

Re: Python 3.13 Gets a JIT

#352

Post by pierrebai »

lifthrasiir wrote: post #113
checker659 wrote: post #89

Earlier quoted context omitted.

M. Anton Ertl and David Gregg. 2004. Retargeting JIT Compilers by using C-Compiler Generated Executable Code. In Proceedings of the 13th International Conference on Parallel Architectures and Compilation Techniques (PACT '04). IEEE Computer Society, USA, 41–50. https://dl.acm.org/doi/10.5555/1025127.1025995

While bears a significant resemblance, Ertl and Gregg's approach is not automatic and every additional architecture requires a significant understanding of the target architecture---including an ability to ensure that fully relocable code can be generated and extracted. In comparison, the copy-and-patch approach can be thought as a simple dynamic linker, and objects generated by unmodified C compilers are far more pr…

The copy-and-patch also assumes the compiler will generate patchable code. For example, on some architecture, have a zero operand might have a smaller or different opcode compared to a more general operand. Same issue for relative jumps or offset ranges. It seems the main difference is that the patch approach also patches jumps to absolute addresses instead of requiring instruction-counter relative code.

Re: Python 3.13 Gets a JIT

#353

Post by tomjakubowski »

HDThoreaun wrote: post #341
dron57 wrote: post #309

Earlier quoted context omitted.

AI heavy lifting isn't just model training. There's about a million data pipelines and processes before the training data gets loaded into a PyTorch tensor.

also done in native code

Ehhh... if you're lucky. I've seen (and maybe even written) plenty of we-didn't-have-time-to-write-this-properly-with-dataframes Python data munging code, banged out once and then deployed to production. I'll take performance gains there.

Re: Python 3.13 Gets a JIT

#354

Post by naasking »

lifthrasiir wrote: post #113
checker659 wrote: post #89

Earlier quoted context omitted.

M. Anton Ertl and David Gregg. 2004. Retargeting JIT Compilers by using C-Compiler Generated Executable Code. In Proceedings of the 13th International Conference on Parallel Architectures and Compilation Techniques (PACT '04). IEEE Computer Society, USA, 41–50. https://dl.acm.org/doi/10.5555/1025127.1025995

While bears a significant resemblance, Ertl and Gregg's approach is not automatic and every additional architecture requires a significant understanding of the target architecture---including an ability to ensure that fully relocable code can be generated and extracted. In comparison, the copy-and-patch approach can be thought as a simple dynamic linker, and objects generated by unmodified C compilers are far more pr…

Full copy of the paper: https://www2.cs.arizona.edu/~collberg/Teaching/553/2011/Reso...

There's also this which seems to use the same technique:

Templates-based portable just-in-time compiler, https://dl.acm.org/doi/abs/10.1145/944579.944588

Nice to see there's still room for innovation in the VM space!

Re: Python 3.13 Gets a JIT

#355

Post by chc4 »

hyperman1 wrote: post #234

The article presents a copy and patch jit as something new, but I remember DOS's quickbasic doing the same thing. It generated very bad assembly code in memory by patching together template assembly blocks with filled in values, with a lot of INT instructions toward the quickbasic runtime, but it did compile, not interprete.

Template JITs in general aren't a new technique, but Copy-and-Patch is a specific method of implementing it (leveraging a build time step to generate the templates from C code + ELF relocations).

Re: Python 3.13 Gets a JIT

#356

Post by amelius »

mike_hearn wrote: post #211
amelius wrote: post #189

Earlier quoted context omitted.

Are there any benchmarks that give an idea of how much this might improve Python's speed?

Well, GraalPython is a Python JIT compiler which can exploit dynamically determined types, and it advertises 4.3x faster, so it's possible to do drastically better than a few percent. I think that's state of the art but might be wrong. That's for this benchmark: https://pyperformance.readthedocs.io/ Note that this is with a relatively small investment as these things go, the GraalPython team is about ~3 people I gues…

This is great info, thanks!

Re: Python 3.13 Gets a JIT

#357

Post by aftbit »

swingingFlyFish wrote: post #299
mg wrote: post #90

I love Python and use it for everything other than web development. One reason is performance. So if Python has a faster future ahead of it: Hurray! The other reason is that the Python ecosystem moved away from stateless requests like CGI or mod_php use and now is completely set on long running processes. Does this still mean you have to restart your local web application after any change you made to it? I heard that…

Python is amazing and shines for Web development. I'd recommend taking a look at https://www.tornadoweb.org/en/stable/index.html . I use this in production on my pet project at https://www.meecal.co/ . Put Nginx in front and you're golden. Definitely take a look, it's come a long way from ten years ago.

I personally like Quart, which is like Flask, but with asyncio. Django is also incredibly popular and has been around forever, so it is very battle-tested.

Re: Python 3.13 Gets a JIT

#358

Post by PeterisP »

hyperman1 wrote: post #234

The article presents a copy and patch jit as something new, but I remember DOS's quickbasic doing the same thing. It generated very bad assembly code in memory by patching together template assembly blocks with filled in values, with a lot of INT instructions toward the quickbasic runtime, but it did compile, not interprete.

TIL! I had used qbasic back in school, but I somehow always assumed that these basics were interpreters.

Re: Python 3.13 Gets a JIT

#359

Post by yowlingcat »

AndrewDucker wrote: post #23
cryptos wrote: post #6

I always wondered how Python can be one of the world's most popular languages without anyone (company) stepping up and make the runtime as fast as modern JavaScript runtimes.

Always interested in replies to this kind of comment, which basically boil down to "Python is so slow that we have to write any important code in C. And this is somehow a good thing." I mean, it's great that you can write some of your code in C. But wouldn't it be great if you could just write your libraries in Python and have them still be really fast?

> basically boil down to "Python is so slow that we have to write any important code in C. And this is somehow a good thing."

I think that's a pretty ignorant interpretation. Python has been built to have a giant ecosystem of useful, feature-complete, stable, well built code that has been used for decades and for which there is no need to reinvent the wheel. If that already describes the universe of libraries that you /need/ to be extremely fast and the rest of your code is IO limited and not CPU limited, why reinvent the wheel?

That makes your comment even more inaccurate because you likely don't need to write any "important" (which you are stretching to mean "fast") code in C -- you utilize existing off the shelf fast libraries that are written in Fortran, CUDA, C, Rust or any other language a pre-existing ecosystem was built in.

Try and think of a language that has mature capabilities for domains as far away as what Django solves for, what pandas solves for, what pytorch solves for, and still has fantastic tooling like jupyter and streamlit. I can't think of any other language that has the combined off the shelf breadth and depth of Python. I don't want to have to write fast code in any language unless forced to, because the vast majority of the time I can customize a great off the shelf package and only write the remaining 1% of glue. I can't see why a professional engineer would 99% of the time would need to take a remotely different approach.

Re: Python 3.13 Gets a JIT

#360

Post by bonzini »

vanderZwan wrote: post #82

I think it's really cool that Haoran Xu and Fredrik Kjolstad's copy-and-patch technique[0] is catching on, I remember discovering it through Xu's blog posts about his LuaJIT remake project[1][2], where he intends to apply these techniques to Lua (and I probably found those through a post here). I was just blown away by how they "recycled" all these battle-tested techniques and technologies, and used it to synthesize…

Copy and patch is a variant of QEMU's original "dyngen" backend by Fabrice Bellard[1][2], with more help from the compiler to avoid the maintainability issues that ultimately led QEMU to use a custom code generator.

[1] https://www.usenix.org/legacy/event/usenix05/tech/freenix/fu...

[2] https://review.gerrithub.io/plugins/gitiles/spdk/qemu/+/5a24...

Post reply on HN