Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cython is 20

blog.behnel.de

Post reply on HN
Print view View original

11–20 of 71 posts

Re: Cython is 20

#11

Post by fname11 »

pjmlp wrote: post #7

While it is nice that this option is available, it would be much better if Python itself would embrance the necessary runtime capabilties to not have to rely on it.

This is not going to happen. GvR has successfully ignored Cython and PyPy for decades and has attached himself to a JIT project at Microsoft (has anything emerged?).

CPython is in the hands of not really productive bigcorp representatives who care about large legacy code bases. My guess is that CPython will be largely the same in 10 years, with the usual widely hyped initiatives that go nowhere ("need for speed etc.").

Re: Cython is 20

#12

Post by linspace »

fname11 wrote: post #11
pjmlp wrote: post #7

While it is nice that this option is available, it would be much better if Python itself would embrance the necessary runtime capabilties to not have to rely on it.

This is not going to happen. GvR has successfully ignored Cython and PyPy for decades and has attached himself to a JIT project at Microsoft (has anything emerged?). CPython is in the hands of not really productive bigcorp representatives who care about large legacy code bases. My guess is that CPython will be largely the same in 10 years, with the usual widely hyped initiatives that go nowhere ("need for speed etc."…

> who care about large legacy code bases

It's clear that Python's main strength is its vast libraries, priority number one is not breaking them. If it could be possible to speed up Python without breaking changes I would be surprised precisely because with so much large codebases speed and efficiency would translate directly to money.

Re: Cython is 20

#13

Post by baq »

kubb wrote: post #3

and i still have no idea what i could use it for...

it's C with Python syntax and syntactic sugar for Python objects on C level, including refcounting, which is the hard part.

if you successfully use numba, probably nothing that you couldn't already do.

if you want something that lives much closer to C, it's perfect.

Re: Cython is 20

#14

Post by dikei »

linspace wrote: post #12
fname11 wrote: post #11

Earlier quoted context omitted.

This is not going to happen. GvR has successfully ignored Cython and PyPy for decades and has attached himself to a JIT project at Microsoft (has anything emerged?). CPython is in the hands of not really productive bigcorp representatives who care about large legacy code bases. My guess is that CPython will be largely the same in 10 years, with the usual widely hyped initiatives that go nowhere ("need for speed etc."…

> who care about large legacy code bases It's clear that Python's main strength is its vast libraries, priority number one is not breaking them. If it could be possible to speed up Python without breaking changes I would be surprised precisely because with so much large codebases speed and efficiency would translate directly to money.

Yeah, it took over a decade to switch from Py2 to Py3 due to the breaking changes it brought. I'd rather not to have such a large change again, ever.

Re: Cython is 20

#15

Post by DaedPsyker »

ergo14 wrote: post #4
kubb wrote: post #3

and i still have no idea what i could use it for...

We speed up our ML code 40 times using it.

Would that be in the data loading that you are getting the most benefit?

I'm curious, since most of the big libraries are already just cuda calls anyway but I'm always interested in anything to speed up the full process.

Re: Cython is 20

#16

Post by jokoon »

Question: I found python "bindings" for SFML, written in cython, and patched them a bit.

I guess Cython is not really made to write bindings, but is it easier to write bindings with cython or cpython?

Re: Cython is 20

#17

Post by chaxor »

pjmlp wrote: post #7

While it is nice that this option is available, it would be much better if Python itself would embrance the necessary runtime capabilties to not have to rely on it.

Don't count on it - switch to Julia instead.

Re: Cython is 20

#18

Post by microtonal »

DaedPsyker wrote: post #15
ergo14 wrote: post #4

Earlier quoted context omitted.

We speed up our ML code 40 times using it.

Would that be in the data loading that you are getting the most benefit? I'm curious, since most of the big libraries are already just cuda calls anyway but I'm always interested in anything to speed up the full process.

I can't speak for the parent commenter, but there is often code processing the input/output of machine learning models that benefits from high-performance implementations. To give two examples:

1. We recently implemented an edit tree lemmatizer for spaCy. The machine learning model predicts labels that map to edit trees. However, in order to lemmatize tokens, the trees need to be applied. I implemented all the tree wrangling in Cython to speed up processing and save memory (trees are encoded as compact C unions):

https://github.com/explosion/spaCy/blob/master/spacy/pipelin...

2. I am working on a biaffine parser for spaCy. Most implementations of biaffine parsing use a Python implementation of MST decoding, which is unfortunately quite slow. Some people have reported that decoding dominates parsing time (rather than applying an expensive transformer + biaffine layer). I have implemented MST decoding in Cython and it barely shows up in profiles:

https://github.com/explosion/spacy-experimental/blob/master/...

Re: Cython is 20

#19

Post by jokoon »

physicsguy wrote: post #2

I love Cython. I really feel like it's the right balance of usability and allowing you to do what you want/need. Want to make your code a bit faster? Write Python with type annotations. Want to call a C library? Just import the header, and then use it from a function. Pybind11 is also great, but quite different in aims - I feel like it's more like a project for C++ programmers wanting to expose functionality to Pytho…

Any benchmarks for type annotations?

I already wrote a few patch for pysfml, which is written in cython, it was a bit awkward, and now I'm asking myself if cython is really the right tool to write bindings, compared to cpython, for example.

Re: Cython is 20

#20

Post by f311a »

pjmlp wrote: post #7

While it is nice that this option is available, it would be much better if Python itself would embrance the necessary runtime capabilties to not have to rely on it.

To write fast Cython, you basically need to write in C and control everything including Python API calls. No runtime will help with this.
Post reply on HN