Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Interfacing Python and C: Advanced “ctypes” Features

dbader.org

Post reply on HN
Print view View original

1–10 of 18 posts

Re: Interfacing Python and C: Advanced “ctypes” Features

#2

Post by notacoward »

Good article. A while back, I wrote a framework to support Python modules in a multi-threaded C program, so the control flow is from C to Python and then back again.

http://www.linuxjournal.com/content/extending-glusterfs-pyth...

For the "embedding" part (C to Python) it's very important to get the GIL stuff right, but it's not totally obvious how to do so and it's hard to debug when you get it wrong. For the "extending" part (Python to C) the issue is going to be maintaining references for Python objects. Dealing with all of the edge cases (e.g. decorators and ctypes-generated function wrappers) was quite educational. Mixing C and Python is definitely a fun exercise.

Re: Interfacing Python and C: Advanced “ctypes” Features

#3

Post by xioxox »

I've always thought ctypes a bit of a hack as you get crashes and so on if you're not constantly careful about updating function definitions. It would be much nicer if the compiler could catch the errors, e.g.

    #include "mylib.h"
    PYTHON_WRAP(myfunc, void, float, float...)
Even with cython (please correct me if I'm wrong), the definition needs to be duplicated in the cython script to use it.

Re: Interfacing Python and C: Advanced “ctypes” Features

#4

Post by mattip »

xioxox wrote: post #3

I've always thought ctypes a bit of a hack as you get crashes and so on if you're not constantly careful about updating function definitions. It would be much nicer if the compiler could catch the errors, e.g. #include "mylib.h" PYTHON_WRAP(myfunc, void, float, float...) Even with cython (please correct me if I'm wrong), the definition needs to be duplicated in the cython script to use it.

You are describing cffi http://cffi.readthedocs.io/en/latest

Re: Interfacing Python and C: Advanced “ctypes” Features

#5

Post by xioxox »

mattip wrote: post #4
xioxox wrote: post #3

I've always thought ctypes a bit of a hack as you get crashes and so on if you're not constantly careful about updating function definitions. It would be much nicer if the compiler could catch the errors, e.g. #include "mylib.h" PYTHON_WRAP(myfunc, void, float, float...) Even with cython (please correct me if I'm wrong), the definition needs to be duplicated in the cython script to use it.

You are describing cffi http://cffi.readthedocs.io/en/latest

Sort of, though I was thinking of more of a system external to python at the C/C++ level. Maybe it's possible to use cffi like this.

Re: Interfacing Python and C: Advanced “ctypes” Features

#6

Post by abainbridge »

Seems like a reasonable article. But that Makefile is a disaster:

1. The clean rule doesn't delete the generated .html file.

2. The clean rule returns an error if some of the output files are missing. It needs a -f adding.

3. It doesn't implement dependency tracking of the header files or the Makefile itself.

4. On my clean Ubuntu 16 LTS install, the .o:.c rule doesn't get called. Instead Make uses its built-in rule which doesn't include -fPIC, so the build fails. I guess it is missing some % symbols.

More controversially, I think Makefiles for simple purposes like this are a form of premature optimization. It would be simpler to create a bash script called build.sh containing:

  CFLAGS="-Wall -Werror -fpic"
  gcc $CFLAGS -shared Point.c Line.c -o libline.so
  gcc $CFLAGS -shared Point.c -o libpoint.so
  ./testPoint.py
  ./testWrappedPoint.py
  ./testLine.py
That's much easier to understand, doesn't depend on Make being installed, rebuilds when a header is changed and doesn't generate .o file litter. The price is that it takes 0.1 seconds longer to run in the case where there's no building to be done and there's no "clean". I think it is a net win.

Re: Interfacing Python and C: Advanced “ctypes” Features

#8

Post by ryanpepper »

mattip wrote: post #4
xioxox wrote: post #3

I've always thought ctypes a bit of a hack as you get crashes and so on if you're not constantly careful about updating function definitions. It would be much nicer if the compiler could catch the errors, e.g. #include "mylib.h" PYTHON_WRAP(myfunc, void, float, float...) Even with cython (please correct me if I'm wrong), the definition needs to be duplicated in the cython script to use it.

You are describing cffi http://cffi.readthedocs.io/en/latest

Boost Python looks somewhat similar to that type of interface too. http://www.boost.org/doc/libs/1_62_0/libs/python/doc/html/tu...
Post reply on HN