C++ Language Interface Foundation (CLIF) • Hacker News Board

Re: C++ Language Interface Foundation (CLIF)

#31

Post by pjmlp » Wed, May 03, 2017, 4:19 AM UTC

pcwalton wrote: post #20

rayiner wrote: post #8

C++ is an extremely challenging language to write a wrapper generator for. I did a C++ to Common Lisp generator about a decade ago for my GSoC project, based on GCC-XML (with the goal of being able to wrap QT). The two challenges you run into are that the GCC C+ ABI is extremely complicated and requires a runtime on top of whatever low-level C FFI you're working with. The second is that many semantics just don't map…

> All that is a roundabout way of saying: for god's sake write your system APIs in C. Or at least expose a C-compatible ABI.

On Windows we get to use COM and .NET bindings, with UWP controls slowly picking up steam.

Much better and we get to use a modern OO API.

Hence why better Windows support is relevant for Rust, if you want it to be more seriously taken by Windows devs.

EDIT: Same applies to mainframes, which also don't follow a C ABI, rather their own native languages.

Re: C++ Language Interface Foundation (CLIF)

#32

Post by mountain_lion » Wed, May 03, 2017, 4:42 AM UTC

gumby wrote: post #28

rayiner wrote: post #8

C++ is an extremely challenging language to write a wrapper generator for. I did a C++ to Common Lisp generator about a decade ago for my GSoC project, based on GCC-XML (with the goal of being able to wrap QT). The two challenges you run into are that the GCC C+ ABI is extremely complicated and requires a runtime on top of whatever low-level C FFI you're working with. The second is that many semantics just don't map…

Hopefully this can read the code well enough to recognize explicit instantiation in C++ code and just generate the python versions of those. One of the (many) painful things about SWIG is trying to keep your .i file in synch with your c++ code.

CLIF does recognize explicit template instatiations, and can instantiate a limited set itself. There is no pain like keeping the two files in sync because clif only allows a very small amount of C++ in its sources--basically just type names.

And cliff takes care to error out when the wrapper description doesn't match the C++. "When in doubt, refuse the temptation to guess."

Our internal clients really like it. Someone described it the other day as "magic".

Re: C++ Language Interface Foundation (CLIF)

#33

Post by malkia » Wed, May 03, 2017, 4:52 AM UTC

malkia wrote: post #23

There are several issues that I usually hit with bindings to C/C++ and other languages: - Calling a native function that itself takes callback, and that callback might be your non-native code. (trampolines?) - Dealing with memory allocation. Who owns what. - Exceptions or long_jmps - Green threads, fibers, etc. - Other?

Also:

  - int64 -> uint64 conversion
  - int64/uint64 -> double conversion

Re: C++ Language Interface Foundation (CLIF)

#34

Post by zvrba » Wed, May 03, 2017, 6:22 AM UTC

rayiner wrote: post #8

C++ is an extremely challenging language to write a wrapper generator for. I did a C++ to Common Lisp generator about a decade ago for my GSoC project, based on GCC-XML (with the goal of being able to wrap QT). The two challenges you run into are that the GCC C+ ABI is extremely complicated and requires a runtime on top of whatever low-level C FFI you're working with. The second is that many semantics just don't map…

> All that is a roundabout way of saying: for god's sake write your system APIs in C.

No. Use an IDL to define component interfaces and a generator for any particular language. That's how C++ components interoperate on Windows. (COM is just a formalization of vtables generated from an IDL file and compiled with midl compiler.)

Re: C++ Language Interface Foundation (CLIF)

#35

Post by _pmf_ » Wed, May 03, 2017, 6:50 AM UTC

obstinate wrote: post #25

gpshead wrote: post #14

I wondered how long it would take before someone posted CLIF here since we haven't put up a blog post about it yet... :) Interesting stat: Around 80% of our the Python C++ extension module wrappings being added to our code base are now being done using CLIF instead of SWIG. We are actively working on forbidding new Python SWIG wrappers from being added and migrating important legacy wrappings off of it onto CLIF. Who…

When are y'all doing Go and Java, though? The fact that this exists just compounds the pain of writing SWIG for the other two languages. The grass really is greener.

SWIG is ... special.

Re: C++ Language Interface Foundation (CLIF)

#36

Post by Cieplak » Wed, May 03, 2017, 6:52 AM UTC

chubot wrote: post #21

rayiner wrote: post #8

C++ is an extremely challenging language to write a wrapper generator for. I did a C++ to Common Lisp generator about a decade ago for my GSoC project, based on GCC-XML (with the goal of being able to wrap QT). The two challenges you run into are that the GCC C+ ABI is extremely complicated and requires a runtime on top of whatever low-level C FFI you're working with. The second is that many semantics just don't map…

Yeah I've grown to appreciate C++ in many ways, but what I noticed at my last job is that the baroque interfaces give it a viral effect. Once you have a little C++ code, the easiest thing to do is to continue writing more C++ code, even if it's not the best tool for that job. It's too hard to use a C++ interface from any other language. Even wrapping it in C is annoying, although as I understand LLVM does exactly tha…

I quite like using Erlang/OTP to host C++ nifs. I feel like it strikes that 10% of code being in C++ balance that you mentioned. I heard it's a popular combo for HFT applications.

http://erlang.org/doc/tutorial/nif.html

Re: C++ Language Interface Foundation (CLIF)

#37

Post by ithkuil » Wed, May 03, 2017, 7:12 AM UTC

mountain_lion wrote: post #32

gumby wrote: post #28

Earlier quoted context omitted.

Hopefully this can read the code well enough to recognize explicit instantiation in C++ code and just generate the python versions of those. One of the (many) painful things about SWIG is trying to keep your .i file in synch with your c++ code.

CLIF does recognize explicit template instatiations, and can instantiate a limited set itself. There is no pain like keeping the two files in sync because clif only allows a very small amount of C++ in its sources--basically just type names. And cliff takes care to error out when the wrapper description doesn't match the C++. "When in doubt, refuse the temptation to guess." Our internal clients really like it. Someon…

"magic" is an overloaded term.

In can mean something so beautifully advanced that you cannot distinguish it from advanced tech and you really don't care how it works, since it works so flawlessly you couldn't bother looking behind the scenes. I.e. a praise.

Or it can mean something that uses some arcane unearthly constructs and at the moment you need to look behind the scenes you find yourself utterly lost. I.e. a critique.

Re: C++ Language Interface Foundation (CLIF)

#38

Post by pjmlp » Wed, May 03, 2017, 8:08 AM UTC

chubot wrote: post #21

rayiner wrote: post #8

C++ is an extremely challenging language to write a wrapper generator for. I did a C++ to Common Lisp generator about a decade ago for my GSoC project, based on GCC-XML (with the goal of being able to wrap QT). The two challenges you run into are that the GCC C+ ABI is extremely complicated and requires a runtime on top of whatever low-level C FFI you're working with. The second is that many semantics just don't map…

Yeah I've grown to appreciate C++ in many ways, but what I noticed at my last job is that the baroque interfaces give it a viral effect. Once you have a little C++ code, the easiest thing to do is to continue writing more C++ code, even if it's not the best tool for that job. It's too hard to use a C++ interface from any other language. Even wrapping it in C is annoying, although as I understand LLVM does exactly tha…

> C++ is sort of like a universal receiver and not a universal donor. It can assimilate any C code...

This is part of the language's design.

Bjarne's goal after being forced to use BCPL instead of Simula was never to use a bare bones language ever again, and C with Classes needed to fit into AT&T's C tooling.

Hence why we also didn't got modules back then and a funky name mangling to fit into those bare UNIX linkers.

As for being an universal donor, the story is a bit different in OSes where the ABI is not C based, e.g. BeOS, Symbian, Genode, Windows (COM, .NET, UWP), OS/400 (TIMI), z/OS (ILC).

Re: C++ Language Interface Foundation (CLIF)

#39

Post by pjmlp » Wed, May 03, 2017, 8:09 AM UTC

mountain_lion wrote: post #16

rayiner wrote: post #8

C++ is an extremely challenging language to write a wrapper generator for. I did a C++ to Common Lisp generator about a decade ago for my GSoC project, based on GCC-XML (with the goal of being able to wrap QT). The two challenges you run into are that the GCC C+ ABI is extremely complicated and requires a runtime on top of whatever low-level C FFI you're working with. The second is that many semantics just don't map…

It makes sense to write system APIs in C, but there are many more things than system APIs, and many of them find it useful to use C++ constructs. (edit) Clang's internals are also several orders of magnitude easier to work with than GCC's internals. In fact, clang is designed to be used as a library, and CLIF leverages that very, very highly. See, for example, https://clang.llvm.org/docs/ClangTools.html . Clang tools…

Also should be noted that outside UNIX world, there are other OSes that don't use C conventions, or are migrating away from them (even if slowly).

Re: C++ Language Interface Foundation (CLIF)

#40

Post by gumby » Wed, May 03, 2017, 8:18 AM UTC

mountain_lion wrote: post #32

gumby wrote: post #28

Earlier quoted context omitted.

Hopefully this can read the code well enough to recognize explicit instantiation in C++ code and just generate the python versions of those. One of the (many) painful things about SWIG is trying to keep your .i file in synch with your c++ code.

CLIF does recognize explicit template instatiations, and can instantiate a limited set itself. There is no pain like keeping the two files in sync because clif only allows a very small amount of C++ in its sources--basically just type names. And cliff takes care to error out when the wrapper description doesn't match the C++. "When in doubt, refuse the temptation to guess." Our internal clients really like it. Someon…

Why do you need a handwritten clif file?