Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

C++ Language Interface Foundation (CLIF)

github.com

Post reply on HN
Print view View original

21–30 of 72 posts

Re: C++ Language Interface Foundation (CLIF)

#21

Post by chubot »

rayiner wrote: post #8

C++ is an extremely challenging language to write a wrapper generator for. I did a C++ to Common Lisp generator about a decade ago for my GSoC project, based on GCC-XML (with the goal of being able to wrap QT). The two challenges you run into are that the GCC C+ ABI is extremely complicated and requires a runtime on top of whatever low-level C FFI you're working with. The second is that many semantics just don't map…

Yeah I've grown to appreciate C++ in many ways, but what I noticed at my last job is that the baroque interfaces give it a viral effect.

Once you have a little C++ code, the easiest thing to do is to continue writing more C++ code, even if it's not the best tool for that job. It's too hard to use a C++ interface from any other language. Even wrapping it in C is annoying, although as I understand LLVM does exactly that for some of its API.

IMO there are a lot of systems where C++ is the best language for about 10% of the code. C++ really is unique in terms of offering zero-cost abstraction. But then the remaining 90% gets written in C++ too. It can be remarkably awkward for many problems, and your build times scale nonlinearly too.

In some way this is inherent ... the compiler's job is to erase all those abstractions and generate straightforward machine code. In other ways it is the fault/result of adhering to the C linking model, which ironically was for interoperability.

C++ is sort of like a universal receiver and not a universal donor. It can assimilate any C code, and thus it gets code in other languages transitively. But other languages can't assimilate it, at least not without great effort.

That said, Clang has a better API than GCC-XML, so the problem might be solved. Athoough honestly C++ just keeps growing more features with C++ 11, 14, 17 that make it harder to interoperate with any other code. It's this huge compile-time language completely separate from the C linking model.

Re: C++ Language Interface Foundation (CLIF)

#22

Post by chubot »

mountain_lion wrote: post #16
rayiner wrote: post #8

C++ is an extremely challenging language to write a wrapper generator for. I did a C++ to Common Lisp generator about a decade ago for my GSoC project, based on GCC-XML (with the goal of being able to wrap QT). The two challenges you run into are that the GCC C+ ABI is extremely complicated and requires a runtime on top of whatever low-level C FFI you're working with. The second is that many semantics just don't map…

It makes sense to write system APIs in C, but there are many more things than system APIs, and many of them find it useful to use C++ constructs. (edit) Clang's internals are also several orders of magnitude easier to work with than GCC's internals. In fact, clang is designed to be used as a library, and CLIF leverages that very, very highly. See, for example, https://clang.llvm.org/docs/ClangTools.html . Clang tools…

How stable are Clang's internals? Are you supposed to be able to write your own Clang tools and link them dynamically, or are they mostly distributed and built together with the Clang source code? (i.e. like Linux drivers where there is no stable interface.)

I know that LLVM is notable for not maintaining API stability -- not sure about Clang.

Re: C++ Language Interface Foundation (CLIF)

#23

Post by malkia »

There are several issues that I usually hit with bindings to C/C++ and other languages:

  - Calling a native function that itself takes callback, and that callback might be your non-native code. (trampolines?)
  - Dealing with memory allocation. Who owns what.
  - Exceptions or long_jmps
  - Green threads, fibers, etc.
  - Other?

Re: C++ Language Interface Foundation (CLIF)

#24

Post by sriram_sun »

m-j-fox wrote: post #9
gumby wrote: post #3

Finally! I have been suffering under SWIG and have been hoping for some time that someone would get the compiler to do this.+ Not to say SWIG isn't an amazing effort, but it's a whole C++ compiler maintained by a very small team. + I already put my time in at the gcc salt mines so no, I didn't try doing this myself.

I'm cautiously excited. The problem with C++ is there's just so many good frameworks but you can't use them without creating more C++. SWIG should be called WIG. And anyway, I can't see it surviving modern C++. As long as I'm ranting, what is modern C++ but a bunch of new languages that are incompatible with C++, each other, and everything else? Forgive me if I'm wrong but this seems like a terrible idea.

> SWIG should be called WIG. Depends on your perspective. The other alternative is (was?) to start typing up those wrapper functions manually.

Re: C++ Language Interface Foundation (CLIF)

#25

Post by obstinate »

gpshead wrote: post #14

I wondered how long it would take before someone posted CLIF here since we haven't put up a blog post about it yet... :) Interesting stat: Around 80% of our the Python C++ extension module wrappings being added to our code base are now being done using CLIF instead of SWIG. We are actively working on forbidding new Python SWIG wrappers from being added and migrating important legacy wrappings off of it onto CLIF. Who…

When are y'all doing Go and Java, though? The fact that this exists just compounds the pain of writing SWIG for the other two languages. The grass really is greener.

Re: C++ Language Interface Foundation (CLIF)

#26

Post by gpshead »

krona wrote: post #6

The repository mentions "other languages" but I could only find Python examples? While I think wrapper generators are a noble idea, I doubt I would choose this over the convenience of IDL-like metaprogramming approaches (e.g. pybind11, or Boost.Python)

Metaprogramming such as pybind11 does is neat. Thanks for the pointer to the project. It still looks like manually written C extension modules to me based off of what I see in https://github.com/pybind/pybind11/blob/master/docs/basics.r.... Just much shorter code with a lot of the hard to get right details taken care of you. Good! Better than the status quo. But it doesn't abstract the problem away very much. (no doubt some will consider that a feature)

What if I also wanted my wrapper available on a non CPython VM? The proper way to use PyPy is not via its fake CPython API support (slow and memory hungry). Imagine if a PyPy Generator were added to CLIF. It'd use the same interface definition Parser but generate an entirely different set of code. In PyPy's case that would probably be a C library wrapping the C++ for generated cffi based Python code to interact with.

Admittedly all hypothetical here until other Python VMs have CLIF Generator implementations.

Re: C++ Language Interface Foundation (CLIF)

#27

Post by mountain_lion »

chubot wrote: post #22
mountain_lion wrote: post #16

Earlier quoted context omitted.

It makes sense to write system APIs in C, but there are many more things than system APIs, and many of them find it useful to use C++ constructs. (edit) Clang's internals are also several orders of magnitude easier to work with than GCC's internals. In fact, clang is designed to be used as a library, and CLIF leverages that very, very highly. See, for example, https://clang.llvm.org/docs/ClangTools.html . Clang tools…

How stable are Clang's internals? Are you supposed to be able to write your own Clang tools and link them dynamically, or are they mostly distributed and built together with the Clang source code? (i.e. like Linux drivers where there is no stable interface.) I know that LLVM is notable for not maintaining API stability -- not sure about Clang.

There are (roughly) two ways of using Clang as a library. The first is a via a C-based API into a libclang.so. This is an API designed to be extremely stable, is fairly powerful, and works relatively well.

That API doesn't expose absolutely everything though, which is one reason it can be so stable. Tools that need more are typically shipped and built along side the clang source code. CLIF is the latter.

You can read about both of them here:

http://clang.llvm.org/docs/Tooling.html

Re: C++ Language Interface Foundation (CLIF)

#28

Post by gumby »

rayiner wrote: post #8

C++ is an extremely challenging language to write a wrapper generator for. I did a C++ to Common Lisp generator about a decade ago for my GSoC project, based on GCC-XML (with the goal of being able to wrap QT). The two challenges you run into are that the GCC C+ ABI is extremely complicated and requires a runtime on top of whatever low-level C FFI you're working with. The second is that many semantics just don't map…

Hopefully this can read the code well enough to recognize explicit instantiation in C++ code and just generate the python versions of those. One of the (many) painful things about SWIG is trying to keep your .i file in synch with your c++ code.

Re: C++ Language Interface Foundation (CLIF)

#29

Post by inetknght »

gpshead wrote: post #14

I wondered how long it would take before someone posted CLIF here since we haven't put up a blog post about it yet... :) Interesting stat: Around 80% of our the Python C++ extension module wrappings being added to our code base are now being done using CLIF instead of SWIG. We are actively working on forbidding new Python SWIG wrappers from being added and migrating important legacy wrappings off of it onto CLIF. Who…

So... someone with basically zero Python experience and a ton of C++ experience could basically "hand that experience down" to his newly minted Python-self?

Re: C++ Language Interface Foundation (CLIF)

#30

Post by solipsism »

malkia wrote: post #23

There are several issues that I usually hit with bindings to C/C++ and other languages: - Calling a native function that itself takes callback, and that callback might be your non-native code. (trampolines?) - Dealing with memory allocation. Who owns what. - Exceptions or long_jmps - Green threads, fibers, etc. - Other?

Callbacks are well supported in SWIG for most languages.
Post reply on HN