Libgccjit.so: an embeddable JIT-compiler based on GCC • Hacker News Board

Re: Libgccjit.so: an embeddable JIT-compiler based on GCC

#31

Post by iskander » Thu, Oct 03, 2013, 8:17 PM UTC

DannyBee wrote: post #21

Jasper_ wrote: post #8

Earlier quoted context omitted.

I work with David. Of course he knows about LLVM. This is his way to bring gcc into the modern era. Also related is his plan for removing global state from gcc: http://dmalcolm.fedorapeople.org/gcc/global-state/ He's seriously invested in cleaning up gcc and making it a competitive compiler. You should also see the API he has for JITing, it's much higher-level than LLVM: https://github.com/davidmalcolm/jittest/blob/m…

Could you explain what you mean by " This is his way to bring gcc into the modern era" and "He's seriously invested in cleaning up gcc and making it a competitive compiler." Optimization wise, GCC is still much more modern than LLVM, and in fact, more modern than a most commercial compilers. Architecture wise, there are warts, but bringing GCC into the modern era architecturally was never a technical/engineering chal…

I work on a runtime compiler with LLVM and gcc backends. LLVM presents me with a nice API to generate its IR in memory. For gcc, I actually write out a ".c" file and then invoke gcc to compile it into a shared library, which I load with dlopen. It works, but it's messy and I would prefer to work with an API like LLVM's. This project is exactly what I've been looking for.

Re: Libgccjit.so: an embeddable JIT-compiler based on GCC

#32

Post by xenonflash » Fri, Oct 04, 2013, 1:11 AM UTC

dmytrish wrote: post #3

hartror wrote: post #2

I wonder if this would have been easier to accomplish with LLVM given that gcc's api is so notorious to deal with?

It looks like LLVM was designed for JIT compilation from the start.

The Unladen Swallow project (faster Python, including an LLVM based JIT) found that LLVM wasn't really designed as a universal JIT compiler, or at least not one that was useful for languages such as Python. That was one of the reasons why they eventually stopped work on the project (although a lot of the non-LLVM work they did was still very good an useful and so the project as a whole was not wasted).

The Pypy developers (another Python with JIT project) looked very, very, carefully at every available JIT out there, and finally ended up writing their own. It was the only way that they could get something that would do what they needed. The results of doing that was performance that was far superior to what the Unladen Swallow project got when using LLVM.

It sounds very simple when you look at it from a distance. "We need a JIT, here's a JIT, let's use it." Then you find out that just because something is called a "JIT" doesn't mean that it will be of any use to what you are trying to do. The subject area is very complex and there will probably never be a universal solution.

What I could see the GCC JIT mode being useful for is things like generating certain critical portions of a program under programmer direction. That is, it would make a nice library that you call to generate very optimized code for specific functions or modules. A good example is how GCC is currently called in the background by a number of Python libraries which dynamically generate C code and compile it for faster execution. Being able to do this more directly via a JIT process could be very convenient. This is perhaps the sort of application that the author has in mind.

Re: Libgccjit.so: an embeddable JIT-compiler based on GCC

#33

Post by DannyBee » Fri, Oct 04, 2013, 6:17 AM UTC

azakai wrote: post #27

DannyBee wrote: post #22

Earlier quoted context omitted.

Right, but unlike GCC, this is not an architectural issue, but a simple implementation issue that could be fixed.

It's an implementation issue, but it is far from simple.

So I guess I disagree. There are known good solutions to this problem. Yes, it's definitely some work actually writing the code right, but it's not like this is a problem that requires engineering brand new solutions. It just requires a good engineer and some time.

I consider that "simple", as on the scale of "engineering complexity", it would be simple, even though on the scale of "engineering time" it may take longer.

Re: Libgccjit.so: an embeddable JIT-compiler based on GCC

#34

Post by belorn » Fri, Oct 04, 2013, 8:51 AM UTC

MichaelGG wrote: post #30

riotingpscifis wrote: post #26

Earlier quoted context omitted.

Unless you are extending the compiler why would you care? GCC consistently produced faster code that LLVM If you are extending the compiler, why would you want to allow companies like apple and oracle to bottle up you hard work and not give anything back? Either by not giving the code back or by patenting parts of the functionality they add and then suing you when you try and use them.

It's not LGPL, so linking to it means your entire project is now GPLv3.

And? The above post asks why companies need to make proprietary code changes to the compiler, or why the community should allow distributors of GCC to patent parts of the compiler and then go suing users who the distributer gave copies to.

Re: Libgccjit.so: an embeddable JIT-compiler based on GCC

#35

Post by georgemcbay » Sun, Oct 06, 2013, 8:19 AM UTC

belorn wrote: post #34

MichaelGG wrote: post #30

Earlier quoted context omitted.

It's not LGPL, so linking to it means your entire project is now GPLv3.

And? The above post asks why companies need to make proprietary code changes to the compiler , or why the community should allow distributors of GCC to patent parts of the compiler and then go suing users who the distributer gave copies to.

Even if you don't make changes to the compiler, if you link to this project you are GPLv3 infected. You simply can't use this without exposing your entire project to GPLv3 infection, the usual LGPL and/or runtime library exceptions don't apply to gcc's core code because it was never intended to be used as a library.

Let's say you have an existing FOSS app that is BSD or MIT licensed and it has its own built-in scripting language. You'd like to build a JIT for that scripting language, you see this library and decide to use it in your project, well... you can't do that without changing your entire project to GPLv3 terms because the combined work created between this and your own code all has to be available under the GPLv3 terms. This is usually solved by making the relevant parts of the project LGPL or granting a runtime library exception but neither of those applies to the core gcc code, so the GPLv3 infection in unavoidable.

Whether or not that issue is important is open to debate and depends upon your software politics, but in practical terms it means very few people will use this in their project unless they are already GPLv3 committed for some reason.