Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ClojureC, a compiler for Clojure that targets C as a backend

github.com

Post reply on HN
Print view View original

31–40 of 89 posts

Re: ClojureC, a compiler for Clojure that targets C as a backend

#31

Post by mullr »

vidarh wrote: post #30
mullr wrote: post #26

This is neat. What worries me about this and similar efforts (like https://github.com/takeoutweight/clojure-scheme ) is that clojure's standard library design assumes that the underlying runtime will be do some kind of polymorphic method inlining. For example: the sequence library is all defined in terms of ISeq, which basically requires a "first" and "rest" to be defined for the data structure in question. These are…

You can do polymorphic method caching/inlining with a hybrid ahead of time / JIT compiler targeting C reasonably easily. The code fragments required for caching at least will be small, and code to generate them at runtime is not a big deal. I'm playing with a Ruby compiler, and Ruby badly needs these type of optimizations to get fast, so I've spent a fair amount of time looking at it. For a fair amount of cases you c…

>E.g. speculatively even looking near call sites by method name

That is devious and fantastic.

> But to get the most performance out of this you're likely to need to be prepared to do some very basic JIT.

Yeah. But the attractive targets here are places where you can't have a JIT: embedded systems and iOS.

Re: ClojureC, a compiler for Clojure that targets C as a backend

#32

Post by vidarh »

smosher wrote: post #29
pat_punnu wrote: post #13

Earlier quoted context omitted.

"more ... error prone" Can you prove that? If not, it's just a baseless opinion.

More moving parts being more error prone is not generally a controversial opinion.

This argument can just as easily be used to support a claim that generating C via string manipulation can very well be less error prone than relying on a huge, complex API like LLVM.

It is not a given that there are "more moving parts" in generating C output from a compiler than in using LLVM.

Re: ClojureC, a compiler for Clojure that targets C as a backend

#33

Post by swannodette »

mullr wrote: post #31
vidarh wrote: post #30

Earlier quoted context omitted.

You can do polymorphic method caching/inlining with a hybrid ahead of time / JIT compiler targeting C reasonably easily. The code fragments required for caching at least will be small, and code to generate them at runtime is not a big deal. I'm playing with a Ruby compiler, and Ruby badly needs these type of optimizations to get fast, so I've spent a fair amount of time looking at it. For a fair amount of cases you c…

>E.g. speculatively even looking near call sites by method name That is devious and fantastic. > But to get the most performance out of this you're likely to need to be prepared to do some very basic JIT. Yeah. But the attractive targets here are places where you can't have a JIT: embedded systems and iOS.

re: JIT on iOS I've heard of people compiling their own JavaScriptCore (to use Ejecta), is this still an issue? Embedded systems is another story, I'd be far more concerned about Clojure's assumptions about GC.

Re: ClojureC, a compiler for Clojure that targets C as a backend

#35

Post by jared314 »

rdmckenzie wrote: post #20
rayiner wrote: post #18

Earlier quoted context omitted.

Generating C is quicker to write and easier to debug (can more easily read intermediate output) and doesn't require accessing a C++ api.

I can't stress the C++ api part of this enough. At the moment I'm building a Pascal compiler in Clojure for course work and when the time came to do byte code generation the first thing I looked to was the LLVM infrastructure both for ease of use and because it would trivially provide me with the ability to target non-x86 platforms. The reality of the matter is that interacting with non-java linked libraries is a rea…

Have you looked at mjolnir[0]? It was part of a presentation at Clojure West this year. I'm not sure if the video is out yet, but the slides are[1].

[0] https://github.com/halgari/mjolnir

[1] https://github.com/strangeloop/clojurewest2013/tree/master/s...

Re: ClojureC, a compiler for Clojure that targets C as a backend

#36

Post by rdmckenzie »

jared314 wrote: post #35
rdmckenzie wrote: post #20

Earlier quoted context omitted.

I can't stress the C++ api part of this enough. At the moment I'm building a Pascal compiler in Clojure for course work and when the time came to do byte code generation the first thing I looked to was the LLVM infrastructure both for ease of use and because it would trivially provide me with the ability to target non-x86 platforms. The reality of the matter is that interacting with non-java linked libraries is a rea…

Have you looked at mjolnir[0]? It was part of a presentation at Clojure West this year. I'm not sure if the video is out yet, but the slides are[1]. [0] https://github.com/halgari/mjolnir [1] https://github.com/strangeloop/clojurewest2013/tree/master/s...

bah just as I was finishing my register allocator...

My Firefox history indicates that I have read the Mjolnir page before, but I don't recall why I didn't use it at the time. Taking another look :-P. Thanks for the link!

Re: ClojureC, a compiler for Clojure that targets C as a backend

#37

Post by saosebastiao »

So if I wanted to write CLI applications in Clojure, is this my best bet? Cause the JVM is about the least suitable platform I have ever worked with for CLI apps...which is most of what I do. I'm constantly in this pickle of wanting to use Clojure but defaulting to Ruby because the JVM is so terrible at it.

Re: ClojureC, a compiler for Clojure that targets C as a backend

#38

Post by vidarh »

mullr wrote: post #31
vidarh wrote: post #30

Earlier quoted context omitted.

You can do polymorphic method caching/inlining with a hybrid ahead of time / JIT compiler targeting C reasonably easily. The code fragments required for caching at least will be small, and code to generate them at runtime is not a big deal. I'm playing with a Ruby compiler, and Ruby badly needs these type of optimizations to get fast, so I've spent a fair amount of time looking at it. For a fair amount of cases you c…

>E.g. speculatively even looking near call sites by method name That is devious and fantastic. > But to get the most performance out of this you're likely to need to be prepared to do some very basic JIT. Yeah. But the attractive targets here are places where you can't have a JIT: embedded systems and iOS.

When you spend a few years speculating about what it would take to efficiently compile Ruby as statically as possible (I love Ruby, but I hate moving parts), devious becomes second nature...

The idea of speculatively looking at method names comes from testing that to create vtables ahead of time for Ruby classes, to avoid hash tables in the common-case.

As it turns out, most method on most Ruby classes are the ones inherited from Object or other standard classes, and the number of classes is usually fairly constrained, so again speculatively looking at method names in the compile-time available source and allocating sparse vtables for the most common names results in relatively little waste.

And it reduces typical method lookup to a vtable lookup for common methods, with expensive method dispatch becoming much more rare. There's the tradeoff between theoretical horrible blowup in vtable waste from apps dynamically adding tons of methods and tons of classes, with a unique vtable slot required for each method name across all classes, vs. falling back to doing hash-table lookups all the way up the inheritance chain for "unusual" method names ones you reach certain thresholds for waste.

You do incur the cost of propagating vtable changes down the inheritance tree when methods are dynamically redefined in other places than leaves, but it is fairly rare to see apps where this happens at a very high rate, and the number of subclasses usually fairly small, so it is likely to be quite cheap. Doing it that way is something I first saw in a technical report by (now) prof. Michael Franz from '93 or '94 on "Protocol Extension" for Oberon.

You can probably also get some decent gains by adding heuristics to give preference to names that appears to be used in loops when picking names for the vtables to reduce the need of any JIT'ing.

Re: ClojureC, a compiler for Clojure that targets C as a backend

#39

Post by mambodog »

saosebastiao wrote: post #37

So if I wanted to write CLI applications in Clojure, is this my best bet? Cause the JVM is about the least suitable platform I have ever worked with for CLI apps...which is most of what I do. I'm constantly in this pickle of wanting to use Clojure but defaulting to Ruby because the JVM is so terrible at it.

You could also use node.js with Clojurescript, if you're willing to accept the compromises of Clojurescript.

Re: ClojureC, a compiler for Clojure that targets C as a backend

#40

Post by bbq »

saosebastiao wrote: post #37

So if I wanted to write CLI applications in Clojure, is this my best bet? Cause the JVM is about the least suitable platform I have ever worked with for CLI apps...which is most of what I do. I'm constantly in this pickle of wanting to use Clojure but defaulting to Ruby because the JVM is so terrible at it.

If the startup slowness is your problem, look into Nailgun: http://www.martiansoftware.com/nailgun/

If that's not your problem with the JVM, what is?

Post reply on HN