ClojureC, a compiler for Clojure that targets C as a backend

  Not everything can (or should) be proved of the drop 
  of a hat in a discussion list

If you say something that doesn't make sense, you're saying that one shouldn't have to back that up?

  It's like asking me to prove why using an XML processor 
  to crete and save a DOM tree would produce more error 
  free results than manually compiling tags as strings.

It is in fact the equivalent of asking you to prove that an XML parser after serializing a DOM tree and then parsing that same document produces a non-equivalent DOM tree to the original.

I think the reason pat_punnu is balking at what you have said is because of this: You can think of C as a serialized form of an AST. In order for parsing to produce a non-equivalent AST to the one used to serialize it, the C grammar must be non-deterministic. The C grammar is not non-deterministic, therefore what you said does not make sense and pat_punnu (somewhat rudely) asked you to back up what you were saying.

Asking people to back up what they have claimed is part of intelligent discourse. This happens often on HN and is one of the reasons I like this community because when the person who makes a non-intuitive or seemingly wrong remark turns out to be correct, I learn something.

UPDATE: I should point out that I'm responding specifically to this sub-thread of the tree and not arguing about whether or not this should target the LLVM IR. I think that would be nice.

> ..argh, installing that on ubuntu that requires 110 packages. All that just for a build system?

Don't install lein using the OS packaging system (apt, rpm, yum, etc.).

Instead, just grab the `lein` script (linked to from http://leiningen.org/ ), put it into your ~/bin, set it executable, and you're all set.

This is neat.

What worries me about this and similar efforts (like https://github.com/takeoutweight/clojure-scheme) is that clojure's standard library design assumes that the underlying runtime will be do some kind of polymorphic method inlining.

For example: the sequence library is all defined in terms of ISeq, which basically requires a "first" and "rest" to be defined for the data structure in question. These are polymorphic: there are different implementations of these for different data structures (list, vector, map, etc). So a dispatch step is required to choose the right one. In clojure-jvm, this is implemented using a java interface; this means the jvm will inline calls to said methods when they're being used in a tight loop. And if you use the standard library, calls to 'first' and 'rest' are going to be inside nearly all of your inner loops.

Compare this to a normal lisp or scheme: 'first' and 'rest' (or 'car' and 'cdr', whatever) are monomorphic. They only work on the linked-list data structure. So compiling these directly down to C functions makes perfect sense and incurs no performance penalty.

So in summary: clojure assumes theres a really smart JIT which is helping things along. This means it's not as suitable for alternate compilation targets as you might want it to be.

I wonder if there's something clever you could do here. Vtables could be reordered based on expected usage, certainly. Clojure can already do some measure of type inference, so this could be used for AOT inlining when it's available. Even if it's not, perhaps several versions of a call could be speculatively generated based on what the compiler does know already. The normal polymorphic inline caching technique could perhaps be abused to apply here. But it's hard to see how any of this can work in absence of a profile or heavy hinting.

(not a compiler writer, just interested in the problem)

> It's not like it's some controversial opinion what he said -- it's both self evident and common place.

As someone who has written more than one compiler, I don't see how it is self-evident at all. It's also not at all that common-place compared to generating C or asm output textually.

> It's about having more structure and less of an ad-hoc pipeline, which helps with better tooling, error prevention, etc.

Those provide some benefits, sure. At the cost of massive amounts of complexity in the case of LLVM.

> The only benefit to using C for something like this is portability, which is something else altogether.

Now it is you who are wrong. Other people have already pointed out, for example, that C provides an easy-to-read intermediate format, and is simple to generate, as other benefits. Not having to deal with a massive C++ codebase is another.

You may disagree that these other benefits are worth it, but for me at least they are (just taking a break from a compiler that generates textual asm because I find even that preferable to dealing with LLVM).

You can do polymorphic method caching/inlining with a hybrid ahead of time / JIT compiler targeting C reasonably easily. The code fragments required for caching at least will be small, and code to generate them at runtime is not a big deal. I'm playing with a Ruby compiler, and Ruby badly needs these type of optimizations to get fast, so I've spent a fair amount of time looking at it.

For a fair amount of cases you can do static analysis to get good guesses at likely types, even for cases where you can't be sure. E.g. speculatively even looking near call sites by method name to see if you can guess the type of objects that will get passed in looks to get you a reasonable chance at guessing at the top contenders to let you speculatively generate inlined versions without creating too much junk. But to get the most performance out of this you're likely to need to be prepared to do some very basic JIT.