Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why is C faster than Java: git vs JGit

marc.info

Post reply on HN
Print view View original

41–50 of 106 posts

Re: Why is C faster than Java: git vs JGit

#41

Post by jaylevitt »

snprbob86 wrote: post #4

I almost skipped this link; I assumed it was typical borring blog noise. It's not. This is an insightful post from the git mailing list which shows some of the real limitations that a top tier developer hits when trying to write Java code as fast as neatly optimized C code. Definitely worth reading.

Yep. The usual "Program X is faster in C than Java" gets a barrage of "That's because you know C better". Shawn is a performance-obsessed Java expert, Eclipse committer and longtime Google coder who works on JGit. If he says Java is slower than C at this, then Java is slower than C at this.

EDIT: but as wcoenen points out, this was written in 2009 and Java 1.7 does a better job with some of this.

Re: Why is C faster than Java: git vs JGit

#42

Post by wh-uws »

durin42 wrote: post #31
willvarfar wrote: post #2

So why do they write and use jgit at google instead of just git?

Because cgit is a bunch of binaries that expect to call each other. That makes it harder to abstract out the storage layer, and we don't use vanilla repositories sitting on a filesystem. Things are backed by some other storage abstraction, which isn't always very posix-filesystem like.

Can you elaborate on the storage abstraction and the repository setup?

Just curious about what advantages there are to make you sacrifice the performance of the cgit binaries. Mostly out of ignorance on the subject.

Re: Why is C faster than Java: git vs JGit

#43

Post by SeanLuke »

I build fairly high-performance Java code. And get hit with three major gotchas which prevent it from approaching C code.

- There's no way to do array access without null pointer and index checks each and every time.

- Generics with basic types, and their unfortunate embedding into syntax (like the new for() syntax), are awful. Boxing and unboxing incur a ludicrously high penalty, and generics push coders away from using arrays. Unlike in C++, generics have been the enemy of performance.

- Poor quality collections classes (ArrayList and HashMap are notoriously bad)

Sure there's a few other things like pointer walking etc. in C, and Java's poor floating point, but the big three above are the killers.

Re: Why is C faster than Java: git vs JGit

#45

Post by mbell »

nknight wrote: post #18

In cases where performance actually matters, just avoid bit-twiddle in high-level languages. It sucks too much. You'll probably waste less time on optimizations by offloading the biggest bottlenecks to C/C++ with the native/extension interfaces in your high-level language of choice. Be careful and stay standards-compliant and you can keep most of the portability and maintenance advantages while picking up some signif…

>You'll probably waste less time on optimizations by offloading the biggest bottlenecks to C/C++ with the native/extension interfaces in your high-level language of choice.

In reality this often requires a heavy refactor to actually work. In Java with JNI for instance the overhead of calling native methods is actually rather high, over 200 cpu cycles in many cases. The stack often has to be re-arranged, a CPU stall is usually caused and in the case of most data types passed to the native function, they have to be copied (last i knew java.nio buffers were the only types that weren't copied).

Point is, just moving your "hot function" to C / C++ and calling with JNI doesn't work unless that function is rarely called and does a lot of work internally. More often the "hot function" is something that is called thousands of times and moving something like that to JNI is just as likely to kill performance as help it. You'd have to abstract away an entire module of work and minimize its call surface to JNI to achieve your goal.

Re: Why is C faster than Java: git vs JGit

#46

Post by axlelonghorn »

wh-uws wrote: post #42
durin42 wrote: post #31

Earlier quoted context omitted.

Because cgit is a bunch of binaries that expect to call each other. That makes it harder to abstract out the storage layer, and we don't use vanilla repositories sitting on a filesystem. Things are backed by some other storage abstraction, which isn't always very posix-filesystem like.

Can you elaborate on the storage abstraction and the repository setup? Just curious about what advantages there are to make you sacrifice the performance of the cgit binaries. Mostly out of ignorance on the subject.

There was a google talk on this posted to HN recently, but I can't find it. In it, one of the directors of the build / testing / code review system at google was talking about how they get things working at scale. Since everyone works out of the HEAD of one Perforce repo, they end up using the map-reduce infrastructure to perform tests in the cloud for each checkout. In line with this, there are too many files, that update too often for every developer to be checking out of the repo, so they use a custom FUSE filesystem to lazily give access to files only when they're needed.

Re: Why is C faster than Java: git vs JGit

#47

Post by obtu »

durin42 wrote: post #31
willvarfar wrote: post #2

So why do they write and use jgit at google instead of just git?

Because cgit is a bunch of binaries that expect to call each other. That makes it harder to abstract out the storage layer, and we don't use vanilla repositories sitting on a filesystem. Things are backed by some other storage abstraction, which isn't always very posix-filesystem like.

Didn't you turn to Dulwich rather than JGit for storage abstraction?

Re: Why is C faster than Java: git vs JGit

#48

Post by dochtman »

malkia wrote: post #44

I wonder if mercurial gets rewritten in "C" whether there would be any speedup.

I'm sure there would be some speedup, the question is whether it would be worth it (and I suppose that can only be adequately be assessed by the developers, who now have to maintain C code instead of Python).

But for some perspective from a former Mercurial developer: lots of the more performance-sensitive code has already been rewritten in C. Rewriting the rest of it would simply be a question of diminishing returns. One thing that would improve is hg's startup time; starting up Python just takes a while, which kind of sucks for command-line programs like VCS clients that tend to have many short-running invocations.

Re: Why is C faster than Java: git vs JGit

#49

Post by nvarsj »

dons wrote: post #36
kmm wrote: post #28

Earlier quoted context omitted.

I've heard the argument before that in need, one can use a FFI to optimize bottlenecks in high-level code, but I've never understood. Won't using a high-level language incur an omnipresent speed slump? And even if a bottleneck exists, how would using a FFI remedy crucial problems in the language, like the absence of unsigned types or that all types are boxed. The types will have to be unboxed anyway, so whether that…

At least in Haskell you have unboxed primitive types, memory mapped IO, bump-pointer allocation, and compilation to direct loops that are often identical to what GCC produces (or very close).

All of those things also exist in hotspot/Java.

Primitive types have been available since the creation of Java. It's up to the programmer to use boxed types or not.

Memory mapped IO - see Java.nio.

Bump-pointer allocation/compilation to direct loops all exist in hotspot.

Re: Why is C faster than Java: git vs JGit

#50

Post by soc88 »

SeanLuke wrote: post #43

I build fairly high-performance Java code. And get hit with three major gotchas which prevent it from approaching C code. - There's no way to do array access without null pointer and index checks each and every time. - Generics with basic types, and their unfortunate embedding into syntax (like the new for() syntax), are awful. Boxing and unboxing incur a ludicrously high penalty, and generics push coders away from u…

> - There's no way to do array access without null pointer and index checks each and every time.

That's not happening anymore for years already.

Post reply on HN