Programming language ray tracing benchmarks project • Hacker News Board

Re: Programming language ray tracing benchmarks project

#41

Post by v_lisivka » Thu, Apr 25, 2019, 7:49 PM UTC

hackerpacker wrote: post #35

z92 wrote: post #33

Earlier quoted context omitted.

Julia was astonishing. It's a high level language that's performing almost like C. Last time I checked, many years back, the spec was changing and the run time did crash. Guess it has gone a long way since. The other one is Lua. My assumption was that it's one of the lightest and fastest language around. Looks like "fastest" isn't true in some cases.

shouldn't rust being faster than C be something of a red flag that they aren't quite the same algorithm? Or that the algorithm is sub-optimal?

Rust is compiled by LLVM, while C compiled by GCC, which is a bit conservative. It's possible to enable same optimizations for gcc and LLVM, so their speed will match.

Re: Programming language ray tracing benchmarks project

#42

Post by steveklabnik » Thu, Apr 25, 2019, 7:51 PM UTC

z92 wrote: post #39

hackerpacker wrote: post #35

Earlier quoted context omitted.

shouldn't rust being faster than C be something of a red flag that they aren't quite the same algorithm? Or that the algorithm is sub-optimal?

shouldn't rust being faster than C be something of a red flag that they aren't quite the same algorithm? Or that the algorithm is sub-optimal? The difference isn't much. And Rust is more like FORTRAN. Maybe a bit faster than C, but can't do the gymnastics with pointers that C can.

With unsafe, you can do anything that C can.

Without unsafe, there’s significantly more aliasing information, which helps optimizations.

Re: Programming language ray tracing benchmarks project

#43

Post by oconnor663 » Thu, Apr 25, 2019, 7:52 PM UTC

z92 wrote: post #39

hackerpacker wrote: post #35

Earlier quoted context omitted.

shouldn't rust being faster than C be something of a red flag that they aren't quite the same algorithm? Or that the algorithm is sub-optimal?

shouldn't rust being faster than C be something of a red flag that they aren't quite the same algorithm? Or that the algorithm is sub-optimal? The difference isn't much. And Rust is more like FORTRAN. Maybe a bit faster than C, but can't do the gymnastics with pointers that C can.

> can't do the gymnastics with pointers that C can

It can if you write the "unsafe" keyword, but there's a pretty strong community norm around not doing that sort of thing, unless you can encapsulate it inside some sort of safe API. And to be fair to C, I think C can close the gap with Rust/Fortran if you use the "restrict" keyword a lot?

Re: Programming language ray tracing benchmarks project

#44

Post by piinbinary » Thu, Apr 25, 2019, 7:56 PM UTC

The haskell version can be made >= 3x faster by making the computations non-lazy, e.g.

    -data Vector3 = Vector3 {vx::Float, vy::Float, vz::Float} deriving (Show)
    +data Vector3 = Vector3 {vx :: !Float, vy :: !Float, vz :: !Float} deriving (Show)

Re: Programming language ray tracing benchmarks project

#45

Post by weberc2 » Thu, Apr 25, 2019, 8:00 PM UTC

Lerc wrote: post #34

weberc2 wrote: post #24

Earlier quoted context omitted.

That's a big jump between OCaml and Go. I'm not familiar with ray tracing, but skimming the source code it mostly looks like it's doing floating point math; it doesn't look like it's using the runtime (no allocations, no virtual function calls, no scheduling, etc), so I'm surprised that Go is performing relatively poorly. I wonder if the performance gap is attributable to some overhead in Go's function calls? I know…

I'm not familiar with go but I seem to recall it is garbage collected. If so it may be something to do with the creation of new vectors on the heap instead of the stack. The compiler would have to determine the full lifetime of the vector value to be able to bump it to the stack. That's an optimization, and sometimes it's just not possible (but probably is here). In the C instance no such optimization is necessary. T…

Is it creating vectors in the hot path? I'm not seeing it. Go does some escape analysis (the optimization you're referring to), but it's pretty conservative.

Re: Programming language ray tracing benchmarks project

#46

Post by remcob » Thu, Apr 25, 2019, 8:00 PM UTC

oconnor663 wrote: post #36

z92 wrote: post #33

Earlier quoted context omitted.

Julia was astonishing. It's a high level language that's performing almost like C. Last time I checked, many years back, the spec was changing and the run time did crash. Guess it has gone a long way since. The other one is Lua. My assumption was that it's one of the lightest and fastest language around. Looks like "fastest" isn't true in some cases.

Different languages' benchmarks might not be equally well-written / optimized. In particular, I'd expect C and Rust to be very close to each other, and a 20% gap between them is a red flag. Rules like "code should be simple, as in, easy to read and understand" are also hard to judge, especially near the top of the list where there's a lot of pressure to optimize. Is SIMD easy to understand? What if it's in a library?…

Not necessarily. Because C allows the pointer manipulation, the compiler can in general not make assumptions about pointer aliasing. This prevents some optimizations.

In Rust, the compiler has more information/control over memory layout/lifetime and can therefore make stronger optimizations.

Automatic vectorization is an area where this helps a lot, and raytracing can benefit a lot here. 20% sounds reasonable to me.

Re: Programming language ray tracing benchmarks project

#47

Post by Someone » Thu, Apr 25, 2019, 8:04 PM UTC

This times performance like this:

  $ time ./crb

That means time spent writing the .ppm file is included.

In the implementations I browsed, that is about a million print calls, each of which might flush the output buffer, and whose performance may depend on locale.

To benchmark ray tracing I would, instead, just output the sum of the pixel values, or set the exit code depending on that value.

Even though ray tracing is cpu intensive, it also wouldn’t completely surprise me if some of the implementations in less mature languages spent significant time writing that output because their programmers haven’t come around to optimizing such code.

Re: Programming language ray tracing benchmarks project

#48

Post by jerf » Thu, Apr 25, 2019, 8:05 PM UTC

weberc2 wrote: post #24

kenhwang wrote: post #14

Ordered by realtime, fastest to slowest for those like me who got annoyed by the scrolling up and down trying to compare: Rust (1.13.0-nightly) 1m32.392s Nim (0.14.2) 1m53.320s C 1m59.116s Julia (0.4.6) 2m01.166s Crystal (0.18.7) 2m01.735s C Double Precision 2m26.546s Java (1.7.0_111) 2m36.949s Nim Double Precision (0.14.2) 3m19.547s OCaml 3m59.597s Go 1.6 6m44.151s node.js (6.2.1) 7m59.041s node.js (5.7.1) 8m49.170s…

That's a big jump between OCaml and Go. I'm not familiar with ray tracing, but skimming the source code it mostly looks like it's doing floating point math; it doesn't look like it's using the runtime (no allocations, no virtual function calls, no scheduling, etc), so I'm surprised that Go is performing relatively poorly. I wonder if the performance gap is attributable to some overhead in Go's function calls? I know…

There's a variety of possibilities. Lerc mentions GC as one possibility, which could definitely be the case. Another one that would be high on my "first guess" list is that everything above it has much better optimizers, and raytracing code is one of the places this is really going to show. Go does basically very little optimization, because it prioritizes fast compilation.

(Where Go "wants" to play is that same benchmark, except including compilation time.)

A couple of the things below Go I suspect are bad implementations. I would expect a warmed-up C# to beat Go if both have reasonable (not super-crazy optimized implementations) or at least be at parity, and Luajit may also be a slow implementation. In both cases because ray-traced code is a great place for a JIT to come out and play. EDIT: Oh, I see C# is Mono, and not the Windows implementation. In that case that makes sense.

Oh, and I find it helpful to look at these things logarithmically. I think it matches real-world experiences somewhat better, even though we truly pay for performance in the linear world. From that perspective, it's still only the second largest. The largest is Haskell to Elixir, which is substantially larger. O'Caml->Go is large, but not crazily so; several other differences come close.

Re: Programming language ray tracing benchmarks project

#49

Post by [deleted] » Thu, Apr 25, 2019, 8:06 PM UTC

weberc2 wrote: post #24

kenhwang wrote: post #14

Ordered by realtime, fastest to slowest for those like me who got annoyed by the scrolling up and down trying to compare: Rust (1.13.0-nightly) 1m32.392s Nim (0.14.2) 1m53.320s C 1m59.116s Julia (0.4.6) 2m01.166s Crystal (0.18.7) 2m01.735s C Double Precision 2m26.546s Java (1.7.0_111) 2m36.949s Nim Double Precision (0.14.2) 3m19.547s OCaml 3m59.597s Go 1.6 6m44.151s node.js (6.2.1) 7m59.041s node.js (5.7.1) 8m49.170s…

That's a big jump between OCaml and Go. I'm not familiar with ray tracing, but skimming the source code it mostly looks like it's doing floating point math; it doesn't look like it's using the runtime (no allocations, no virtual function calls, no scheduling, etc), so I'm surprised that Go is performing relatively poorly. I wonder if the performance gap is attributable to some overhead in Go's function calls? I know…

[deleted]

Re: Programming language ray tracing benchmarks project

#50

Post by alkonaut » Thu, Apr 25, 2019, 8:18 PM UTC

jerf wrote: post #48

weberc2 wrote: post #24

Earlier quoted context omitted.

That's a big jump between OCaml and Go. I'm not familiar with ray tracing, but skimming the source code it mostly looks like it's doing floating point math; it doesn't look like it's using the runtime (no allocations, no virtual function calls, no scheduling, etc), so I'm surprised that Go is performing relatively poorly. I wonder if the performance gap is attributable to some overhead in Go's function calls? I know…

There's a variety of possibilities. Lerc mentions GC as one possibility, which could definitely be the case. Another one that would be high on my "first guess" list is that everything above it has much better optimizers, and raytracing code is one of the places this is really going to show. Go does basically very little optimization, because it prioritizes fast compilation. (Where Go "wants" to play is that same benc…

There are multiple ”levels” of performance in play here, and which level a language performs on depends on the language, runtime and implementation.

The most naive level is e.g allocating heap objects for vectors, rays etc. On that level the algorithm is probably bounded by pointed chasing, cache misses and GC.

The next level up is an allocation-free loop (at least)

The best level is an optimized and allocation free. If the implementation isn’t allowed to optimize (use SoA instead of AoS, manually vectorize, unroll etc) then the winning languages will be the ones that have sophisticated compilers such as those with LLVM backends.

As an example: The C# example should be on the second level here - but it has a poor implementation (looks like it’s ported from java or written by a java developer) so it’s actually stuck on the first naive level.