New Computer Language Benchmarks Game metric: time + source code size
benchmarksgame-team.pages.debian.net
New Computer Language Benchmarks Game metric: time + source code size
1–10 of 59 posts
Re: New Computer Language Benchmarks Game metric: time + source code size
#2[deleted]
Re: New Computer Language Benchmarks Game metric: time + source code size
#3I'd be interested to see "C compiled with Clang" added as another language to the benchmark games. In part, digging into Clang vs gcc benchmarks is always interesting, and in part, as Rust & Clang share the same LLVM backend, it would shed light on how much of the C vs Rust difference is from frontend language stuff vs backend code gen stuff.
Re: New Computer Language Benchmarks Game metric: time + source code size
#4For comparing multiple implementations of a single benchmark in a single language, this sort of data would be interesting as a 2D plot, to see how many lines it takes to improve performance by how much. But for cross-language benchmarking this seems somewhat confounding, as the richness of standard libraries varies between languages (and counting the lines of external dependencies sounds extremely annoying, not only because you have to decide whether to include standard libraries (including libc), you also need to find a way not to penalize those for having many lines devoted to tests).
Re: New Computer Language Benchmarks Game metric: time + source code size
#5Geometric mean of (time + gzipped source code size in bytes) seems statistically wrong.
What if you shifted time to nanoseconds ? Or source code size in terms of Megabytes. The rankings could change. The culprit is the '+'
I would think Geometric mean of (time x gzipped source code size) is the correct way to compare languages together. It would not matter what the units of time or size are in that case.
[Here the geometric mean is the geometric mean of (time x gzipped size) of all benchmark programs of a particular language.]
Re: New Computer Language Benchmarks Game metric: time + source code size
#6For comparing multiple implementations of a single benchmark in a single language, this sort of data would be interesting as a 2D plot, to see how many lines it takes to improve performance by how much. But for cross-language benchmarking this seems somewhat confounding, as the richness of standard libraries varies between languages (and counting the lines of external dependencies sounds extremely annoying, not only…
And when you want to make the code readable you try to space things out, split things in small functions, use longer and clear variables name. I guess they are asking for running the code trough a minifier so their implementation gains some points.
Re: New Computer Language Benchmarks Game metric: time + source code size
#7This presentation is pretty bad, there should be more context, some kind of color scheme or labels instead of text in the background, spacing between the languages represented, other benchmarks than the geometric mean, etc.
Re: New Computer Language Benchmarks Game metric: time + source code size
#8Geometric mean of (time + gzipped source code size in bytes) seems statistically wrong. What if you shifted time to nanoseconds ? Or source code size in terms of Megabytes. The rankings could change. The culprit is the '+' I would think Geometric mean of (time x gzipped source code size) is the correct way to compare languages together. It would not matter what the units of time or size are in that case. [Here the ge…
Yep this is correct. Adding disparate units is almost always nonsensical. You can confirm with a scientific calculator like insect:
$ insect '5s + 10MB'
Conversion error:
Cannot convert unit MB (base units: bit)
to unit s
$ insect '5s * 10MB'
50 s·MBRe: New Computer Language Benchmarks Game metric: time + source code size
#9For comparing multiple implementations of a single benchmark in a single language, this sort of data would be interesting as a 2D plot, to see how many lines it takes to improve performance by how much. But for cross-language benchmarking this seems somewhat confounding, as the richness of standard libraries varies between languages (and counting the lines of external dependencies sounds extremely annoying, not only…
And when you want to make the code readable you try to space things out, split things in small functions, use longer and clear variables name. I guess they are asking for running the code trough a minifier so their implementation gains some points.
On other benchmarks they measure the size of source code after it's been run through compression, as a way to normalize that. Not sure if that's been done here, but it should be.
Re: New Computer Language Benchmarks Game metric: time + source code size
#10For comparing multiple implementations of a single benchmark in a single language, this sort of data would be interesting as a 2D plot, to see how many lines it takes to improve performance by how much. But for cross-language benchmarking this seems somewhat confounding, as the richness of standard libraries varies between languages (and counting the lines of external dependencies sounds extremely annoying, not only…
I'm not sure I see the problem. What does it matter that program A is shorter than program B because language A has a richer standard library? Program A still required less code.