Live data from Hacker News

Benchmarking 20 programming languages on N-queens and matrix multiplication

github.com

51–60 of 194 posts

Re: Benchmarking 20 programming languages on N-queens and matrix multiplication

#51
post #5

What is this supposed to demonstrate? There appears to be roughly the same code structure, ported to every language, while for some languages, arbitrary optimizations are introduced (such as using `array` instead of `list` in Python). But nobody working in Python uses matrix multiplication code written in Python. They use NumPy, which is a de facto standard library for people working in the relevant fields. It's as m…

[deleted]

Re: Benchmarking 20 programming languages on N-queens and matrix multiplication

#53
post #41

Earlier quoted context omitted.

Generally, the point of language benchmarks is to show how the languages compare at solving the same problem, without external libraries. Including external libraries is pointless since any language can call any library, ultimately, so at best you'd be comparing the FFI overhead. So this shouldn't be taken as "how fast does a real-world Python program do at matrix multiplication", since of course no one writes real-w…

> But it can show the relative speed of pure Python at purely computational tasks. But that's irrelevant if nobody uses "pure Python" for computational tasks. It's like asking "how well do these languages run on a Lisp machine from 1979?". It simply has no relevance to real-world considerations today.

Every time you use a for loop, addition or multiplication operators, array indexing, assingment to numerical variable in Python, you are using exactly what is benchmarked in matmul.

Re: Benchmarking 20 programming languages on N-queens and matrix multiplication

#54
Surprised how well JS (node and others) seem to come out when I've had firsthand experience of switching from JS to Go to speed up an algorithm type question and had the Go version crank through a bruteforce much much faster.

Maybe I made an accidental optimization in my language translation, or maybe there are some operations that are much slower in JS and these benchmarks didn't hit any of them.

Re: Benchmarking 20 programming languages on N-queens and matrix multiplication

#55

Earlier quoted context omitted.

Yes, but the author claims the longest JIT warmup is 0.3 seconds, so it's not an important issue in these benchmarks that take several seconds.

I strongly suspect that the author may have confused the JIT warmup (hard to measure, as you need to ensure that the performance figure have reached the stable point) from the startup overhead (easy to measure).

...and many jitted vms do not even reach a stable point at all, there was a big paper on this a couple of years ago.

Re: Benchmarking 20 programming languages on N-queens and matrix multiplication

#58
post #54

Surprised how well JS (node and others) seem to come out when I've had firsthand experience of switching from JS to Go to speed up an algorithm type question and had the Go version crank through a bruteforce much much faster. Maybe I made an accidental optimization in my language translation, or maybe there are some operations that are much slower in JS and these benchmarks didn't hit any of them.

Go's AOT compiler is actually not that sophiscated (comparable to -O1 in most C compilers, possibly even worse). Was your program running only for a fraction of a second? Then the JIT may haven't fully warmed up and even a basic AOT compiler has a better chance to win.

Re: Benchmarking 20 programming languages on N-queens and matrix multiplication

#59

Earlier quoted context omitted.

IMO, uncompressed bytes is a better representation, because it can be used to compare relative expressive power for the particular problem. I'd bet Python cleans house here, but the write-only languages are a wild card.

Why would uncompressed bytes be better? Using a good compression algorithm better approximates the statistical entropy of the code which is at least correlated with e.g., Kolmogorov complexity.

because it often sits uncompressed on the drive

Re: Benchmarking 20 programming languages on N-queens and matrix multiplication

#60
post #25

Aren’t JIT languages at a disadvantage since they are benchmarked through the CLI rather than using a benchmarking library to allow JIT to warmup?

What's CLI?

Presumably command line interface/terminal. I can type `dotnet run` in the terminal and provide some options to runa .NET program for example.
Post reply on HN