81–90 of 194 posts
Earlier quoted context omitted.
Why would uncompressed bytes be better? Using a good compression algorithm better approximates the statistical entropy of the code which is at least correlated with e.g., Kolmogorov complexity.
because it often sits uncompressed on the drive
Earlier quoted context omitted.
"Expressive power" is a very subjective term, and uncompressed size is a bad proxy as it includes too many variables specific to coding conventions. Compressed size with a stupid enough algorithm (here gzip) is meant to reduce these variables. The true Kolmogorov complexity in comparison can't be computed, and too smart algorithms can start to infer enough about the language itself.
Kolmogorov complexity is absolutely the wrong metric. It doesn't account for big-O, timing, and many other production requirements. You'll never have a perfect metric here, but human readable size of code base is well justified. Do you write minified javascript?
For emitting programs, if we assume the program is already fully formed in my brain and I'm just transcribing it, then we would like an accurate physical model of my hands moving over a QWERTY keyboard that can tell us how many joules and milliseconds I will use for each keystroke to type the given sequence of symbols. Ideally we would have a complete model or simulation of my brain so we can measure how many neurons need to fire for each keystroke as well. We don't have that, so we could measure my average milliseconds per character and multiply it by the total count of characters, but a language that is just made up of one symbol function names is probably harder to type than one made out of English words and there's also all the typing mistakes I will make. This is what the simple compression algorithm (gzip) is an attempt to normalize for, that a more verbose but more predictable language is as fast to write and read than an overly terse one. gziping is an imitation of a complex model.
Earlier quoted context omitted.
It demonstrates that Python needs libraries like NumPy. Few problems are more heavily optimized than matrix multiplication in practice, so comparing matrix multiplication benchmarks across languages with NumPy is not representative of real-world performance for most programming use cases. It also means that adding performance to an existing Python program requires dropping into a different language, which is not only…
> It demonstrates that Python needs libraries like NumPy. People use matrix multiplication libraries (often written in Assembly) from every language if they really care about performance. That's because such libraries incorporate 100 PhD theses' worth of tricks that no individual can hope to reinvent in the course of solving another problem. There is absolutely nothing special about Python in this context. > It also…
Assume the author knows about BLAS, and that the point is to benchmark the language not the FFI. Most people don't actually spend their time bashing vectors together.
Neat! The Julia matmul implementation has its rows and columns flipped though - unlike C, Julia uses row-major matrices. This has large implications for speed. Also, the code may be much faster if you enable SIMD in the function, which is disabled in the code because a) the code unnecessarily checks bounds at every index instead of at the top of the function, and b) float SIMD is opt-in since SIMD changes the roundin…
Earlier quoted context omitted.
"Expressive power" is a very subjective term, and uncompressed size is a bad proxy as it includes too many variables specific to coding conventions. Compressed size with a stupid enough algorithm (here gzip) is meant to reduce these variables. The true Kolmogorov complexity in comparison can't be computed, and too smart algorithms can start to infer enough about the language itself.
Kolmogorov complexity is absolutely the wrong metric. It doesn't account for big-O, timing, and many other production requirements. You'll never have a perfect metric here, but human readable size of code base is well justified. Do you write minified javascript?
Yes [1] [2] [3]?
[1] https://js1024.fun/demos/2020/46/readme
[2] https://js1024.fun/demos/2022/18/readme
[3] https://github.com/lifthrasiir/roadroller/blob/442caa4/index...
Jokes aside, human doesn't read each (uncompressed) byte anyway. The number of tokens would have been much better than the number of bytes, but even this is unclear because a single token can have multiple perceived words (e.g. someLongEnoughIdentifier) and multiple tokens can even be perceived as a single word for some cases (e.g. C/C++ `#define` is technically two tokens long, but no human would perceive it as such). I would welcome a more realistic estimate than the gzipped size, but I'm confident that it won't be the number of uncompressed bytes.
Earlier quoted context omitted.
> Generally, the point of language benchmarks is to show how the languages compare at solving the same problem, without external libraries. If this were the only concern, it should be valid to create a blob of binary and call that function for the optimal performance. (Python's ctypes makes this very easy, for example.) So you want an idiomatic solution instead, and Numpy for matrix computation is considered idiomati…
No, creating a binary blob is not a realistic option, and it's still not a Python/Java/C++ whatever solution. It would be handwritten assembly that you use the FFI facilities to call into. So, it's not fair game in any honest benchmark. And even if using a C library is idiomatic Python, it still has no place in a language benchmark. It's a C library, not a Python implementation.
Once again, "realistic" is subjective and I would say no "realistic" user will try to multiply arbitrarily-sized matrices in pure Python. (I can see small enough matrices, like 3x3 or 4x4, might be different.) And...
> So, it's not fair game in any honest benchmark. And even if using a C library is idiomatic Python, it still has no place in a language benchmark. It's a C library, not a Python implementation.
...you have correctly figured out that it's unfair for anyone using your (vague) definition of "realistic". It's also true that your proposal is also unfair for anyone using my definition of "idiomatic" however. I can try to defend my definition with quantative arguments, but I don't feel like doing so. In fact I tend to ignore most "language benchmarks" because it is virtually impossible to make them reasonably fair. This one is no exception.
Best "language benchmarks" tend to be more like language showcases with useful commentaries, there will be no single winner but you will get a good sense of pros and cons of each language-implementation-strategy combination. They are generally not advertised as "benchmarks", of course.
Earlier quoted context omitted.
Generally, the point of language benchmarks is to show how the languages compare at solving the same problem, without external libraries. Including external libraries is pointless since any language can call any library, ultimately, so at best you'd be comparing the FFI overhead. So this shouldn't be taken as "how fast does a real-world Python program do at matrix multiplication", since of course no one writes real-w…
> But it can show the relative speed of pure Python at purely computational tasks. But that's irrelevant if nobody uses "pure Python" for computational tasks. It's like asking "how well do these languages run on a Lisp machine from 1979?". It simply has no relevance to real-world considerations today.
For the love of god, log graph tiny values with large values please. :)
Earlier quoted context omitted.
The rust code is very unidiomatic, not only because of the Vec of Vecs which I’d say, even if it’s the obvious naive approach, no one experienced wouldn’t choose over a flat slice, the implementation itself is very naive and unidiomatic.
Also it's using checked indexing, which apart from being not idiomatic, is also going to slow things down. A fairer comparison would be to use the unchecked indexing variants.