211–220 of 234 posts
Hm, I tried comparing the simple.hs vs simple.py (with ghc 8.10.7 which I happened to have here, their stack.yaml uses 8.10.4) and it's 5.7s vs 1.9s, not quite as stark a difference. Maybe because I'm on Linux?
Re: the Rust optimized implementation, I was able to get ~20-25% better performance by rewriting the for loops as iterators, using Rust's byte range pattern matching, and a buffered writer, which seems crazy, but it's true. I chalked it up to some crazy ILP/SIMD tricks the compiler is doing. I even submitted a PR[0], but Ben decided he was tired of maintaining and decided to archive the project (which fair enough!).…
Also, for loops desugar to into_iter, but the extra closure does provide an additional bump https://doc.rust-lang.org/book/ch13-04-performance.html
Earlier quoted context omitted.
Out of curiosity, what's the difference in runtime when compared to Andrew's optimized version on your machine? For your other solution, if the time save is consistent from your machine to OP's (a big if), the Rust solution bumps up to 4th place.
> Out of curiosity, what's the difference in runtime when compared to Andrew's optimized version on your machine? Results on an M1: my "idiomatic" version is 1.32 times faster than Andrew's original optimized version, whereas the optimized C version is 1.13 times faster than my "idiomatic" version. So, all things being equal, that'd make Rust 3rd, just ahead of C++, and behind Zig and C, if I'm reading the results co…
I've seen folks flub the measuring for this benchmark by testing much smaller inputs, for example.
For those curious about Rust optimization, the “optimized” version makes three changes compared to the “idiomatic” version: * It uses byte strings instead of UTF-8 strings. In my opinion, that’s not an optimization, that’s changing the problem. Depending on the question you’re asking, only one of the two can be correct. * It uses a faster hash algorithm. It’s not the first time this came up in a benchmark article. Ru…
No, that's very wrong. ripgrep has rich Unicode support for example, but represents file contents as byte strings. UTF-8 strings vs byte strings is an implementation detail.
I think you might benefit from reading the "bonus" Rust submission: https://github.com/benhoyt/countwords/blob/8553c8f600c40a462...
IMO, Ben kind of glossed over the bonus submission. But I personally think it was the biggest point in favor of Rust for a real world version of this task.
How about extending this to comparing programming language efficiency from a developer's standpoint, looking at bytes of code, versus the execution time? I took the source code file size from the repository, and the runtime from the blog post. Then I made an arbitrary overall "PAIN SCORE" (lower is better) by multiplying code size * runtime. I suggest this is a worthwhile metric simply because lower is better on both…
It would probably be prudent to strip comments before measuring the code sizes: I did not check all files, but noticed that the simple rust code consists of about 60% comments (in terms of bytes), mostly due to a verbose blurb. It also spends about 15% of the non-comment bytes on printing nicer error messages, which is an interesting choice for a micro-benchmark. The simple C and FORTH versions similarly have a lot o…
Indeed, a naively analysis based on code size without taking comments into account is pretty obviously wrong.
I'm curious why go is faster than rust in the benchmarks. My best theory is that the program doesn't run long enough to trigger a GC, so rust is paying the cost of freeing memory, but go isn't. (Using an arena allocator in rust would probably produce similar behavior). Or maybe the difference is just noise? It's hard to tell without more details on how the benchmarks were run.
I wrote comments explaining things: https://github.com/benhoyt/countwords/blob/8553c8f600c40a462...
You can tell it will be BS just from the title treating C and C++ as if they were one language, not two very different languages. Other comments here reveal how, exactly.
from collections import Counter
from re import finditer
word_counts = Counter()
with open(PATH, encoding = 'utf-8') as file:
doc = file.read()
word_counts.update(match.group().lower() for match in finditer(r'\w+', doc))
Earlier quoted context omitted.
> Out of curiosity, what's the difference in runtime when compared to Andrew's optimized version on your machine? Results on an M1: my "idiomatic" version is 1.32 times faster than Andrew's original optimized version, whereas the optimized C version is 1.13 times faster than my "idiomatic" version. So, all things being equal, that'd make Rust 3rd, just ahead of C++, and behind Zig and C, if I'm reading the results co…
Can you share precisely how you're running the code and measuring timings? I've seen folks flub the measuring for this benchmark by testing much smaller inputs, for example.
I just took a look on Linux and my code that is now running 1.43x faster on the Mac is only 1.02x faster on Ubuntu 22.04. But, again, it's really that it's a Rust commercial -- by leaning on the stdlib, it's possible to get straightforward, but still really fast code.
I have no doubt you could make it faster than me and have!
Re: the Rust optimized implementation, I was able to get ~20-25% better performance by rewriting the for loops as iterators, using Rust's byte range pattern matching, and a buffered writer, which seems crazy, but it's true. I chalked it up to some crazy ILP/SIMD tricks the compiler is doing. I even submitted a PR[0], but Ben decided he was tired of maintaining and decided to archive the project (which fair enough!).…
$ cd /tmp/
$ git clone -b ag/test-kimono https://github.com/BurntSushi/countwords
$ cd countwords/rust/
$ ./bench
Summary
'./optimized-trie/target/release/countwords Your 'fast-simple' version is a different story though. Personally, I would not put that in the "simple" classification as outlined by the OP. It is IMO very much not the first program someone would write for this. Bringing in 'hashbrown' and futzing with the hash table lookup/insert is definitely a perf tweak that you probably wouldn't want to do unless you had to, because it makes the code more complex. The 'Box' cements it as an 'optimized' variant.
Now it does look like a simpler variant of the "optimized" Rust program I wrote. I'm pretty impressed. I don't think I ever would have broken out of my local optima to discover that program. In particular, your approach appears to do 3 (and some change) passes over each buffer: 1) read up to '\n' (just the first line), 2) UTF-8 validation, 3) make lowercase and 4) split on whitespace. I realize chunking all of that up confers a lot of benefits, but still, 4 passes and it still being faster than 1 pass is very interesting.
I've glanced at the profiles of each program but haven't been able to solidly conclude anything more precisely about where exactly the benefit is. At this point, my next step would be to slowly translate my version into yours, and benchmark each step of the way until I could isolate the key change (assuming it is one change). But alas, I have run out of time this evening. :-)
Kudos!
(The other interesting thing to note here is that my 'trie' variant is now the fastest Rust submission. Previously, it was slower than the optimized variants on my older CPU. That's pretty neat.)