[deleted]
Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
81–90 of 234 posts
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#82How about extending this to comparing programming language efficiency from a developer's standpoint, looking at bytes of code, versus the execution time? I took the source code file size from the repository, and the runtime from the blog post. Then I made an arbitrary overall "PAIN SCORE" (lower is better) by multiplying code size * runtime. I suggest this is a worthwhile metric simply because lower is better on both…
It'd be interesting to write code in a concise, easy to learn/grok language and then transpile to a faster systems language and recompute the PAIN score. I spent a few months last year pursuing that approach (py2many on github).
We'd need a stdlib that works across languages for the approach to be successful. nimpylib and similar libraries in other languages are a step in that direction.
Alternatively, the python stdlib itself could be rewritten in python and transpiled to other languages. Perhaps it'd help alternative python implementations in terms of compatibility.
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#83For those curious about Rust optimization, the “optimized” version makes three changes compared to the “idiomatic” version: * It uses byte strings instead of UTF-8 strings. In my opinion, that’s not an optimization, that’s changing the problem. Depending on the question you’re asking, only one of the two can be correct. * It uses a faster hash algorithm. It’s not the first time this came up in a benchmark article. Ru…
Ad uft8: it would change the problem only if the definition of word breaks changed. E.g., if a word break is defined by whitespace, then it probably doesn't.
What happens when the ASCII character for space is embedded within a multibyte UTF-8 codepoint?
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#84Earlier quoted context omitted.
Ad uft8: it would change the problem only if the definition of word breaks changed. E.g., if a word break is defined by whitespace, then it probably doesn't.
What happens when the ASCII character for space is embedded within a multibyte UTF-8 codepoint?
If UTF-8 allowed that, then a solution would split words wrong according to UTF-8 rules, but it still would not matter, since the problem statement says that "it's okay to only support ASCII for the whitespace handling and lowercase operation".
So if your solution gets word splitting (or lowercasing) wrong for non-ASCII input, it still gets a pass according to my reading.
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#85Your Perl implementation is almost as fast as c++? Something weird is going on here. Maybe you are bottlenecked on disk read speeds or something.
Why is that surprising? The perl interpreter is fast and dispatches just about everything to heavily optimized C. Just like python does, it is only better at it.
This is the kind of thing where perl really shines.
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#86Earlier quoted context omitted.
Ad uft8: it would change the problem only if the definition of word breaks changed. E.g., if a word break is defined by whitespace, then it probably doesn't.
What happens when the ASCII character for space is embedded within a multibyte UTF-8 codepoint?
UTF-8 is designed so that this doesn’t happen: All bytes in a multibyte codepoint have the high bit set, placing them outside of the ASCII range.
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#87Earlier quoted context omitted.
Ad uft8: it would change the problem only if the definition of word breaks changed. E.g., if a word break is defined by whitespace, then it probably doesn't.
What happens when the ASCII character for space is embedded within a multibyte UTF-8 codepoint?
This never happens. In UTF-8, bit 7 is set in all valid multibyte representations; ASCII characters cannot appear in them. Now, you can have an ASCII character in the middle of a multibyte sequence, but this isn't valid UTF-8 (and should be treated as an error by the decoder). "Words" generated by a splitter would also have invalid UTF-8 if they were split on a space coming in between bytes in a multibyte sequence, but these can detected. Word splitting will not generate invalid words from valid input, or vice versa; and if you know you have valid input, splitting on space, \t, \n, \r, etc. is perfectly cromulent in UTF-8.
Now if you want to also split on U+200B (ZERO WIDTH SPACE), U+202F (NARROW NO-BREAK SPACE), etc... hoo boy.
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#88This is a rather meaningless comparison, since the differences are going to be dominated by: 1) The choice of libraries/datatypes used for strings and word->count map 2) How the source file is split into words - probably library function again, although in C/C++ one could choose to implement a super-optimized low level version that would blow the others away IMO a performance comparison between languages is only mean…
I understand what you're saying. However if you look at it not from the point of the performance figures per se, but rather which language, when written idiomatically, can solve a realistic problem performantly, then I think it makes more sense.
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#89Wow. Swift, touted as "safe by design and (...) runs lightning-fast"[1] is more of a screw-up than I thought. Almost twice as slow as Lua and behind even Pascal and Forth. [1] https://developer.apple.com/swift/
Swift is in the same or better performance bracket then C#/Java and unsafe usage can be close to performance of C (maybe 1.3-2x slower). I'm not sure about ranting for poor single test results :P
The optimized version of C# is not really optimized as there are things left in the table that could be much close to C++.
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#90This is a rather meaningless comparison, since the differences are going to be dominated by: 1) The choice of libraries/datatypes used for strings and word->count map 2) How the source file is split into words - probably library function again, although in C/C++ one could choose to implement a super-optimized low level version that would blow the others away IMO a performance comparison between languages is only mean…
Do you have a particular example problem in mind that might demonstrate language performance better? One of the Benchmark Game programs maybe?