I had a 1 minute look at the "optimized" C code - it's calling malloc for every word.
Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
131–140 of 234 posts
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#132I had a 1 minute look at the "optimized" C code - it's calling malloc for every word.
Doesn't malloc return a pointer? Sounds unsafe
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#133Another 1 minute lookin... multiplication on every byte.. really?
If this is supposed to be fast, get one big buffer for the words. Get another if you run out.
The hash function can be the word itself for words up to length 8.
Longer words would need a mult per 8 bytes.
You can do branchless strchr.
Roll your own memmove - just always copy 24 bytes and terminate at the length you already know.
The whole point of using C or C++ is that you can do non idiomatic things when you have to.
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#134I had a 1 minute look at the "optimized" C code - it's calling malloc for every word.
Doesn't malloc return a pointer? Sounds unsafe
That's not the point. It's perfectly safe and there's nothing else in C.
The point is you don't have to call this function for every single word. It takes time and pointer increment doesn't
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#135Earlier quoted context omitted.
As I wrote elsewhere in this thread (though I was downvoted), I suspect the reason is that this benchmark is measuring the total execution time of a single run of an executable. This would include any time spent by the executable to bootstrap its runtime environment, initiate a virtual machine, and whatever else it needs to do in addition to the relevant code at hand to process the string. Such a test will favor impl…
I was going to make a general comment about this too. It’s a huge penalty for Java too and has no correlation for how well it performs outside of toy benchmarks
… unless you’re using Java in the same way, such as an Azure Function or AWS Lambda. Or simply as a command-line tool.
IMHO, VM startup time should be included and the first few passes (before JIT kicks in) should also be included.
Java has supposedly “nearly C-like performance” until you read the fine print.
(This should apply to C# as well.)
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#136I had a 1 minute look at the "optimized" C code - it's calling malloc for every word.
>I had a 1 minute look at the "optimized" C code - it's calling malloc for every word.
Neither simple.c nor optimized.c call malloc "for every word". They only call malloc when inserting a previously-unseen word into the counting map.
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#137> We usually think of I/O as expensive, but I/O isn’t the bottleneck here...The tokenization and hash table operations are the bottleneck by a long shot Interesting. This is something I'll have to keep in mind. Best practice is always been to consider I/O the slowest. But, it's true... times have changed. Maybe we shouldn't make this assumption anymore.
He didn't clear the disk cache when profiling so of course the IO overhead was negligible. Had he flushed the cache between runs the numbers would have been very different.
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#138I would not call it performance comparison at all. When Python call functions written in C it is not Python's performance. Write those functions using plain Python and then see the results. Sure for this basic example in the article it does not matter from a practical standpoint. But when you need to step away from canned cases suddenly Python's performance sucks big time.
So we should disqualify the C and C++ impls because some libc functions are implemented using ASM, right?
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#139Earlier quoted context omitted.
and, is it a compiled language at all? versus a script language of some kind.
Is what a compiled language? For Knuth's program in the Programming Pearls article? He wrote it in Pascal, so not a scripting language.
I thought he wrote it in his own language, WEB.
Re: Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
#140This is a rather meaningless comparison, since the differences are going to be dominated by: 1) The choice of libraries/datatypes used for strings and word->count map 2) How the source file is split into words - probably library function again, although in C/C++ one could choose to implement a super-optimized low level version that would blow the others away IMO a performance comparison between languages is only mean…
exactly, is mostly comparing different C libraries implementations.