Earlier quoted context omitted.
What you are seeing is different regex engines and capabilities, and grep's focus on pure speed and optimization of a common case and Perl's focus on versatility. I see very similar results between Perl and grep, and you can see this by also including egrep, which allows slightly more complex expressions: [root@stats ~]# time perl -ne 'print if /number 123456/' But what happens if we use a slightly more complex expre…
Your results are interesting and I'd be curious to know why the grep degrades so badly on that last regex. But the original benchmark was ridiculously biased in favor of Perl by not actually doing anything in Perl. If Perl is actually being competitive in the unfair benchmark, the benchmark should be made more fair by actually putting some logic in Perl, and writing the equivalent logic in C. At that point, you would…
I really do think it has to do with grep swapping out regex implementations based on features needed. The last regex matches a variable length string, so it may trigger a much more complex and/or cpu-intensive regex engine to be used.
> If Perl is actually being competitive in the unfair benchmark, the benchmark should be made more fair by actually putting some logic in Perl, and writing the equivalent logic in C. At that point, you would start to see C win again (modulo any inherent inefficiencies in grep's regex engine).
I see your point, but I think it's less relevant than you suppose. Regular expressions are first class citizens in Perl, just as much as Arrays and Hashes. This doesn't just mean that the syntax has some niceties, but you can actually call Perl code within the regex itself[1], and even use this feature to build a more complex regular expression as you parse[2]. Complaining that Perl uses a regex and it isn't Perl is sort of like complaining Perl is using hashes, and any fair benchmark between C and Perl should just stick to Arrays.
> Another way of putting this is: your regex wasn't actually an XML parser. Things that are actually XML parsers were slower. This is not too surprising.
Yes. I didn't want to give the impression a wrote a general purpose XML parser that beat all the C implementations I could find. I still think it's interesting that well formed regular expressions are performant enough in this circumstance to make them a preferred alternative of the many options. I could have written a simple parser in C that would have been faster, but the solution I ended up with is quite fast, robust, and very, very easy to debug.
> That would probably mean that SAX bindings for Perl to a C parser would have to take a full SAX attribute hash and turn it into a Perl attribute hash. Still, 10x is pretty bad.
I think it's more related to the fact that the actions of the regex parsing implementation when optimized sufficiently is very close in implementation to C code that steps through a char array looking for the record beginning and ending indicators, and for the content between them, and then steps through the record looking for data items and saving the key and value for each. The main benefit of using a regex in Perl is that I get what is probably a fairly close approximation (all things considered) of a C implementation of that without having to write any C, and without having to marshal data back and forth to a library to accomplish it, with very simple and concise code.
There are other tasks which obviously aren't going to be nearly as efficient in Perl, but this exchange was spurred by someone talking about Perl being very fast for a Perl-type problem, which this definitely is.