What you are seeing is different regex engines and capabilities, and grep's focus on pure speed and optimization of a common case and Perl's focus on versatility.
I see very similar results between Perl and grep, and you can see this by also including egrep, which allows slightly more complex expressions:
[root@stats ~]# time perl -ne 'print if /number 123456/'
But what happens if we use a slightly more complex expression?
[root@stats ~]# time perl -ne 'print if /number [1]23456/'
The difference becomes much less pronounced. What if we make the expression just a bit more complex?
[root@stats ~]# time perl -ne 'print if /number [1]23456[0-9]*/'
So, now we have the Perl regex engine fairly static across extra complexity while grep and egrep are seeing order of magnitude time increases, and are
much slower than Perl at this point. I suspect your first benchmark was the result of a specific optimization grep has that Perl doesn't, or it may be that grep was able to switch to using a DFA regex for that first case, while Perl doesn't both with a completely different regex implementation for special cases like that.
Anecdata: I needed to process a large amount of XML a while back, to the point where a week spent testing and optimizing XML parsing libraries in Perl was worth it, because it could shave weeks or months off the processing time. The winner? A regex that captured attributes and content and assigned name/value pairs directly out to a hash. This was only possible because the XML was highly normalized, but it was actually over 10 times faster than the closest competitor for XML parsing I could fine, and I checked all the libXML libXML2, and SAX libraries I could get my hands on.
In the end, it was something as simple as the following approximation:
while (my ($doc) = $xml =~ /$get_record_xml_re/) {
my %hash = $get_record_xml =~ /$record_begin_re$capture_name_and_value_pairs_re$record_end_re/;
process_record( \%hash );
}