Live data from Hacker News

\d less efficient than [0-9]

stackoverflow.com

11–20 of 80 posts

Re: \d less efficient than [0-9]

#12
post #4
post #2

Short answer: \d includes all the Unicode characters from http://www.fileformat.info/info/unicode/category/Nd/list.htm

Is that actually a good thing? If I'm using \d to validate numbers (for example to check before string to int conversion, or IP address, phone number, or any other use), other unicode digits are not helpful to me. It's great to support unicode, but I don't think the \d should have been extended this way. Add a \ud or something.

Given that the category is specifically "decimal digit", I think it's good, so long as the number parsing code accepts them all too.

Re: \d less efficient than [0-9]

#13
post #2

Short answer: \d includes all the Unicode characters from http://www.fileformat.info/info/unicode/category/Nd/list.htm

oh wow I had no idea that "full width digits" can actually be handled properly. (U+FF10 ~ U+FF19)

Or improperly. If you expect \d to be a shorthand for 0-9, your string can also contain junk.

Re: \d less efficient than [0-9]

#14
post #9

I was a bit surprised that Perl does not seem to be matching Unicode digits. Anyone know why? $ echo '0' | perl -pe 'print "yes: " if m/\d/' yes: 0 $ echo '੧' | perl -pe 'print "yes: " if m/\d/' ੧

Try:

    utf8::upgrade($string)
And/or:

    use feature 'unicode_strings'

Re: \d less efficient than [0-9]

#16
post #5
post #3

Earlier quoted context omitted.

...at least in C# regexes.

Anyone know if this happens in other languages?

Doesn't appear to in JavaScript:

    "੧".match(/\d/); //null
(Incidentally, this may explain the finding from http://stackoverflow.com/a/16622773/172322, as to why adding the RegexOptions.ECMAScript flag in the C# code eliminates the performance gap)

Re: \d less efficient than [0-9]

#17
post #9

I was a bit surprised that Perl does not seem to be matching Unicode digits. Anyone know why? $ echo '0' | perl -pe 'print "yes: " if m/\d/' yes: 0 $ echo '੧' | perl -pe 'print "yes: " if m/\d/' ੧

You have to tell perl to expect utf8 from stdin (switch -C).

  $ echo '੧' | perl -C -pe 'print "yes: " if m/\d/'
  and
  $ perl -e 'use utf8; print "yes\n" if "੧" =~ m/\d/;'
both work :)
Post reply on HN