The test code creates a new regex every time, would be interesting to see how it works with a compiled and reused regex.
It compiles it once and then matches it against 10000 strings.
[deleted]
\d less efficient than [0-9]
51–60 of 80 posts
Re: \d less efficient than [0-9]
#52Noted here that modifying \d to only include [0-9] yields \d more efficient
http://stackoverflow.com/a/16622773/1943429
Re: \d less efficient than [0-9]
#53Earlier quoted context omitted.
Not true for Go http://play.golang.org/p/ls96RxJxpz
Considering go's vocal support for UTF8 I'm surprised at this behavior and curious to the reason for excluding it.
Supporting UTF8 and correctly handling unicode are very, very different beasts. The former is absolutely trivial, the latter is extremely difficult.
Go is vocal about the former, but seems to not give a shit about the latter.
Re: \d less efficient than [0-9]
#54Short answer: \d includes all the Unicode characters from http://www.fileformat.info/info/unicode/category/Nd/list.htm
Is that actually a good thing? If I'm using \d to validate numbers (for example to check before string to int conversion, or IP address, phone number, or any other use), other unicode digits are not helpful to me. It's great to support unicode, but I don't think the \d should have been extended this way. Add a \ud or something.
Yes it's a good thing. There are other places in the world that don't just use ascii. If you want European style numbers just use [0-9]
Re: \d less efficient than [0-9]
#55Earlier quoted context omitted.
Considering go's vocal support for UTF8 I'm surprised at this behavior and curious to the reason for excluding it.
Supporting UTF8 and correctly handling unicode are very, very different beasts. The former is absolutely trivial, the latter is extremely difficult. Go is vocal about the former, but seems to not give a shit about the latter.
I'm not particularly fond of go, but "correctly handling unicode" can be subjective and case-dependent... I think making only minimal guarantees and punting to the application is often the only sane course.
Re: \d less efficient than [0-9]
#56There seems to be a tiny bit of difference in Ruby too. This code:
require 'benchmark'
def random_string(length)
result = (1..length).map { (65+rand(26)).chr }.join
result[rand(length)] = rand(10).to_s if rand > 0.5
result
end
Benchmark.bmbm do |b|
b.report("\\d") do
(1..1000).count { random_string(1000).match(/\d/) }
end
b.report("[0-9]") do
(1..1000).count { random_string(1000).match(/[0-9]/) }
end
b.report("[0123456789]") do
(1..1000).count { random_string(1000).match(/[0123456789]/) }
end
end
~/Code/ruby% ruby regex.rb
Rehearsal ------------------------------------------------
\d 0.690000 0.000000 0.690000 ( 0.712500)
[0-9] 0.690000 0.000000 0.690000 ( 0.703990)
[0123456789] 0.680000 0.010000 0.690000 ( 0.705759)
--------------------------------------- total: 2.070000sec
user system total real
\d 0.710000 0.000000 0.710000 ( 0.791722)
[0-9] 0.700000 0.000000 0.700000 ( 0.708210)
[0123456789] 0.690000 0.010000 0.700000 ( 0.713355)Re: \d less efficient than [0-9]
#57Earlier quoted context omitted.
Supporting UTF8 and correctly handling unicode are very, very different beasts. The former is absolutely trivial, the latter is extremely difficult. Go is vocal about the former, but seems to not give a shit about the latter.
I'm not particularly fond of go, but "correctly handling unicode" can be subjective and case-dependent... I think making only minimal guarantees and punting to the application is often the only sane course.
> "correctly handling unicode" can be subjective and case-dependent...
So is correctly handling integers.
> I think making only minimal guarantees and punting to the application is often the only sane course.
That is completely and utterly crazy, the average developer has neither the knowledge nor the resources to make anything but a mess out of it without proper tools and APIs. Even with these (including a complete implementation of the unicode standard and its technical reports) unicode is already complex enough to deal with.
Re: \d less efficient than [0-9]
#58Earlier quoted context omitted.
I'm not particularly fond of go, but "correctly handling unicode" can be subjective and case-dependent... I think making only minimal guarantees and punting to the application is often the only sane course.
> "correctly handling unicode" can be subjective and case-dependent... So is correctly handling integers. > I think making only minimal guarantees and punting to the application is often the only sane course. That is completely and utterly crazy, the average developer has neither the knowledge nor the resources to make anything but a mess out of it without proper tools and APIs. Even with these (including a complete…
Of course it's not "completely and utterly crazy."
Not every app needs to deal with the enormous complexities implied by "full unicode support", and given the huge cost of that, there's a real place for a minimalist approach. If all I do with unicode is input strings from the user, store them in a database, and then later spit them out, I don't need to be able to do Turkish case-conversion, and I may not want to pay the cost of making it possible.
Certainly tools and APIs help for those cases where an app needs to do the sort of complicated text-processing that warrants "full" unicode support, but it's not at all clear that the proper place for such support is in the base language libraries. It's quite reasonable for the language implementors to say "if you want to do X, we'll support that, but if you want to do Y and Z, please use external library L."
Re: \d less efficient than [0-9]
#59Earlier quoted context omitted.
> "correctly handling unicode" can be subjective and case-dependent... So is correctly handling integers. > I think making only minimal guarantees and punting to the application is often the only sane course. That is completely and utterly crazy, the average developer has neither the knowledge nor the resources to make anything but a mess out of it without proper tools and APIs. Even with these (including a complete…
Of course it's not "completely and utterly crazy." Not every app needs to deal with the enormous complexities implied by "full unicode support", and given the huge cost of that, there's a real place for a minimalist approach. If all I do with unicode is input strings from the user, store them in a database, and then later spit them out, I don't need to be able to do Turkish case-conversion, and I may not want to pay…
> Not every app needs to deal with the enormous complexities implied by "full unicode support", and given the huge cost of that, there's a real place for a minimalist approach.
Not sure what point you're trying to make, I never said all applications had to make full use of all possible Unicode APIs, I said the language must expose them. Because if it doesn't, those who should use them will never become aware of them let alone use them.
> If all I do with unicode is input strings from the user, store them in a database, and then later spit them out, I don't need to be able to do Turkish case-conversion, and I may not want to pay the cost of making it possible.
So?
> It's quite reasonable for the language implementors to say "if you want to add numbers, we'll support that, but if you want to subtract or divide them, please use external library L."
Really?
Then again, considering Go's embedded contempt for non-US locales (see: datetime patterns) I'm not even sure why we're having this discussion, and since it's obvious they don't care for a non-US world it make sense that they wouldn't care for processing text.
And at the end of the day, you agree that Go has no provision for unicode handling, you just think it's all fine and dandy.
Re: \d less efficient than [0-9]
#60I wonder what kind of security vulnerabilities could be looming in validators not expecting non-ascii 0-9 digits and using this regex?
[deleted]