C# strings silently kill your SQL Server indexes in Dapper

Post by gpvos » Sat, Mar 07, 2026, 5:16 AM UTC

kstrauser wrote: post #37

wvenable wrote: post #33

Earlier quoted context omitted.

Blame the Unicode consortium for not coming up UTF-8 first (or, really, at all). And for assuming that 65526 code points would be enough for everyone. So many problems could be solved with a time machine.

The first draft of Unicode was in 1988. Thompson and Pike came up with UTF-8 in 1992, made an RFC in 1998. UTF-16 came along in 1996, made an RFC in 2000. The time machine would've involved Microsoft saying "it's clear now that USC-2 was a bad idea, so let's start migrating to something genuinely better".

Post by [deleted] » Sat, Mar 07, 2026, 5:33 AM UTC

SigmundA wrote: post #42

simonask wrote: post #28

Earlier quoted context omitted.

English has plenty of Unicode — claiming otherwise is such a cliché… Unicode is a requirement everywhere human language is used, from Earth to the Boöotes Void.

I am talking about coded values, like Status = 'A', 'B' or 'C' Taking double the space for this stuff is a waste of resources and nobody usually cares about extended characters here in English language systems at least they just want something more readable than integers when querying and debugging the data. End users will see longer descriptions joined from code tables or from app caches which can have unicode.

Post by wvenable » Sat, Mar 07, 2026, 5:35 AM UTC

SigmundA wrote: post #42

simonask wrote: post #28

Earlier quoted context omitted.

English has plenty of Unicode — claiming otherwise is such a cliché… Unicode is a requirement everywhere human language is used, from Earth to the Boöotes Void.

I am talking about coded values, like Status = 'A', 'B' or 'C' Taking double the space for this stuff is a waste of resources and nobody usually cares about extended characters here in English language systems at least they just want something more readable than integers when querying and debugging the data. End users will see longer descriptions joined from code tables or from app caches which can have unicode.

Post by wvenable » Sat, Mar 07, 2026, 5:39 AM UTC

SigmundA wrote: post #41

wvenable wrote: post #17

Earlier quoted context omitted.

I wouldn't consider it a defect in the optimizer; it's doing exactly what it's told to do. It cannot convert an nvarchar to varchar -- that's a narrowing conversion. All it can do is convert the other way and lose the ability to use the index. If you think that there is no danger converting an nvarchar that contains only ASCII to varchar then I have about 70+ different collations that say otherwise.

Can you give an example whats dangerous about converting a nvarchar with only ascii (0-127) then using the index otherwise fallback to a scan? If we simply went to UTF-8 collation using varchar then this wouldn't be an issue either, which is why you would use varchar in 2026, best of both worlds so to speak.

Post by wvenable » Sat, Mar 07, 2026, 6:25 AM UTC

kstrauser wrote: post #37

wvenable wrote: post #33

Earlier quoted context omitted.

Blame the Unicode consortium for not coming up UTF-8 first (or, really, at all). And for assuming that 65526 code points would be enough for everyone. So many problems could be solved with a time machine.

The first draft of Unicode was in 1988. Thompson and Pike came up with UTF-8 in 1992, made an RFC in 1998. UTF-16 came along in 1996, made an RFC in 2000. The time machine would've involved Microsoft saying "it's clear now that USC-2 was a bad idea, so let's start migrating to something genuinely better".