Strings and the CLR – A Special Relationship • Hacker News Board

Re: Strings and the CLR – A Special Relationship

#21

Post by jsingleton » Wed, Jun 01, 2016, 12:44 PM UTC

matthewwarren wrote: post #13

jsingleton wrote: post #12

Nice post. There are many things that aren't possible in C# but can be expressed in pure IL. Sigil [0] is an IL generator used in the high-performance JSON serializer Jil [1], which takes advantage of this. https://github.com/kevin-montrose/Sigil https://github.com/kevin-montrose/Jil

I've seen Jil/Sigil before, but not thought about it in this way. Are you saying that you could exactly replicate a C# System.String using IL, or am I mis-understanding?

No, I wasn't suggesting that you could implement the same high-performance System.String in IL, especially considering the assembly language used. I was just saying that there are many other things that can't be done in C# but are possible in .NET, using other languages.

BTW there is a typo: "passed in my the calling code" should probably be "passed in by the calling code".

I wonder if the new C# 6 string interpolation also uses string builder like format does. e.g.

  $"Name = {name}, hours = {hours:hh}"

https://msdn.microsoft.com/en-us/library/Dn961160.aspx

This post seems to suggest it uses concatenation, which may be bad for performance in some situations due to the immutability.

https://weblogs.asp.net/bleroy/c-6-string-interpolation-is-n...

Re: Strings and the CLR – A Special Relationship

#22

Post by matthewwarren » Wed, Jun 01, 2016, 1:08 PM UTC

jsingleton wrote: post #21

matthewwarren wrote: post #13

Earlier quoted context omitted.

I've seen Jil/Sigil before, but not thought about it in this way. Are you saying that you could exactly replicate a C# System.String using IL, or am I mis-understanding?

No, I wasn't suggesting that you could implement the same high-performance System.String in IL, especially considering the assembly language used. I was just saying that there are many other things that can't be done in C# but are possible in .NET, using other languages. BTW there is a typo: "passed in my the calling code" should probably be "passed in by the calling code". I wonder if the new C# 6 string interpolati…

I just did a quick test with Reflector and it looks like string interpolation is turned into String.Format(..) calls, which in turn will use StringBuilder. This makes sense as String.Format(..) understands all the string formatting placeholders.

> BTW there is a typo: "passed in my the calling code" should probably be "passed in by the calling code".

Thanks for spotting that, I'll fix it in a bit

Re: Strings and the CLR – A Special Relationship

#23

Post by asveikau » Wed, Jun 01, 2016, 2:16 PM UTC

MarkSweep wrote: post #4

ygra wrote: post #2

Another point for this, besides optimization, is easier interop with existing technologies. The memory layout is identical to Windows' BSTR, as far as I know, and also includes a terminating U+0000. All of that makes it possible to marshal a CLR string as is in many circumstances without the need to copy or convert the characters around.

There is an amusing comment in the CLR about it's support for odd length BSTRs[0]. There can be an extra wchar after the null terminator to store the extra byte. It looks like if you use the automatic BSTR marshaler it creates a copy[1], though the CLR is so big it's hard for me to say that with confidence. Taking the address of a string with the fixed statement in C# of course won't allocate and that pointer can be…

I would think, especially for out parameters and return values, you would need to copy to interop with BSTRs because they are supposed to live on the COM heap (SysAllocString) and CLR wouldn't use that by default.

Re: Strings and the CLR – A Special Relationship

#24

Post by matthewwarren » Wed, Jun 01, 2016, 3:44 PM UTC

Someone wrote: post #6

So, are there high-level languages that make it easy to write classes that include length-varies-by-instance member(s) in their memory layout? That could be a huge efficiency gain because of cache misses.

Thanks for asking this, I didn't even think about other languages and if they made this possible or not

Re: Strings and the CLR – A Special Relationship

#25

Post by needusername » Wed, Jun 01, 2016, 4:36 PM UTC

Someone wrote: post #6

So, are there high-level languages that make it easy to write classes that include length-varies-by-instance member(s) in their memory layout? That could be a huge efficiency gain because of cache misses.

There is a proposal for Java http://objectlayout.org/

Re: Strings and the CLR – A Special Relationship

#26

Post by adzm » Wed, Jun 01, 2016, 6:43 PM UTC

matthewwarren wrote: post #11

MarkSweep wrote: post #4

Earlier quoted context omitted.

There is an amusing comment in the CLR about it's support for odd length BSTRs[0]. There can be an extra wchar after the null terminator to store the extra byte. It looks like if you use the automatic BSTR marshaler it creates a copy[1], though the CLR is so big it's hard for me to say that with confidence. Taking the address of a string with the fixed statement in C# of course won't allocate and that pointer can be…

I love comments like that, it's almost a history lesson on VB and the CLR!

Then you'd probably love to know why the OLE DATE epoch is 1899-12-30...

also the B in BSTR is for Basic!

https://blogs.msdn.microsoft.com/ericlippert/2003/09/16/eric...

Re: Strings and the CLR – A Special Relationship

#27

Post by nblumhardt » Wed, Jun 01, 2016, 10:42 PM UTC

Great post. So the follow-up question is, how much work would be involved in mirroring all that in a native UTF-8 encoded string type? :-)

Windows interop, and 8/16 conversions, would obviously be an expense, but ~half the storage requirements of UTF-16 have to represent a substantial CPU/RAM saving.

Guess it's too much cruft and complexity to introduce into .NET now, but who knows? https://twitter.com/terrajobst/status/717935598904807424

Re: Strings and the CLR – A Special Relationship

#28

Post by ed_blackburn » Thu, Jun 02, 2016, 9:04 AM UTC

This makes me wonder..

Can one extend the CLR (modulalary) and access via: extern and [MethodImplAttribute(MethodImplOptions.InternalCall)]?

Native UTF-8 is an obvious example but the scope could be far greater in scope such as native interop for a hardware device (IoT), funky filesystems or hypervisors?

Re: Strings and the CLR – A Special Relationship

#29

Post by matthewwarren » Thu, Jun 02, 2016, 11:12 AM UTC

nblumhardt wrote: post #27

Great post. So the follow-up question is, how much work would be involved in mirroring all that in a native UTF-8 encoded string type? :-) Windows interop, and 8/16 conversions, would obviously be an expense, but ~half the storage requirements of UTF-16 have to represent a substantial CPU/RAM saving. Guess it's too much cruft and complexity to introduce into .NET now, but who knows? https://twitter.com/terrajobst/sta…

Yeah I think the legacy interop is the tricky part, see "Why does C# use UTF-16 for strings?" (http://blog.coverity.com/2014/04/09/why-utf-16/#.V1AT9vkguUl) for example.

Interestingly enough Java just implemented compact strings that can be ISO-8859-1/Latin-1 (one byte per character) or as UTF-16 (two bytes per character). See http://openjdk.java.net/jeps/254 and https://www.infoq.com/news/2016/02/compact-strings-Java-JDK9 for more info.

Re: Strings and the CLR – A Special Relationship

#30

Post by matthewwarren » Thu, Jun 02, 2016, 11:13 AM UTC

ed_blackburn wrote: post #28

This makes me wonder.. Can one extend the CLR (modulalary) and access via: extern and [MethodImplAttribute(MethodImplOptions.InternalCall)]? Native UTF-8 is an obvious example but the scope could be far greater in scope such as native interop for a hardware device (IoT), funky filesystems or hypervisors?

Good question, I don't know how much (if at all) it can be extended.

Those [MethodImplAttribute(MethodImplOptions.InternalCall)] calls are still within the CoreCLR codebase, they're just from the managed part -> un-managed part.