Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We don't need a string type (2013)

mortoray.com

Post reply on HN
Print view View original

11–20 of 70 posts

Re: We don't need a string type (2013)

#11

Post by arcbyte »

ncmncm wrote: post #4

The article is an argument against types, in general. The point that characters can be stored in other containers is meaningless: the question is whether, conceptually, a specific sequence of character values distinct from another sequence has compile-time meaning. It does. Therefore, it needs a type. Such a sequence has numerous special characteristics. In particular, element at [i] often has an essential connection…

I actually read it as a argument FOR types and against modern languages choice to make the String class a weak proxy for typeless byte arrays. See all the arguments (in this HN comments no less!) for just using utf8 byte arrays as strings.

Hes saying semantically there's no difference between arrays and string classes except that with string classes we let you do all kinds of dangerous byte manipulation that we would never dream of with any other type. Moreover, most of the uses for this dangerous access aren't real usages because if you're manipulating strings you're almost certainly actually manipulating code points. So why wouldn't you just use a code point array and give yourself real type safety instead?

Re: We don't need a string type (2013)

#12

Post by foo_barrio »

37ef_ced3 wrote: post #5

Go's immutable UTF-8 string type is one of the nice things about the language A Go string is almost exactly like this C struct: struct String { uint8_t* addr; ptrdiff_t len; }; The language guarantees you can't modify the bytes in memory range [addr, addr+len) Go's garbage collection makes it simple and natural to have one string alias ("point into", "overlap") part of another string. This works because strings are i…

Java's sub-strings used to work sort of like this but was changed to use copy semantics. The structure was to have a "char[]" and an "offset" into that array. This allowed sub-strings to share the underlying array. However if you had a 1 char sub-string to a 1 GB array, the underlying array was never trimmed for garbage collection.

In the case of a 1 char substring to a 1 GB string, is Go smart enough to free the rest of the array and keep only the 1 char?

Re: We don't need a string type (2013)

#13

Post by bobthepanda »

sfvisser wrote: post #10
pca006132 wrote: post #2

I think the problem is that, a lot of time when we deal with strings, we are thinking about ASCII strings instead of other encoding like UTF-8. If we treat them as ASCII strings, an array of characters would make sense, but it is not that simple for other encoding. One of the languages that considered the issue is Rust. In rust, we don't really index into strings, but use iterators or other methods to do the operatio…

I really don’t think many programmers nowadays actually think this.

I would hazard that very few people think about what an underlying String is at all.

String encoding is something I encountered as a problem in college, but is up there with implementing a homemade red-black tree in terms of “things that are asked in interviews but have little to no bearing on my day-to-day.”

Re: We don't need a string type (2013)

#14

Post by 37ef_ced3 »

foo_barrio wrote: post #12
37ef_ced3 wrote: post #5

Go's immutable UTF-8 string type is one of the nice things about the language A Go string is almost exactly like this C struct: struct String { uint8_t* addr; ptrdiff_t len; }; The language guarantees you can't modify the bytes in memory range [addr, addr+len) Go's garbage collection makes it simple and natural to have one string alias ("point into", "overlap") part of another string. This works because strings are i…

Java's sub-strings used to work sort of like this but was changed to use copy semantics. The structure was to have a "char[]" and an "offset" into that array. This allowed sub-strings to share the underlying array. However if you had a 1 char sub-string to a 1 GB array, the underlying array was never trimmed for garbage collection. In the case of a 1 char substring to a 1 GB string, is Go smart enough to free the res…

In Go, as in C, there's no magic. A programmer using Go thinks of a string variable as a pointer/length pair, and knows what will happen. Just like with slices

If you keep a pointer into an allocation (in your example, a small Go string pointing into a much larger Go string) the allocation is preserved by the garbage collector

You should explicitly copy the substring out (instead of aliasing the underlying string) if retaining the underlying string causes you a problem

Re: We don't need a string type (2013)

#15

Post by KMag »

DougBTX wrote: post #8
jrimbault wrote: post #6

Earlier quoted context omitted.

I'd arguee Go's string type is "somewhat unusable"* since it doesn't enforce the guarantees it says/implies it does. The byte slice it points to is not guaranteed to be valid utf8. * of course to a degree, let's be reasonable, it's usable in a _lot_ of contexts, but I like my types to actually mean something.

Go doesn't guarantee any encoding for strings, very deliberately (so that, eg, they can be used to represent file names).

Filesystem paths are not strings. Linux doesn't enforce an encoding. Windows at least didn't used to enforce proper use of conjugate UTF-16 pairs (see WTF-8 encoding).

I think OS X does perform UTF-8 normalization, which might include sanity checking and rejecting malformed UTF-8, but I'm not sure.

A byte array (or a ref-counted singly-linked list of immutable byte arrays to save space/copying) is a much better representation for a file system path. That doesn't have great interaction with GUIs, but there are other corner cases that are often problematic for GUIs. In high school, one of my friends had a habit of putting games on the school library computers, and renaming them to names with non-printable characters using alt+number pad. (He used 129, IIRC, which isn't assigned a character in CP-1252.) The Windows 95 graphical shell would convert the non-printable characters to spaces for display, but when the librarian tried to delete the games, it would pass the display name to the kernel, which would complain that the presented path didn't exist.

Re: We don't need a string type (2013)

#16

Post by irogers »

String should be an interface/protocol. When I log a message, I want to pass a string. If I have to append large strings for a log message I don't want to run out of memory, I should be able to pass a rope/cord [1]. We've known how to abstract this for forever and should work to optimize our compilers/runtimes accordingly. I'm not aware of a language which has got this right, for example, Java has the ugly CharSequence interface that nobody uses. StringProtocol in Swift (can I implement it?) makes you pay a character tax rather than to just pass a string. Rust/C++ give various non-abstracted types.

[1] https://en.wikipedia.org/wiki/Rope_(data_structure)

Re: We don't need a string type (2013)

#17

Post by foo_barrio »

37ef_ced3 wrote: post #14
foo_barrio wrote: post #12

Earlier quoted context omitted.

Java's sub-strings used to work sort of like this but was changed to use copy semantics. The structure was to have a "char[]" and an "offset" into that array. This allowed sub-strings to share the underlying array. However if you had a 1 char sub-string to a 1 GB array, the underlying array was never trimmed for garbage collection. In the case of a 1 char substring to a 1 GB string, is Go smart enough to free the res…

In Go, as in C, there's no magic. A programmer using Go thinks of a string variable as a pointer/length pair, and knows what will happen. Just like with slices If you keep a pointer into an allocation (in your example, a small Go string pointing into a much larger Go string) the allocation is preserved by the garbage collector You should explicitly copy the substring out (instead of aliasing the underlying string) if…

Okay I understand. In Java, the default String class has the option to "intern" a string which just maintains a list of strings that can be shared.

The change was made because often the devs were unaware of the string manipulation taking place in a third party library (eg XML/JSON/HTML parsing). You'd see the memory balloon, investigate and notice that String/char[] instances were dominating your heap. Instead of changing the entire implementation of the standard String class, they changed the semantics of the "substring()" call from O(1) to O(n) + memory side-effects.

Re: We don't need a string type (2013)

#18

Post by shadowgovt »

I think the author started from an assertion ("This primary difference between a C++ ‘string’ and ‘vector’ is really just a historical oddity that many programs don’t even need anymore") that highlights an error in the C++ model of strings, not in the way we must think about strings.

Contrast NSString in Cocoa (https://developer.apple.com/documentation/foundation/nsstrin...). The Cocoa string is extremely opaque; it's basically an object. And under the hood, that opacity allows for piles of optimization that are unsafe if the developer is allowed to treat the thing as just a vector of bytes or codepoints. Under the hood, Cocoa does all kinds of fanciness to the memory representation of the string (automatically building and cutting cords, "interning" short strings so that multiple copies of the string are just pointers to the same memory, caching of some transforms under the assumption that if it's needed once, it's often needed again).

Taken this way, one can even start to talk about things like "Why does 'indexing' into a string always return a character, instead of, say, a word?" and other questions that are harder to get into if one assumes a string is just 'vector of characters' or 'vector of bytes.'

Re: We don't need a string type (2013)

#19

Post by ncmncm »

arcbyte wrote: post #11
ncmncm wrote: post #4

The article is an argument against types, in general. The point that characters can be stored in other containers is meaningless: the question is whether, conceptually, a specific sequence of character values distinct from another sequence has compile-time meaning. It does. Therefore, it needs a type. Such a sequence has numerous special characteristics. In particular, element at [i] often has an essential connection…

I actually read it as a argument FOR types and against modern languages choice to make the String class a weak proxy for typeless byte arrays. See all the arguments (in this HN comments no less!) for just using utf8 byte arrays as strings. Hes saying semantically there's no difference between arrays and string classes except that with string classes we let you do all kinds of dangerous byte manipulation that we would…

I did not get that at all. Anyway a code point array would not serve the purpose: most possible sequences of valid code points are not valid strings.

A variable-size array of code points is also useful, just as, in C++, a std::vector is useful, but that doesn't make it a string.

That C++ std::string is wrong for what we now think of as strings is a whole other argument. People once hoped that std::string or std::string might be the useful string, but they were disappointed. C++ does not have a useful string type at this time, but there is ongoing work on one. It should appear in C++26.

Re: We don't need a string type (2013)

#20

Post by jerf »

KMag wrote: post #15
DougBTX wrote: post #8

Earlier quoted context omitted.

Go doesn't guarantee any encoding for strings, very deliberately (so that, eg, they can be used to represent file names).

Filesystem paths are not strings. Linux doesn't enforce an encoding. Windows at least didn't used to enforce proper use of conjugate UTF-16 pairs (see WTF-8 encoding). I think OS X does perform UTF-8 normalization, which might include sanity checking and rejecting malformed UTF-8, but I'm not sure. A byte array (or a ref-counted singly-linked list of immutable byte arrays to save space/copying) is a much better repre…

It is not clear to me if you're elaborating or think you're disagreeing, but that is what Go does. It is generally assumed in Go that strings are UTF-8, but in practice what they actually are are just bags of bytes. Nothing really "UTF-y" will happen to them until you directly call UTF functions on them, which may produce new strings.

It's something that I don't think could work unless your language is as recent as Go, and perhaps even Go 1.0 was pushing it, but it is an increasingly viable answer. For as thin as Go's encoding support really is in some sense, it has almost never caused me any trouble. The contexts where you are actively unsafe in assuming UTF-8 are decreasing, and the ones that are going to survive are the ones where there's some sort of explicit label, like in email. (Not that those are always trustworthy either.)

Post reply on HN