[deleted]
We don't need a string type (2013)
21–30 of 70 posts
Re: We don't need a string type (2013)
#22Earlier quoted context omitted.
Go doesn't guarantee any encoding for strings, very deliberately (so that, eg, they can be used to represent file names).
Filesystem paths are not strings. Linux doesn't enforce an encoding. Windows at least didn't used to enforce proper use of conjugate UTF-16 pairs (see WTF-8 encoding). I think OS X does perform UTF-8 normalization, which might include sanity checking and rejecting malformed UTF-8, but I'm not sure. A byte array (or a ref-counted singly-linked list of immutable byte arrays to save space/copying) is a much better repre…
Posix thinks paths are strings. See https://pubs.opengroup.org/onlinepubs/009695399/functions/op...
Re: We don't need a string type (2013)
#23Earlier quoted context omitted.
I actually read it as a argument FOR types and against modern languages choice to make the String class a weak proxy for typeless byte arrays. See all the arguments (in this HN comments no less!) for just using utf8 byte arrays as strings. Hes saying semantically there's no difference between arrays and string classes except that with string classes we let you do all kinds of dangerous byte manipulation that we would…
I did not get that at all. Anyway a code point array would not serve the purpose: most possible sequences of valid code points are not valid strings. A variable-size array of code points is also useful, just as, in C++, a std::vector is useful, but that doesn't make it a string. That C++ std::string is wrong for what we now think of as strings is a whole other argument. People once hoped that std::string or std::stri…
> most possible sequences of valid code points are not valid strings.
Could you clarify? In what way are they not valid strings?
Re: We don't need a string type (2013)
#24Go's immutable UTF-8 string type is one of the nice things about the language A Go string is almost exactly like this C struct: struct String { uint8_t* addr; ptrdiff_t len; }; The language guarantees you can't modify the bytes in memory range [addr, addr+len) Go's garbage collection makes it simple and natural to have one string alias ("point into", "overlap") part of another string. This works because strings are i…
Java's sub-strings used to work sort of like this but was changed to use copy semantics. The structure was to have a "char[]" and an "offset" into that array. This allowed sub-strings to share the underlying array. However if you had a 1 char sub-string to a 1 GB array, the underlying array was never trimmed for garbage collection. In the case of a 1 char substring to a 1 GB string, is Go smart enough to free the res…
I wonder how hard the JVM folks looked into specialized weak references for solving this issue. The mark phase would treat all Strings with zero offset and full length as strongly referencing the byte[], and weakly otherwise. At the end of each full GC, you could iterate over all of your Strings (custom allocate/compact them to their own ranges of the heap for faster scanning), use some heuristics and probabilistic sampling to select some of the weakly reachable byte[]s for size reduction. A specialized copy/compact pass over the Strings could in-place replace byte[] references and fix up offsets.
You'd probably also want to modify String.equals() to internally mutate equal strings to point to the same byte[], preferring smaller offsets, and when offsets are equal, preferring lower addresses. This is a light weight lazy version of the background String byte[] interning done by some JVMs.
Re: We don't need a string type (2013)
#25String should be an interface/protocol. When I log a message, I want to pass a string. If I have to append large strings for a log message I don't want to run out of memory, I should be able to pass a rope/cord [1]. We've known how to abstract this for forever and should work to optimize our compilers/runtimes accordingly. I'm not aware of a language which has got this right, for example, Java has the ugly CharSequen…
Erlang/Elixir's iolists which are heavily utilized in Phoenix's templating engine are a rope and are extremely efficient (for a dynamic language). Phoenix's templating is very fast.
Re: We don't need a string type (2013)
#26The article is an argument against types, in general. The point that characters can be stored in other containers is meaningless: the question is whether, conceptually, a specific sequence of character values distinct from another sequence has compile-time meaning. It does. Therefore, it needs a type. Such a sequence has numerous special characteristics. In particular, element at [i] often has an essential connection…
You can mess up any ordered sequence in this way.
Re: We don't need a string type (2013)
#27Earlier quoted context omitted.
I did not get that at all. Anyway a code point array would not serve the purpose: most possible sequences of valid code points are not valid strings. A variable-size array of code points is also useful, just as, in C++, a std::vector is useful, but that doesn't make it a string. That C++ std::string is wrong for what we now think of as strings is a whole other argument. People once hoped that std::string or std::stri…
> most possible sequences of valid code points are not valid strings. Could you clarify? In what way are they not valid strings?
Some code points are characters. Others are operators with constrained contexts in which they operate. Sufficiently long random sequences of characters and these context-specific operators are likely to apply the operators in invalid contexts. Invalid characters mean invalid strings.
For instance, there are code points that are effectively operators that add continental European accents (umlaut, accent grave, etc.) to Latin characters. (Also, there are redundant code points for accented characters.) There's a whole set of code points that are combinators for primitive components of Han characters, etc. (Also, there are redundant code points for pre-composed Han characters.) One way of writing Korean syllables strictly requires triplets of individual jamo components: initial consonant jamo, vowel jamo, and final consonant jamo. (Also, there are redundant code points for every valid triple-jamo syllable in Korean.)
A Han character with an ancient Greek digamma in its "radical" position, a poo emoji inside a box, a thousand umlauts, all three French accents, a Hangul jamo vowel sticking through its center, a Hebrew vowel point, and a Thai tone mark is not a valid character. Any string containing invalid characters is not a valid string.
Re: We don't need a string type (2013)
#28The article is an argument against types, in general. The point that characters can be stored in other containers is meaningless: the question is whether, conceptually, a specific sequence of character values distinct from another sequence has compile-time meaning. It does. Therefore, it needs a type. Such a sequence has numerous special characteristics. In particular, element at [i] often has an essential connection…
Let me respond to you again in a different way, this time referencing some unicorn definitions I like (https://stackoverflow.com/a/27331885).
I don't think we can have a meaningful conversation in terms of characters so I'm going to ignore that and reference your last paragraph. You seem to be arguing that string as a type has use when viewing it as a collection of methods that allow access to Code Points given an underlying storage of Code Units. The article is arguing that unless you're writing a unicode encoder/decoder, you probably don't care about manipulating Code Units (except that modern languages have given you these byte arrays that you reference the length of for memory purposes). What you really usually care about is searching, replacing, concating, and cutting collections of Code Points. But languages have only given you this hodge podge grouping of Code Unit arrays and specialty methods for Code Point access so thats what you're used to dealing with and of course you want some kind of abstraction, like a string type, to deal with so you don't end up with the scenario you describe where you screw up a Code Unit sequence trying to manipulate a Code Point.
So the final point is that unless you're working with unicode encoding/decoding, you really only care about Code Points. And once you create a String class that only exposes Code Points, you have got something equivalent to a simple array.
Re: We don't need a string type (2013)
#29Earlier quoted context omitted.
Filesystem paths are not strings. Linux doesn't enforce an encoding. Windows at least didn't used to enforce proper use of conjugate UTF-16 pairs (see WTF-8 encoding). I think OS X does perform UTF-8 normalization, which might include sanity checking and rejecting malformed UTF-8, but I'm not sure. A byte array (or a ref-counted singly-linked list of immutable byte arrays to save space/copying) is a much better repre…
Posix thinks paths are strings. See https://pubs.opengroup.org/onlinepubs/009695399/functions/op...
"String" has multiple meanings in this context. In the context of that manpage, it means "nul-terminated array of char" which is the C language meaning. In the context of what you're replying to, a "string" is a sequence of bytes (octets) in a specific Unicode Transformation Format. Those are very different things when it comes to programmatic manipulation of those things.
Re: We don't need a string type (2013)
#30String should be an interface/protocol. When I log a message, I want to pass a string. If I have to append large strings for a log message I don't want to run out of memory, I should be able to pass a rope/cord [1]. We've known how to abstract this for forever and should work to optimize our compilers/runtimes accordingly. I'm not aware of a language which has got this right, for example, Java has the ugly CharSequen…
Can't agree more.
Java in particular suffers greatly from Object toString with a weak contract and no global String interface. If String were an interface instead of an implementation than any method signature could accept multiple implementations. This allows for really effective type aliases which even support strong typing so if you have a signature with multiple String values you can use the strong types to ensure you don't transpose arguments.