Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

We don't need a string type (2013)

mortoray.com

Post reply on HN
Print view View original

41–50 of 70 posts

Re: We don't need a string type (2013)

#41

Post by donaldihunter »

hollasch wrote: post #31

Curious. I have to come to exactly the opposite conclusion — that we should drop the idea of a fixed-length character type, and instead _only_ have (Unicode) string types. Actually, I'd prefer something like `std::text` to finally be free of the baggage of "string". Operations on text should work on logical text concepts. For example, something like `someText.firstCharacter()` would have a return type of `text`, with…

This!

Raku introduced the concept of NFG - Normal Form Grapheme - as a way to represent any Unicode string in its logical ‘visual character’ grapheme form. Sequences of combining characters that don’t have a canonical single codepoint form are given a synthetic codepoint so that string methods including regexes can operate on grapheme characters without ever causing splitting side effects.

Of course there are methods for manipulating at the codepoint level as well.

Re: We don't need a string type (2013)

#42

Post by lisper »

donaldihunter wrote: post #39
lisper wrote: post #34

Earlier quoted context omitted.

I fully endorse the general idea here, but this: > `someText.firstCharacter()` would have a return type of `text`, with logical length 1 is a huge mistake. There are operations that make sense on characters that do not make sense on texts whose length happens to be 1. The most obvious of these is inquiring about the numerical value of the unicode code point of a character. Conflating characters and texts-of-length-1…

Only if you ignore the rest of what the post said. First it should make things easy for ‘normal’ tasks, then it should make everything else possible. > Basically, users should definitely _not_ need to understand the deeper details of Unicode. They shouldn't need to understand and worry about different entities such as code units, code points, graphemes, and the like, though they should be able to extract such encodin…

Except that the "users" of a string type are programmers, and a "normal task" for a programmer often requires things like this. I'll give you an example from a project I am currently working on: a spam filter. One of the things my filter does is count the number of Chinese characters in a string. I implement this as n1<=ord(c)<=n2 where n1 and n2 are integers representing the start and end of the range of Unicode Chinese characters. This seems like a "normal task" to me and I don't see how conflating characters and texts-of-length-1 would make this any easier.

Re: We don't need a string type (2013)

#44

Post by dang »

DougBTX wrote: post #3

The date should be (2013) not (2018), as that dates it before Rust 1.0 (which does have a UTF-8 string type) and before the Julia 1.0 release date (which implements UTF-8 strings as arrays with irregularly spaced indexes, eg, the valid indexes may be 1, 2, 4, 5, if the character at 2 takes up two bytes). Both would be interesting examples to compare against if this article was written today.

I've fixed the date now. Actually the date at the top of the article "2013-08-13" is in a font that somehow makes it look like 2018. I had to squint a couple times to make sure I was reading it right! The year in the URL is easier to read.

Re: We don't need a string type (2013)

#45

Post by BlueTemplar »

hollasch wrote: post #31

Curious. I have to come to exactly the opposite conclusion — that we should drop the idea of a fixed-length character type, and instead _only_ have (Unicode) string types. Actually, I'd prefer something like `std::text` to finally be free of the baggage of "string". Operations on text should work on logical text concepts. For example, something like `someText.firstCharacter()` would have a return type of `text`, with…

> Actually, I'd prefer something like `std::text` to finally be free of the baggage of "string". Operations on text should work on logical text concepts. For example, something like `someText.firstCharacter()` would have a return type of `text`, with logical length 1. It's _data_ length is variable, since a Unicode character is variable length.

I don't see how you came to the "opposite conclusion" when the author basically says the same thing ?

Re: We don't need a string type (2013)

#46

Post by BlueTemplar »

bobthepanda wrote: post #13
sfvisser wrote: post #10

Earlier quoted context omitted.

I really don’t think many programmers nowadays actually think this.

I would hazard that very few people think about what an underlying String is at all. String encoding is something I encountered as a problem in college, but is up there with implementing a homemade red-black tree in terms of “things that are asked in interviews but have little to no bearing on my day-to-day.”

Really, they don't run into string/character issues regularly ? Because I do...

Re: We don't need a string type (2013)

#47

Post by BlueTemplar »

The author has these followup blogposts :

2013 : https://mortoray.com/2013/11/27/the-string-type-is-broken/

2014 : https://mortoray.com/2014/03/17/strings-and-text-are-not-the...

(See also : https://thehardcorecoder.com/2014/04/15/data-text-and-string... )

2016 : https://mortoray.com/2016/04/28/what-is-the-length-of-a-stri...

Re: We don't need a string type (2013)

#48

Post by BlueTemplar »

shadowgovt wrote: post #18

I think the author started from an assertion ("This primary difference between a C++ ‘string’ and ‘vector’ is really just a historical oddity that many programs don’t even need anymore") that highlights an error in the C++ model of strings, not in the way we must think about strings. Contrast NSString in Cocoa ( https://developer.apple.com/documentation/foundation/nsstrin... ). The Cocoa string is extremely opaque; i…

Today I learned that Python does interning of shorts strings too :

https://news.ycombinator.com/item?id=26097732

Re: We don't need a string type (2013)

#49

Post by Koshkin »

lisper wrote: post #34
hollasch wrote: post #31

Curious. I have to come to exactly the opposite conclusion — that we should drop the idea of a fixed-length character type, and instead _only_ have (Unicode) string types. Actually, I'd prefer something like `std::text` to finally be free of the baggage of "string". Operations on text should work on logical text concepts. For example, something like `someText.firstCharacter()` would have a return type of `text`, with…

I fully endorse the general idea here, but this: > `someText.firstCharacter()` would have a return type of `text`, with logical length 1 is a huge mistake. There are operations that make sense on characters that do not make sense on texts whose length happens to be 1. The most obvious of these is inquiring about the numerical value of the unicode code point of a character. Conflating characters and texts-of-length-1…

Functors are everywhere. That's why we need monads!

Re: We don't need a string type (2013)

#50

Post by Koshkin »

37ef_ced3 wrote: post #5

Go's immutable UTF-8 string type is one of the nice things about the language A Go string is almost exactly like this C struct: struct String { uint8_t* addr; ptrdiff_t len; }; The language guarantees you can't modify the bytes in memory range [addr, addr+len) Go's garbage collection makes it simple and natural to have one string alias ("point into", "overlap") part of another string. This works because strings are i…

Oh well. The could've (should've?) used the layout of _bstr_t instead.
Post reply on HN