Live data from Hacker News

The string type is broken (2013)

mortoray.com

11–20 of 45 posts

Re: The string type is broken (2013)

#11
post #3

Any Swift developers who know how it fares on these things? I've heard it does much better, largely by virtue of the Character type being an extended grapheme cluster, but does it handle all these cases right?

All of the things I know the answers to off the top of my head it passes, but I’d have to check a few of the others (e.g. the “ffl” ligature) I’d have to check.

Re: The string type is broken (2013)

#12
post #6

Earlier quoted context omitted.

What does "table stakes" mean? Never heard that before.

"Baseline expectation". I think it comes from poker or something

Table stakes are a required minimum bet in a game (often poker) - the meaning being if you can’t provide at least this you can’t play. So if a new language doesn’t support Unicode well it won’t get any adoption.

Re: The string type is broken (2013)

#13
post #4

Rust's string type doesn't have the mentioned problems: https://play.rust-lang.org/?version=stable&mode=debug&editio... Though I would bet this mainly has to do with being a language born in the age of the web. It seems like something that would be very hard to back-port (as opposed to designing for it from the beginning), but proper handling of Unicode is practically table-stakes for a new language in today's world.…

It does for multi-codepoint characters (such as the rainbow flag): https://play.rust-lang.org/?version=stable&mode=debug&editio...

Elixir gets it right: https://replit.com/@natanbc/reverse

Re: The string type is broken (2013)

#14
post #13
post #4

Rust's string type doesn't have the mentioned problems: https://play.rust-lang.org/?version=stable&mode=debug&editio... Though I would bet this mainly has to do with being a language born in the age of the web. It seems like something that would be very hard to back-port (as opposed to designing for it from the beginning), but proper handling of Unicode is practically table-stakes for a new language in today's world.…

It does for multi-codepoint characters (such as the rainbow flag): https://play.rust-lang.org/?version=stable&mode=debug&editio... Elixir gets it right: https://replit.com/@natanbc/reverse

Yeah, another commenter mentioned grapheme-clusters, though apparently there's a readily available crate that adds support: https://stackoverflow.com/questions/58770462/how-to-iterate-...

According to this they added it to the standard library at one point but then broke it back out because the lookup tables were too large

Re: The string type is broken (2013)

#15
post #9
post #3

Any Swift developers who know how it fares on these things? I've heard it does much better, largely by virtue of the Character type being an extended grapheme cluster, but does it handle all these cases right?

I recently did a little survey of string types across programming languages, and found that they display an immense variety of designs for a datatype so fundamental. Swift stakes out one of the most interesting points in the design space by completely encapsulating the internal representation and requiring all accessors to be explicit in whether they operate on the string's encoding (e.g. as UTF-8), its Unicode code…

Java's MUTF-8 makes me want to eject it into the Sun. Makes interop with JNI extra fun.

Re: The string type is broken (2013)

#16
post #4

Rust's string type doesn't have the mentioned problems: https://play.rust-lang.org/?version=stable&mode=debug&editio... Though I would bet this mainly has to do with being a language born in the age of the web. It seems like something that would be very hard to back-port (as opposed to designing for it from the beginning), but proper handling of Unicode is practically table-stakes for a new language in today's world.…

this isn't quite set up the way the article was mentioning. Here's with it set up correctly:

https://play.rust-lang.org/?version=stable&mode=debug&editio...

It looks like it still misses some of the cases (specifically with decomposed vs precomposed).

As a side note, it looks like the playground editor also doesn't correctly render some of the lengths (the cursor is 1 character off on the decomposed case)

Re: The string type is broken (2013)

#18
post #4

Rust's string type doesn't have the mentioned problems: https://play.rust-lang.org/?version=stable&mode=debug&editio... Though I would bet this mainly has to do with being a language born in the age of the web. It seems like something that would be very hard to back-port (as opposed to designing for it from the beginning), but proper handling of Unicode is practically table-stakes for a new language in today's world.…

this isn't quite set up the way the article was mentioning. Here's with it set up correctly: https://play.rust-lang.org/?version=stable&mode=debug&editio... It looks like it still misses some of the cases (specifically with decomposed vs precomposed). As a side note, it looks like the playground editor also doesn't correctly render some of the lengths (the cursor is 1 character off on the decomposed case)

Huh, I'd never heard of decomposed/precomposed before and I kind of skimmed over them in the original article. Looking at your example, I still can't figure out what the difference is between the two (I guess the Playground editor itself just can't represent the difference?). I'll have to go read up on that concept.

Edit: Having read about them, I guess I assumed that if my keyboard inputs an ë it's actually inputting those two underlying characters. Or to put it differently, I assumed there weren't two separate representations for the same character; that the "decomposed" and "precomposed" versions are one in the same, at the byte level. Learned something h̶o̶r̶r̶i̶f̶y̶i̶n̶g̶ new!

Re: The string type is broken (2013)

#19
Rust 1.53 seems to handle these correctly - feedback welcome.

    let s = String::from("noël");
    println!("Printable? {}", s);
    println!("Countable? {}", s.chars().count());
    println!("Reversable? {}", s.chars().rev().collect::());
    println!("First three characters? {}", s.chars().take(3).collect::());
Output:

    Printable? noël
    Countable? 4
    Reversable? lëon
    First three characters? noë
Repo for the "noël" example and "cats" example: https://github.com/joelparkerhenderson/demo-rust-string-issu...

Re: The string type is broken (2013)

#20
Reversing strings is such an odd thing to focus on. I have written dozens of implementations of string reversers for school and interviews and following books, but I would be hard pressed to come up with any real case where I needed to reverse a string. And then, it was because I was abusing a string to store some data that was not really text, at which point I really didn't care that it textually supported reversing Unicode code points.
Post reply on HN