Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The string type is broken

mortoray.com

Post reply on HN
Print view View original

71–80 of 230 posts

Re: The string type is broken

#71

Post by lnanek2 »

this_user wrote: post #35

The string type isn't broken. If anything these "X is broken" posts are broken. Taking one special case, finding problems with that case and deducing that the whole concept must therefore be discarded is just silly. Strings work fine for the vast majority of use cases. No technology is free of flaws and engineering decisions are almost always based on weighting the pros and cons and choosing a solution that on balanc…

That's OK, sounds like he is writing a new language, so screwing up on the strings implementation is par for the course. Languages and databases don't typically get correct string handling for many years later after they are born, if ever. Supporting all the unicode and other character set insanity takes years of work. Asking someone writing a language to get strings right is like asking a five year old to obtain a drivers license.

Re: The string type is broken

#72

Post by lmm »

agravier wrote: post #39
drdaeman wrote: post #18

That's because in a truly sane languages there should be a distinction between data type and its implementation. Then it would be not "string" type, that's broken, but an implementation of "string" type.

I agree, it seems like a much saner thing to do. Now that you make me think of that, I do not know many instances of this. I just could think of https://github.com/clojure-numerics/core.matrix upon which I stumbled recently. Do you have other example of efforts to separate a type from its implementations?

Most collection libraries (e.g. the Java one) work like this - you have List as an interface and can use LinkedList or ArrayList or so on. I particularly like scala's approach to factory-like methods combined with this; Seq is an interface, as is List, with implementations like LinkedList. But you can do any of LinkedList(1, 2, 3), List(1, 2, 3), or Seq(1, 2, 3) - and get back a LinkedList, a List (which will be an implementation-selected implementation, possibly LinkedList), or a Seq (which again will be an implementation-selected implementation, possibly LinkedList).

Re: The string type is broken

#73

Post by Renaud »

pilif wrote: post #57

A nitpick from the article >This spells trouble for languages using UTF-16 encodings (Java, C#, JavaScript). if they were using UTF-16, this wouldn't be a problem as UTF-16 can be used to perfectly well encode code points outside of the BMP (at the cost of losing ability for O(1) access to specific code points of course. If you need to know what the n-th code point is, you have to scan the string until the n-th posit…

Exactly, UTF-16 perfectly defines surrogate pairs for code points that do not fit into the 16 bit plane. A perfect implementation of UTF-16 should have no problem, unfortunately, most are broken when it comes to surrogate pairs.

Many languages pre-date the introduction of UTF-16 and implemented 16 bit string encoding as UCS-2, and still do.

Then there are oddities like VBA using UTF-16 internally, but converting all strings going through the Win32 API as 8-bit (relying on the current code page for character translation!)...

Re: The string type is broken

#74

Post by zwily »

chiph wrote: post #58
billpg wrote: post #23

Do people really need to reverse strings in the real world? I don't think I've ever written code to do that outside of homework assignments and interviews.

I've been waiting for someone to ask me to reverse a string in an interview, so I can tell them why the code I just wrote for them (using the XOR trick, which is what they're usually expecting), is wrong.

When I've asked people to reverse a char* in the past, it's just been to see if they understand the basics of pointers. The XOR "trick" hasn't been impressive since high school. :)

Re: The string type is broken

#76

Post by lelf »

judofyr wrote: post #6

In many languages it's difficult fixing the string type without breaking existing code. In Ruby: String#upcase only handles ASCII (by spec), #length counts codepoints, #reverse reverses codepoints. You can use UnicodeUtils if you need "full" Unicode support: >> UnicodeUtils.upcase("baffle") => "BAFFLE" >> graphemes = UnicodeUtils.each_grapheme("noe\u0308l").to_a >> graphemes.reverse.join => "lëon" >> graphemes.size =>…

> String#upcase only handles ASCII (by spec)

Bad for Ruby

> You can use UnicodeUtils if you need "full" Unicode support:

Oh, sure

  Betty:~ lelf$ ruby -r unicode_utils/u -e 'puts UnicodeUtils.each_grapheme("A‮͜CB‬D").to_a.reverse.join'
  D‬BC͜‮A
So, "full" (it's not) Unicode support won't help you if you have little idea about what you're doing (like indexing stringه҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿҈̿s)

Re: The string type is broken

#77

Post by jeltz »

pilif wrote: post #57

A nitpick from the article >This spells trouble for languages using UTF-16 encodings (Java, C#, JavaScript). if they were using UTF-16, this wouldn't be a problem as UTF-16 can be used to perfectly well encode code points outside of the BMP (at the cost of losing ability for O(1) access to specific code points of course. If you need to know what the n-th code point is, you have to scan the string until the n-th posit…

> Most languages that claim Unicode support still only have UCS-2 libraries (Python 3 is a notable exception)

Most non-JVM languages[1] actually use UTF-8 as the internal encoding so they should not suffer from this. Python 3 does not use UTF-16 either, it selects an encoding based on the contents of the string.

http://www.python.org/dev/peps/pep-0393/

1. I think .NET too uses UCS-2 or UTF-16, but I am not a Windows developer.

Re: The string type is broken

#78

Post by TorKlingberg »

XorNot wrote: post #10
unwind wrote: post #5

Earlier quoted context omitted.

I also doubt the validity of the upper-casing, it feels like in an internationalization/localization context, converting a string to all upper case is not a valid thing to be doing. Not all languages (or even characters) have a well-defined upper-case versions of their glyphs. Even if they all did, I would expect the interpretation by a (human) reader to vary culturally.

The usual goal is to apply a consistent transform though, to smooth out interpretation differences - i.e. when looking for command input I either lowercase or uppercase things to smooth over the fact that "yes" "YES" "Yes" are all completely valid ways of saying the same thing with those characters. If there's only one way of expressing the thing - i.e. a single chinese character - then it would be valid to do nothin…

For that use case it is better to compare case insensitively with "yes" instead of converting the input to lower case first.

Re: The string type is broken

#79

Post by delinka »

Now let's take the lower case of "BAFFLE" - should we get "baffle" or should the string class/function/wtfe attempt to recognize that a ligature can replace "ffl" and return to us "baffle"? More generally, should the string library ever attempt to replace letter with ligatures? Should this be yet another option?

And as I type this, another issue manifests: the spelling correction can't even recognize baffle as a properly spelled word; it highlights the 'ba' and ignores the rest.

Re: The string type is broken

#80

Post by riffraff »

agravier wrote: post #39
drdaeman wrote: post #18

That's because in a truly sane languages there should be a distinction between data type and its implementation. Then it would be not "string" type, that's broken, but an implementation of "string" type.

I agree, it seems like a much saner thing to do. Now that you make me think of that, I do not know many instances of this. I just could think of https://github.com/clojure-numerics/core.matrix upon which I stumbled recently. Do you have other example of efforts to separate a type from its implementations?

doesn't every statically typed imperative language do this, and recommend it?
Post reply on HN