Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Swift UTF-8 String

swift.org

Post reply on HN
Print view View original

81–90 of 97 posts

Re: Swift UTF-8 String

#81

Post by asveikau »

nicky0 wrote: post #52
sonnyblarney wrote: post #22

Earlier quoted context omitted.

Totally see what you are saying (the underlying issue is language itself), but it's pragmatically obtainable.

The think is, strings that make sense has been a major goal of Swift, that's why they have been iterating on it so heavily. FWIW I think Swift's is probably one of the cleanest String abstractions out there with respect to unicode correctness balanced against efficiency and usability.

To be clear I applaud successful attempts to make it better, what I doubt is a possibility to make the problem completely go away without users of the api ever having to think.

Re: Swift UTF-8 String

#82

Post by WorldMaker »

garganzol wrote: post #76
WorldMaker wrote: post #74

Earlier quoted context omitted.

> UTF-16 allows to quickly index a char in a string covering the most languages that way. Nope. UTF-16 is not UCS-2 anymore for multiple reasons. You cannot assume you can index a UTF-16 string as if it was UCS-2 anymore. UCS-4 (32-bit) is a possible option, though not generally recommended either. There are multiple tricks for getting great performance in UTF-8 string processing such as codepoint maps of various sor…

Never made a claim that UTF-16 is UCS-2. "Most languages" does not mean "all languages". Still, many popular platforms treat it like that in terms of char indexing. They are able to get away with it since a surrogate char is a rare guest in UTF-16. P.S. It's surprising how easily HN crowd downvotes something when it falls out of a whimsical "popular contemporary view on things". Having a lot of years of text processi…

You definitely can't claim "most languages" today with the commonality of emoji and emoji-based sub-languages/communication forms/media. Surrogate characters are no longer rare in UTF-16 in average usage on the internet and that's impacting everything.

Even if you are restricting "most languages" to mean "most programming languages", there's an increasing rise of emoji in comments alone for languages that support UTF-8 and some languages are picking up increasing emoji usage in places like identifier names.

Platforms that treat UTF-16 like UCS-2 and allow raw char indexing rather than codepoint-oriented traversal are wrong in 2019. (It's been wrong since the very definition of UTF-16, such as RFC 2781 in 2000.) UTF-16 is a disappointing hack because it allowed UCS-2 platforms the luxury of pretending that the UCS-2 era never ended. UTF-8 at least leaves you very aware that the UCS-2 emperor has no clothes.

Re: Swift UTF-8 String

#83

Post by eridius »

pjmlp wrote: post #26
eridius wrote: post #7

Earlier quoted context omitted.

Unless you consider reference counting to be a form of GC¹, Swift is not a GC'ed language. ¹It technically is, but most people don't mean reference counting when they say GC.

It doesn't matter what people think, rather CS definition. Software Engineering is not about what people think, rather what is technically correct.

It certainly does matter because the question is what did the grandparent mean.

Re: Swift UTF-8 String

#84

Post by eridius »

pjmlp wrote: post #41
arebours wrote: post #33

Earlier quoted context omitted.

Well.. what is technically correct depends on a definition. And there are customary/practical definitions too. And sometimes definitions are tainted by context, like in "is this gc or ref counting language".

Which should be "is this tracing GC or ref counting language". Naturally when in some juriditions one is allowed to call themselves engineers after a 6 months bootcamp we land in such customary/practical definitions.

Even wikipedia admits that most people mean tracing GC when they say GC.

What exactly are you trying to accomplish here?

Re: Swift UTF-8 String

#85

Post by sonnyblarney »

masklinn wrote: post #67
sonnyblarney wrote: post #22

Earlier quoted context omitted.

Totally see what you are saying (the underlying issue is language itself), but it's pragmatically obtainable.

Ah, so your solution is the same old canard: just ignore the complexity and call it pragmatic.

No. How about a string that returns a Unicode character length?

That's not 'ignoring complexity' and it's pragmatic.

That would help in English as it's really hard today just to deal emojis.

Re: Swift UTF-8 String

#86

Post by pjmlp »

eridius wrote: post #84
pjmlp wrote: post #41

Earlier quoted context omitted.

Which should be "is this tracing GC or ref counting language". Naturally when in some juriditions one is allowed to call themselves engineers after a 6 months bootcamp we land in such customary/practical definitions.

Even wikipedia admits that most people mean tracing GC when they say GC. What exactly are you trying to accomplish here?

Advocating learning CS properly.

Re: Swift UTF-8 String

#87

Post by asveikau »

sonnyblarney wrote: post #85
masklinn wrote: post #67

Earlier quoted context omitted.

Ah, so your solution is the same old canard: just ignore the complexity and call it pragmatic.

No. How about a string that returns a Unicode character length? That's not 'ignoring complexity' and it's pragmatic. That would help in English as it's really hard today just to deal emojis.

The point I tried to make: I suspect if you dug really deep into what is a character or glyph you might find "character length" is an imprecise and messy notion, with differing performance costs associated with differing definitions which may or may not apply to you.

So even if you pick one definition and decide to live with it always... You'll still have to know what tradeoffs you (or an API designer or implementer) made.

Re: Swift UTF-8 String

#88

Post by sonnyblarney »

asveikau wrote: post #87
sonnyblarney wrote: post #85

Earlier quoted context omitted.

No. How about a string that returns a Unicode character length? That's not 'ignoring complexity' and it's pragmatic. That would help in English as it's really hard today just to deal emojis.

The point I tried to make: I suspect if you dug really deep into what is a character or glyph you might find "character length" is an imprecise and messy notion, with differing performance costs associated with differing definitions which may or may not apply to you. So even if you pick one definition and decide to live with it always... You'll still have to know what tradeoffs you (or an API designer or implementer)…

First - a combined unicode length is definitely universally useful. Second - a grapheme length is universally useful. Third - I didn't say 'pick one definition' - you did. A simple function which took as a parameter some kind of linguistic construct would be immensely useful in cases of choice/ambiguity.

'It's complicated' is the utterly the wrong answer as to why we 'can't have better strings'.

Strings in almost every programming language were designed before Emjojis and true internationalization. If we designed them today, they would have support for dealing with most of the problems we face.

And FYI - there are not infinite corner cases. They are limited, and we can develop constructs for dealing with all of them.

Re: Swift UTF-8 String

#89

Post by masklinn »

sonnyblarney wrote: post #85
masklinn wrote: post #67

Earlier quoted context omitted.

Ah, so your solution is the same old canard: just ignore the complexity and call it pragmatic.

No. How about a string that returns a Unicode character length? That's not 'ignoring complexity' and it's pragmatic. That would help in English as it's really hard today just to deal emojis.

> No. How about a string that returns a Unicode character length?

That doesn't mean anything. Do you mean a code unit, a codepoint, a grapheme cluster, if the latter is it legacy, extended or tailored, how do you handle the locale dependency, … All of them can be useful in some context, all of them are length. And if anything, the truly useful one is the first one (because it tells you how much room you need to store something).

> That's not 'ignoring complexity' and it's pragmatic.

It's true that it's not ignoring complexity, because you're ignorant of it to start with. Which isn't really different.

> That would help in English as it's really hard today just to deal emojis.

It… really isn't.

Re: Swift UTF-8 String

#90

Post by asveikau »

sonnyblarney wrote: post #88
asveikau wrote: post #87

Earlier quoted context omitted.

The point I tried to make: I suspect if you dug really deep into what is a character or glyph you might find "character length" is an imprecise and messy notion, with differing performance costs associated with differing definitions which may or may not apply to you. So even if you pick one definition and decide to live with it always... You'll still have to know what tradeoffs you (or an API designer or implementer)…

First - a combined unicode length is definitely universally useful. Second - a grapheme length is universally useful. Third - I didn't say 'pick one definition' - you did. A simple function which took as a parameter some kind of linguistic construct would be immensely useful in cases of choice/ambiguity. 'It's complicated' is the utterly the wrong answer as to why we 'can't have better strings'. Strings in almost eve…

There are myriad problems today that 99% of developers couldn't be bothered to think about, but can be a source of bugs. eg. I'm going to guess every time you call tolower() have you haven't thought "I wonder what this does to Turkish I", but it's still there. An API won't fix us from having to think, and having ambiguities occasionally become bugs. (The fact that you've acknowledged the need for multiple string length functions -- I agree with it, but it also means you'll sometimes use the wrong one.)

As I said in another comment, I still applaud efforts to improve the status quo.

Post reply on HN