Live data from Hacker News

Swift UTF-8 String

swift.org

91–97 of 97 posts

Re: Swift UTF-8 String

#91
post #83
post #26

Earlier quoted context omitted.

It doesn't matter what people think, rather CS definition. Software Engineering is not about what people think, rather what is technically correct.

It certainly does matter because the question is what did the grandparent mean.

Some people want to improve their skills, others don't.

Re: Swift UTF-8 String

#92

Earlier quoted context omitted.

No. How about a string that returns a Unicode character length? That's not 'ignoring complexity' and it's pragmatic. That would help in English as it's really hard today just to deal emojis.

> No. How about a string that returns a Unicode character length? That doesn't mean anything. Do you mean a code unit, a codepoint, a grapheme cluster, if the latter is it legacy, extended or tailored, how do you handle the locale dependency, … All of them can be useful in some context, all of them are length. And if anything, the truly useful one is the first one (because it tells you how much room you need to store…

Totally wrong. Your implied solution is to just have every single developer, everywhere, to be a linguistics expert and to handle the myriad of international cases for basically everything.

This is totally the wrong approach.

Language is shifty, but there are definitely a series of tools that should be integrated into every 'String' that can facilitate most problems.

"That doesn't mean anything" This is false. It absolutely means something, you can't brush it aside by inventing a separate series of specific measurements. Should we have those other measurements - yes, in many cases, we probably should have them as well -> that's the whole point.

Re: Swift UTF-8 String

#93

Earlier quoted context omitted.

No. How about a string that returns a Unicode character length? That's not 'ignoring complexity' and it's pragmatic. That would help in English as it's really hard today just to deal emojis.

> No. How about a string that returns a Unicode character length? That doesn't mean anything. Do you mean a code unit, a codepoint, a grapheme cluster, if the latter is it legacy, extended or tailored, how do you handle the locale dependency, … All of them can be useful in some context, all of them are length. And if anything, the truly useful one is the first one (because it tells you how much room you need to store…

Totally wrong. Your implied solution is to just have every single developer, everywhere, to be a linguistics expert and to handle the myriad of international cases for basically everything.

This is totally the wrong approach.

Language is complex and sometimes ambiguous, but there are definitely a series of tools that should be integrated into every 'String' that can facilitate most problems.

Not every company is trying to satisfy Hindi speakers.

"That doesn't mean anything" This is false. It absolutely means something, you can't brush it aside by inventing a separate series of related measurements. Should we have those other measurements? - yes, in many cases, we probably should have them as well - which the whole point. We desperately need a series of common tools built into String, some of them which provide more nuance.

"It's true that it's ignoring complexity, because you're ignorant to start with"

Well, since I have a background in NLP, I've developed writing recognition and word prediction algorithms, as well as language models for dozens of languages for products used by millions of people every day ... maybe I'm not as ignorant as your snide remark suggests?

"> That would help in English as it's really hard today just to deal emojis.

It… really isn't."

Uh, yes, it absolutely would help, and it's painfully obvious. In most languages, which cover the exceeding majority of markets for most apps made, simply knowing the character length (i.e. 'visible character' as to avoid confusion over messy nomenclature here) is very useful. In Latin languages, after Unicode normalization and other kinds of cleaning, Emojis present a very common case for confusion.

There are very small set of tools (or rather, a slightly better, more standardized String approach) which would solve most of the problems for most companies developing for most markets. There are even a few extra tools which would solve, pragmatically all of them.

Re: Swift UTF-8 String

#94
post #29

Earlier quoted context omitted.

It doesn't have automatic currying. What code/docs are you getting that impression from?

Swift used to have a special syntax for defining automatically curried functions, but it was removed in Swift 3: https://github.com/apple/swift-evolution/blob/master/proposa...

Indeed, and the compiler still had the relicts of that functionality at least until Swift 4. I'm wondering how they got the impression Swift still has currying, given it sounded like a recent "discovery".

Re: Swift UTF-8 String

#95

Earlier quoted context omitted.

I would assume good debug-tooling should make spotting such cycles quite easy, why is this even considered a problem?

It’s not easy and iOS developers spend a lot of time explicitly breaking ref cycles in code and tracking down resource leaks. I’ve seen a lot of code from experienced iOS devs that gets it wrong.

What kinds of mistakes do you see? Forgetting to use capture lists entirely? Using them improperly? Not making delegates weak properties?

Re: Swift UTF-8 String

#96
post #75

I have never written a single line of iOS/macos code, but I'm interested in Swift because it's a cool language. Can anyone explain why this is done now and not when Swift was first released? I mean, UTF-8 was already the clear winner when Swift started. Is it some obj-c compat story?

The commonly-accepted wisdom among Unicode people was that UTF-8 is good for transmission and storage, but bad for in memory representation. It now appears pretty conclusive that this was wrong.

They are still partially right, and I'd go so far as to say were totally right if efficiency wasn't a concern. From the programmers point of view utf-32 is still by far the easiest memory representation to use.

Back in the day, when the Unicode people were pushing it as an alternative to whatever the ISO standard is that solved the same problem, they argued it was better than the ISO standard because the was only one encoding, UCS2 whereas ISO defined 8, 16 and 32 bit encodings. They said they could get away with that because 16 bits was enough for all modern languages (read: languages in use). Windows, Java and Javascript among others drank the kool-aid.

But they were wrong, 16 bits was not enough. USC2 morphed into UTF-16, which is actually worse than UTF-8 because if has the byte ordering problem.

To make matters worse, Unicode decided to abandon grapheme == code point. (This means there is ä and ä. They look identical, but one is '\xe4' and the other is a combining diaeresis, '\61\u0308', and the latter is now preferred.) I don't know why that did that as it makes string handling fragile, but I notice the encoding chosen for UTF-16 drastically reduced the number of code points available, so I wonder if they were worried about running out. The end result means that makes utf-32 actually harder to parse than utf-8 (which is quite an achievement), so you may as well use utf-8 for everything.

Re: Swift UTF-8 String

#97
post #37
post #35

I wonder how long Windows can hold UTF-16 as the default native encodings. They recently added UTF-8 locale.

CP65001 has existed for quite a while. There just have been plenty of places where it never really worked (I think the console was one of them). Nonetheless, UTF-16 will most likely stay. They might add wrappers around all API functions that convert arguments first, so we'd get CreateWindow8 alongside CreateWindowW and CreateWindowA, but I wouldn't hold my breath. The duality of API functions was to enable programs t…

The problem is, the rest of the world decided to settle on UTF-8. We have some normalization disagreements but it's UTF-8 anyway.

Now we have surrogate pairs and the IVS, there is no advantage of using UTF-16 at all.

I will not surprise the Microsoft someday convert everything including kernel to filesystem to UTF-8 someday... or more likely they lost the market share.

Post reply on HN