Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Swift UTF-8 String

swift.org

Post reply on HN
Print view View original

61–70 of 97 posts

Re: Swift UTF-8 String

#61

Post by cageface »

millstone wrote: post #50
cageface wrote: post #30

Earlier quoted context omitted.

If GC pauses aren't an issue for your app then you're better off with a full-blown GC because it's a lot easier to code for. Reference counting, as used in Swift, forces you to think a lot about cases where you might be creating reference cycles and memory leaks. GCs can figure this out for you and take that burden off the programmer.

One of the major advantages of reference counting is that it's easier to integrate with other languages. For example you can manipulate Swift objects from C via CoreFoundation, which then allows them to be used from C++, Rust, whatever. GC'd languages have some support (JNI, etc) but they require handles, pinning, write barriers, etc. which ends up being more complicated and less flexible.

Right, which is one of the reasons it makes sense for Apple to use it, since they have a lot of lower level C API to interact with.

For more general purpose backend work this is not usually a major consideration though, which is one of the reasons I predict most backend work will continue to be done in a GC language.

Re: Swift UTF-8 String

#62

Post by vbezhenar »

ygra wrote: post #37
ezoe wrote: post #35

I wonder how long Windows can hold UTF-16 as the default native encodings. They recently added UTF-8 locale.

CP65001 has existed for quite a while. There just have been plenty of places where it never really worked (I think the console was one of them). Nonetheless, UTF-16 will most likely stay. They might add wrappers around all API functions that convert arguments first, so we'd get CreateWindow8 alongside CreateWindowW and CreateWindowA, but I wouldn't hold my breath. The duality of API functions was to enable programs t…

When the world uses UTF-8 and you have to constantly recode from/to with every WinAPI call, that's definitely an unnecessary overhead.

Re: Swift UTF-8 String

#63

Post by danpalmer »

saagarjha wrote: post #59
skrebbel wrote: post #44

I have never written a single line of iOS/macos code, but I'm interested in Swift because it's a cool language. Can anyone explain why this is done now and not when Swift was first released? I mean, UTF-8 was already the clear winner when Swift started. Is it some obj-c compat story?

Objective-C strings are UTF-16, so it’s likely this was the case. The String ABI has diverged since then, but there is a need to finalize those changes now due to ABI stability.

And worth noting, Objective-C’s strings (actually Foundation’s NSString) is UTF-16 because at the time it was implemented UTF-8 had become as universal as it is now. UTF-16 still seemed like a possible option. And changing the existing strings from 16-8 would have been a huge undertaking - Swift adds an interoperability layer and more flexible compiler/language that makes the transition much easier, not to mention a whole new standard library that has been designed to make this transition work well.

Re: Swift UTF-8 String

#64

Post by badestrand »

vbezhenar wrote: post #62
ygra wrote: post #37

Earlier quoted context omitted.

CP65001 has existed for quite a while. There just have been plenty of places where it never really worked (I think the console was one of them). Nonetheless, UTF-16 will most likely stay. They might add wrappers around all API functions that convert arguments first, so we'd get CreateWindow8 alongside CreateWindowW and CreateWindowA, but I wouldn't hold my breath. The duality of API functions was to enable programs t…

When the world uses UTF-8 and you have to constantly recode from/to with every WinAPI call, that's definitely an unnecessary overhead.

Obviously it is unnecessary but I doubt it affects the perceived speed of the average desktop application in any way.

Re: Swift UTF-8 String

#65

Post by ChrisSD »

ygra wrote: post #37
ezoe wrote: post #35

I wonder how long Windows can hold UTF-16 as the default native encodings. They recently added UTF-8 locale.

CP65001 has existed for quite a while. There just have been plenty of places where it never really worked (I think the console was one of them). Nonetheless, UTF-16 will most likely stay. They might add wrappers around all API functions that convert arguments first, so we'd get CreateWindow8 alongside CreateWindowW and CreateWindowA, but I wouldn't hold my breath. The duality of API functions was to enable programs t…

To be clear, the Windows filesystem isn't even UTF-16. It's UCS-2, which is simply a sequence of 16 bit values. The difference is that not all sequences are valid UTF-16 (see surrogate pairs).

Thus you have to be very careful when handling filesystem paths.

Re: Swift UTF-8 String

#66

Post by masklinn »

kevin_thibedeau wrote: post #47
faragon wrote: post #34

Wise move, in my opinion. I hope Python will do that too, eventually.

Python implemented multi-representation strings in 3.3. It picks the best encoding from ASCII, UTF8, UCS2, or UCS4. UTF8 is the preferred encoding for strings exposed to the C API. https://www.python.org/dev/peps/pep-0393/

CPython provides but does not use UTF8 internally, it's a cache for PyUnicode_AsUTF8 (formerly _PyUnicode_AsString), which exists to convert Python strings to char* in order to interoperate with C libraries. It is not the canonical representation of the string.

pypy, on the other hand, made this change in the latest release (7.1.0): https://twitter.com/pypyproject/status/1095971192513708032

Re: Swift UTF-8 String

#67

Post by masklinn »

sonnyblarney wrote: post #22
asveikau wrote: post #20

Earlier quoted context omitted.

I don't think a lot of people realize that Unicode itself, in any representation, is complex, and there is no one abstraction that will make it always easy in all cases. So I think your goal is unobtainable. Some of that complexity is because world languages themselves are not simple. (eg. If you were to ditch Unicode and start from scratch, it would remain hard to do bi-di text, to pick a random example.)

Totally see what you are saying (the underlying issue is language itself), but it's pragmatically obtainable.

Ah, so your solution is the same old canard: just ignore the complexity and call it pragmatic.

Re: Swift UTF-8 String

#68

Post by orbifold »

ilovecaching wrote: post #12
bbrian wrote: post #6

Earlier quoted context omitted.

Google are beginning to use it for Tensorflow: https://www.youtube.com/watch?v=s65BigoMV_I There's a course currently running in University of San Francisco run by Fast.ai which is using Swift-Tensorflow taught by Chris Lattner for two lessons: https://www.usfca.edu/data-institute/certificates/deep-learn... Not exactly what you asked, but pertinent.

I don't expect that will do very well. ML is already crowded, and Python is already solidly at the top. Having worked in ML myself, no one would switch if it meant dealing with a real statically typed language they could hack away at easily in a repl or jupyter notebooks, or access mypy, numpy, scikit, etc. ML is an ecosystem of tools that are all in Python currently.

In Swift you can just write

import Python

var np = Python.import(“numpy”)

And can then use numpy as you would normally. Same goes for plotting.

Re: Swift UTF-8 String

#69

Post by ken »

kevin_thibedeau wrote: post #47
faragon wrote: post #34

Wise move, in my opinion. I hope Python will do that too, eventually.

Python implemented multi-representation strings in 3.3. It picks the best encoding from ASCII, UTF8, UCS2, or UCS4. UTF8 is the preferred encoding for strings exposed to the C API. https://www.python.org/dev/peps/pep-0393/

The "kind" field is one of [uninitialized, Latin-1, UCS-2, UCS-4]. I'm not sure how it would pick UTF-8.

(In memory usage, the Python way is strictly worse: adding one 4-byte character to an ASCII string forces the entire string to be upgraded to UCS-4. The advantage of the Python way is O(1) access to codepoints, but that's an operation that rarely comes up in practice, when dealing with Unicode strings.)

I'm not sure how that would be compatible with Python, either. Either you're giving up string[index], or you're causing this one subscript operation to take O(n) time, or you need a new index type just for strings (which is what Swift does). None of these seem very 'Pythonic'.

EDIT: Apparently recent versions of pypy solve this by computing their own index into your string, which seems like it would be terrible for memory usage, but I look forward to their blog post about it.

Re: Swift UTF-8 String

#70

Post by BigWhale »

pjmlp wrote: post #53
saagarjha wrote: post #48

Earlier quoted context omitted.

There's still stuff going on on the GitHub repository: https://github.com/SwiftCommunityPodcast/podcast

So in one and half months they haven't found time nor a volunteer willing to do another podcast? My favourite podcasts have weekly and bi-weekly updates.

Yeah, he can't even release a podcast episode whenever you want him to. Does he not know he won't make it to your list of favorite podcasts? Why does he even wake up? Lattner is such a loser.
Post reply on HN