141–150 of 315 posts
Earlier quoted context omitted.
> for western programmers And by western you mean american, right? You can't even use ASCII in the UK --- '£'.
£ is still just a byte
Earlier quoted context omitted.
Because it’s never actually useful. You can’t use that information to know how much actual space it takes (in storage) as nobody sane stores UTF-32, you can’t use it to know much much logical space it takes (aka the user’s interpretation), you can’t use it to know how much visual space it takes (not that you can ever get that), and you can’t use it to segment or process the text. A length in codepoints gives you noth…
It is useful: When iterating over a string in Python (which I hope you agree _is_ useful?), you get that many parts.
That’s… not useful?
I can’t say I remember ever caring knowing how many items I would be getting during an iteration[0]. If I want to set an iteration limit I can just… do that, using `islice` or some such.
[0] in python anyway, in lower level language there can be a utility in order to pre-allocate an output collection
Earlier quoted context omitted.
Because it’s never actually useful. You can’t use that information to know how much actual space it takes (in storage) as nobody sane stores UTF-32, you can’t use it to know much much logical space it takes (aka the user’s interpretation), you can’t use it to know how much visual space it takes (not that you can ever get that), and you can’t use it to segment or process the text. A length in codepoints gives you noth…
It is useful: When iterating over a string in Python (which I hope you agree _is_ useful?), you get that many parts.
Earlier quoted context omitted.
Because it’s never actually useful. You can’t use that information to know how much actual space it takes (in storage) as nobody sane stores UTF-32, you can’t use it to know much much logical space it takes (aka the user’s interpretation), you can’t use it to know how much visual space it takes (not that you can ever get that), and you can’t use it to segment or process the text. A length in codepoints gives you noth…
It is useful: When iterating over a string in Python (which I hope you agree _is_ useful?), you get that many parts.
Not often. There’s almost nothing useful you can correctly do with a sequence of code points.
I encountered some real world unicode/emoji breakdown recently. I set my surname in a webapp to an emoji country flag because I needed a way to communicate where I was. Elsewhere in the app, it showed surnames as just their initial, e.g. "John S". There, mine showed as a featureless black flag rather than the flag I set. Presumably because that is the first codepoint of several that make up the flag.
The country flags are each made of two Unicode code points, which Unicode calls Regional Indicator Symbols. There are twenty six, one for each of the Latin capital letters A through Z. These are used to encode a flag by writing the ISO two letter country code from ISO-3166-1 e.g. F + R is France, you get a French flag.
Given your black flag experience, and the fact this is an English language forum, I'd guess maybe you wanted a flag for some entity that isn't a UN member state or some sort of recognised similar entity (e.g. the European flag EU symbolising the continent of Europe) and thus doesn't have an ISO two letter code, such as California or Wales. Those are built from a waving black flag plus their long ISO-3166-2 region code
Earlier quoted context omitted.
I agree, "length" is an ambiguous function name. It should probably not exist and instead you have functions with units in the name: .sizeBytes, .widthCharacters, .widthResAdjPixels, and so on. Back when the world was ASCII you could get away with just .length because the numbers would always be the same, but with Unicode and all of the other complications of the modern world it isn't sufficient.
length is not ambiguous at all. Its the number of elements in the array. A string in python3 is an array of unicode code points, so the length of a string is the number of unicode code points. If you want the number of bytes, you need to encode the string in a unicode format (utf8, utf16 or utf32) to get a bytes object, which is an array of bytes. Then you can get the length of that. Remember, one of the big accompli…
It can't be used to determine length in bytes (important for storage or network transmission), it can't be used to determine number of displayed characters, it can't be used to safely split a string at some position.
The only reason it has caught on is that it is easy to encode into UTF-8 and UTF-16, and that anything more interesting generally requires a language context and even a font.
I hope that future languages will get rid of this single string abstraction, and instead offer two completely separate types:
- symbol strings, which would only be usable for programming purposes and should probably be limited to ASCII
- text strings, which would be intended for human display purposes, with full Unicode support, and have APIs which answer things like "in the specified Culture, what is the length of human-recognizable characters of this string" or "what is the seventh human-recognizable characters in this string in the specified culture"
There's no reason to pay the conceptual cost of Unicode for representing field names or enums (and yes, I don't believe supporting Unicode identifiers is a good idea for a programming language; and note that I am not a native English speaker, and while I do use an alphabet, ASCII is missing some of the letters&symbols I use in my native Romanian). And there's no reason to settle for the misleading safety of Unicode code points when trying to process human displayable text.
> Python 3’s approach is unambiguously the worst one, though. Did I miss the part where he explains this take? It's made up of 5 valid unicode code units. For a language where you're not supposed to need to know the byte size semantics, the correct length should be 5. What am I missing? The close second being 17, because length in bytes. Is another fine way to represent this data, e.g. what a successful write of some…
In the words of the article: “The choice of UTF-32 (or Python 3-style code point sequences) arises from wanting the wrong thing.”
“Not needing to know the byte size semantics” seems reasonable, but it simply isn’t a useful goal. The things it makes easier or faster (knowing how many code points there are, and O(1) indexing by code point) are things you shouldn’t be doing—and when you have to interact with the rest of the world, you now have a more expensive encoding step that is always needed, rather than just sometimes if you’d chosen UTF-8 or even UTF-16.
I have read somewhere that you should learn 2 or 3 programming languages from the get go. If you learn one, you run the risk of letting it's shape dictate how you mentally model computation. At some point someone who learned a dynamically typed programming language first is bound to find out why data types matter.
I started with JavaScript and went to c/logic gates/assembly, and it wasn't too bad I had to learn what pointers were, but it was OK, haha. That said, maybe people who don't have "The Knack" would be better off learning a slightly harder language first..
Earlier quoted context omitted.
Characters in context are printable or non-printable/formatting marks right? I agree they probably meant grapheme clusters, but grapheme clusters can vary dramatically in width so the point of the conversation was to explain why a bounding box was a better approximation of their goals.
They do very in width, but with a proportional font that’s true even with ASCII text. What grapheme clusters tells you is how many times you have to press the arrow key/backspace to get to the beginning of the string.