Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

hsivonen.fi

Post reply on HN
Print view View original

251–260 of 287 posts

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#251

Post by simonask »

jibal wrote: post #102
simonask wrote: post #86

Earlier quoted context omitted.

This is American imperialism at its worst. I'm serious. Lots of people around the world learn programming from sources in their native language, especially early in their career, or when software development is not their actual job. Enforcing ASCII is the same as enforcing English. How would you feel if all cooking recipes were written in French? If all music theory was in Italian? If all industrial specifications we…

It's neither American nor imperialism -- those are both category mistakes. Andreas Rumpf, the designer of Nim, is Austrian. All the keywords of Nim are in English, the library function names are in English, the documentation is in English, Rumpf's book Mastering Nim is in English, the other major book for the language, Nim In Action (written by Dominik Picheta, nationality unknown but not American) is in English ...…

I mean, the keywords of a programming language have to be in some language (unless you go the cursed route of Excel). I'm arguing against the position that non-ASCII identifiers should be disallowed.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#252

Post by simonask »

nitwit005 wrote: post #213
simonask wrote: post #98

Earlier quoted context omitted.

Well, the problem is that what you are advocating is also that knowing Latin would be a prerequisite for studying medicine, which it isn't anywhere. That's the equivalent. Doctors learn a (very limited) Latin vocabulary as they study and work. You are severely underestimate how far you can get without any real command of the English language. I agree that you can't become really good without it, just like you can't d…

You don't need to become fluent in Greek and Latin, but if you want to be able to read your patent's diagnosis, you're absolutely going to need to know the terms used. The standard names are in those languages. And frequently, there is no other name. There are a lot of diseases, and no language has names for all of them.

Sure, you can also look them up though, because it is a limited vocabulary.

Identifiers in code are not a limited vocabulary, and understanding the structure of your code is important, especially so when you are in the early stages of learning.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#253

Post by simonask »

ksenzee wrote: post #162
simonask wrote: post #86

Earlier quoted context omitted.

This is American imperialism at its worst. I'm serious. Lots of people around the world learn programming from sources in their native language, especially early in their career, or when software development is not their actual job. Enforcing ASCII is the same as enforcing English. How would you feel if all cooking recipes were written in French? If all music theory was in Italian? If all industrial specifications we…

I don’t see much difference between the amount of Italian you need for music and the amount of English you need for programming. You can have a conversation about it in your native language, but you’ll be using a bunch of domain-specific terms that may not be in your native language.

I agree, but we're talking about identifiers in code you write yourself here. Not the limited vocabulary of keywords, which are easy to memorize in any language. Standard libraries may trip you up, but documentation for those may be available in your native language.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#254

Post by rurban »

jibal wrote: post #102
simonask wrote: post #86

Earlier quoted context omitted.

This is American imperialism at its worst. I'm serious. Lots of people around the world learn programming from sources in their native language, especially early in their career, or when software development is not their actual job. Enforcing ASCII is the same as enforcing English. How would you feel if all cooking recipes were written in French? If all music theory was in Italian? If all industrial specifications we…

It's neither American nor imperialism -- those are both category mistakes. Andreas Rumpf, the designer of Nim, is Austrian. All the keywords of Nim are in English, the library function names are in English, the documentation is in English, Rumpf's book Mastering Nim is in English, the other major book for the language, Nim In Action (written by Dominik Picheta, nationality unknown but not American) is in English ...…

> The reason for this lack of handling of Unicode linguistically is simply to make the lexer smaller and faster.

No, it is actually for security reasons. Once you allow non-ASCII identifiers, identifiers will become non identifiable. Only zig recognized that. Nim allows insecure identifiers. https://github.com/rurban/libu8ident/blob/master/doc/c11.md#...

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#255

Post by slavik81 »

deathanatos wrote: post #163
mid-kid wrote: post #104

Earlier quoted context omitted.

Yeah I have no idea what is wrong with that. Python simply operates on arrays of codepoints, which are a stable representation that can be converted to a bunch of encodings including "proper" utf-8, as long as all codepoints are representable in that encoding. This also allows you to work with strings that contain arbitrary data falling outside of the unicode spectrum.

> which are a stable representation that can be converted to a bunch of encodings including "proper" utf-8, as long as all codepoints are representable in that encoding. Which, to humor the parent, is also true of raw bytes strings. One of the (valid) points raised by the gist is that `str` is not infallibly encodable to UTF-8, since it can contain values that are not valid Unicode. > This also allows you to work wit…

The Python language developers themselves thought that their code only needed to operate on str and later realized that it needed to handle arbitrary bytes.

It's a common mistake. A lot of code was written using str despite users needing it to operate on UnicodeWithBullshit. PEP 383 was a necessary escape hatch to fix countless broken programs.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#256

Post by kiitos »

zahlman wrote: post #223
kiitos wrote: post #216

Earlier quoted context omitted.

"character" is not a well defined concept in the context of this discussion do you mean "byte"? or "rune"?

I mean what the Unicode Consortium says it means. https://www.unicode.org/glossary/#character

I see, yup!

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#257

Post by jibal »

rurban wrote: post #254
jibal wrote: post #102

Earlier quoted context omitted.

It's neither American nor imperialism -- those are both category mistakes. Andreas Rumpf, the designer of Nim, is Austrian. All the keywords of Nim are in English, the library function names are in English, the documentation is in English, Rumpf's book Mastering Nim is in English, the other major book for the language, Nim In Action (written by Dominik Picheta, nationality unknown but not American) is in English ...…

> The reason for this lack of handling of Unicode linguistically is simply to make the lexer smaller and faster. No, it is actually for security reasons. Once you allow non-ASCII identifiers, identifiers will become non identifiable. Only zig recognized that. Nim allows insecure identifiers. https://github.com/rurban/libu8ident/blob/master/doc/c11.md#...

Reading is fundamental. I was referring to the Nim lexer. Obviously the reason that it "allows insecure identifiers" is not "actually for security reasons". It is, as I stated, for reasons of performance ... I know this from reading the code and the author's statements.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#258

Post by jibal »

jibal wrote: post #102
simonask wrote: post #86

Earlier quoted context omitted.

This is American imperialism at its worst. I'm serious. Lots of people around the world learn programming from sources in their native language, especially early in their career, or when software development is not their actual job. Enforcing ASCII is the same as enforcing English. How would you feel if all cooking recipes were written in French? If all music theory was in Italian? If all industrial specifications we…

It's neither American nor imperialism -- those are both category mistakes. Andreas Rumpf, the designer of Nim, is Austrian. All the keywords of Nim are in English, the library function names are in English, the documentation is in English, Rumpf's book Mastering Nim is in English, the other major book for the language, Nim In Action (written by Dominik Picheta, nationality unknown but not American) is in English ...…

P.S. The response is a https://en.wikipedia.org/wiki/Motte-and-bailey_fallacy

The motte: non-ASCII identifiers should be allowed

The bailey: disallowing non-ASCII identifiers is American imperialism at its worst

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#259

Post by crazygringo »

PapaPalpatine wrote: post #202
zahlman wrote: post #154

Earlier quoted context omitted.

> I have, on the other hand, always wanted the string length. In an environment that supports advanced Unicode features, what exactly do you do with the string length?

I don’t know about advanced Unicode features… but I use them all the time as a backend developer to validate data input. I want to make sure that the password is between a given number of characters. Same with phone numbers, email addresses, etc. This seems to have always been known as the length of the string. This thread sounds like a bunch of scientists trying to make a simple concept a lot harder to understand.

Practically speaking, for maximum lengths, you generally want to limit code points or bytes, not characters. You don't want to allow some ZALGO monstrosity in a password that is 5 characters but 500 bytes.

For exact lengths, you often have a restricted character set (like for phone numbers) and can validate both characters and length with a regex. Or the length in bytes works for 0–9.

Unless you're involved in text layout, you actually usually don't wind up needing the exact length in characters of arbitrary UTF-8 text.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#260

Post by capitainenemo »

zahlman wrote: post #205
capitainenemo wrote: post #190

Earlier quoted context omitted.

Heh. It always starts this way... then they confidently send me something that breaks on testing it, then half a dozen more iterations, then "python2 is doing the wrong thing" or, "I could get this working but it isn't worth the effort" but sure, let's do this one more time. Could be they were all missing something obvious - wouldn't know, I avoid python personally, apart from when necessary like with LLM glue. https…

> then they confidently send me something that breaks on testing it, then half a dozen more iterations, then "python2 is doing the wrong thing or, 'I could get this working but it isn't worth the effort'" It almost works as-is in my testing. (By the way, there's a typo in the usage message.) Here is my test process: #!/usr/bin/env python import random, sys, time def out(b): # ASCII 0..7 for the second digit of the co…

Sorry, I'm not a python guy, do you have a script you'd like me to run against python3? Just toss me a pastebin link, and ideally the version of python3 to run, since half the python3 scripts on my system seem to require a different version of python3 from the other half and a variety of isolated sets of python libs in virtual environments (heck, pip even warns you not to try installing libs globally so everyone can use same set these days). I'd rather not try to follow a set of suggestions and then be told I did it wrong.

As for typo, yep. But then, I've left this script essentially untouched for a couple of decades since I was given it.

Post reply on HN