Live data from Hacker News

It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

hsivonen.fi

281–287 of 287 posts

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#281
post #270

Earlier quoted context omitted.

Humans speak many different languages. Not all of them are English, and not all of them have writing systems which make it meaningful to talk about "string length" without disambiguating further.

Why exactly would I care about humans not speaking any of the languages I speak?

My point is that "the human way" is not specific to the way you think about these things.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#282
post #56
post #31

Earlier quoted context omitted.

Are you referring to Unicode? Because UTF-8 is simple and relatively straight forward to parse. Unicode definitely has its faults, but on the whole it‘s great. I‘ll take Unicode w/ UTF-8 any day over the mess of encodings we had before it. Needless to say, Unicode is not a good fit for every scenario.

Just as an example of what I am talking about, this is my current UTF-8 parser which I have been using for a few years now. bool utf_append_plaintext(utf* result, const char* text) { #define msk(byte, mask, value) ((byte & mask) == value) #define cnt(byte) msk(byte, 0xc0, 0x80) #define shf(byte, mask, amount) ((byte & mask) = 3) && (nxt[0] == 0xef) && (nxt[1] == 0xbb) && (nxt[2] == 0xbf)) nxt += 3; while (nxt Not exa…

I don't know what your code is doing exactly. For comparison, here's my utf8 decoder (for a single codepoint):

    static UnicodeCodepoint utf8_decode(u8 const bytes[static 4], u8 *out_num_consumed) {
        u8 const flipped = ~bytes[0];
        if (flipped == 0) {
            // Because __builtin_clz is UB for value 0.
            // When his happens, the UTF-8 is malformed.
            *out_num_consumed = 1;
            return 0;
        }
        
        u8 const num_ones = __builtin_clz(flipped) & 0x07;
        u8 const num_bytes_total = num_ones > 1 ? num_ones : 1;
        u8 const main_byte_shift = num_ones + 1;
        UnicodeCodepoint value = bytes[0] & (0xFF >> main_byte_shift);
        
        for (u8 i = 1; i > 6 != 2) {
                // Not a valid continuation byte.
                *out_num_consumed = i;
                return 0;
            }
            
            value = (value 

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#283

Earlier quoted context omitted.

In unicode the default is still SS [1] while the Germans seem to have changed it to ẞ [2]. That means now it's the same on every system, but once the unicode standard changes and some systems get updated and others not there will be different behavior of len("ß".upper()) around. I don't know how or if systems deal with this, but ß should be printed as ss if ß is unavailable in the font. It's possible this is complete…

"In unicode the default is still SS [1] while the Germans seem to have changed it to ẞ [2]." Where does the source corroborate that claim? Can you give is a hint where to find the source?

page 48: > E3: Bei Schreibung mit Großbuchstaben ist neben der Verwendung des Groß buchstabens ẞ auch die Schreibung SS möglich: Straße – STRAẞE – STRASSE.

While in older versions [1] it was the other way around:

> E3: Bei Schreibung mit Großbuchstaben schreibt man SS. Daneben ist auch die Verwendung des Großbuchstabens ẞ möglich. Beispiel: Straße – STRASSE – STRAẞE.

[1] https://www.rechtschreibrat.com/DOX/rfdr_Regeln_2016_redigie...

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#284

Earlier quoted context omitted.

I mean, the keywords of a programming language have to be in some language (unless you go the cursed route of Excel). I'm arguing against the position that non-ASCII identifiers should be disallowed.

> I'm arguing against the position that non-ASCII identifiers should be disallowed. Maybe I'm tired, but I've read this multiple times and can't quite figure out your desired position. I *think* you are in favor of non -ASCII identifiers? Like I said, I must be tired.

He says that disallowing non-ASCII identifiers is "American imperialism at its worst".

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#285
post #132
post #126

Earlier quoted context omitted.

You are in the wrong job if you don’t want to think about “nerd shit” while programming.

Idk it pays my bills and I have success at work so I must do something right.

Posting "Miss me with that nerd shit" in this thread certainly isn't doing anything right.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#286

Earlier quoted context omitted.

> in the global international connected computing world it doesn’t fit at all. I disagree. Not all text is human prose. For example, there is nothing wrong with an programming language that only allows ASCII in the source code and many downsides to allowing non-ASCII characters outside string constants or comments.

> For example, there is nothing wrong with an programming language that only allows ASCII in the source code and many downsides to allowing non-ASCII characters outside string constants or comments. That's a tradeoff you should carefully consider because there are also downsides to disallowing non-ASCII characters. The downsides of allowing non-ASCII mostly stem from assigning semantic significance to upper/lowercase…

But in Julia it also makes code a pain to write outside of a Julia aware IDE. Even in such an IDE I now need to remember what the shorthand for Euclidean division is to type it. (It would be great if I had a toggle that could switch all unicode for which there is a shorthand to the shorthand...)

Not saying the trade-off isn't worth it, but I do feel like there is a tendency to overuse unicode somewhat in Julia.

Re: It’s not wrong that "\u{1F926}\u{1F3FC}\u200D\u2642\uFE0F".length == 7 (2019)

#287

Earlier quoted context omitted.

This is American imperialism at its worst. I'm serious. Lots of people around the world learn programming from sources in their native language, especially early in their career, or when software development is not their actual job. Enforcing ASCII is the same as enforcing English. How would you feel if all cooking recipes were written in French? If all music theory was in Italian? If all industrial specifications we…

Well I'm not American and I can tell you that we do not see English source code as imperialism. In fact it's awesome that we have one common very simple character set and language that works everywhere and can do everything. I have only encountered source code using my native language (German) in comments or variable names in highly unprofessional or awful software and it is looked down upon. You will always get an u…

So source code needs to be UTF8 because it contains comments and string literals. And filenames need to be bytes. It seems that both of these are orthogonal to the question of whether non-ASCII code is desirable...

Restricting the program part to ASCII is fine for me, but as a fellow German it's also important to recognize that we don't loose much by not having ä cömplete sät of letters. Everyone can write comprehensible German using ASCII characters only. So I would listen to what people from languages that really don't fit into ASCII have to say.

Post reply on HN