Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Charset="WTF-8"

wtf-8.xn--stpie-k0a81a.com

Post reply on HN
Print view View original

351–360 of 463 posts

Re: Charset="WTF-8"

#351

Post by n2d4 »

wruza wrote: post #299
n2d4 wrote: post #238

Earlier quoted context omitted.

> Is this necessary to correctly represent human language? Yes! As soon as you have any invisible characters (eg. RTL or LTR marks, which are required to represent human language), you will be able to encode any data you want.

How many direction marks can we see in this hidden text?

None — it's tag characters instead, which are used to represent emojis. But there's no difference! Either you can smuggle text in Unicode, or you can't. It's quite binary, and you don't gain advantages from having "fewer ways" to smuggle text, but you certainly gain advantages from having emojis in your characterset.

Re: Charset="WTF-8"

#352

Post by foldr »

dcow wrote: post #329
afiori wrote: post #328

Earlier quoted context omitted.

I am not saying to only allow [a-zA-Z ]+ in names, what I am Saying is that it is ok to block names like "'; drop table users;" or " " rel="nofollow">https://bad.site.net/> " if part of your business is to distribute that data to other consumers.

And I’m arguing, rhetorically, what if your name produces a syntax error—or worse means something semantically devious—in the query language I’m using? Not all problems look like script tags and semicolons.

It's a question of intent. There aren't any hard and fast rules, but if someone has chosen their company name specifically in order to cause problems for other people using your service, then it's reasonable to make them change it.

Re: Charset="WTF-8"

#353

Post by cudder »

caseyohara wrote: post #109
Macha wrote: post #97

Earlier quoted context omitted.

Honestly I wish we could just abolish first and last name fields and replace them with a single free text name field since there's so many edge cases where first and last is an oversimplification that leads to errors. Unfortunately we have to interact with external systems that themselves insist on first and last name fields, and pushing it to the user to decide which is part of what name is wrong less often than str…

I did this in the product where I work. We operate globally so having separate first and last name fields was making less sense. So I merged them into a singular full name field. The first and only people to complain about that change were our product marketing team, because now they couldn’t “personalize” emails like `Hi ,`. I had the hardest time convincing them that while the concept of first and last names are co…

I tried this too, and a customer angrily asked why they can't sort their report alphabetically by last name. Sigh.

Re: Charset="WTF-8"

#354

Post by throw0101a »

MrJohz wrote: post #82
throw310822 wrote: post #35

Earlier quoted context omitted.

I know multiple developers who would just say "well it's their fault, they have to change name then".

I worked with an office of Germans who insisted that ASCII was sufficient. The German language uses letters that cannot be represented in ASCII. In fairness, they mostly wanted stuff to be in English, and when necessary, to transliterate German characters into their English counterparts (in German there is a standardised way of doing this), so I can understand why they didn't see it was necessary. I just never unders…

> I just never understood why I, as the non-German, was forever the one trying to convince them that Germans would probably prefer to use their software in German...

I've heard that German is often one of the first localizations of (desktop) software because there were often super-long words in the translations of various concepts, so if you wanted to test typeface rendering and menu breakage it was good language to run through your QA for that.

Re: Charset="WTF-8"

#355

Post by crazygringo »

lmm wrote: post #249
crazygringo wrote: post #14

Earlier quoted context omitted.

If you just use the {Alphabetic} Unicode character class (100K code points), together with a space, hyphen, and maybe comma, that might get you close. It includes diacritics. I'm curious if anyone can think of any other non-alphabetic characters used in legal names around the world, in other scripts? I wondered about numbers, but the most famous example of that has been overturned: "Originally named X Æ A-12, the chi…

> I'm curious if anyone can think of any other non-alphabetic characters used in legal names around the world, in other scripts? Some Japanese names are written with Japanese characters that do not have Unicode codepoints. (The Unicode consortium claims that these characters are somehow "really" Chinese characters just written in a different font; holders of those names tend to disagree, but somehow the programmer co…

Ha, well I don't think we need to worry about validating characters if they can't be typed in a text box in the first place. ;)

But very interesting thanks!

Re: Charset="WTF-8"

#356

Post by throw0101a »

sandreas wrote: post #106
MrJohz wrote: post #82

Earlier quoted context omitted.

I worked with an office of Germans who insisted that ASCII was sufficient. The German language uses letters that cannot be represented in ASCII. In fairness, they mostly wanted stuff to be in English, and when necessary, to transliterate German characters into their English counterparts (in German there is a standardised way of doing this), so I can understand why they didn't see it was necessary. I just never unders…

You should have asked how they would encode the german currency sign (€ for euro) in ASCII or its german counterpart latin1/iso-8859-1... It's not possible. However I bet they would argument to use iso-8859-15 (latin9 / latin0) with the international currency sign (¤) instead or insist that char 128 of latin1 is almost always meant as €, so just ignore the standard in these cases and use a new font. This would only f…

> international currency sign (¤)

TIL:

* https://en.wikipedia.org/wiki/Currency_sign_(generic)

Re: Charset="WTF-8"

#357

Post by Muromec »

soco wrote: post #311
Muromec wrote: post #277

Earlier quoted context omitted.

That’s the most basic consideration for names, unless you only show it to the user themselves — other people have to be able to read it at least somehow. Which one is why the bag of unicode bytes approach is as wrong as telling Stęphań he has an invalid name.

And here we go again, engineers expecting the world should behave fitting their framework du jour. Unfortunately, the real world doesn't care about our engineering bubble and goes on with life - where you can be called !xóõ Kxau or ꦱꦭꦪꦤ or X Æ A-12.

I can be called what I want and in fact I have perfectly reasonable name that doesn't fit neither ASCII nor FN+LN convention. The thing is, your website accepting whatever utf8 blob my name can be serialized to today, without actually understanding it, makes my life worse, not better.

Re: Charset="WTF-8"

#358

Post by Muromec »

hooby wrote: post #331
MrJohz wrote: post #82

Earlier quoted context omitted.

I worked with an office of Germans who insisted that ASCII was sufficient. The German language uses letters that cannot be represented in ASCII. In fairness, they mostly wanted stuff to be in English, and when necessary, to transliterate German characters into their English counterparts (in German there is a standardised way of doing this), so I can understand why they didn't see it was necessary. I just never unders…

There are some valid reasons to use software in English as a German speaker. Main among those is probably translations. If you can speak English, you might be better of using the software in English, as having to deal with the English language can often be less of hassle, than having to deal with inconsistent, weird, or outright wrong translations. Even high quality translations might run into issues, where the same…

UX translations are broken most of the time for most of the software and not just in German. People just pretend it's working and okay, when it's not.

And then developers just do N > 1 ? "things" : "thing" without thinking twice, not use pgettext and all the other things.

Re: Charset="WTF-8"

#359

Post by teddyh »

speleding wrote: post #339
ddulaney wrote: post #241

Earlier quoted context omitted.

There's at least one major exception to this: Unicode normalization. It's possible for the same logical character to have two different sets of code points (for example, a-with-umlaut as a single character, vs a followed by umlaut combining diacritic). Related, distinguishing between the "a" character in Latin, Greek, Cyrillic, and the handful of other times it shows up throughout Unicode. This comes up in at least 3…

Something we just ran in to: There are two UTF-8 codepoints for the @ character, the normal one and "Full width At Sign U+FF20". It took a lot of head scratching to understand why several Japanese users could not be found with their email address when I was seeing their email right there in the database.

There are actually two more: U+FE6B and U+E0040.

Re: Charset="WTF-8"

#360

Post by voidUpdate »

xigoi wrote: post #80
ginko wrote: post #49

Under GDPR you have the legal right for your name to be stored and processed with the correct spelling in the EU. https://gdprhub.eu/index.php?title=Court_of_Appeal_of_Brusse...

This seems to only apply to banks.

Apparently it didn't apply to my bank when I tried to change my name with them, according to them anyone could have made my change of name form, and having ID and correctly putting in the PIN for my card wasn't enough
Post reply on HN