Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Charset="WTF-8"

wtf-8.xn--stpie-k0a81a.com

Post reply on HN
Print view View original

321–330 of 463 posts

Re: Charset="WTF-8"

#321

Post by Aeolun »

scripturial wrote: post #218
Aeolun wrote: post #210

Earlier quoted context omitted.

I mean, we can just make the languages simpler? We can also remove all the hundred different ways to pronounce English sounds. All elementary students will thank you for it xD

You can make a language simpler but old books still exist. I guess if we burn all old books and disallow a means to print these old books again, people would be happy?

Reprint them with new spelling? We have 500 year old books that are unreadable. 99.99% of all books published will not be relevant to anyone that isn’t consuming them right at that moment anyway.

Lovers can read the lord of the rings in the ‘original’ spelling.

Re: Charset="WTF-8"

#322

Post by arghwhat »

Muromec wrote: post #254
kgeist wrote: post #232

My rule of thumb is to treat strings as opaque blobs most of the time. The only validation I'd always enforce is some sane length limit, to prevent users from shoving entire novels inside. If you treat your strings as opaque blobs, and use UTF8, most of internationalization problems go away. Imho often times, input validation is an attempt to solve a problem from the wrong side. Say, when XSS or SQL injections are fo…

You can treat names as byte blobs for as long as you don't use them for their purpose -- naming people. Suppose you have a unicode blob of my name in your database and there is a problem and you need to call me and say hi. Would your customer representative be able to pronounce my name somewhat correctly? >I think there're very few exceptions to this, probably something law-related, or if you have to interact with so…

> Suppose you have a unicode blob of my name in your database and there is a problem and you need to call me and say hi. Would your customer representative be able to pronounce my name somewhat correctly?

You cannot pronounce the name regardless of whether it is written in ASCII. Pronouncing a name requires at the very least knowledge of the language it originated in, and attempts at reading it with an English pronunciation can range from incomprehensible to outright offensive.

The only way to correctly deal with a name that you are unfamiliar with the pronunciation of is to ask how it is pronounced.

You must store and operate on the person's name as is. Requiring a name modified, or modifying it automatically, is unacceptable - in many cases legal names must be represented accurately as your records might be used for e.g. tax or legal reasons later.

Re: Charset="WTF-8"

#323

Post by dcow »

afiori wrote: post #310
rafram wrote: post #272

Earlier quoted context omitted.

Sanitizing your strings immediately before display is all well and good until you need to pass them to some piece of third-party software that is very dumb and doesn’t sanitize them. You’ll argue that it’s the vendor’s fault, but the vendor will argue that nobody else allows characters like that in their name inputs! See the Companies House XSS injection situation, where their rationale for forcing a business to chan…

Forbidding users to use your service to propagate "litte bobby tables" pseudo-pranks is likely a good choice. The choice is different if like most apps you are almost only a data sink, but if you are also a data source for others it pays to be cautious.

I think it’s more of an ethical question than anything. There will always be pranksters and there will never be perfect input validation for names. So who do you oppress? The people with uncommon names? Or the pranksters? I happen to think that if you do your job right, the pranksters aren’t really a problem. So why oppress those with less common names?

Re: Charset="WTF-8"

#325

Post by Muromec »

cowsandmilk wrote: post #304
swiftcoder wrote: post #296

Earlier quoted context omitted.

Not really, no. A lot of us only really have to deal with English-adjacent input (i.e. European languages that share the majority of character forms with English, or cultures that explicitly Anglicise their names when dealing with English folks). As soon as you have to deal with users with a radically different alphabet/input-method, the wheels tend to come off. Can your CSR reps pronounce names written in Chinese lo…

You can analyze the name and direct a case to a CSR who can handle it. May be unrealistic for a 1-2 person company, but every 20+ person company I’ve worked at has intentionally hired CSRs with different language abilities.

First of, no you can't infer language preference from a name. The reasonable and well meaning assumption about my name on a good day makes me only sad and irritated.

And even if you could, I don't know if you actually do it by looking at what you signup form asks me to input.

Re: Charset="WTF-8"

#326

Post by arghwhat »

Intermernet wrote: post #301
Muromec wrote: post #254

Earlier quoted context omitted.

You can treat names as byte blobs for as long as you don't use them for their purpose -- naming people. Suppose you have a unicode blob of my name in your database and there is a problem and you need to call me and say hi. Would your customer representative be able to pronounce my name somewhat correctly? >I think there're very few exceptions to this, probably something law-related, or if you have to interact with so…

Many Japanese companies require an alternative name entered in half width kana to alleviate this exact problem. Unfortunately, most Japanese websites have a million other UX problems that overshadow this clever solution to the problem.

This is a problem specific to languages using Chinese characters where most only know some characters and therefore might not be able to read a specific one. Furigana (which is ultimately what you're providing in a separate field here) is often used as a phonetic reading aid, but still requires you to know Japanese to read and pronounce it correctly.

The only generic solution I can think of would be IPA notation, but it would be entirely unreasonable to expect someone to know the IPA for their name, just as it would be unreasonable to expect a random third party to know how to read IPA and replicate the sounds it described.

Re: Charset="WTF-8"

#327

Post by robin_reala »

samatman wrote: post #192
robin_reala wrote: post #120

Earlier quoted context omitted.

I know an Åsa who became variously Åsa, Aasa and Asa after moving to a non-Scandinavian country. That took a while to untangle, and caused some of the problems you describe.

Spelled with an Angstrom, or with a Latin Capital Letter A With Ring Above?

The second. It’s the 27th letter of the Swedish alphabet.

Re: Charset="WTF-8"

#328

Post by afiori »

dcow wrote: post #323
afiori wrote: post #310

Earlier quoted context omitted.

Forbidding users to use your service to propagate "litte bobby tables" pseudo-pranks is likely a good choice. The choice is different if like most apps you are almost only a data sink, but if you are also a data source for others it pays to be cautious.

I think it’s more of an ethical question than anything. There will always be pranksters and there will never be perfect input validation for names. So who do you oppress? The people with uncommon names? Or the pranksters? I happen to think that if you do your job right, the pranksters aren’t really a problem. So why oppress those with less common names?

I am not saying to only allow [a-zA-Z ]+ in names, what I am Saying is that it is ok to block names like "'; drop table users;" or "" rel="nofollow">https://bad.site.net/>" if part of your business is to distribute that data to other consumers.

Re: Charset="WTF-8"

#329

Post by dcow »

afiori wrote: post #328
dcow wrote: post #323

Earlier quoted context omitted.

I think it’s more of an ethical question than anything. There will always be pranksters and there will never be perfect input validation for names. So who do you oppress? The people with uncommon names? Or the pranksters? I happen to think that if you do your job right, the pranksters aren’t really a problem. So why oppress those with less common names?

I am not saying to only allow [a-zA-Z ]+ in names, what I am Saying is that it is ok to block names like "'; drop table users;" or " " rel="nofollow">https://bad.site.net/> " if part of your business is to distribute that data to other consumers.

And I’m arguing, rhetorically, what if your name produces a syntax error—or worse means something semantically devious—in the query language I’m using? Not all problems look like script tags and semicolons.

Re: Charset="WTF-8"

#330

Post by rmrfchik »

Well, the labels of input fields are written in English yet user enters his name in native language.

What's the reason of having name at all? You can call the person by this name. But if I write you my name in my language, what you (not knowing how to read it) can do? Only "hey, still-don't-know-you, here is your info".

In my foreign passport I have name __transliterated__ to Latin alphabet. Shouldn't this be the case for other places?

Post reply on HN