Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

eng.getwisdom.io

Post reply on HN
Print view View original

131–140 of 231 posts

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#131

Post by iforgotpassword »

Avamander wrote: post #76
Iv wrote: post #59

Earlier quoted context omitted.

Disclaimer: as a heavy user of unicode, using it for both French and Japanese, I love it and see how important it is in the world. I just ranted about why ASCII is important to programmers: https://news.ycombinator.com/item?id=21760540 and this is a perfect example. ASCII has almost a 1-to-1 mapping between screen representation and byte representation. Once you know your font will differentiate between 1 i L | l and…

I'm personally incredibly annoyed by just the idea of "Unicode is hard, let's do ASCII", most of the world is non-ASCII, it's just annoying and sad to still see systems that fail when people try to use their native languages. UTF-8 should be the default pretty much everywhere, there are quite easy ways to avoid homograph attacks, those attacks are a poor excuse to discriminate against non-anglosphere.

> there are quite easy ways to avoid homograph attacks, those attacks are a poor excuse to discriminate against non-anglosphere.

No. There are these kind of attacks every now and then to this day. Maybe of you're not following itsec they fly under your radar, but getting this right is exceptionally hard. And besides these kind of attack, every major os had multiple bugs just in the processing of Unicode that could at least be used for DoS attacks.

So saying it's easy to avoid any sort of abuse of Unicode seems quite ridiculous.

Go ahead and support it for messages, display names and whatnot, but for the love of god, limit the login name of users to ASCII. Don't assume that your Python/Go/JavaScript lib for Unicode handles sanitizing and canonicalization properly. It doesn't. And even if it has only a minor bug that doesn't lead to direct issues, the next update of the lib might fix the problem and now you have to deal with the fact that your db might contain data that was processed with the old faulty lib and now gets compared to the properly processed output of the new version. Just don't. Use it as opaque data for displaying, as GP said, but never as an identifier for anything.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#132

Post by grenoire »

Avamander wrote: post #75
Thorentis wrote: post #30

Earlier quoted context omitted.

So it is! It seems to even fool Chrome. If you search for "delivered" on the page the search box says "1/4" but entering will only take you to the 2 real ones, not the Turkish i ones which it has presumably counted.

The alternative is worse though. Characters with umlauts matched with characters with no umlauts. E.g. searching for "rõõsa" will find both "rõõsa" and "roosa", incredibly annoying.

Mind that an umlaut is the double dots (Ö, Ü, Ä, etc.) above letters; accents are any of the ones that are attached to the 'base set.'

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#133

Post by iforgotpassword »

ComputerGuru wrote: post #102
tim333 wrote: post #81

Earlier quoted context omitted.

I was thinking that. Unicode is great for presenting text to users but for file names, email addresses, code and the like ASCII has a lot going for it.

No, it doesn’t. What’s a typical corporate email address? First.Last@...? F.Last@...? More than half the world’s population does not use the Latin alphabet.

And how's that supposed to work? Every country has developed some form of transliteration to the Latin alphabet. Its simple enough that anybody can memorize these additional 26 letters. A form of least common denominator. Anyone on this planet can scribble down their email address on a piece of paper and equally type one on their computer. Now imagine you'd write down a Chinese email address with characters. How am I supposed to type it? How's the Chinese going to type in the Arab's address? If I receive a mail from them, how can I confirm it's really them from the address? I can't read Chinese! But the Arab and the Chinese can read the Latin alphabet.

Case in point: we have punycode for over a decade now. China has their own native TLD. Still pretty much every Chinese website out there gets created under the cn TLD or even com, and uses Latin characters only. How come?

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#134

Post by pwdisswordfish2 »

ComputerGuru wrote: post #103
scythe wrote: post #82

Earlier quoted context omitted.

So here's the thing. When you display a word written entirely in Cyrillic characters, it would be wrong for it not to display as normal text. But when you display a word that contains eight Latin characters and one Cyrillic character, couldn't the display create some sort of warning? Highlight the section-mismatched character with a box? It's not normal in any language to mix alphabets.

> it’s not normal in any language to mix alphabets Decidedly not true. Languages like French and Turkish and German use their own alphabet but they look like the typical “American” Latin alphabet but aren’t. How do you decide if it’s English with non-American letters thrown in or not? Languages like Arabic are transliterated with English letters and Arabic numerals. Other Transliteration dialects use random Arabic ch…

By the ‘script’ property, which every Unicode code point has?

It's hardly bulletproof, but it can certainly stop some of the more obvious shenanigans.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#135

Post by iraldir »

iforgotpassword wrote: post #133
ComputerGuru wrote: post #102

Earlier quoted context omitted.

No, it doesn’t. What’s a typical corporate email address? First.Last@...? F.Last@...? More than half the world’s population does not use the Latin alphabet.

And how's that supposed to work? Every country has developed some form of transliteration to the Latin alphabet. Its simple enough that anybody can memorize these additional 26 letters. A form of least common denominator. Anyone on this planet can scribble down their email address on a piece of paper and equally type one on their computer. Now imagine you'd write down a Chinese email address with characters. How am I…

Counterpoint: A lot of people in China are as familiar with the latin alphabet as you are with the Cyrillic alphabet: Source: https://newrepublic.com/article/117608/chinese-number-websit...

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#136

Post by Symbiote »

_jal wrote: post #19
microcolonel wrote: post #11

Earlier quoted context omitted.

We use three versions of the email address internally: the exact verified address used at signup or the last valid email change, a normalized version of that (for identity) without + mailboxes, lowercased, de-accented, stripped of dots and other inert punctuation, and normalized in a number of other ways... and then of course the email parameter (only used during registration). We accomplish this with a slightly more…

> stripped of dots and other inert punctuation The period thing is a Gmail feature, not a standard. some.email@mydomain and someemail@mydomain most certainly do not deliver to the same mailbox.

It is actually an antifeature, since it is non-standard and leads to various attacks.

For example, a malicious user can register

  some.email@gmail.com
  so.meemail@gmail.com
  someem.ail@gmail.com
  so.mee.mail@gmail.com
  somee.mail@gmail.com
  so..mee.mail@gmail.com
  som..eem.ail@gmail.com
  so.meemail@gmail.com
  somee.mail@gmail.com
with a service, which will (quite reasonably) send a "Welcome" email. That results in a flood of emails to the GMail user.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#137

Post by qwerty456127 »

> 'ß'.toLowerCase() // 'ss'

Why? First of all ß is already lowercase, why should toLowerCase() change it? It also is a normal letter having both uppercase (ẞ) and lowercase (ß) forms so converting between the cases can be made be trivial and quirk-free. Arguably the most common word you will encounter ß in is Straße (street) where it already is lowercase - will "Straße".toLowerCase() turn it into "strasse"? WTF? "Straße".lower() returns "straße" in Python which seems reasonable (nevertheless "Straße".upper() actually returns "STRASSE" ignoring the existence of the uppercase ẞ (U+1E9E)). Why should it behave different in JavaScript? (Because JavaScript is different, I know, just a rhetorical question)

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#138

Post by founderling »

BoppreH wrote: post #36

I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical…

    opaque byte arrays, only available operation is
    rendering into a bounded area
Goodbye web then. Because the whole request you get from a client is nothing but strings.

So when you run an onlineshop and a customer orders "7" screwdrivers - then you are screwed. Because what does an "opaque byte array" of "opaque byte arrays" cost? How much shipping will that be?

But at least you have a brand new customer: Henry@gmail.com. Since you do not lowercase the email and do not recognize him as henry@gmail.com which he used in his last order.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#139

Post by enlyth »

qwerty456127 wrote: post #137

> 'ß'.toLowerCase() // 'ss' Why? First of all ß is already lowercase, why should toLowerCase() change it? It also is a normal letter having both uppercase (ẞ) and lowercase (ß) forms so converting between the cases can be made be trivial and quirk-free. Arguably the most common word you will encounter ß in is Straße (street) where it already is lowercase - will "Straße".toLowerCase() turn it into "strasse"? WTF? "Str…

I must be missing something because I tried three browsers and all their JavaScript engines evaluate 'ß'.toLowerCase() as 'ß', not 'ss'.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#140

Post by tomcooks »

qwerty456127 wrote: post #137

> 'ß'.toLowerCase() // 'ss' Why? First of all ß is already lowercase, why should toLowerCase() change it? It also is a normal letter having both uppercase (ẞ) and lowercase (ß) forms so converting between the cases can be made be trivial and quirk-free. Arguably the most common word you will encounter ß in is Straße (street) where it already is lowercase - will "Straße".toLowerCase() turn it into "strasse"? WTF? "Str…

"We are living in America, Amerika ist wunderbar"
Post reply on HN