Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

eng.getwisdom.io

Post reply on HN
Print view View original

221–230 of 231 posts

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#221

Post by leipert »

80386 wrote: post #220
gumby wrote: post #80

Earlier quoted context omitted.

Barely. If you look at it it is clearly just what its name says: a lowercase medial s combined with a lowercase z. The uppercase version exists, as a parallel commentator noted, basically as a typographic utility. Fraktur had/has other such lower case ligatures (tz, ch, sch, ss ( not ß) et al) but for some reason only ß survived into Latin script as a full fledged letter. I have a lot of old (mostly 20th century) boo…

Fraktur ß is a ligature of s and z; the current form came into use in Latin-script German because the Latin script already had ß, for the ligature of s and s. Some sort of orthographic device is needed - s between two vowels means the first vowel is long and the consonant is voiced, and ss between two vowels means the first vowel is short and the consonant is voiceless. So it's useful to have a different case for whe…

Not really _needed_. Swiss orthography doesn’t use _ß_

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#222

Post by nybble41 »

ChrisSD wrote: post #169
BoppreH wrote: post #91

Earlier quoted context omitted.

It's not just Unicode weirdness, it's the whole concept of string operations. Examples: 1) Naive CSV libraries that break when given a field containing a comma itself. Same for injections. 2) Indexing things by strings (file names, user names, etc) relies on humans typing and reading with 100% accuracy. 3) Control characters are still present. Try to generate a file name containing every ASCII character (from 0x01 to…

For 3), that's not the case on Windows filesystems as these characters are not allowed in file and folder names. On Linux, in addition to control characters, the shell interprets the `-` character specially. This means that if a filename starts with `-` you may have surprising results even if the filename is quoted.

> On Linux, in addition to control characters, the shell interprets the `-` character specially.

It's not the shell that interprets the leading `-` character specially, but rather the program receiving the string. The convention of marking the end of option processing with `--` helps, for programs that support it; you can also prefix relative paths with `./`.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#223

Post by paulddraper »

lookdangerous wrote: post #117
amenghra wrote: post #68

Earlier quoted context omitted.

Mr. dot is evil is the first thing we taught new engineers at Facebook during the security engineering on boarding. There’s a whole slide deck (filled with real code snippets) with examples where a string was used instead of a better suited object representation and lead to a security flaw. We eventually build xhp/jsx to get rid of strings-holding-html data, but that was just scratching the surface of bugs caused by…

Mr. Dot as in the php stir concatenation operator?

That's my guess as well.

Speaking of "evil" broken abstractions...

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#224

Post by angst » 

  >'ß'.toLowerCase() // 'ss'
  "ß"
  >'ß'.toLowerCase() === 'SS'.toLowerCase() // true
  false
  >// Note the Turkish dotless i
  >'John@Gıthub.com'.toUpperCase() === 
  'John@Github.com'.toUpperCase()
  true
Chrome 79.0.3945.79 (Windows, 64bit) seems to differ from the proposed results for the first two statements. If the proposed results are what should indeed happen by unicode standard then I wonder chrome is not fully implementing them?

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#225

Post by jhanschoo »

thaumasiotes wrote: post #217
camgunz wrote: post #213

Earlier quoted context omitted.

Ugh, I was trying to get a good comparison of translations, but too bad I guess. Yeah I mean, there's no question ideographic languages have more meaning per character than alphabetic languages. But other comparisons and considerations aren't as obvious: - how do ideographic languages compare with each other? - do people using ideographic languages write more? - are ideographic languages as effective at compression a…

> are ideographic languages as effective at compression as general (or special) compression algorithms? That one we know; the compression algorithms are more compressive. For example, compressed Chinese text takes up less space than the same text uncompressed. Ideographic languages are still languages that real humans have to use, and they feature redundancy because that helps everyone. Compression algorithms have th…

> do people using ideographic languages write more?

A related question is if people using ideographic languages write electronically when they do, and I suspect yes.

Language Log has several articles on the phenomenon where Chinese seem to be forgetting how to write characters due to IT input methods. https://languagelog.ldc.upenn.edu/nll/?p=7142

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#226

Post by hsivonen »

thaumasiotes wrote: post #210
hsivonen wrote: post #191

Earlier quoted context omitted.

For writing globally appropriate software, UTF-8 is the best option that exists in practice. Note that “Asian” is quite a gamut in terms of UTF-8 size: Chinese is at the very compact end of the spectrum and the Burmese script at the other end: https://hsivonen.fi/string-length/#counts

> For writing globally appropriate software, UTF-8 is the best option that exists in practice. OK, I lied. China doesn't use GB2312 anymore. They use its update, GB18030, which carries arbitrary Unicode data just as UTF-8 does... except that, like GB2312, it puts the Asian characters in two bytes and the European characters in three. It's certainly not obvious to me that UTF-8 is better for globally appropriate softw…

GB18030 has no 3-byte sequences. Only 1, 2, and 4.

There are more important qualities than optimizing byte length. Processing GB18030 as Unicode scalar values involves lookup tables. A single byte error cascades potentially further than in UTF-8.

If optimizing Chinese byte length is really important for you, UTF-16 is easier to work with than GB18030.

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#227

Post by Liquid_Fire »

scythe wrote: post #82
BoppreH wrote: post #36

I love Unicode, but I'm more and more coming to the conclusion that strings are evil and should be treated as opaque byte arrays, whose only available operation is rendering into a bounded area. I now see any other string operation as code smell. It's scary how much of our infrastructure relies on strings, given how few guarantees string operations actually give. Take files names, for example. Two visually identical…

So here's the thing. When you display a word written entirely in Cyrillic characters, it would be wrong for it not to display as normal text. But when you display a word that contains eight Latin characters and one Cyrillic character, couldn't the display create some sort of warning? Highlight the section-mismatched character with a box? It's not normal in any language to mix alphabets.

> It's not normal in any language to mix alphabets.

This is not true.

For example, my native language has suffix-based articles (there is no separate word "the"; instead there is a suffix attached to the noun). When including a foreign word in the middle of a sentence, some people will attach the article suffix directly to the foreign word, or at best with a hyphen (this is likely not correct "by the book", but some people write like this anyway). With your suggestion, this will show a warning when it is perfectly normal usage.

Furthermore, some Asian languages like Japanese do not use spaces, so when they mix in foreign words or names written in their original script how do you tell that it's intended to be a separate word?

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#228

Post by c0l0 »

londons_explore wrote: post #209
lazyjones wrote: post #164

Earlier quoted context omitted.

> Total non-optimisation. Lots of non-thought too. Sending e-mail directly from the place where web requests are processed isn't very smart. What if the SMTP subsystem is currently down or very slow? How many e-mails will you send if an attacker starts 1000 parallel web requests for a password reset? A saner way to do these things is to just set a flag ("password reset requested") in the user database there and do th…

Are designers like you the reason password reset mails frequently take 30 seconds plus to arrive? I expect things on the web to be damn near instant - I want to click that password reset button and hear a synchronous "ding" of an arriving mail. I don't want to wait for some cron job to run once per minute. If you want to send mail off the serving path, at least use a push based queue so there is no scheduling or poll…

Yeah... and don't forget that the "R" in "SMTP" stands for "Real-time".

Re: Hacking GitHub's Auth with Unicode's Turkish Dotless 'I'

#229

Post by thaumasiotes »

jhanschoo wrote: post #225
thaumasiotes wrote: post #217

Earlier quoted context omitted.

> are ideographic languages as effective at compression as general (or special) compression algorithms? That one we know; the compression algorithms are more compressive. For example, compressed Chinese text takes up less space than the same text uncompressed. Ideographic languages are still languages that real humans have to use, and they feature redundancy because that helps everyone. Compression algorithms have th…

> do people using ideographic languages write more? A related question is if people using ideographic languages write electronically when they do, and I suspect yes. Language Log has several articles on the phenomenon where Chinese seem to be forgetting how to write characters due to IT input methods. https://languagelog.ldc.upenn.edu/nll/?p=7142

Handwriting isn't exactly uncommon in China. Nobody's forgetting how to write characters they write all the time, and if they do need to write a character they can't quite remember, they can squiggle something and rely on the reader to understand it.

Handwriting is actually a moderate-level problem for me -- I can't recognize most handwritten characters. Often a special handwriting form will be used.

Post reply on HN