Nice post, quite interesting. I think Rob Pike may have one of the coolest email addresses in the world (r@google.com).
UTF-8 history (2003)
41–50 of 58 posts
Re: UTF-8 history (2003)
#42Nice post, quite interesting. I think Rob Pike may have one of the coolest email addresses in the world (r@google.com).
I've heard his email causes issues for tons of internal systems at Google because a lot are coded with the expectation of a minimum of 3 characters for the left part of the email. May just be urban legend though.
Re: UTF-8 history (2003)
#43Rob's initial claim is that Ken came up with UTF-8 entirely from scratch, without even looking at IBM's proposal.
But Ken's oldest document from Sep 2, 1992 has his changes simply appended after the original FSS-UTF document from IBM (starting at "We define 7 byte types", as mentioned in the email), and notes the changes from original spec.
Then the final document that he sent out (Sep 8) is basically the FSS-UTF document with his changes applied (this is also mentioned in the email!).
There are two changes, basically:
1. Use 10 instead of 1 as the prefix for continuation bytes, so you can synchronize from an arbitrary location.
2. Once the bits are reassembled, use the value as-is instead of adding a bias constant, which simplifies the code at the cost of a tiny bit of packing efficiency.
Given the documentation provided, I would say that the fairest description of the development of UTF-8 is that IBM came up with the initial design, and Plan 9 made two improvements on it to produce the final UTF-8 design.
Re: UTF-8 history (2003)
#44back from 2003... how is possible? even nowadays I still find places which miss the Ñ when printing documents...
Re: UTF-8 history (2003)
#45Nice post, quite interesting. I think Rob Pike may have one of the coolest email addresses in the world (r@google.com).
Back when I worked at Google, a friend of mine got an unpleasant email from r@ because his Python script had a bug: it sent alerts to the individual characters of my-friend@google.com
Re: UTF-8 history (2003)
#46Earlier quoted context omitted.
Yes, the ultra-compact encoding will not be self-sequencing. But the bit-6-continuation variant yields more bits per byte, which would give better compression in many languages. Regarding efficiency, you still have to read every character, with the difference being one check every 1, 2, 3, or 4 characters in a more complex algorithm vs a check on every character in a simpler algorithm (I haven't checked to see which…
> huge data wastage when certain languages are encoded Is there a language that consistently uses codepoints with more than 2 bytes? It bothers me a bit that UTF-8 is not an infinitely extendable encoding. But that also isn't an important objection, because it is finite, but huge.
Kanji, hiragana, katakana etc. At least 3 bytes with some 4-byte sequences. This compares unfavorably to for example Shift-JIS.
Re: UTF-8 history (2003)
#47I believe these are photos of the exact diner. I can't confirm though. https://www.flickr.com/photos/ajstarks/albums/72157631470798... . Source: https://news.ycombinator.com/item?id=19565980
Fixed flickr url: https://www.flickr.com/photos/ajstarks/sets/7215763147079887...
Re: UTF-8 history (2003)
#48Earlier quoted context omitted.
I've heard his email causes issues for tons of internal systems at Google because a lot are coded with the expectation of a minimum of 3 characters for the left part of the email. May just be urban legend though.
Do "+" suffixes work for google.com addresses? Is r+pike@google.com the same account?
Re: UTF-8 history (2003)
#49such as examples/UTF-8-demo.txt a beautiful demo file for testing terminal emulators
or the wcwidth.c file I discovered was the origin of the same c function found in all OS's, i forked & maintain in python form as the public "wcwidth" module,
anyway just wanted to point out the treasure trove of the parent folder!
Re: UTF-8 history (2003)
#50With all due respect, it seems to me that the documents contradict the claim in the initial email. Rob's initial claim is that Ken came up with UTF-8 entirely from scratch, without even looking at IBM's proposal. But Ken's oldest document from Sep 2, 1992 has his changes simply appended after the original FSS-UTF document from IBM (starting at "We define 7 byte types", as mentioned in the email), and notes the change…