Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

UTF-8 history (2003)

doc.cat-v.org

Post reply on HN
Print view View original

11–20 of 37 posts

Re: UTF-8 history (2003)

#11

Post by nostrademons »

anonymars wrote: post #8
flohofwoe wrote: post #3

WinNT missing out on UTF-8 and instead going with UCS-2 for their UNICODE text encoding might have been "the other" billion dollar mistake in the history of computing ;) There was a 9 month time window between the invention of UTF-8 and the first release of WinNT (Sep 1992 to Jul 1993). But ok fine, UTF-8 didn't really become popular until the web became popular. But then missing the other opportunity to make the tra…

"UTF-8 was first officially presented at the USENIX conference in San Diego, from January 25 to 29, 1993" ( https://en.wikipedia.org/wiki/UTF-8 ) Hey team, we're working to release an ambitious new operating system in about 6 months, but I've decided we should burn the midnight oil to rip out and redo all of the text handling we worked on to replace it with something that was just introduced at a conference.. Oh and…

Interestingly, there is another story on the HN front page about Steve Wozniak doing exactly that for the Apple I:

https://news.ycombinator.com/item?id=45265240

Re: UTF-8 history (2003)

#12

Post by masfuerte »

bobmcnamara wrote: post #10
flohofwoe wrote: post #3

WinNT missing out on UTF-8 and instead going with UCS-2 for their UNICODE text encoding might have been "the other" billion dollar mistake in the history of computing ;) There was a 9 month time window between the invention of UTF-8 and the first release of WinNT (Sep 1992 to Jul 1993). But ok fine, UTF-8 didn't really become popular until the web became popular. But then missing the other opportunity to make the tra…

Can't imagine they would've wanted to change encoding between Win3.1 and NT3.1.

But they did?

Re: UTF-8 history (2003)

#13

Post by zahlman »

ummonk wrote: post #9
bikeshaving wrote: post #6

There are socio-economic reasons why the early computing boom (ENIAC, UNIVAC, IBM mainframes, early programming languages like Fortran and COBOL) was dominated by the US: massive wartime R&D, university infrastructure, and a large domestic market. But I wonder if the Anglophone world also had an orthographic advantage as well. English uses 26 letters with no diacritics, compared to other languages like Chinese (thous…

The same applies to why China had all the building blocks (pun intended) of the printing press but it was perfected by Gutenberg in Europe, where the number of glyphs was much more manageable.

Indeed. Even if you try to split hanzi into parts it's far more unwieldy (https://en.wikipedia.org/wiki/Kangxi_radicals).

Re: UTF-8 history (2003)

#14

Post by pklausler »

bikeshaving wrote: post #6

There are socio-economic reasons why the early computing boom (ENIAC, UNIVAC, IBM mainframes, early programming languages like Fortran and COBOL) was dominated by the US: massive wartime R&D, university infrastructure, and a large domestic market. But I wonder if the Anglophone world also had an orthographic advantage as well. English uses 26 letters with no diacritics, compared to other languages like Chinese (thous…

We got lots done with 6-bit pre-ASCII encodings, actually, like CDC Display Code and Univac's Fieldata. It's more than enough for 26 letters, 10 digits, and lots of punctuation. And there are faint echoes of these early character sets remaining in ASCII -- a zero byte is ^@, for example, because @ was the zero-valued Fieldata "master space" character, which distinguished EXEC 8 control cards from source code and data cards.

Re: UTF-8 history (2003)

#15

Post by kps »

bikeshaving wrote: post #6

There are socio-economic reasons why the early computing boom (ENIAC, UNIVAC, IBM mainframes, early programming languages like Fortran and COBOL) was dominated by the US: massive wartime R&D, university infrastructure, and a large domestic market. But I wonder if the Anglophone world also had an orthographic advantage as well. English uses 26 letters with no diacritics, compared to other languages like Chinese (thous…

Computer character codes descended directly from pre-computer codes, either teletype or punched card. The advantage holds back through printing to writing itself; having a small, fixed set of glyphs that can represent anything is just better.

Re: UTF-8 history (2003)

#17

Post by andrewl-hn »

ivanjermakov wrote: post #7
flohofwoe wrote: post #3

WinNT missing out on UTF-8 and instead going with UCS-2 for their UNICODE text encoding might have been "the other" billion dollar mistake in the history of computing ;) There was a 9 month time window between the invention of UTF-8 and the first release of WinNT (Sep 1992 to Jul 1993). But ok fine, UTF-8 didn't really become popular until the web became popular. But then missing the other opportunity to make the tra…

Windows using CP-125X encodings by default in many countries instead of a UTF-8 did a lot of damage, at least in my experience.

For many European languages like French or German the switch from local CP-encodings meant that only some characters like å, ñ, ç, etc. would require extra bytes. And thus the switch to UTF-8 was a no-brainer.

On the other hand, Cyrillic and Greek are two examples of short alphabets that allowed combining them with ASCII into a single-byte encoding for countries like Greece, Bulgaria, Russia, etc. For those locations switching to UTF-8 meant that you need extra bytes for all characters in a local language, and thus higher storage, memory, and bandwidth requirements for all computing. So, non-Unicode encodings stuck there for a lot longer.

Re: UTF-8 history (2003)

#18

Post by Rendello »

xenadu02 wrote: post #5

It is worth reading the history of the proposal. The final form is superior to the others so someone was doing a lot of editing! Take the final and second form where the use of multiple letters was eliminated, instead using "v" to indicate bits of the encoded character. I also chuckle at the initial implementation's note about the desire to delete support for 4/5/6 byte versions. Someone was still laboring under the…

They pretty much got their wish, bytes 5 and 6 are gone, along with half of byte 4!

The RFC that restricted it: https://www.rfc-editor.org/rfc/rfc3629#page-11

A UTF-8 playground: https://utf8-playground.netlify.app/

Re: UTF-8 history (2003)

#19

Post by duskwuff »

pklausler wrote: post #14
bikeshaving wrote: post #6

There are socio-economic reasons why the early computing boom (ENIAC, UNIVAC, IBM mainframes, early programming languages like Fortran and COBOL) was dominated by the US: massive wartime R&D, university infrastructure, and a large domestic market. But I wonder if the Anglophone world also had an orthographic advantage as well. English uses 26 letters with no diacritics, compared to other languages like Chinese (thous…

We got lots done with 6-bit pre-ASCII encodings, actually, like CDC Display Code and Univac's Fieldata. It's more than enough for 26 letters, 10 digits, and lots of punctuation. And there are faint echoes of these early character sets remaining in ASCII -- a zero byte is ^@, for example, because @ was the zero-valued Fieldata "master space" character, which distinguished EXEC 8 control cards from source code and data…

> a zero byte is ^@, for example, because...

A zero byte is ^@ because 0x00 + 64 = '@'. The same pattern holds for all C0 control codes.

Re: UTF-8 history (2003)

#20

Post by jcranmer »

bikeshaving wrote: post #6

There are socio-economic reasons why the early computing boom (ENIAC, UNIVAC, IBM mainframes, early programming languages like Fortran and COBOL) was dominated by the US: massive wartime R&D, university infrastructure, and a large domestic market. But I wonder if the Anglophone world also had an orthographic advantage as well. English uses 26 letters with no diacritics, compared to other languages like Chinese (thous…

> English uses 26 letters with no diacritics, compared to other languages like Chinese (thousands of characters), Hindi (50+ letters), or French/German (latin with diacritics).

The English language has diacritics (see words like naïve, façade, résumé, or café). It's just that the English language uses them so rarely that they are largely dropped in any context where they are hard to introduce. Note that this adaptation to lack-of-diacritic can be found in other Latin script languages: French similarly is prone to loss-of-diacritic (especially in capital letters), whereas German has alternative spelling rules (e.g., Schroedinger instead of Schrödinger).

Post reply on HN