Live data from Hacker News

The Elegance of the ASCII Table

danq.me

21–30 of 189 posts

Re: The Elegance of the ASCII Table

#21
I would be remiss not to post a link to the late Bob Bemer's[0] website.

https://web.archive.org/web/20150801005415/http://bobbemer.c...

He was considered the "father of ASCII". Hr wrote very well and gives clear explanations for the motivations behind the design of ASCII.

[0] https://en.m.wikipedia.org/wiki/Bob_Bemer

Re: The Elegance of the ASCII Table

#22
post #13

I heard someone describe the ASCII table as a state machine. Guess I could understand that as a state machine needed to parse it? This is surprisingly hard to search for but I was wondering if anyone knows what they were talking about.

They might be talking about using escape sequences to map additional codepoints into ASCII. It was designed to be extensible. See: https://web.archive.org/web/20150810075144/http://bobbemer.c...

Re: The Elegance of the ASCII Table

#23

Earlier quoted context omitted.

That's because the intended purpose is either useless (for machine control characters) or useless and logically impossible (for delimiters). What do you do if you have a record that includes a record separator character? Given that you have this problem anyway, why do you want a character dedicated to achieving the same thing that a comma achieves?

The record separator isn't on people's keyboards, so it's less likely to show up where it's not expected. Also it's less likely to legitimately occur in something like a name, so there are many users of CSVs who can say they will never need to consider data containing a record separator, and they will be right more often than those who never consider data containing a comma. Of course, the fact that record separators…

In the DOS days, you could "type" control characters by pressing Ctrl and the corresponding letter key, Ctrl+M is Carriage Return, Ctrl+H is Backspace, Ctrl+Z is End Of File, etc.

It was probably possible to type an RS with Ctrl+Shift+. and the others with similar combos.

Re: The Elegance of the ASCII Table

#24
> That, I’m afraid, is because ASCII was based not on modern computer keyboards but on the shifted positions of a Remington No. 2 mechanical typewriter – whose shifted layout was the closest compromise we could find as a standard at the time, I imagine.

According to Wikipedia¹, American typewriters were pretty consistent with keyboard layout until the IBM Selectric electric typewriter. Apparently "small" characters (like apostrophe, double-quote, underscore, and hyphen) should be typed with less pressure to avoid damaging the platen, and IBM decided the Selectric could be simpler if those symbols were grouped on dedicated keys instead of sharing keys with "high pressure" symbols, so they shuffled the symbols around a bit, resulting in a layout that would look very familiar to a modern PC user.

Because IBM electric typewriters were so widely used (at least in English speaking countries), any computer company that wanted to sell to businesses wanted a Selectric-style layout, including the IBM PC.

Meanwhile, in other countries where typewriters in general weren't so popular or useful, the earliest computers had ASCII-style punctuation layout for simplicity, and later computers didn't have any pressing need to change, so they stuck with it. Japanese keyboards, for example, are still ASCII-style to this day.

¹: https://en.wikipedia.org/wiki/IBM_Selectric#Keyboard_layout

Re: The Elegance of the ASCII Table

#25
post #13

I heard someone describe the ASCII table as a state machine. Guess I could understand that as a state machine needed to parse it? This is surprisingly hard to search for but I was wondering if anyone knows what they were talking about.

Bespoke hardware for text handling isn't a thing these days but would have been in the 60's and 70's. A table layout that can be easily decoded in hardware simplifies the necessary circuitry for responding to control characters or converting binary numbers to/from decimal when the microprocessor hadn't been invented yet.

Re: The Elegance of the ASCII Table

#26

Earlier quoted context omitted.

The record separator isn't on people's keyboards, so it's less likely to show up where it's not expected. Also it's less likely to legitimately occur in something like a name, so there are many users of CSVs who can say they will never need to consider data containing a record separator, and they will be right more often than those who never consider data containing a comma. Of course, the fact that record separators…

In the DOS days, you could "type" control characters by pressing Ctrl and the corresponding letter key, Ctrl+M is Carriage Return, Ctrl+H is Backspace, Ctrl+Z is End Of File, etc. It was probably possible to type an RS with Ctrl+Shift+. and the others with similar combos.

you can still type them -- alt + 030(for instance) on the keypad will insert that RS character. In Windows at least -- not sure about the other OS.

Re: The Elegance of the ASCII Table

#27

Earlier quoted context omitted.

The record separator isn't on people's keyboards, so it's less likely to show up where it's not expected. Also it's less likely to legitimately occur in something like a name, so there are many users of CSVs who can say they will never need to consider data containing a record separator, and they will be right more often than those who never consider data containing a comma. Of course, the fact that record separators…

> Also it's less likely to legitimately occur in something like a name, so there are many users of CSVs who can say they will never need to consider data containing a record separator, and they will be right more often than those who never consider data containing a comma. No, they'll be right exactly as often, 0% of the time. But their mistake will show up less frequently, causing more problems when it does. As soon…

> No, they'll be right exactly as often, 0% of the time.

> But their mistake will show up less frequently, causing more problems when it does.

Enough people use CSVs (and have limited, small-scale use-cases) that I'd be willing to bet "less frequently" means never for at least 1% of people who use CSVs.

I don't know whether the chance of no problems is worth the increased difficulty of problems that do occur - considering that balance feels a bit silly because if you're aware there could be a problem in a context where you could choose between commas and unit separators, you could just add validation or escaping.

Re: The Elegance of the ASCII Table

#28

Earlier quoted context omitted.

> Also it's less likely to legitimately occur in something like a name, so there are many users of CSVs who can say they will never need to consider data containing a record separator, and they will be right more often than those who never consider data containing a comma. No, they'll be right exactly as often, 0% of the time. But their mistake will show up less frequently, causing more problems when it does. As soon…

> No, they'll be right exactly as often, 0% of the time. > But their mistake will show up less frequently, causing more problems when it does. Enough people use CSVs (and have limited, small-scale use-cases) that I'd be willing to bet "less frequently" means never for at least 1% of people who use CSVs. I don't know whether the chance of no problems is worth the increased difficulty of problems that do occur - consid…

> considering that balance feels a bit silly because if you're aware there could be a problem in a context where you could choose between commas and record separators, you could just add validation or escaping.

As soon as you have validation or escaping, having a record separator character loses its entire purpose. The existence of the character is predicated on the idea that you don't have to do that, and that idea is false.

That's why the character is never used. It's a conceptual mistake that was accidentally enshrined in a series of encoding standards that had enough free space to accommodate it.

Re: The Elegance of the ASCII Table

#29

I always lament that since at least 1980s or so, it seems the vast majority of the control characters were never used for their intended purpose. Instead, we crudely use commas and tabs as delimiters instead of something like RS (#30).

I did some ETL work that used the ASCII delimiter characters. It was very enjoyable. I didn't have to worry about escaping or parsing escaped strings. The control codes were guaranteed to be illegal in input. It was refreshing.

Could you do the same with TSV? A lot of datasets can either prohibit tabs in data, or convert it to spaces in early ingestion.

Re: The Elegance of the ASCII Table

#30
unfortunately this page is based on mackenzie's book. mackenzie is the ibm guy who spent decades trying to kill ascii, promoting its brain-damaged ebcdic as a superior replacement (because it was more compatible, at least if you were already an ibm customer). he spends most of his fucking book trumpeting the virtues of ebcdic actually

bob bemer more or less invented ascii. he was also an ibm guy before mackenzie's crowd pushed him out of ibm for promoting it. he wrote a much better book about the history of ascii which is also freely available online, really more a pamphlet than a book, called "a story of ascii": https://archive.org/details/ascii-bemer/page/n1/mode/2up

tom jennings, who invented fido, also wrote a history of ascii, called 'an annotated history of some character codes or ascii: american standard code for information infiltration'; it's no longer online at his own site, but for the time being the archive has preserved it: https://web.archive.org/web/20100414012008/http://wps.com/pr...

jennings's history is animated by a palpable rage at mackenzie's self-serving account of the history of ascii, partly because bemer hadn't really told his own story publicly. so jennings goes so far as to write punchcard codes (and mackenzie) out of ascii's history entirely, deriving it purely from teletypewriter codes—from which it does undeniably draw many features, but after all, bemer was a punchcard guy, and ascii's many excellent virtues for collation show it

as dwheeler points out, the accomplished informatics archivist eric fischer has also written an excellent history of the evolution of ascii. though, unlike bemer, fischer wasn't actually at the standardization meetings that created ascii, he is more careful and digs deeper than either bemer or jennings, so it might be better to read him first: https://archive.org/details/enf-ascii/

it would be a mistake to credit ascii entirely to bemer; aside from the relatively minor changes in 01967 (including making lowercase official), the draft was extensively revised by the standards committees in the years leading up to 01963, including dramatic improvements in the control-character set

for the historical relationship between ascii character codes and keyboard layouts, see https://en.wikipedia.org/wiki/Bit-paired_keyboard

Post reply on HN