Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Elegance of the ASCII Table

danq.me

Post reply on HN
Print view View original

61–70 of 189 posts

Re: The Elegance of the ASCII Table

#61

Post by p_l »

WalterBright wrote: post #52

Too bad we now have Unicode, an elegant castle covered with ugly graffiti and ramshackle addons. For example: 1. normalization 2. backwards running text (hey, why not add spiral running text?) 3. fonts 4. invisible characters 5. multiple code points with the same glyph 6. glyphs defined by multiple code points (gee, I thought Unicode was to get away with that mess from code pages!) 7. made up languages (Elvish? Come…

How to say you don't know what Unicode is for without saying it.

1, 2, 4, 5, 6, and, unfortunately, 8, all fall under "ability to encode written text from all human languages". And that includes historical. Some of the issues (5 & 6) are due semantic difference even if the resulting glyph looks the same. Unfortunately you can't expect programmers to understand pesky little thing like languages having different writing, so you end up with normalisation to handle the fact that one system sent "a + ogonek accent" and another (properly) sent "a with ogonek" (these print the same but are semantically different!), and now you need to figure out normalisation in order to be able to compare strings.

7. just like 8 are down to proposal of specific new forms of writing to add to Unicode. Elvish had one since 1997 but only now got a tentative "we will talk about it". Klingon, which is IIRC more complete language including native speakers (...weird things happened sometimes) does not have outside of private use area.

Emojis were added because they were used with incompatible encodings first, even before unicode happened, and without including something like SIXEL into unicode they were unrepresentable (and with SIXEL would lose semantic information)

Re: The Elegance of the ASCII Table

#62

Post by chthonicdaemon »

WalterBright wrote: post #52

Too bad we now have Unicode, an elegant castle covered with ugly graffiti and ramshackle addons. For example: 1. normalization 2. backwards running text (hey, why not add spiral running text?) 3. fonts 4. invisible characters 5. multiple code points with the same glyph 6. glyphs defined by multiple code points (gee, I thought Unicode was to get away with that mess from code pages!) 7. made up languages (Elvish? Come…

All languages are made up. For that matter, all glyphs are made up, too.

Re: The Elegance of the ASCII Table

#63

Post by zokier »

Retr0id wrote: post #60
lmm wrote: post #55

Earlier quoted context omitted.

> For everyone else, UTF is a blessing. Except people who want to use Japanese and not have it render weirdly, something that was easy in any internationalised software that used the traditional codepage system, but is practically impossible in Unicode-based software.

Where can I learn more about this issue?

https://en.wikipedia.org/wiki/Han_unification

Re: The Elegance of the ASCII Table

#64

Post by p_l »

Retr0id wrote: post #60
lmm wrote: post #55

Earlier quoted context omitted.

> For everyone else, UTF is a blessing. Except people who want to use Japanese and not have it render weirdly, something that was easy in any internationalised software that used the traditional codepage system, but is practically impossible in Unicode-based software.

Where can I learn more about this issue?

Probably referring to so-called "Han unification" which tried to use same codepoints for different glyphs to reduce code space for ideograms derived from Chinese ones.

But that only causes confusion because you need to provide external information which way to interpret them, just like a code page

Re: The Elegance of the ASCII Table

#65

Post by zokier »

I think that adopting ASCII as the general purpose text encoding was one of the great mistakes of early computing. It originated as control interface for teletypes and such, and that's arguably where it should have remained. For storing and processing (plain) text ASCII doesn't really fit that well, control characters are a hindrance and the code space would have been useful for additional characters. The ASCII set of printables was definitely a compromise formed by the limited code space.

Re: The Elegance of the ASCII Table

#66

Post by kragen »

jart wrote: post #54
kragen wrote: post #32

Earlier quoted context omitted.

del is not a printing character. it's a control character. if you run a paper tape full of del characters through a teletype it does not print anything. it has to have that bit pattern, even though it greatly complicates the mechanics of the teletype (which has to do all the digital logic with cams and levers) because that way it can be punched over any character on the paper tape to delete it a figure caption in thi…

> so the only way to mark a character as invalid was to rewind the tape and punch out all the holes in that position So that's why \177 (DEL) is the loneliest control character. Wow. Thank you!

happy to help

Re: The Elegance of the ASCII Table

#67

Post by augusto-moura »

Useful tip, on linux (not sure about other *nixes) you can view the ascii table by opening its manpage:

  man ascii
It's been useful to me more than once every year, mostly to know about shell escape codes and when doing weird character ranges in regex and C.

It can be a bit confusing, but the gist is that you have 2 chars being show in each line, I would prefer a view where you see the same char with shift and/or ctrl flags, but you can only ask so much

Re: The Elegance of the ASCII Table

#68

Post by keybored »

thaumasiotes wrote: post #15
BobbyTables2 wrote: post #12

I always lament that since at least 1980s or so, it seems the vast majority of the control characters were never used for their intended purpose. Instead, we crudely use commas and tabs as delimiters instead of something like RS (#30).

That's because the intended purpose is either useless (for machine control characters) or useless and logically impossible (for delimiters). What do you do if you have a record that includes a record separator character? Given that you have this problem anyway, why do you want a character dedicated to achieving the same thing that a comma achieves?

> What do you do if you have a record that includes a record separator character?

This comes up every time. Options:

1. You disallow it. And you might as well disallow all the control codes except the carriage return, line feed, and other “spacing” characters. Because what are they doing in the data proper? They are in-band signals.

2. You use the Escape character to escape them

3. Weirdest option: if you really want to nest in a limited way you can still use the group and file separator characters

Re: The Elegance of the ASCII Table

#69

Post by keybored »

penteract wrote: post #16
thaumasiotes wrote: post #15

Earlier quoted context omitted.

That's because the intended purpose is either useless (for machine control characters) or useless and logically impossible (for delimiters). What do you do if you have a record that includes a record separator character? Given that you have this problem anyway, why do you want a character dedicated to achieving the same thing that a comma achieves?

The record separator isn't on people's keyboards, so it's less likely to show up where it's not expected. Also it's less likely to legitimately occur in something like a name, so there are many users of CSVs who can say they will never need to consider data containing a record separator, and they will be right more often than those who never consider data containing a comma. Of course, the fact that record separators…

I can’t think of a case where someone would write a control character like that into something intended for text on purpose. So you might as well disallow it.

Re: The Elegance of the ASCII Table

#70

Post by tivert »

pwg wrote: post #14
jolmg wrote: post #10

> So when you’re reading 7-bit ASCII, if it starts with 00, it’s a non-printing character. Otherwise it’s a printing character. > The first printing character is space; it’s an invisible character, but it’s still one that has meaning to humans, so it’s not a control character (this sounds obvious today, but it was actually the source of some semantic argument when the ASCII standard was first being discussed). Hmm..…

You also have to keep in mind the "interface" for 1962-1968. The printer teletype machine. The "control codes" were to "control" the printhead. So "carriage return" meant move the "print carriage" back to the left margin. "New line" meant move the paper platen one line height of rotation to move the paper to the next line. In that context, "back space" was "move print head one space left" (rather more like a "reverse…

> Tab's being control characters then make a bit more sense, in that they cause the printhead to jump some fixed distance to the right.

Isn't that incorrect? Tab doesn't jump a "fixed distance to the right," it jumps a variable distance to the next tab-stop to the right.

Post reply on HN