Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Elegance of the ASCII Table

danq.me

Post reply on HN
Print view View original

151–160 of 189 posts

Re: The Elegance of the ASCII Table

#151

Post by Findecanor »

WalterBright wrote: post #52

Too bad we now have Unicode, an elegant castle covered with ugly graffiti and ramshackle addons. For example: 1. normalization 2. backwards running text (hey, why not add spiral running text?) 3. fonts 4. invisible characters 5. multiple code points with the same glyph 6. glyphs defined by multiple code points (gee, I thought Unicode was to get away with that mess from code pages!) 7. made up languages (Elvish? Come…

> 2. backwards running text (hey, why not add spiral running text?)

Unicode encodes code points in logical order rather than visual order: the order in which text is supposed to be collated and spoken rather than the visual order.

One tricky issue is when both directions exist in the same text. Unicode can encode nesting of text in one direction within another. For example, text consisting of an English word and a Hebrew word can be encoded as either the English embedded in Hebrew or the Hebrew embedded in English: both would render the same but collate differently.

Is there a better way?

Re: The Elegance of the ASCII Table

#152

Post by Dylan16807 »

thaumasiotes wrote: post #28
penteract wrote: post #27

Earlier quoted context omitted.

> No, they'll be right exactly as often, 0% of the time. > But their mistake will show up less frequently, causing more problems when it does. Enough people use CSVs (and have limited, small-scale use-cases) that I'd be willing to bet "less frequently" means never for at least 1% of people who use CSVs. I don't know whether the chance of no problems is worth the increased difficulty of problems that do occur - consid…

> considering that balance feels a bit silly because if you're aware there could be a problem in a context where you could choose between commas and record separators, you could just add validation or escaping. As soon as you have validation or escaping, having a record separator character loses its entire purpose. The existence of the character is predicated on the idea that you don't have to do that, and that idea…

> As soon as you have validation or escaping, having a record separator character loses its entire purpose.

Not true. Validation is easier than escaping.

Re: The Elegance of the ASCII Table

#153

Post by Dylan16807 »

NoMoreNicksLeft wrote: post #128
senkora wrote: post #125

Fun fact: sorting ASCII numerically puts all the uppercase letters first, followed by all the lowercase letters (ABC... abc...). A more typical dictionary ordering would be more like AaBbCc... (or to even consider A and a at the same sort level and only use them to break ties if the words are otherwise identical). The order used by ASCII is sometimes called "ASCIIbetical", which I think is wonderful. https://en.wikti…

I thought the point of that was that a single bitflip makes an uppercase lower, or vice versa...

It can't be "the" point, because AaBbCc would also let you use a single bit to control case, the bottom bit.

Re: The Elegance of the ASCII Table

#154

Post by MisterTea »

WalterBright wrote: post #52

Too bad we now have Unicode, an elegant castle covered with ugly graffiti and ramshackle addons. For example: 1. normalization 2. backwards running text (hey, why not add spiral running text?) 3. fonts 4. invisible characters 5. multiple code points with the same glyph 6. glyphs defined by multiple code points (gee, I thought Unicode was to get away with that mess from code pages!) 7. made up languages (Elvish? Come…

Most of this is pretty useful for reproducing a wide gamut of human language. It gets completely fucked when it comes to fonts with png's embedded in svg's and other INSANE matryoshka doll nesting of bitmap/vector rendering technologies.

I also half hate emoji as it pollutes human writable text with bitmaps that are difficult to reproduce by hand on paper with a writing instrument - it's not text. I say half hate as it allows us a standard set of icons to use that can be easily rendered in-line with text or on their own.

Re: The Elegance of the ASCII Table

#155

Post by WalterBright »

p_l wrote: post #61
WalterBright wrote: post #52

Too bad we now have Unicode, an elegant castle covered with ugly graffiti and ramshackle addons. For example: 1. normalization 2. backwards running text (hey, why not add spiral running text?) 3. fonts 4. invisible characters 5. multiple code points with the same glyph 6. glyphs defined by multiple code points (gee, I thought Unicode was to get away with that mess from code pages!) 7. made up languages (Elvish? Come…

How to say you don't know what Unicode is for without saying it. 1, 2, 4, 5, 6, and, unfortunately, 8, all fall under "ability to encode written text from all human languages". And that includes historical. Some of the issues (5 & 6) are due semantic difference even if the resulting glyph looks the same. Unfortunately you can't expect programmers to understand pesky little thing like languages having different writin…

> How to say you don't know what Unicode is for without saying it.

I know what its original mission was, which was a character set.

It's been mangled beyond recognition - by including semantic information which is in the purview of context, and presentation information (italics, fonts) which is in the purview of markup languages and layout information (backwards text) which is also in the purview of markup.

> you can't expect programmers to understand pesky little thing like languages having different writing,

But you're requiring programmers to understand all the complicated normalization rules? Normalization is a totally unnecessary feature. Just use the normalized code points. Done.

> these print the same but are semantically different!

Think about what this means. How ever did people manage to read and understand printed books? The semantic meaning comes from the context, not the glyph. For example, I can use `a` to mean the `ahh` sound, or the `ayy` sound, or mean a variable in algebra. How can I know which? The context.

It is totally impossible to add every meaning a glyph has.

> would lose semantic information

Unicode is supposed to be a character set. That's it. Characters do not have semantic information without context.

Oh, and here's some drawkcab text I wrote without any help from Unicode at all.

I had to add some code into the D compiler to reject Unicode text direction "characters" because they can be used to invisibly insert malware into ordinary code.

Adding toy "languages" should be for people having fun, not Unicode.

Re: The Elegance of the ASCII Table

#156

Post by WalterBright »

mnau wrote: post #91
WalterBright wrote: post #52

Too bad we now have Unicode, an elegant castle covered with ugly graffiti and ramshackle addons. For example: 1. normalization 2. backwards running text (hey, why not add spiral running text?) 3. fonts 4. invisible characters 5. multiple code points with the same glyph 6. glyphs defined by multiple code points (gee, I thought Unicode was to get away with that mess from code pages!) 7. made up languages (Elvish? Come…

As someone that whose native language isn't representable purely by ASCII, I celebrate it. Plus the first 128 codepoints are same as ASCII in UT-8. Is Unicode kind of messy? Sure, but that's just natural consequences of writing systems being messy. Every point you made was for a sensible reason that is in a scope of Unicode mission (representing all text in all writing systems).

I'm sure that books can be printed in your language without any need for semantic information in the characters.

Re: The Elegance of the ASCII Table

#157

Post by WalterBright »

Findecanor wrote: post #151
WalterBright wrote: post #52

Too bad we now have Unicode, an elegant castle covered with ugly graffiti and ramshackle addons. For example: 1. normalization 2. backwards running text (hey, why not add spiral running text?) 3. fonts 4. invisible characters 5. multiple code points with the same glyph 6. glyphs defined by multiple code points (gee, I thought Unicode was to get away with that mess from code pages!) 7. made up languages (Elvish? Come…

> 2. backwards running text (hey, why not add spiral running text?) Unicode encodes code points in logical order rather than visual order: the order in which text is supposed to be collated and spoken rather than the visual order. One tricky issue is when both directions exist in the same text. Unicode can encode nesting of text in one direction within another. For example, text consisting of an English word and a He…

I've seen newspapers with txet sdrawkcab in them. Note that the last sentence has text in both directions.

I didn't need Unicode for that - nobody does.

    Uni
    ! c
    edo
Why is there no Unicode markup for that?

Re: The Elegance of the ASCII Table

#158

Post by mnau »

WalterBright wrote: post #156
mnau wrote: post #91

Earlier quoted context omitted.

As someone that whose native language isn't representable purely by ASCII, I celebrate it. Plus the first 128 codepoints are same as ASCII in UT-8. Is Unicode kind of messy? Sure, but that's just natural consequences of writing systems being messy. Every point you made was for a sensible reason that is in a scope of Unicode mission (representing all text in all writing systems).

I'm sure that books can be printed in your language without any need for semantic information in the characters.

Yes, they can.

Is it a problem that they do? I don't think so. Using semantic symbols seems far better option. Most fonts simply map multiple codepoints to a single glyph while dealing with all fun stuff like ligatures and all fun from GSUB tables (and fun company tables in fonts).

Honestly, I see semantic information as an absolute win and good choice. If unicode didn't contain it, it would have to be somewhere else (or making ratehr unpleasant choices like having fj together). It's an illusion that it wouldn't. People want pretty text. Rest of the world doesn't care. They want pretty text everywhere.

Instead of hating unicode, there would be hating "glyph points" plus "markup" (that would be literally everwere, from email to form editors) have all kinds of problems.

Re: The Elegance of the ASCII Table

#159

Post by snvzz »

Gormo wrote: post #140
snvzz wrote: post #75

The ASCII table is defective; it is missing a dedicated code for newline. CR and LF aren't dedicated, and have precise cursor movement meanings, rather than being a logical line ender. There was a proposal in the 80s to reassigning the -otherwise useless- VT (vertical tab) character for the purpose. Unfortunately unfruitful.

> Unfortunately unfruitful. Fortunately unfruitful, since if it had gained adoption, there'd be a mix of three different line endings (and combinations thereof) in widespread use, instead of two.

>widespread use

We are talking about the early 80s at worst. It would have consolidated by now.

Re: The Elegance of the ASCII Table

#160

Post by augusto-moura »

fitsumbelay wrote: post #136
augusto-moura wrote: post #67

Useful tip, on linux (not sure about other *nixes) you can view the ascii table by opening its manpage: man ascii It's been useful to me more than once every year, mostly to know about shell escape codes and when doing weird character ranges in regex and C. It can be a bit confusing, but the gist is that you have 2 chars being show in each line, I would prefer a view where you see the same char with shift and/or ctrl…

strange: on MacOS 14.5 I get output for `man ascii` but `ascii` goes "command not found"

Not sure why anyone would downvote your comment, because it is a genuine question

`man` is basically manual documentation on anything on your system, not only commands. Most commands do have a manpage for them, but it is not a requirement. The argument of the command is just the file name for the document

Post reply on HN