Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ASCII and Unicode quotation marks

cl.cam.ac.uk

Post reply on HN
Print view View original

111–120 of 195 posts

Re: ASCII and Unicode quotation marks

#111

Post by alanh »

mxfh wrote: post #74
mark-r wrote: post #43

Earlier quoted context omitted.

Those should be added to the document, with a note that they are NOT quotes!

I doubt this will ever be updated, since it's reference for a very specific, 20 year old, code interpretation related proposal und not meant for type setting. But for anything related to contemporary typesetting on the web I recommend Practical Typography and especially the Type Composition chapter: http://practicaltypography.com/type-composition.html#links Including notes on quotes and apostrophes: http://practicalt…

Hmm. I was going to link the classic https://alistapart.com/article/emen (and I guess I just did), but I noticed a disclaimer at the top pointing out that it “is now obsolete.” And how! It proclaims that not enough text editors support UTF-8 yet, which thankfully hasn’t been true in ages

Re: ASCII and Unicode quotation marks

#112

Post by jrochkind1 »

ComputerGuru wrote: post #92
diamondo25 wrote: post #71

Earlier quoted context omitted.

I've been trying to push TSV (tab separated values) as a standard response/implementation when they ask for CSV. "Yes but its comma separated!", sure is, but text can contain commas... I have seen issues with Google Spreadsheets not recognizing the tabs however... Excel doesnt know what to do with a TSV either. But both have a complete wizard for parsing CSV...

> but text can contain commas Erm.. text can contain tabs, too. This problem was solved so, so long ago when all the various ANSI/ASCII/whatever encodings were compiled by specifically reserving not one but two characters precisely to serve as field and record delimiters. 0x30 and 0x31 solved not only the problem of having commas or tabs in your text preventing you from treating them as field delimiters, but also all…

I find no matter what you do you will _sometimes_ need escaping. There will, eventually, come a time when you want to embed an ASCII 30 (0x usually means hex, it's actually decimal code 30, but hex 0x1E) RS Record Separator in some record delimited by 30 RS. So you'll need some method of escaping anyway. Or it'll be annoying.

I have spent some time working with MARC 21 binary encoding (used for library cataloging records) which uses ASCII 0x1D, 0x1E, and 0X1F as delimiters. I would def not call it appreciably more _convenient_ than a more modern 'text' record format. If it has benefits, convenience isn't really one of them.

Re: ASCII and Unicode quotation marks

#113

Post by jrochkind1 »

Pxtl wrote: post #106
drdeca wrote: post #59

Earlier quoted context omitted.

I think it would make the parsing a tiny bit more inconvenient though

Yup, suddenly your parser needs to keep a count of how many nested strings deep it is.

ruby does that. computers are pretty fast now.

Re: ASCII and Unicode quotation marks

#114

Post by int_19h »

tedunangst wrote: post #63
hk__2 wrote: post #58

Earlier quoted context omitted.

You don’t need escaping in «This is a string containing an «embedded» quoted string».

Debatable whether that would actually work in practice. /* nested /* comments */ don't work */

They don't work in C, but that's an arbitrary decision made by its designers. There are many languages that have balanced comments that can be nested. In OCaml, for example, this is legal:

    (* nested (* comments *) work *)

Re: ASCII and Unicode quotation marks

#115

Post by 13of40 »

delinka wrote: post #99
13of40 wrote: post #97

CSB: Years ago I was working on a team that developed a scripting language and we had this recurring problem where someone would write up a code sample in a Word document and it would break if you cut and pasted it because all of the single and double quotes would be Unicode. My boss was this tough guy who tried to snap the whole team to a standard of strictly disabling that behavior in all of our Office applications…

Or ... use a text editor?

You know a lot of PMs who write specs in notepad?

Re: ASCII and Unicode quotation marks

#116

Post by jrochkind1 »

moron4hire wrote: post #81
kps wrote: post #33

Earlier quoted context omitted.

Yes. I consider it a mistake of Unicode that combining characters follow rather than precede the base character. If they preceded, most dead keys could simply generate the appropriate combining character, rather than requiring complicated input method support. (And finding the end of a sequence of multiple combining character wouldn't require lookahead.)

The Unicode way makes sorting easier. Your way would require special knowledge about the characters to know that ä should sort directly after a, rather than directly before ë.

In fact this requires special language-specific knowledge anyway (which unicode provides in some tables and algorithms actually). In some languages ä should sort exactly as if it were 'a'. "aa", "äb", "ac". In others it should sort as a distinct letter (but not necessarily between 'a' and 'b'). Different Latin languages sort differently, I'm not sure if exact UTF-8 (or UTF 16 or UTF 32) byte ordering is actually appropriate collation in any latin-alphabet language.

But I do suspect it had something to do with ascii compatibility, I don't recall what. Very little of unicode is accidental, there's usually some reason for whatever in it.

Re: ASCII and Unicode quotation marks

#117

Post by jiggunjer »

PeterisP wrote: post #31
koolba wrote: post #28

Earlier quoted context omitted.

It doesn't need to. You can put the mdash directly in comments: — As opposed to regular dash: -

Well, since my keyboard doesn't have an mdash key, if there's no support for something like — or --- then I can't use mdashes.

Keyboards and typing aids can enter any character if you configure them properly.

Re: ASCII and Unicode quotation marks

#118

Post by int_19h »

evincarofautumn wrote: post #93
ttepasse wrote: post #90

The usage of of an accent as syntax in markup and programming languages annoys me to no end. And it will still be used, to this day, the latest example are template string in Javascript. • It is semantically idiotic because it's an accent, not a character. • It is visually annoying because you almost can't see the thing. • It is bad for usability, because on non-US keyboards the accents are implemented as dead keys.…

Same, I’ve never cared for it. For these reasons I’ve decided to take a stand and avoid using the grave accent for anything in a programming language I’m working on. Same goes for the dollar sign, because it’s somewhat Americentric, and as a currency character it doesn’t have any great semantic or mnemonic value except for, well, currency units. I guess you could argue for $trings (BASIC) or $calars (Perl) if you hav…

> Same goes for the dollar sign, because it’s somewhat Americentric, and as a currency character it doesn’t have any great semantic or mnemonic value except for, well, currency units.

By that metric, wouldn't & be too Anglo-centric, and # be too Euro-centric? There are layouts out there on which neither is readily available.

Re: ASCII and Unicode quotation marks

#119

Post by xelxebar »

eesmith wrote: post #48
lisper wrote: post #41

The fact that ASCII does not have balanced quotes is one of the great catastrophes of computing. It makes everything more complicated than it needs to be, from embedding code in strings to parsing CSV files, to regexps. For example, if I want to embed a quoted string in another quoted string, I have to escape the inner quotes like so: "This is string containing an embedded \"quoted\" string" Then I have to think abou…

The complexity might be minimized, but not avoided. You would still need an escape mechanism for something like «She said «The \» key on the server doesn't work.»» ASCII did add , [], and {}, any of which could have been used for quoted strings, had the programming language designers chosen that option. https://en.wikipedia.org/wiki/String_literal#Paired_delimite... points out that PostScript and Tcl have a string li…

> You would still need an escape mechanism for ...

I think this is actually desirable, since in your case the escape denotes different semantics. The unescaped pairs act like quotation operators while the escaped version is a character literal.

Re: ASCII and Unicode quotation marks

#120

Post by userbinator »

Pxtl wrote: post #106
drdeca wrote: post #59

Earlier quoted context omitted.

I think it would make the parsing a tiny bit more inconvenient though

Yup, suddenly your parser needs to keep a count of how many nested strings deep it is.

Don't forget that "parser" also includes human brains, which tend to not be that great at parsing nested things.

To use formal language theory, strings containing escape characters are regular, i.e. parseable with a finite-state machine. Allowing nesting means you need a stack to find the matching pair.

Post reply on HN